Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix: granted fix-check for the filter hold

garrytanec1749c3e824
diff --git a/docs/architecture/key-files/files-and-sync-2.md b/docs/architecture/key-files/files-and-sync-2.md
index 047b20e509..8f4e3d3a50 100644
--- a/docs/architecture/key-files/files-and-sync-2.md
+++ b/docs/architecture/key-files/files-and-sync-2.md
@@ -26,7 +26,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/persistence/sync-prescreen.ts` — #6712 PR 1 (L1): the discovery prescreen and drift baseline. After `discoverManagedSync` freezes identities, `prescreenSyncEntries` reads the pinned blobs (`readTreeBlobs`, one recursive listing for a large set) and the import provenance of every Git import of an existing page, and `classifyBaseline` marks each `prescreened` (disk blob, revision and `syncImportParserFor` fingerprint all match; bypassed under `--full` and for held paths) or `drift` (`database`: the page moved, the file did not; `both`/`unknown` in a tree walk or for a `file_drift` hold). A page whose current revision a committed managed-sync request wrote, or whose writer's publication wrote to its file (a write-through `git` effect), is not guarded (`fileBackedRevisions`), and drifted pages under `storage.keep_db` prefixes leave the manifest as drift candidates (`releaseKeepDbDrift`, `recordDriftSkip`). The run passes prescreened entries through `waivePrescreenedRun` (`sync-waivers.ts`); `freezeEntry` holds a drifted import `file_drift` (`fileDriftHold`, `sync-page-fault.ts`) unless `sameContentAtOrigin` proves it a no-op. `movedDriftHolds` re-screens drift holds whose page moved. Tests: `test/sync-prescreen-drift-6712.test.ts` (both engines).
- `src/core/persistence/sync-image.ts` — #6529: managed sync's image branch. With multimodal embedding on, `freezeEntry` (`sync-run.ts`) freezes an image entry as its bytes in base64 (`frozenImageIntent`; `rawHash` stays the file sha256, so the working-tree check and the hash no-op are unchanged) and `prepareManagedSyncMutation` (`sync-prepare.ts`) hands it to `prepareManagedSyncImage`, which runs `importImageFile` with `prepare` (the coordinated path `gbrain import` uses in `import-prepare.ts`): the importer decides the hash no-op, OCRs and embeds the image at preparation inside the sync budget (`--no-embed` skips both; the caption chunk's text vector is deferred to `embed --stale`), and the apply records `source_path`, releases the path's hold and re-reads the snapshot. A company plan refuses (`profile_incompatible`), a screening preparation (`sameContentAtOrigin`, the no-op waiver in `sync-waivers.ts` and `unchangedManagedImport` in `import-mutations.ts` return early for images) is refused rather than paid for, newer working-tree bytes or a vanished file refuse `source_changed`, and a Git rename publishes the new slug and soft-deletes the old page. `import-file.ts` records `source_path` on the legacy importer's hash-skip path too (`refreshSourcePath`), so no image import leaves it NULL. Tests: `test/managed-sync-images-5493.test.ts` (both engines), `test/embed-stale-images-managed-6529.test.ts`.
- `src/core/persistence/sync-streamed.ts` — Wave B PR 2: managed sync of a Markdown file over the 24 MiB single-string ceiling. With `pages.segmented_storage` on, `freezeEntry` (`sync-run.ts`) freezes such a file by reference instead of holding it by its size: `streamedSyncBodyRef` hashes the sha256 and the Git blob id as a stream (sha1 or sha256 by the pinned commit's length) into `intent.body_ref` (`git_blob`, or `input_file` for an uncommitted working-tree file); the journal carries no content, and a committed entry whose working tree does not hold its pinned blob, a file over `pages.max_bytes` or a non-Markdown file is held as before. `prepareManagedSyncMutation` (`sync-prepare.ts`) hands it to `prepareManagedSyncStreamed`, which streams it through `importStreamedMarkdownFile` (`page-body/streamed-import.ts`, `coordinated`; `moved` on a rename keeps an implicit type following the new path; `slug` is the admitted page, the one recording the file as its origin, as the inline import targets) and publishes database-only, recording the origin through `bindOrigin` (B-14: a renamed page keeping its origin takes the file, no twin at the path's slug); a rename moves the page first and re-seals an unchanged body (`sealSegmentedPage`). A file the streamed path refuses (a facts fence) fails its request with the refusal's code and the run holds it (`streamedRefusalHold` in `sync-screen.ts`); a dry run lists it in `screen_skipped` as `page_body_segmented`; `groupableIntent` (`sync-group.ts`) keeps it out of bulk groups. Tests: `test/large-pages/managed-sync-streamed.test.ts` (both engines), `managed-sync-streamed-crash.test.ts` (owner killed at `consumer:prepared`), `managed-sync-streamed-memory.slow.test.ts` (VmHWM), `streamed-sync-origin-b14.test.ts` (both engines).
-- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `filterOwnedRefusal` (GST-2312, #9024), checked first on the same two paths, holds `content_filter_owned` a blob starting with git-crypt's header or a committed blob whose path names a Git filter driver other than `lfs` that the Git config defines (`pinnedFilterDriver` in `sync-screen.ts`, at the pinned commit or a company plan's revision, read with `readFilterDrivers`' batched `git check-attr --source` (git 2.40+) and one `git config --get-regexp`, neither of which runs a program). `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes; a page-body fault (`PAGE_BODY_FAULT_CODES`: `page_body_segmented`, `page_body_rows_unreadable`, e.g. a segmented page whose segment rows don't cover its body) is held `page_write_failed` with `write_failure.code` set to that code, which `sync status` classifies by the `freeze_page_body_*` rows of `SYNC_FAULT_TABLE`; any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.
+- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `filterOwnedRefusal` (GST-2312, #9024), checked first on the same two paths, holds `content_filter_owned` a blob starting with git-crypt's header or a committed blob whose path names a Git filter driver other than `lfs`, configured here or not (`pinnedFilterDriver` in `sync-screen.ts`, at the pinned commit or a company plan's revision, read with `readFilterDrivers`' batched `git check-attr --source` on git 2.40+, or `read-tree` into a temporary index and `check-attr --cached` on an older git; `NO_REPOSITORY_PROGRAMS` keeps `core.fsmonitor` and hooks off these reads). `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes; a page-body fault (`PAGE_BODY_FAULT_CODES`: `page_body_segmented`, `page_body_rows_unreadable`, e.g. a segmented page whose segment rows don't cover its body) is held `page_write_failed` with `write_failure.code` set to that code, which `sync status` classifies by the `freeze_page_body_*` rows of `SYNC_FAULT_TABLE`; any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.
- `src/core/url-clone.ts` — gbrain-owned `sources add --url` clones (#6946, GBRA-130 C-2). `isOwnedClone` (config `managed_clone` or a `local_path` under the default clone dir) and `defaultCloneDir`; `urlCloneReadOnly` is the one predicate (owned clone on a brain without managed persistence) the fence writer (`fence-write.ts`, `databaseOnlyReason: 'url_clone'`), write-through (`skipped: 'url_clone'`), timeline write-through, unfenced-facts adoption, the empty-fence guard and `sourceMirrorReadOnly` consult, so nothing gbrain writes lands in a checkout it updates with `git pull --ff-only`. `urlClonePullBlock` reads the tracked changes of a clone whose pull failed and returns `pull_blocked_local_changes` with an `ask_user` fix naming the preview `gbrain repair url-clone-fences --source <id>`; `formatUrlClonePullBlock` renders it for sync's stderr (`sync/preflight.ts`) and `cycle.ts` puts it in the sync phase detail as `pull_blocked`. Test: `test/pathological/c2-url-clone-fences.test.ts`.
- `src/core/url-source-origin.ts`, `src/commands/sources-record-origin.ts` and `src/commands/doctor/checks/url-source-origin.ts` — who added a URL source (`sources.config.added_by_remote_caller`). Both add paths write it (`true` for a remote caller, `false` for the local operator); only an explicit `true` reads as remote. URL sources added before that record existed carry none, and nothing stored tells a remote caller's add from the local operator's (same row, same default clone directory); a clone outside `$GBRAIN_HOME/clones/<id>` (or a `clones/<id>` under an earlier home) proves a local add. A URL clone counts when `isOwnedClone` accepts it, so an unmarked clone at the default directory is listed too. `urlSourcesWithoutOrigin` (read-only) feeds the wave check `url_source_origin` (host-only, `resolution: 'operator'`, `warn` with an `ask_user` fix whose argv is the preview). `recordSourceOrigin(engine, ids, { origin, apply, remote })` backs `gbrain sources record-origin <id>... --remote|--local [--apply] [--json]`: it throws `trusted_local_only` unless `remote === false`, previews unless `apply`, writes `true` (`--remote`) or `false` (`--local`) in one transaction that re-reads the row `FOR UPDATE` and refuses `source_changed` if its path or URL moved; on a managed brain that transaction runs inside `withTopologyLocks` after `lockTopologyRows`, the locks a reclone takes (a held lock refuses the id `source_busy`), and `runManagedSourceClone` takes the config it clones with from its own locked admission read, so a record and a reclone never interleave. `--remote --apply` also writes `LFS_OFF_CLONE_CONFIG` into the repository's shared config (`--git-common-dir`; `git config --file`, not run in the checkout) before the row, reads the effective values back (`lockDownGaps`) and throws when a worktree-scoped or included setting overrides one, and locks down an already-remote source whose checkout lacks it (no git-lfs refusal for it unless Git cannot place the checkout, `lfs_unknown`); each key's existing values are removed before its one value is written, in a `config.lock` copy renamed over the config in one step; a throw is reported for its id as `write_failed` and the batch continues. It never removes a `true`, and refuses `--remote` for a checkout with any tracked `filter=lfs` path or a committed `.gitattributes` declaring `filter=lfs` outside a comment line (`lfs_tracked`, worded per case; read with `hardenedGitSync` `ls-files -- ':(attr:filter=lfs)'` plus every committed `.gitattributes` blob from `ls-tree -z` records split at their first tab, then `cat-file`, so clone-local attributes cannot hide one, and read again under the locks), or one it cannot inspect (`lfs_unknown`, whose reason names the cause: checkout missing, not its own repository's top level by a realpath comparison, owned by another user (`safe.directory`), or a Git read failed), because a lock-down reclone would not match the stored writer manifest there. An unborn HEAD with an empty index (a clone of an empty repository) has no committed attributes and passes. Without git-lfs files the tracked bytes are the same, so no manifest is written. Pinned by `test/url-source-origin.test.ts` (PGLite and Postgres).
diff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.md
index dfe07d0d0f..13b2e26e2e 100644
--- a/docs/guides/write-refusals.md
+++ b/docs/guides/write-refusals.md
@@ -108,7 +108,7 @@ of making a duplicate.
| `file_unreadable` (sync hold) | `file_unreadable` | A managed sync held one file because it is not a readable regular file inside the source root: a symlink (to a file outside the root or inside it), a non-regular file, or a permission or loop error. Sync never reads or follows it. Until GBRA-130 the whole sync stopped at that file on every run and the rest of the source never imported; now only the file waits. A source root that is itself replaced by a symlink still refuses the whole sync. | Make the file a readable regular file in the checkout (replace the symlink with the file, fix its permissions) or exclude it with `sync.exclude`, and commit. A commit that changes it re-screens the hold on the next sync; after a permission fix run `gbrain sources retry-held <source>`, then the same sync. |
| <a id="page_write_failed"></a>`page_write_failed` (sync hold) | `page_write_failed` | A managed sync held one file because its write failed the same way three times (the same code, origin, error class and SQLSTATE in this source), with an error gbrain cannot classify: an owner exception (for example Postgres 54000 when a frontmatter tag is too long for its index), or a refusal the receipt does not explain. Until GBRA-130 every sync stopped `blocked_by_failures` at that file and the rest of the source never imported; now the file is held and the rest of the source syncs. The first two failures still stop the run, so a one-off defect never holds a file. | Read the receipt the hold names (`gbrain write-request <request_id>`, read-only). Fix the file when the error names its content (shorten the tag, split the page), or report the receipt with the gbrain version. Then `gbrain sources retry-held <source>` and the same sync. An upgrade re-tests the file on its own. A hold whose `meta.write_failure.origin` is `freeze` (GBRA-133) is different: sync could not even prepare the file (an unexpected error while reading or screening it, before any request was admitted), so it holds it on the first failure and names the error in the hold message; the next syncs re-test it on their own until it has been held 3 times, so a transient fault clears without help. When most files of one run fail that way the run stops instead (the cause is not the files). |
| <a id="content_null_bytes"></a>`content_null_bytes` (sync hold) | `content_null_bytes` | A managed sync held one Markdown file because its text holds NUL bytes (U+0000). Page text is stored as UTF-8 without NUL bytes. The usual cause is a UTF-16 file (a Windows "Unicode" save; the hold says so when the file starts with a UTF-16 byte order mark), a binary file named `.md` or a crash-padded file. Until GBRA-133 the whole managed sync stopped at that file on every run (`storage_error`, SQLSTATE 22P05) and the rest of the source never imported; now only the file waits. With `sync.holds=fail` the run stops with `content_null_bytes` instead of holding. | Save a UTF-16 file as UTF-8, remove the NUL bytes, or remove a binary file (or leave it out with `sync.exclude`), then commit and `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. |
-| <a id="content_filter_owned"></a>`content_filter_owned` (sync hold) | `content_filter_owned` | A managed sync held one committed file because Git stores it through a filter: its path has a `filter` attribute naming a clean/smudge driver your Git config defines, or its bytes start with git-crypt's header. The committed bytes are what the filter's clean program wrote (git-crypt ciphertext, for one), not the text in your checkout, and gbrain does not run a repository's filter programs during sync, so it cannot turn them into text. Sync reads the attribute with `git check-attr` (at the synced commit with git 2.40 or newer) and the driver with `git config`, neither of which runs a program; a `filter` attribute whose driver has no config changes nothing, since Git then stores the plain text. Before, a filtered file imported its stored bytes as the page text, and a git-crypt file was held `content_null_bytes`. A page imported from such a file before this release is left as it is until the file changes. Git LFS pointers are not this hold. With `sync.holds=fail`, and on a company-brain source, the run stops with `content_filter_owned` instead of holding. | Leave the file out with `sync.exclude`, or commit it without the filter so Git stores its text, then `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. |
+| <a id="content_filter_owned"></a>`content_filter_owned` (sync hold) | `content_filter_owned` | A managed sync held one committed file because Git stores it through a filter: its path has a `filter` attribute at the synced commit naming a clean/smudge driver (whether or not this checkout's Git config defines it, since the commit may have been cleaned on another machine), or its bytes start with git-crypt's header. The committed bytes are what the filter's clean program wrote (git-crypt ciphertext, for one), not the text in your checkout, and gbrain does not run a repository's filter programs during sync, so it cannot turn them into text. Sync reads the attribute at the synced commit with `git check-attr --source` (git 2.40 or newer) or, on an older git, from a temporary index of that commit with `check-attr --cached`, with `core.fsmonitor` and hooks switched off, so no program runs. A plain-text file under a `filter` attribute nobody uses is held too: the hold is the safe side and loses nothing. Before, a filtered file imported its stored bytes as the page text, and a git-crypt file was held `content_null_bytes`. A page imported from such a file before this release is left as it is until the file changes. Git LFS pointers are not this hold. With `sync.holds=fail`, and on a company-brain source, the run stops with `content_filter_owned` instead of holding. | Leave the file out with `sync.exclude`, or commit it without the filter so Git stores its text, then `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. For a plain-text file under a filter attribute nobody uses, remove the attribute from `.gitattributes`, commit, and run `gbrain sources retry-held <source>`: a commit that changes only `.gitattributes` does not re-screen the file. |
| <a id="git_object_missing"></a>`git_object_missing` (sync hold) | `git_object_missing` | A managed sync held one committed file because the commit names a Git object the checkout no longer has (an interrupted `git gc`, a partial clone without its promisor remote, disk corruption) and the working-tree copy differs from it or is gone. When the working-tree bytes hash to the committed blob id, sync imports them and holds nothing. Until GBRA-133 the whole managed sync stopped at that file on every run. | Run `git fsck --connectivity-only` in the checkout to see what is missing, restore it with `git fetch` (a partial or shallow clone) or from a backup, then `gbrain sync --source <source> --no-pull`. Every sync re-tests the hold. |
| <a id="path_not_utf8"></a>`path_not_utf8` (sync hold) | `path_not_utf8` | A managed sync held one committed file because its name is not valid UTF-8 (a Latin-1 name from an older system, an archive or a network share). Git hands the name back with U+FFFD for the bytes that are not UTF-8, and no Git read or file matches that decoded name, so nothing is imported from it. Until GBRA-133 the managed sync either stopped at that file on every run or skipped it with no hold; now only the file waits and the hold names it. With `sync.holds=fail` the run stops with `path_not_utf8` instead of holding. | Rename the file to a UTF-8 name (`git -c core.quotePath=true ls-files` shows its exact bytes; `convmv -f latin1 -t utf8` converts a Latin-1 name), commit, then `gbrain sync --source <source> --no-pull`. The rename clears the hold. |
| <a id="write_interrupted"></a>`write_interrupted` | `write_interrupted` | A transient interruption kept ending this managed write: the database ended its own statements (SQLSTATE `57014`, a statement timeout or cancel) at preparation or inside its publication transaction, or its publication ran past `persistence.publication_ceiling_ms` and its grace and the owner cancelled it. Each interruption throws the preparation away, but it is load or latency, not the write's content, so it has its own budget: the request is claimed again after a backoff (1 s, 2 s, 4 s, 8 s, then 15 s, scheduled on the request as `next_attempt_at`; no claim, root lock or connection is held while it waits, and `gbrain sources writer status` names the waiting head `transient_backoff`) up to `persistence.max_transient_retries` times (default 5, 0 to 20), and the next interruption finishes it `failed` instead of preparing it again without limit. Interruptions never spend `persistence.max_preparation_attempts`, which preparations cut off at their deadline keep ([`preparation_stalled`](#preparation_stalled)). Nothing was published: a worktree-backed write's canonical file is restored before the failure is recorded. Contention with other transactions (`55P03` lock timeout, `40001` serialization failure, `40P01` deadlock) is released as `database_contention` uncounted, and a dropped connection or a database that is restarting or out of connections is never counted. The receipt's message and owner-only `error_detail` name the interruptions (`attempts`), the retry limit (`limit`, `limit_key`), the `cause` (`statement_cancel` or `publication_ceiling`) and the SQLSTATE (`sqlstate`, public). A managed sync member is held like [`preparation_stalled`](#write_interrupted-hold): the run moves past it and the rest of the source lands, `gbrain sync status` lists the hold (class `page`, safe action `retry`, [runbook row](sync-unblock-runbook.md)), and `gbrain sources retry-held` re-admits it. A group's shared read the database cancels is charged once, to the first member that settles with it, never to every member; and the charge commits with the claim's release, so a crash between them never counts it twice. With `persistence.preparation_deadlines` off nothing is counted and no backoff applies. | Run `gbrain sources writer status --source <source> --json` (read-only: the owner, its pool, what its writes wait on), then submit the write again with a new `request_id` once the database is less loaded; for a held sync file, run `gbrain sources retry-held <source>` and the same sync with its options. A `publication_ceiling` cause on a page that legitimately takes longer to publish is the user's call: ask before raising `persistence.publication_ceiling_ms` (or the database `statement_timeout` for a `statement_cancel`). If the same write ends this way again, report its request id with `gbrain doctor --json`. Do not raise `persistence.max_transient_retries` to get past it. |
diff --git a/scripts/core-world-reads-baseline.tsv b/scripts/core-world-reads-baseline.tsv
index 1155ac0ad1..0f18f6acc7 100644
--- a/scripts/core-world-reads-baseline.tsv
+++ b/scripts/core-world-reads-baseline.tsv
@@ -743,6 +743,7 @@ src/core/persistence/staging.ts fs_import 1 W5: repo-less persistence for hosts
src/core/persistence/switches.ts module_collection 2 W6: per-engine state, or mark as a constant table
src/core/persistence/switches.ts process_env 1 W6: settings plane out of the core
src/core/persistence/sync-authority.ts als_store 1 W2: engine scope (decision E) replaces ad hoc async context
+src/core/persistence/sync-blobs.ts fs_import 1 W7: filesystem boundary moves to the local app (the temporary index a git before 2.40 reads attributes from, GST-2312)
src/core/persistence/sync-blobs.ts spawn_import 1 W7: process-wide effects move to src/cli
src/core/persistence/sync-capacity-preflight.ts fs_import 1 W5: repo-less persistence for hosts
src/core/persistence/sync-discovery.ts fs_import 1 W5: repo-less persistence for hosts
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index e31c45b74a..ae17074ae7 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1501,7 +1501,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.635.0', id: 'read_scope_params_gbra431', text: 'An MCP read now refuses an unknown parameter whose name is, or is nearest to, a source scope or a filter (`source` for `source_id`, `sources` for `source_ids`, `type` for `types`, `tag`, `since`, `entity`, `visibility`, `min_trust`) with `invalid_params` naming the declared parameter, whatever `mcp.strict_params` says; it used to ignore it and answer unnarrowed. Other unknown read parameters keep the warn grace period. `gbrain serve --http` now applies `GBRAIN_HTTP_RATE_LIMIT_IP`, `GBRAIN_HTTP_RATE_LIMIT_TOKEN` and `GBRAIN_HTTP_MAX_BODY_BYTES` to `/mcp` when they are set (429 with Retry-After, 413); unset, it keeps no rate limit and a 4 MiB body cap.' },
{ since: '0.60.637.0', id: 'sync_restored_file_gst2201', text: 'On a managed source, a page sync deleted because its file was gone comes back on the next sync once the file returns (a `git revert`, a branch or detached checkout and back, a later commit that re-adds it with the same or edited bytes, a file missing from disk for one working-tree sync). It used to stay deleted (`deleted_same`, or a `file_drift` hold whose suggested fix removed the file) while `gbrain sync status` said synced. The first sync after the upgrade reads the whole source once, so pages an older version left deleted beside their file come back and such holds clear without a manual step. A page a person deleted keeps its deletion.' },
{ since: '0.60.637.0', id: 'purge_keeps_committed_file_gst2201', text: 'The retention purge (the autopilot purge phase, the `purge` job and `gbrain pages purge-deleted`) never removes a soft-deleted page\'s file that is committed at its source\'s last synced commit or at HEAD. A page sync deleted whose file is back is left for the sync that restores it (counted as deferred); any other such page is removed database-only (receipt `write_through.skipped: committed_file`) and its file stays. A page whose file is gone, or a person\'s delete that never removed its file (for example because another page shared it), purges as before. A page purge (`gbrain delete <slug> --purge` and the retention purge) now also redacts the timeline text the page\'s `add_timeline_entry` receipts kept (the entry\'s summary and source, `bullet`), compacted ones included.' },
- { since: 'TBD', id: 'sync_filter_owned_hold_9024', text: 'A managed sync now holds a committed file that Git stores through a filter (a path whose `filter` attribute names a driver the Git config defines, Git LFS excepted, or a file starting with git-crypt\'s header) as `content_filter_owned`, and the rest of the source syncs. Before, such a file imported its stored bytes (git-crypt ciphertext, for one) as the page text, or a git-crypt file was held `content_null_bytes`. A `filter` attribute whose driver has no config changes nothing, and with git 2.40 or newer the attribute is read at the synced commit. gbrain still never runs a repository\'s filter programs during sync; the hold names git-crypt when the header matches and says to leave the file out with `sync.exclude` or commit it without the filter. A page an older version already imported from such a file is left as it is until the file changes; then the file is held. With `sync.holds=fail`, and on a company-brain source (which never holds), the run stops with `content_filter_owned`. Classic (unmanaged) sync is unchanged.' },
+ { since: 'TBD', id: 'sync_filter_owned_hold_9024', text: 'A managed sync now holds a committed file that Git stores through a filter (a path whose `filter` attribute at the synced commit names a driver other than Git LFS, whether or not this checkout configures that driver, or a file starting with git-crypt\'s header) as `content_filter_owned`, and the rest of the source syncs. Before, such a file imported its stored bytes (git-crypt ciphertext, for one) as the page text, or a git-crypt file was held `content_null_bytes`. gbrain still never runs a repository\'s filter programs (or `core.fsmonitor`) during sync; the hold names git-crypt when the header matches and says to leave the file out with `sync.exclude` or commit it without the filter, and for a plain-text file under a filter attribute nobody uses, to remove the attribute and run `gbrain sources retry-held`. A page an older version already imported from such a file is left as it is until the file changes; then the file is held. With `sync.holds=fail`, and on a company-brain source (which never holds), the run stops with `content_filter_owned`. Classic (unmanaged) sync is unchanged.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/persistence/sync-blobs.ts b/src/core/persistence/sync-blobs.ts
index 9fdc3cf12d..c517b2de18 100644
--- a/src/core/persistence/sync-blobs.ts
+++ b/src/core/persistence/sync-blobs.ts
@@ -8,6 +8,9 @@
* `git show` per entry.
*/
import { execFileSync } from 'node:child_process';
+import { mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
import { gitChildEnv } from '../git-env.ts';
import { PAGE_MAX_BYTES } from '../page-size-policy.ts';
@@ -16,9 +19,15 @@ export const SYNC_READ_BOUND = PAGE_MAX_BYTES;
export interface TreeBlob { oid: string; size: number }
-function git(gitRoot: string, args: string[], input?: string, maxBuffer = 32 * 1024 ** 2): Buffer {
- return execFileSync('git', ['-c', 'core.quotepath=false', '--literal-pathspecs', '-C', gitRoot, ...args],
- { env: gitChildEnv(), timeout: 60_000, maxBuffer, stdio: ['pipe', 'pipe', 'pipe'], ...(input !== undefined ? { input } : {}) });
+/**
+ * GST-2312 (#9024): the configuration every read here runs with, so no repository-configured program starts: `check-attr`
+ * reads the index, and reading the index runs `core.fsmonitor`.
+ */
+export const NO_REPOSITORY_PROGRAMS: readonly string[] = ['-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=/dev/null'];
+
+function git(gitRoot: string, args: string[], input?: string, maxBuffer = 32 * 1024 ** 2, env: NodeJS.ProcessEnv = {}): Buffer {
+ return execFileSync('git', ['-c', 'core.quotepath=false', ...NO_REPOSITORY_PROGRAMS, '--literal-pathspecs', '-C', gitRoot, ...args],
+ { env: gitChildEnv(env), timeout: 60_000, maxBuffer, stdio: ['pipe', 'pipe', 'pipe'], ...(input !== undefined ? { input } : {}) });
}
/** At this many paths `readTreeBlobs` lists the whole tree once instead of path chunks (#6712: the prescreen of a tree walk). */
@@ -70,12 +79,20 @@ export function readBlobBytes(gitRoot: string, blobs: TreeBlob[]): Map<string, B
let checkAttrSource: boolean | undefined;
+/** Test seam: `false` takes the path of a git older than 2.40; returns the restore. */
+export function __setCheckAttrSourceForTests(value: boolean | undefined): () => void {
+ const previous = checkAttrSource;
+ checkAttrSource = value;
+ return () => { checkAttrSource = previous; };
+}
+
/**
- * GST-2312 (#9024): the Git filter driver of each path at commit `target`, null where no `filter` attribute names a driver
- * the Git config defines (`filter.<driver>.clean`, `.smudge` or `.process`): Git stores a path whose driver is not
- * configured as its plain bytes. A committed blob a configured filter owns holds what the filter's clean program stored
- * (git-crypt ciphertext, for one), not the text of the checkout, and sync never runs a repository's filters. Only
- * `git check-attr` (attribute files at `target` from git 2.40, the checkout's before) and `git config` run.
+ * GST-2312 (#9024): the Git filter driver each path's `filter` attribute names at commit `target`, null where none does,
+ * whether or not this checkout's Git config defines the driver (the commit may have been cleaned on another machine). A
+ * committed blob a filter owns holds what the filter's clean program stored (git-crypt ciphertext, for one), not the text
+ * of the checkout, and sync never runs a repository's filters. Git 2.40 and newer read the attributes with
+ * `check-attr --source=<target>`; an older git reads `target` into a temporary index (`read-tree`) and asks
+ * `check-attr --cached` there. Neither runs a program (`NO_REPOSITORY_PROGRAMS`).
*/
export function readFilterDrivers(gitRoot: string, target: string, gitPaths: string[]): Map<string, string | null> {
const out = new Map<string, string | null>();
@@ -84,13 +101,17 @@ export function readFilterDrivers(gitRoot: string, target: string, gitPaths: str
const [major, minor] = (/git version (\d+)\.(\d+)/.exec(git(gitRoot, ['version']).toString('utf8')) ?? ['', '0', '0']).slice(1).map(Number);
checkAttrSource = major! > 2 || major === 2 && minor! >= 40;
}
- let config = '';
- try { config = git(gitRoot, ['config', '-z', '--get-regexp', '^filter\\.']).toString('utf8'); } catch (error) { if ((error as { status?: number }).status !== 1) throw error; }
- const configured = new Set(config.split('\0').map(record => record.split('\n')[0]!).filter(key => /\.(clean|smudge|process)$/.test(key))
- .map(key => key.slice('filter.'.length, key.lastIndexOf('.'))));
- for (let i = 0; i < gitPaths.length; i += 2000) {
- const fields = git(gitRoot, ['check-attr', ...(checkAttrSource ? [`--source=${target}`] : []), '-z', '--stdin', 'filter'], gitPaths.slice(i, i + 2000).join('\0') + '\0').toString('utf8').split('\0');
- for (let at = 0; at + 2 < fields.length; at += 3) out.set(fields[at]!, configured.has(fields[at + 2]!) ? fields[at + 2]! : null);
+ const scratch = checkAttrSource ? null : mkdtempSync(join(tmpdir(), 'gbrain-attributes-'));
+ const env = scratch ? { GIT_INDEX_FILE: join(scratch, 'index') } : {};
+ try {
+ if (scratch) git(gitRoot, ['read-tree', target], undefined, undefined, env);
+ for (let i = 0; i < gitPaths.length; i += 2000) {
+ const fields = git(gitRoot, ['check-attr', ...(scratch ? ['--cached'] : [`--source=${target}`]), '-z', '--stdin', 'filter'],
+ gitPaths.slice(i, i + 2000).join('\0') + '\0', undefined, env).toString('utf8').split('\0');
+ for (let at = 0; at + 2 < fields.length; at += 3) out.set(fields[at]!, ['unspecified', 'unset', 'set'].includes(fields[at + 2]!) ? null : fields[at + 2]!);
+ }
+ } finally {
+ if (scratch) rmSync(scratch, { recursive: true, force: true });
}
return out;
}
diff --git a/src/core/persistence/sync-discovery.ts b/src/core/persistence/sync-discovery.ts
index 5b0b198572..78b0a3c6c7 100644
--- a/src/core/persistence/sync-discovery.ts
+++ b/src/core/persistence/sync-discovery.ts
@@ -33,7 +33,7 @@ import { assertManagedSyncActive } from './sync-authority.ts';
import { decideOriginTwins, readOriginClaimants, slugPathOccupied, type OriginConflict } from './origin-twins.ts';
import { isReservedSkillBundlePath } from '../skill-reserved-paths.ts';
import { holdBasisMoved, holdRescreenDue, readGitHoldRetryPaths, readGitSourceHolds, readSyncImportProvenances } from './sync-holds.ts';
-import { readBlobContents, readTreeBlobs, SYNC_READ_BOUND } from './sync-blobs.ts';
+import { NO_REPOSITORY_PROGRAMS, readBlobContents, readTreeBlobs, SYNC_READ_BOUND } from './sync-blobs.ts';
import { gitChildEnv } from '../git-env.ts';
import { readOriginEvidence } from './sync-origin-evidence.ts';
import { sourceDbOnlyVerdict } from './source-storage.ts';
@@ -834,5 +834,5 @@ export function readSyncContent(discovery: SyncDiscovery, entry: SyncEntry): str
argv: ['gbrain', 'sync', '--no-pull', '--working-tree', '--source', discovery.sourceId] } });
return bytes.toString('utf8');
}
- return syncGit(discovery.gitRoot, ['show', `${discovery.target}:${syncGitPath(discovery, entry.path)}`]);
+ return syncGit(discovery.gitRoot, [...NO_REPOSITORY_PROGRAMS, 'show', `${discovery.target}:${syncGitPath(discovery, entry.path)}`]);
}
diff --git a/src/core/persistence/sync-entry-fault.ts b/src/core/persistence/sync-entry-fault.ts
index ccf6f511c5..b8e5e43af0 100644
--- a/src/core/persistence/sync-entry-fault.ts
+++ b/src/core/persistence/sync-entry-fault.ts
@@ -57,13 +57,15 @@ const GIT_CRYPT_HEADER = '\u0000GITCRYPT\u0000';
* GST-2312 (#9024): the `content_filter_owned` refusal of a blob that starts with git-crypt's header, or of a committed
* blob whose path names a Git filter `driver` (Git LFS excepted: its pointers keep their own hold), or null. Such bytes
* are what the filter's clean program stored, not the text of the checkout, and sync never runs a repository's filters.
+ * `held` is false where the run stops instead (`sync.holds=fail`, a company-brain source).
*/
-export function filterOwnedRefusal(path: string, content: string, driver: string | null): { code: 'content_filter_owned'; message: string } | null {
+export function filterOwnedRefusal(path: string, content: string, driver: string | null, sourceId: string, held: boolean): { code: 'content_filter_owned'; message: string } | null {
const gitCrypt = content.startsWith(GIT_CRYPT_HEADER);
if (!gitCrypt && (driver === null || driver === 'lfs')) return null;
return { code: 'content_filter_owned', message: `${path} is ${gitCrypt ? 'encrypted with git-crypt' : `stored through the Git filter "${driver}"`}, so its committed bytes are not the text in your checkout. `
- + "gbrain does not run a repository's filter programs during sync, so sync held this file and the rest of the source synced. "
- + 'Leave it out with sync.exclude, or commit it without the filter so Git stores its text, then sync again.' };
+ + `gbrain does not run a repository's filter programs during sync, so ${held ? 'sync held this file and the rest of the source synced' : 'this sync stopped at this file'}. `
+ + 'Leave it out with sync.exclude, or commit it without the filter so Git stores its text, then sync again.'
+ + (gitCrypt ? '' : ` If no filter "${driver}" is in use and the file is plain text, remove the filter attribute from .gitattributes, commit, and run gbrain sources retry-held ${sourceId}: a commit that changes only .gitattributes does not re-screen the file.`) };
}
/**
diff --git a/src/core/persistence/sync-holds.ts b/src/core/persistence/sync-holds.ts
index 8fd4ba6f2a..bb118f099a 100644
--- a/src/core/persistence/sync-holds.ts
+++ b/src/core/persistence/sync-holds.ts
@@ -912,8 +912,8 @@ export function gitHoldFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'c
verify: { argv: ['gbrain', 'sources', 'status', source, '--json'] } };
case 'content_filter_owned':
return { argv: ['gbrain', 'sources', 'status', source, '--json'], consent: [], actor: 'user', requires_exclusive: false, docs: gitHoldDocs(record.code),
- user_message: `${record.path} is stored in Git through a filter (git-crypt encryption or another clean/smudge driver), so its committed bytes are not the text in the checkout. gbrain does not run a repository's filter programs during sync, so sync holds it and the rest of source ${source} keeps syncing. Leave it out with sync.exclude, or commit it without the filter so Git stores its text, then run gbrain sync --source ${source} --no-pull.`,
- why: "Running a repository's filter programs would let the repository run code during sync, so the stored bytes cannot become the page; the hold re-screens when the file changes.",
+ user_message: `${record.path} is stored in Git through a filter (git-crypt encryption or another clean/smudge driver), so its committed bytes are not the text in the checkout. gbrain does not run a repository's filter programs during sync, so sync holds it and the rest of source ${source} keeps syncing. Leave it out with sync.exclude, or commit it without the filter so Git stores its text, then run gbrain sync --source ${source} --no-pull. If the filter is not in use and the file is plain text, remove the filter attribute from .gitattributes, commit, and run gbrain sources retry-held ${source}: a commit that changes only .gitattributes does not re-screen the file.`,
+ why: "Running a repository's filter programs would let the repository run code during sync, so the stored bytes cannot become the page; the hold re-screens when the file changes or on gbrain sources retry-held.",
verify: { argv: ['gbrain', 'sources', 'status', source, '--json'] } };
case 'git_object_missing':
return { argv: ['gbrain', 'sources', 'status', source, '--json'], consent: [], actor: 'user', requires_exclusive: false, docs: gitHoldDocs(record.code),
diff --git a/src/core/persistence/sync-run.ts b/src/core/persistence/sync-run.ts
index ec26baab5c..67bc0835c8 100644
--- a/src/core/persistence/sync-run.ts
+++ b/src/core/persistence/sync-run.ts
@@ -411,7 +411,7 @@ async function freezeEntryOnce(engine: BrainEngine, cursor: Cursor, key: string,
assertActive();
} else content = imageBytes ? frozenImageIntent(imageBytes).content : entry.action === 'import' && !oversize && !bodyRef ? pinnedContent(cursor, entry) : null;
const filtered = !screening && content !== null && !image // GST-2312 (#9024) with sync.holds=fail
- ? filterOwnedRefusal(entry.path, content, entry.working ? null : pinnedFilterDriver(cursor, entry.path)) : null;
+ ? filterOwnedRefusal(entry.path, content, entry.working ? null : pinnedFilterDriver(cursor, entry.path), cursor.sourceId, false) : null;
if (filtered) throw syncRunRefusal('content_filter_owned', 'A synced file is stored through a Git filter.', retry, filtered.message);
const nul = !screening && content !== null && !image ? nullBytesRefusal(entry.path, content) : null; // GBRA-133 S-39 with sync.holds=fail
if (nul) throw syncRunRefusal('content_null_bytes', 'A synced Markdown file holds NUL bytes.', retry, nul.message);
diff --git a/src/core/persistence/sync-screen.ts b/src/core/persistence/sync-screen.ts
index 5b84c84e5f..1f8e940929 100644
--- a/src/core/persistence/sync-screen.ts
+++ b/src/core/persistence/sync-screen.ts
@@ -392,7 +392,7 @@ export async function screenFrozenImport(engine: BrainEngine, input: FrozenImpor
// GST-2312 (#9024): a committed blob a Git filter owns is held alone; sync never runs the filter that would turn it into text.
const { gitRoot, target, entries } = input.cursor;
const filtered = filterOwnedRefusal(entry.path, content, entry.working || !gitRoot || !target || !entries ? null
- : pinnedFilterDriver({ ...input.cursor, gitRoot, target, entries }, entry.path));
+ : pinnedFilterDriver({ ...input.cursor, gitRoot, target, entries }, entry.path), input.cursor.sourceId, true);
if (filtered) return heldEntry(entry, slug, pageId, filtered, content, blob);
// GBRA-133 S-39: text holding NUL cannot be stored or carried in the write intent; only this file is held. A slug conflict
// still names the hold (#6749): its message carries the path alone, and the slug-conflicts lane reports NULL_BYTES as still_invalid.
diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txt
index 679e199fa7..128791fc78 100644
--- a/test/postgres-unit-arms.txt
+++ b/test/postgres-unit-arms.txt
@@ -555,6 +555,7 @@ test/sync-exits-l4-6712.test.ts
test/sync-fenced-slices-gbra133.test.ts
test/sync-file-drift-agrees-7931.test.ts
test/sync-filter-owned-9024.test.ts
+test/sync-filter-owned-company-9024.test.ts
test/sync-freshness-6858.test.ts
test/sync-frontmatter-recovery.test.ts
test/sync-frontmatter-slug-conflict.test.ts
diff --git a/test/sync-filter-owned-9024.test.ts b/test/sync-filter-owned-9024.test.ts
index 208a4bcf0f..e70ea42042 100644
--- a/test/sync-filter-owned-9024.test.ts
+++ b/test/sync-filter-owned-9024.test.ts
@@ -3,7 +3,7 @@ import { execFileSync } from 'node:child_process';
import { chmodSync, existsSync, mkdirSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import type { BrainEngine } from '../src/core/engine.ts';
-import { __setPinnedWindowForTests } from '../src/core/persistence/sync-blobs.ts';
+import { __setCheckAttrSourceForTests, __setPinnedWindowForTests } from '../src/core/persistence/sync-blobs.ts';
import { readGitSourceHolds } from '../src/core/persistence/sync-holds.ts';
import { performManagedSync } from '../src/core/persistence/sync-run.ts';
import { managedBrain } from './helpers/managed-brain.ts';
@@ -12,17 +12,22 @@ import { testBackends } from './helpers/test-backends.ts';
// GST-2312 (#9024): a committed file a Git filter owns (git-crypt, any clean/smudge driver) is held alone under
// content_filter_owned; the rest of the source imports, the watermark moves, and no filter program ever runs.
-const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
+const git = (root: string, ...args: string[]) => execFileSync('git', ['-c', 'core.fsmonitor=false', '-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
const commit = (root: string, message: string) => { git(root, 'add', '-A'); git(root, '-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', message); };
const note = (title: string) => '---\ntype: concept\ntitle: ' + title + '\n---\n\nThe body of ' + title + '.\n';
const OPTS = { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true };
const live = async (engine: BrainEngine) => (await engine.executeRaw<{ slug: string }>("SELECT slug FROM pages WHERE source_id='default' AND deleted_at IS NULL ORDER BY slug")).map(row => row.slug);
const holds = async (engine: BrainEngine) => ((await readGitSourceHolds(engine, { sourceIds: ['default'] }))[0]?.holds ?? []) as Array<{ path: string; code: string; message: string }>;
-/** A filter `secret` on secret/** whose clean stores base64 (after git-crypt's header when `gitCrypt`) and whose smudge leaves a marker. */
+/**
+ * A filter `secret` on secret/** whose clean stores base64 (after git-crypt's header when `gitCrypt`) and whose smudge
+ * leaves a marker, and a `core.fsmonitor` that leaves `<marker>.fsmonitor` (the test's own Git calls switch it off).
+ */
function filtered(root: string, gitCrypt: boolean): string {
const bin = join(root, '..', 'bin'), marker = join(root, '..', 'filter-ran');
mkdirSync(bin, { recursive: true });
+ writeFileSync(join(bin, 'fsmonitor'), '#!/bin/sh\ntouch ' + marker + '.fsmonitor\nexit 1\n'); chmodSync(join(bin, 'fsmonitor'), 0o755);
+ git(root, 'config', 'core.fsmonitor', join(bin, 'fsmonitor'));
writeFileSync(join(bin, 'clean'), '#!/bin/sh\ntouch ' + marker + '.clean\n' + (gitCrypt ? "printf '\\000GITCRYPT\\000'; " : '') + 'base64\n');
writeFileSync(join(bin, 'smudge'), '#!/bin/sh\ntouch ' + marker + '\n' + (gitCrypt ? 'tail -c +11 | ' : '') + 'base64 -d\n');
chmodSync(join(bin, 'clean'), 0o755); chmodSync(join(bin, 'smudge'), 0o755);
@@ -61,6 +66,7 @@ for (const backend of testBackends()) describe('[' + backend + '] a committed fi
expect(await live(engine)).toEqual(['ok', 'ok2', 'seed']);
expect((await engine.executeRaw<{ c: string }>("SELECT last_commit AS c FROM sources WHERE id='default'"))[0]!.c).toBe(git(root, 'rev-parse', 'HEAD'));
expect(existsSync(marker)).toBe(false);
+ expect(existsSync(marker + '.fsmonitor')).toBe(false);
} finally { restore(); }
}), 240_000);
}
@@ -73,6 +79,7 @@ for (const backend of testBackends()) describe('[' + backend + '] a committed fi
expect(error?.code).toBe('content_filter_owned');
expect(await live(engine)).not.toContain('secret/one');
expect(existsSync(marker)).toBe(false);
+ expect(existsSync(marker + '.fsmonitor')).toBe(false);
}), 240_000);
test('a clean/smudge driver under sync.holds=fail refuses the run with content_filter_owned', () => brain(async ({ engine, root }) => {
@@ -83,6 +90,7 @@ for (const backend of testBackends()) describe('[' + backend + '] a committed fi
expect(error?.code).toBe('content_filter_owned');
expect(await live(engine)).not.toContain('secret/one');
expect(existsSync(marker)).toBe(false);
+ expect(existsSync(marker + '.fsmonitor')).toBe(false);
}), 240_000);
test('committing the file without the filter clears the hold and imports its text', () => brain(async ({ engine, root }) => {
@@ -104,24 +112,34 @@ for (const backend of testBackends()) describe('[' + backend + '] a committed fi
expect((await holds(engine)).filter(hold => hold.path === 'locked.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
}), 240_000);
- test('a filter attribute whose driver Git has no config for imports the file as text', () => brain(async ({ engine, root }) => {
+ test('a filter attribute whose driver no config defines still holds the file, naming the attribute and retry-held', () => brain(async ({ engine, root }) => {
expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
writeFileSync(join(root, '.gitattributes'), 'notes/** filter=unconfigured\n');
mkdirSync(join(root, 'notes')); writeFileSync(join(root, 'notes/plain.md'), note('plain'));
commit(root, 'unconfigured filter');
await performManagedSync(engine, OPTS);
- expect((await holds(engine)).filter(hold => hold.code === 'content_filter_owned')).toEqual([]);
- expect(await live(engine)).toContain('notes/plain');
+ const held = (await holds(engine)).filter(hold => hold.path === 'notes/plain.md');
+ expect(held.map(hold => hold.code)).toEqual(['content_filter_owned']);
+ expect(held[0]!.message).toContain('remove the filter attribute from .gitattributes');
+ expect(held[0]!.message).toContain('gbrain sources retry-held default');
+ expect(await live(engine)).not.toContain('notes/plain');
}), 240_000);
- test('the filter attribute is read at the synced commit, not from an uncommitted .gitattributes', () => brain(async ({ engine, root }) => {
- expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
- filtered(root, false);
- writeFileSync(join(root, '.gitattributes'), '');
- await performManagedSync(engine, OPTS);
- expect((await holds(engine)).filter(hold => hold.path === 'secret/one.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
- expect(await live(engine)).not.toContain('secret/one');
- }), 240_000);
+ for (const source of [true, false]) {
+ test('the filter attribute is read at the synced commit, not from an uncommitted .gitattributes (' + (source ? 'check-attr --source' : 'git before 2.40: a temporary index') + ')', () => brain(async ({ engine, root }) => {
+ const restore = __setCheckAttrSourceForTests(source);
+ try {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ const marker = filtered(root, false);
+ writeFileSync(join(root, '.gitattributes'), '');
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.path === 'secret/one.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
+ expect(await live(engine)).toEqual(['ok', 'seed']);
+ expect(existsSync(marker)).toBe(false);
+ expect(existsSync(marker + '.fsmonitor')).toBe(false);
+ } finally { restore(); }
+ }), 240_000);
+ }
test('a Git LFS path is not held content_filter_owned (its pointers keep their own handling)', () => brain(async ({ engine, root }) => {
expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
diff --git a/test/sync-filter-owned-company-9024.test.ts b/test/sync-filter-owned-company-9024.test.ts
new file mode 100644
index 0000000000..9947d80328
--- /dev/null
+++ b/test/sync-filter-owned-company-9024.test.ts
@@ -0,0 +1,70 @@
+/**
+ * GST-2312 (#9024) on a company-brain source, which never holds: a committed file a Git filter owns stops the managed
+ * connect with content_filter_owned, nothing imports it, and neither the filter nor `core.fsmonitor` runs.
+ */
+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
+import { randomUUID } from 'node:crypto';
+import { execFileSync } from 'node:child_process';
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { inspectCompanyBrain } from '../src/core/company-brain/inspection.ts';
+import { connectCompanyBrain } from '../src/core/company-brain/runtime.ts';
+import { __setCheckAttrSourceForTests } from '../src/core/persistence/sync-blobs.ts';
+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';
+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';
+import { testBackends } from './helpers/test-backends.ts';
+import { withEnv } from './helpers/with-env.ts';
+
+const home = mkdtempSync(join(tmpdir(), 'gbrain-company-9024-'));
+const backends = testBackends();
+const engines: BrainEngine[] = [];
+let closePostgres: (() => Promise<void>) | undefined;
+const git = (root: string, ...args: string[]) => execFileSync('git', ['-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=/dev/null', '-c', 'commit.gpgsign=false', '-c', 'user.name=Example', '-c', '[email protected]', '-C', root, ...args], { encoding: 'utf8' }).trim();
+beforeAll(async () => {
+ if (backends.includes('pglite')) { const pglite = new PGLiteEngine(); await pglite.connect({}); await pglite.initSchema(); engines.push(pglite); }
+ if (backends.includes('postgres')) { const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!); engines.push(pg.engine); closePostgres = pg.close; }
+}, 120_000);
+afterAll(async () => {
+ await withEnv({ GBRAIN_HOME: home }, async () => {
+ for (const engine of engines) { await disposePersistenceConsumer(engine); if (engine.kind === 'pglite') await engine.disconnect(); }
+ await closePostgres?.();
+ });
+ rmSync(home, { recursive: true, force: true });
+});
+
+for (const backend of backends) describe('[' + backend + '] a company-brain source with a Git-filtered file (#9024)', () => {
+ for (const source of [true, false]) test('the managed connect stops with content_filter_owned; no filter and no fsmonitor runs (' + (source ? 'check-attr --source' : 'git before 2.40: a temporary index') + ')', async () => withEnv({ GBRAIN_HOME: home }, async () => {
+ const restore = __setCheckAttrSourceForTests(source);
+ try {
+ const engine = engines[backends.indexOf(backend)]!;
+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');
+ const root = mkdtempSync(join(home, 'repo-')), bin = mkdtempSync(join(home, 'bin-')), marker = join(bin, 'ran');
+ writeFileSync(join(bin, 'clean'), '#!/bin/sh\ntouch ' + marker + '.clean\nbase64\n');
+ writeFileSync(join(bin, 'smudge'), '#!/bin/sh\ntouch ' + marker + '.smudge\nbase64 -d\n');
+ writeFileSync(join(bin, 'fsmonitor'), '#!/bin/sh\ntouch ' + marker + '.fsmonitor\nexit 1\n');
+ for (const name of ['clean', 'smudge', 'fsmonitor']) chmodSync(join(bin, name), 0o755);
+ git(root, 'init', '-q');
+ git(root, 'config', 'filter.secret.clean', join(bin, 'clean')); git(root, 'config', 'filter.secret.smudge', join(bin, 'smudge'));
+ writeFileSync(join(root, '.gitattributes'), 'customers/secret.md filter=secret\n');
+ mkdirSync(join(root, 'people')); mkdirSync(join(root, 'customers'));
+ writeFileSync(join(root, 'people/operator.md'), '---\ntype: person\ntitle: Example Operator\n---\n# Example Operator\nOwns the account.\n');
+ writeFileSync(join(root, 'customers/secret.md'), '---\ntype: customer\ntitle: Example Secret\n---\n# Example Secret\nA synthetic account.\n');
+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'Synthetic source fixture');
+ expect(existsSync(marker + '.clean')).toBe(true);
+ // A checkout where the filter never ran on checkout (a server clone without the key) holds the stored bytes.
+ writeFileSync(join(root, 'customers/secret.md'), execFileSync('git', ['-C', root, 'cat-file', 'blob', 'HEAD:customers/secret.md']));
+ git(root, 'config', 'core.fsmonitor', join(bin, 'fsmonitor'));
+ const plan = await inspectCompanyBrain({ path: root, profile: 'company-brain' });
+ expect(plan.ready).toBe(true);
+ const sourceId = 'company-' + randomUUID().slice(0, 8);
+ const result = await connectCompanyBrain(engine, { brainId: 'company-example', sourceId, path: root, plan, remote: false, requestId: randomUUID() });
+ expect(result).toMatchObject({ ok: false, code: 'content_filter_owned' });
+ expect(await engine.getPage('customers/secret', { sourceId })).toBeNull();
+ expect(existsSync(marker + '.smudge')).toBe(false);
+ expect(existsSync(marker + '.fsmonitor')).toBe(false);
+ } finally { restore(); }
+ }), 120_000);
+});