Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: lock-down lands where Git reads it, and already-remote sources get it
garrytan7c66b8fa107f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2bce84cba6..31cbadd8e3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,11 +16,12 @@ identifiers and attribution are available in the pre-removal Git revision### Itemized changes-- `src/core/url-source-origin.ts`: on a managed brain each apply runs inside `withTopologyLocks` and `lockTopologyRows` (a held lock or a running refresh refuses `source_busy`); `--remote --apply` writes the clone-time lock-down (`lfs.fetchexclude=*`, `core.hooksPath=/dev/null`, `lfs.locksverify=false`) into the checkout's `.git/config` with `git config --file` before the row; a throw is reported per id as `write_failed`, saying whether the checkout already got all or part of the lock-down, and the batch continues. The committed-attributes read splits each `ls-tree -z` record at its first tab, reads only blobs, and ignores comment lines; a declared-only filter refuses `lfs_tracked` with its own wording. An unborn HEAD with an empty index counts as no committed attributes. The top-level comparison realpath-normalizes both sides. `lfs_unknown` reasons name the cause: checkout missing, not its own repository's top level (an enclosing repository, or none), or a Git read failed.+- `src/core/url-source-origin.ts`: on a managed brain each apply runs inside `withTopologyLocks` and `lockTopologyRows` (a held lock or a running refresh refuses `source_busy`); `--remote --apply` writes the clone-time lock-down (`lfs.fetchexclude=*`, `core.hooksPath=/dev/null`, `lfs.locksverify=false`) into the checkout's `.git/config` with `git config --file` before the row, into the repository's shared config (`--git-common-dir`, so a linked worktree is covered), then reads back the values Git uses there and refuses `write_failed`, naming the key, when a worktree-scoped or included setting overrides one; a source already recorded as remote whose checkout lacks the lock-down gets it (`recorded`), and one that has it stays `already_recorded`; a throw is reported per id as `write_failed`, saying whether the checkout already got all or part of the lock-down, and the batch continues. The committed-attributes read splits each `ls-tree -z` record at its first tab, reads only blobs, and ignores comment lines; a declared-only filter refuses `lfs_tracked` with its own wording. An unborn HEAD with an empty index counts as no committed attributes. The top-level comparison realpath-normalizes both sides. `lfs_unknown` reasons name the cause from Git's own error: checkout missing, not its own repository's top level (an enclosing repository, or none), owned by another user (`safe.directory`), or a Git read failed (for example a corrupt `.git`).+- `src/core/hardened-git.ts`: `rev-parse --git-common-dir` and `--path-format=absolute` join the read-only allow-list, so the lock-down finds a linked worktree's shared config.- `src/core/persistence/topology-clone.ts`: a managed clone or reclone takes the source config it clones with from its locked admission read, not from the read before it took its locks.- `src/core/behavior-change-notice.ts`: one row for the changed `record-origin` behavior.- Docs: `docs/guides/repair.md#url-source-origin` (checkout lock-down and why, `source_busy`, `write_failed`, `lfs_tracked` and `lfs_unknown` causes), the key-files row in `docs/architecture/key-files/files-and-sync-2.md`, `docs/architecture/canonical-writers.tsv`.-- Tests: `test/url-source-origin.test.ts` (PGLite and Postgres): a record attempted while a reclone holds its locks is refused and writes nothing; a reclone that read the source before a record landed clones with the recorded settings; the existing checkout gets the lock-down on apply and not on preview; a tab-named directory's `.gitattributes` is read; commented rules are ignored and a declared-only rule is worded as declared; an empty-repository clone records; each `lfs_unknown` cause is named; a mid-batch write failure is reported for its id while the others record; a record during an active refresh is refused `source_busy`; a lock-down that fails part way is reported as a partial change.+- Tests: `test/url-source-origin.test.ts` (PGLite and Postgres): a record attempted while a reclone holds its locks is refused and writes nothing; a reclone that read the source before a record landed clones with the recorded settings; the existing checkout gets the lock-down on apply and not on preview; a tab-named directory's `.gitattributes` is read; commented rules are ignored and a declared-only rule is worded as declared; an empty-repository clone records; each `lfs_unknown` cause is named; a mid-batch write failure is reported for its id while the others record; a record during an active refresh is refused `source_busy`; a lock-down that fails part way is reported as a partial change; a worktree-scoped override is refused `write_failed` naming the key; a linked worktree gets the lock-down where Git reads it; a source recorded as remote before the lock-down existed gets it on the next apply; a corrupt `.git` and a `safe.directory` refusal each get their own `lfs_unknown` reason.## [0.60.620.0] - 2026-11-09**A diverged managed checkout now publishes its local commits on refresh by default when Git durability is on, after a secret scan of those commits.** Commits a script made with plain `git commit` in a managed checkout used to strand there whenever the remote moved: every `gbrain sources refresh` refused `refresh_diverged` and asked the user, unless `sources.refresh_publish_diverged` was set to `true`. With the setting unset, a worktree whose Git durability is on (`gbrain sources writer git-durability <source> --enable`, or the legacy hook) now publishes them itself: the local commits merged with the upstream in memory, never a rebase or force-push, then the fast-forward. The publish child scans, under its publish lock and for the exact commits it pushes, what they add: commit messages, added lines (including a value a later commit removed) and merge-commit conflict resolutions. A value that looks like a secret, added text too large to scan, or a commit that changes `.gbrain-scan-allow` or a `.gitattributes` refuses the new `publish_secret_found` naming the file, line, rule and commit, never the value, with nothing pushed; commits the scan cannot read in full refuse `publish_scan_unreadable` instead. The rewritten-upstream refusal and the `--expect-remote` bound are unchanged. `false` turns it off; `true` still publishes with durability off.diff --git a/docs/architecture/key-files/files-and-sync-2.md b/docs/architecture/key-files/files-and-sync-2.mdindex cd305b0bd6..200652064b 100644--- a/docs/architecture/key-files/files-and-sync-2.md+++ b/docs/architecture/key-files/files-and-sync-2.md@@ -29,5 +29,5 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes; a page-body fault (`PAGE_BODY_FAULT_CODES`: `page_body_segmented`, `page_body_rows_unreadable`, e.g. a segmented page whose segment rows don't cover its body) is held `page_write_failed` with `write_failure.code` set to that code, which `sync status` classifies by the `freeze_page_body_*` rows of `SYNC_FAULT_TABLE`; any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.- `src/core/url-clone.ts` — gbrain-owned `sources add --url` clones (#6946, GBRA-130 C-2). `isOwnedClone` (config `managed_clone` or a `local_path` under the default clone dir) and `defaultCloneDir`; `urlCloneReadOnly` is the one predicate (owned clone on a brain without managed persistence) the fence writer (`fence-write.ts`, `databaseOnlyReason: 'url_clone'`), write-through (`skipped: 'url_clone'`), timeline write-through, unfenced-facts adoption, the empty-fence guard and `sourceMirrorReadOnly` consult, so nothing gbrain writes lands in a checkout it updates with `git pull --ff-only`. `urlClonePullBlock` reads the tracked changes of a clone whose pull failed and returns `pull_blocked_local_changes` with an `ask_user` fix naming the preview `gbrain repair url-clone-fences --source <id>`; `formatUrlClonePullBlock` renders it for sync's stderr (`sync/preflight.ts`) and `cycle.ts` puts it in the sync phase detail as `pull_blocked`. Test: `test/pathological/c2-url-clone-fences.test.ts`.-- `src/core/url-source-origin.ts`, `src/commands/sources-record-origin.ts` and `src/commands/doctor/checks/url-source-origin.ts` — who added a URL source (`sources.config.added_by_remote_caller`). Both add paths write it (`true` for a remote caller, `false` for the local operator); only an explicit `true` reads as remote. URL sources added before that record existed carry none, and nothing stored tells a remote caller's add from the local operator's (same row, same default clone directory); a clone outside `$GBRAIN_HOME/clones/<id>` (or a `clones/<id>` under an earlier home) proves a local add. A URL clone counts when `isOwnedClone` accepts it, so an unmarked clone at the default directory is listed too. `urlSourcesWithoutOrigin` (read-only) feeds the wave check `url_source_origin` (host-only, `resolution: 'operator'`, `warn` with an `ask_user` fix whose argv is the preview). `recordSourceOrigin(engine, ids, { origin, apply, remote })` backs `gbrain sources record-origin <id>... --remote|--local [--apply] [--json]`: it throws `trusted_local_only` unless `remote === false`, previews unless `apply`, writes `true` (`--remote`) or `false` (`--local`) in one transaction that re-reads the row `FOR UPDATE` and refuses `source_changed` if its path or URL moved; on a managed brain that transaction runs inside `withTopologyLocks` after `lockTopologyRows`, the locks a reclone takes (a held lock refuses the id `source_busy`), and `runManagedSourceClone` takes the config it clones with from its own locked admission read, so a record and a reclone never interleave. `--remote --apply` also writes `LFS_OFF_CLONE_CONFIG` into the checkout's own `.git/config` (`git config --file`, not run in the checkout) before the row; a throw is reported for its id as `write_failed` and the batch continues. It never removes a `true`, and refuses `--remote` for a checkout with any tracked `filter=lfs` path or a committed `.gitattributes` declaring `filter=lfs` outside a comment line (`lfs_tracked`, worded per case; read with `hardenedGitSync` `ls-files -- ':(attr:filter=lfs)'` plus every committed `.gitattributes` blob from `ls-tree -z` records split at their first tab, then `cat-file`, so clone-local attributes cannot hide one, and read again under the locks), or one it cannot inspect (`lfs_unknown`, whose reason names the cause: checkout missing, not its own repository's top level by a realpath comparison, or a Git read failed), because a lock-down reclone would not match the stored writer manifest there. An unborn HEAD with an empty index (a clone of an empty repository) has no committed attributes and passes. Without git-lfs files the tracked bytes are the same, so no manifest is written. Pinned by `test/url-source-origin.test.ts` (PGLite and Postgres).+- `src/core/url-source-origin.ts`, `src/commands/sources-record-origin.ts` and `src/commands/doctor/checks/url-source-origin.ts` — who added a URL source (`sources.config.added_by_remote_caller`). Both add paths write it (`true` for a remote caller, `false` for the local operator); only an explicit `true` reads as remote. URL sources added before that record existed carry none, and nothing stored tells a remote caller's add from the local operator's (same row, same default clone directory); a clone outside `$GBRAIN_HOME/clones/<id>` (or a `clones/<id>` under an earlier home) proves a local add. A URL clone counts when `isOwnedClone` accepts it, so an unmarked clone at the default directory is listed too. `urlSourcesWithoutOrigin` (read-only) feeds the wave check `url_source_origin` (host-only, `resolution: 'operator'`, `warn` with an `ask_user` fix whose argv is the preview). `recordSourceOrigin(engine, ids, { origin, apply, remote })` backs `gbrain sources record-origin <id>... --remote|--local [--apply] [--json]`: it throws `trusted_local_only` unless `remote === false`, previews unless `apply`, writes `true` (`--remote`) or `false` (`--local`) in one transaction that re-reads the row `FOR UPDATE` and refuses `source_changed` if its path or URL moved; on a managed brain that transaction runs inside `withTopologyLocks` after `lockTopologyRows`, the locks a reclone takes (a held lock refuses the id `source_busy`), and `runManagedSourceClone` takes the config it clones with from its own locked admission read, so a record and a reclone never interleave. `--remote --apply` also writes `LFS_OFF_CLONE_CONFIG` into the repository's shared config (`--git-common-dir`; `git config --file`, not run in the checkout) before the row, reads the effective values back (`lockDownGaps`) and throws when a worktree-scoped or included setting overrides one, and locks down an already-remote source whose checkout lacks it; a throw is reported for its id as `write_failed` and the batch continues. It never removes a `true`, and refuses `--remote` for a checkout with any tracked `filter=lfs` path or a committed `.gitattributes` declaring `filter=lfs` outside a comment line (`lfs_tracked`, worded per case; read with `hardenedGitSync` `ls-files -- ':(attr:filter=lfs)'` plus every committed `.gitattributes` blob from `ls-tree -z` records split at their first tab, then `cat-file`, so clone-local attributes cannot hide one, and read again under the locks), or one it cannot inspect (`lfs_unknown`, whose reason names the cause: checkout missing, not its own repository's top level by a realpath comparison, or a Git read failed), because a lock-down reclone would not match the stored writer manifest there. An unborn HEAD with an empty index (a clone of an empty repository) has no committed attributes and passes. Without git-lfs files the tracked bytes are the same, so no manifest is written. Pinned by `test/url-source-origin.test.ts` (PGLite and Postgres).- `src/core/write-through.ts` — shared atomic disk write-through for the canonical ingestion path. `resolvePageWriteTarget(engine, slug, sourceId, { includeDeleted? })` (active rows by default; `includeDeleted: true` resolves a tombstone's recorded `source_path` for the purge remediation retry) is the exported single source of truth for the ONE file a (source, slug) pair lives in on disk: a source with its own `local_path` writes at that tree's ROOT (never nested under `.sources/`); a source without one nests under the host repo (`sync.repo_path`, default at root, non-default under `.sources/<id>/`) unless that path is another source's own working tree (leak guard → `source_repo_belongs_to_other_source`; a live source whose `local_path` is relative counts as owning every tree, and a relative `local_path` or `sync.repo_path` target is `repo_not_found`); the page's recorded `source_path` (or a contained absolute `file://` `source_uri`; a relative or host-only `file://` names no local file and is read as absent) is preferred over a slug-derived name so writes land in the file of record instead of minting a twin; and `isWriteTargetContained` rejects hostile rows escaping the tree (`path_escapes_source_root`). The ok-result also carries `sourcePathToBind` — the target expressed in the file scanner's `source_path` convention for the source's pinned `slug_root_mode` (or the mode its first sync would pin, via `scannerSlugRootMode`): GIT-ROOT-relative under `git-root` when the scan root sits inside a git repo, scan-root-relative under `source-root` and for non-git or git-root scan roots, so a put-born page binds the same form managed sync and delete-reconcile key on. Every recorded-path reader (write targets, takes, reconcile, audit, export restore, doctor) decodes with the same mode (`resolveSourceLocalFilePath(..., slugRootMode)`): under `source-root` the scope prefix is removed only from the legacy Git-root form whose remainder names the page slug, and basename rows resolve under the unstripped slug directory. The resolver is shared by `writePageThrough`, the facts fence writer (`src/core/facts/fence-write.ts`), and fence forget (`src/core/facts/forget.ts`) — the fence appends to the page's file, so all writers MUST compute the identical path or the fence lands in a file sync never reads back and the next extract_facts reconcile deletes the fence-owned DB rows. `writePageThrough(engine, slug, {sourceId?, frontmatterOverrides?, inlineOnly?, logger?, expectedFileBytes?, deferCommit?})` resolves the target through it, re-reads the just-written DB row (`getPage`), renders an inline page via `serializePageToMarkdown`, and writes the `.md`. A page stored in segments is never rendered whole: its file is measured from its segments at the snapshot's revision (`unreadPageFile`; a page that moved since refuses `revision_conflict`), checked against `PUSH_SCAN_MAX_FILE_BYTES` before `source_path` is bound (a body already over the cap by `body_bytes` is refused without streaming), streamed through `stageStreamedContent` on the engine it was given (so a caller inside a transaction works) keeping the existing file's mode, and renamed over the target after the file-changed check; `frontmatterOverrides` on such a page return an error result, and `inlineOnly` (the derived writer) skips it as `segmented_page` so the brain has a committable artifact that round-trips through `gbrain sync`. Rendering from the row keeps the serialized content consistent at write time. The fsync-backed `atomicWriteFileSync` stages and renames the file after required source-path bookkeeping; lock/lease checks run before publication. `put_page` holds the root lock before importing and invokes write-through inside its database transaction, so ordinary filesystem rejection rolls back existing content, tags, chunks, and versions. A crash or database COMMIT failure after rename can still leave canonical Markdown ahead of the index; this is not distributed ACID or a durable queue. Never throws — returns `WriteThroughResult { written, path?, content? (the bytes written; inline pages only), skipped?: 'disabled_by_config' | 'url_clone' | 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root' | 'case_insensitive_collision' | 'file_changed' | 'over_push_scan_cap' | 'segmented_page', warning?, error? }` (`file_changed`: the caller's `expectedFileBytes` preimage, null for absent, no longer matches the file under the source filesystem lock, so the file is left untouched) so the caller decides messaging + exit codes. `withNoRepoWriteThroughWarning(result, sourceId)` decorates an unwritten `no_repo_configured` result with `warning` (the write landed only in the database; bind the token/server to a git-backed source or configure `local_path`/`sync.repo_path`); `put_page` applies it for untrusted callers only (`ctx.remote !== false`), so local CLI responses and the `disabled_by_config` operator opt-out stay silent. `isWriteThroughDisabled` (shared with the fence lane, ~30s per-engine cache) honors the `sync.write_through` opt-out. Trust gating (subagent sandbox, dry-run) stays at the CALLER. On a durability-hardened repo (`isDurabilityHardened` — the gbrain post-commit hook is installed, i.e. the user ran `gbrain sources harden`), a successful write is best-effort COMMITTED via `commitWriteThroughFile` (path-limited `git commit -- <file>`, never sweeps unrelated edits; the hook then background-pushes) unless the caller passes `deferCommit` (the derived write-through commits its batch itself, #8073; `deletePageThrough` takes it too) so write-through content reaches git instead of accumulating uncommitted forever; result carries `committed?: boolean`. Unhardened repos keep write-only behavior. `deletePageThrough(engine, slug, {sourceId?, target?, logger?})` is the delete-side sink (sole caller: the `delete_page` op): resolves the same target, unlinks the `.md`, and on a hardened repo best-effort commits the removal path-limited through the same helper with subject `gbrain: delete write-through <slug>`, reporting `committed?: boolean` on `DeleteThroughResult` (absent when unhardened or when the commit did not land — the unlink still stands; a never-committed page leaves nothing to commit, so no empty commit). Pinned by `test/delete-write-through-commit.test.ts`. Consumers: `put_page` op and `gbrain brainstorm/lsd --save` via `persistSavedIdea`. Pinned by `test/write-through.test.ts` + `test/write-through-commit.test.ts`. The `put_page` op is that deciding caller for direct writes: a skip outside the deliberate DB-only set (`no_repo_configured`/`disabled_by_config`/`subagent_sandbox`/`dry_run`) throws `storage_error` inside the import transaction, rolling back the page revision, tags, chunks and version snapshot instead of deleting a just-created row afterward. #6946: on a gbrain-owned `--url` clone of an unmanaged brain (`urlCloneReadOnly`), `writePageThrough` returns `skipped: 'url_clone'` and `deletePageThrough` returns `removed: false, skipped: 'url_clone'`; `timeline-write-through.ts` skips the same way.diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex d87ecf12e2..a91d47c79c 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -2842,7 +2842,15 @@ in the existing checkout's own `.git/config` now (`lfs.fetchexclude=*`,two choices: a reclone happens only when a checkout is lost or replaced, sowithout it the checkout would keep fetching git-lfs objects from the endpointits repository names, and keep running hooks, while doctor already reads-green. Only `.git/config` changes, never a tracked file; a gbrain frontmatter+green. The settings go into the repository's shared config (the one+`git rev-parse --git-common-dir` names, so a linked worktree gets them too),+and the command then reads back the values Git uses in the checkout: when a+worktree-scoped or included setting still overrides one, it refuses+`write_failed`, names the key, and says the settings were already written.+A source already recorded as remote whose checkout lacks the lock-down (one+recorded before this) gets it on `--remote --apply` and reports `recorded`;+one that has it stays `already_recorded`, and the record is never removed.+Only `.git/config` changes, never a tracked file; a gbrain frontmatterhook wired through `core.hooksPath` stops running there, as on any clone aremote caller adds. `--local` records a local add and changes no checkout, anddoctor stops listing either. Each apply re-reads the source inside one@@ -2873,7 +2881,9 @@ refuses, writing nothing:and the reason names which cause applies: the checkout is missing (restoreit, then run this again); the clone directory is not the top level of itsown Git repository, because Git resolves it to an enclosing repository or to- none (check what the directory holds); or Git could not list or read it. A+ none (check what the directory holds); Git refuses the checkout because+ another user owns it (`safe.directory`); or Git could not list or read it+ (for example a corrupt `.git`). Aclone of an empty repository (no commit yet) has no committed attributes,so it is not refused.- `custom_clone_dir`: the clone sits where only the local CLI can clone.diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex 955fa5a8c7..1e614b63b6 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1489,7 +1489,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.618.0', id: 'refresh_replan_own_commits_8828', text: 'A `gbrain sources refresh` whose checkout HEAD moved while it waited for writes admitted before its fence no longer refuses `refresh_source_changed` when every commit since its start is one of gbrain\'s own (in the commit ledger) and the upstream branch holds them: it takes HEAD as its start and fast-forwards to the upstream. A HEAD move that includes a commit gbrain did not make, or gbrain\'s commits the upstream does not hold yet, still refuses as before.' },{ since: '0.60.619.0', id: 'managed_commit_push_unpublished_gst1923', text: 'On a managed checkout, `gbrain git commit-push -m <msg> -- <paths>` now reports how many commits the upstream lacks as last fetched (`unpublished` in `--json`, `null` when there is no upstream) and says nothing was pushed, instead of answering `nothing to commit; <remote>/<branch> at -` and logging `[push] ok-already-on-remote` for a commit that already existed. It writes no push-log line on a managed checkout. It still never pushes there; `gbrain git commit-push --push-only --keep-checkout --path <checkout>` publishes.' },{ since: '0.60.620.0', id: 'refresh_publish_diverged_default_gbra386', text: 'A diverged managed checkout now publishes its local commits on `gbrain sources refresh` by default when the worktree\'s Git durability is on: `sources.refresh_publish_diverged` unset now means on there (it used to mean off), merged with the upstream in memory, never rebased or forced. The refresh publish first scans what the local commits add (messages, added lines, merge resolutions) and refuses the new `publish_secret_found` naming the file, line and rule, never the value, when one looks like a secret; `gbrain git commit-push --push-only --scan-secrets` runs the same gate. Unset with durability not on still refuses, naming `gbrain sources writer git-durability <source> --enable`; `false` turns it off.' },- { since: 'TBD', id: 'record_origin_checkout_lockdown_8888', text: '`gbrain sources record-origin <id>... --remote --apply` now also turns git-lfs downloads and hooks off in each recorded source\'s existing checkout (its `.git/config`: `lfs.fetchexclude=*`, `core.hooksPath=/dev/null`, `lfs.locksverify=false`), not only on later reclones. On a managed brain it takes the same worktree locks a reclone takes, refusing the id `source_busy` while a reclone, refresh or write holds them, and a reclone reads the record under its own locks. A write that fails is reported for its own id (`write_failed`) instead of ending the run. The git-lfs check reads a committed `.gitattributes` under a directory whose name holds a tab, ignores commented-out `filter=lfs` lines, says when a committed rule only declares an LFS filter, records a clone of an empty repository instead of refusing it, and names the cause of an `lfs_unknown` refusal.' },+ { since: 'TBD', id: 'record_origin_checkout_lockdown_8888', text: '`gbrain sources record-origin <id>... --remote --apply` now also turns git-lfs downloads and hooks off in each recorded source\'s existing checkout (its `.git/config`: `lfs.fetchexclude=*`, `core.hooksPath=/dev/null`, `lfs.locksverify=false`), not only on later reclones, in the repository\'s shared config, and refuses `write_failed` naming the key when a worktree-scoped or included setting still overrides one; a source already recorded as remote gets the lock-down on the next `--remote --apply` when its checkout lacks it. On a managed brain it takes the same worktree locks a reclone takes, refusing the id `source_busy` while a reclone, refresh or write holds them, and a reclone reads the record under its own locks. A write that fails is reported for its own id (`write_failed`) instead of ending the run. The git-lfs check reads a committed `.gitattributes` under a directory whose name holds a tab, ignores commented-out `filter=lfs` lines, says when a committed rule only declares an LFS filter, records a clone of an empty repository instead of refusing it, and names the cause of an `lfs_unknown` refusal.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/hardened-git.ts b/src/core/hardened-git.tsindex e39940ef64..c55b212fd7 100644--- a/src/core/hardened-git.ts+++ b/src/core/hardened-git.ts@@ -106,7 +106,7 @@ const LOG_SHAPES: ReadonlyArray<readonly string[]> = [REASON_LOG_OPTIONS, HISTOR* listed here refuses.*/const ALLOWED_OPTIONS: ReadonlyMap<string, ReadonlySet<string>> = new Map([- ['rev-parse', new Set(['--show-toplevel', '--absolute-git-dir', '--verify', '--quiet', '--show-object-format'])],+ ['rev-parse', new Set(['--show-toplevel', '--absolute-git-dir', '--git-common-dir', '--path-format=absolute', '--verify', '--quiet', '--show-object-format'])],['ls-files', new Set(['-s', '--stage', '--eol', '-z', '--cached', '--others', '--exclude-standard'])],['ls-tree', new Set(['-r', '-l', '-z', '--full-tree'])],['cat-file', new Set(['--batch', '--batch-check'])],diff --git a/src/core/url-source-origin.ts b/src/core/url-source-origin.tsindex c3b227b3b1..5b6fe49f50 100644--- a/src/core/url-source-origin.ts+++ b/src/core/url-source-origin.ts@@ -57,7 +57,7 @@ export async function urlSourcesWithoutOrigin(engine: BrainEngine): Promise<Arrareturn rows.filter(row => originState(row) === 'unknown').map(row => ({ id: row.id, local_path: row.local_path! }));}-type LfsState = { state: 'none' | 'tracked' | 'declared' } | { state: 'unknown'; cause: 'missing' | 'not_top_level' | 'git_failed' };+type LfsState = { state: 'none' | 'tracked' | 'declared' } | { state: 'unknown'; cause: 'missing' | 'not_top_level' | 'unsafe_owner' | 'git_failed' };const GIT_LIMITS = { timeoutMs: 30_000, maxBytes: 64 * 1024 * 1024 };const sameDirectory = (a: string, b: string): boolean => { try { return realpathSync.native(a) === realpathSync.native(b); } catch { return false; } };const declaresLfs = (attributes: string): boolean => attributes.split('\n')@@ -74,8 +74,11 @@ const declaresLfs = (attributes: string): boolean => attributes.split('\n')function lfsTracked(root: string): LfsState {if (!existsSync(root)) return { state: 'unknown', cause: 'missing' };const gitFailed = { state: 'unknown', cause: 'git_failed' } as const;- const top = hardenedGitSync(root, ['rev-parse', '--show-toplevel'], GIT_LIMITS);- if (!top.ok) return top.reason === 'exit' ? { state: 'unknown', cause: 'not_top_level' } : gitFailed;+ const top = hardenedGitSync(root, ['rev-parse', '--show-toplevel'], { ...GIT_LIMITS, stderr: true });+ if (!top.ok) {+ if (/not a git repository/i.test(top.stderr ?? '')) return { state: 'unknown', cause: 'not_top_level' };+ return /dubious ownership/i.test(top.stderr ?? '') ? { state: 'unknown', cause: 'unsafe_owner' } : gitFailed;+ }if (!sameDirectory(top.stdout.toString('utf8').replace(/\n$/, ''), root)) return { state: 'unknown', cause: 'not_top_level' };const listed = hardenedGitSync(root, ['ls-files', '-z', '--cached', '--', ':(attr:filter=lfs)'], GIT_LIMITS);if (!listed.ok) return gitFailed;@@ -99,16 +102,33 @@ function lfsTracked(root: string): LfsState {return { state: 'none' };}-/** Writes the clone-time lock-down (`LFS_OFF_CLONE_CONFIG`) into the checkout's own .git/config, without running Git in the checkout. */-function lockDownCheckout(root: string): void {- const gitDir = hardenedGitSync(root, ['rev-parse', '--absolute-git-dir'], GIT_LIMITS);- if (!gitDir.ok) throw new Error(`Git could not locate the repository of ${root}`);- const file = join(gitDir.stdout.toString('utf8').replace(/\n$/, ''), 'config');- for (const setting of LFS_OFF_CLONE_CONFIG.filter((_, at) => at % 2 === 1)) {- const at = setting.indexOf('=');- execFileSync('git', ['config', '--file', file, setting.slice(0, at), setting.slice(at + 1)],- { cwd: dirname(file), env: hardenedGitEnvironment(), stdio: 'pipe', timeout: GIT_LIMITS.timeoutMs });- }+const LOCK_DOWN = LFS_OFF_CLONE_CONFIG.filter((_, at) => at % 2 === 1).map(setting => [setting.slice(0, setting.indexOf('=')), setting.slice(setting.indexOf('=') + 1)] as const);++/** The lock-down keys whose value Git reads in the checkout (worktree config and includes counted) is not the clone-time one. */+function lockDownGaps(root: string): string[] {+ return LOCK_DOWN.filter(([key, value]) => {+ try { return execFileSync('git', ['-C', root, 'config', '--get', key], { env: hardenedGitEnvironment(), stdio: 'pipe', timeout: GIT_LIMITS.timeoutMs }).toString('utf8').replace(/\n$/, '') !== value; }+ catch { return true; }+ }).map(([key]) => key);+}++/**+ * Writes the clone-time lock-down (`LFS_OFF_CLONE_CONFIG`) into the repository's shared config (`--git-common-dir`, so a linked+ * worktree's repository gets it too), without running Git in the checkout, then reads back what Git uses there and throws when a+ * worktree-scoped or included setting still overrides a key. `progress` says how far it got: `started` before the first write,+ * `written` once every key is in the file.+ */+function lockDownCheckout(root: string, progress: { state: 'none' | 'started' | 'written' | 'done' }): void {+ const commonDir = hardenedGitSync(root, ['rev-parse', '--path-format=absolute', '--git-common-dir'], GIT_LIMITS);+ if (!commonDir.ok) throw new Error(`Git could not locate the repository of ${root}.`);+ const file = join(commonDir.stdout.toString('utf8').replace(/\n$/, ''), 'config');+ progress.state = 'started';+ for (const [key, value] of LOCK_DOWN)+ execFileSync('git', ['config', '--file', file, key, value], { cwd: dirname(file), env: hardenedGitEnvironment(), stdio: 'pipe', timeout: GIT_LIMITS.timeoutMs });+ progress.state = 'written';+ const gaps = lockDownGaps(root);+ if (gaps.length) throw new Error(`Git in the checkout at ${root} still reads another value for ${gaps.join(', ')} (a worktree-scoped or included config overrides ${file}).`);+ progress.state = 'done';}export interface RecordOriginResult {@@ -132,6 +152,7 @@ function lfsRefusal(id: string, root: string): RecordOriginResult | null {const unconfirmed = 'so gbrain cannot confirm that no file is tracked with git-lfs; nothing changed.';if (lfs.cause === 'missing') return refusal(id, 'lfs_unknown', `Source ${id}'s checkout at ${root} is missing, ${unconfirmed} Restore the checkout, then run this again.`);if (lfs.cause === 'not_top_level') return refusal(id, 'lfs_unknown', `Source ${id}'s clone directory ${root} is not the top level of its own Git repository (Git resolves it to an enclosing repository, or to none), ${unconfirmed} Check what that directory holds before you run this again.`);+ if (lfs.cause === 'unsafe_owner') return refusal(id, 'lfs_unknown', `Git refuses to read source ${id}'s checkout at ${root} because another user owns it (safe.directory), ${unconfirmed} Check who owns that checkout, then run this again as that user or after fixing its ownership.`);return refusal(id, 'lfs_unknown', `Git could not list or read source ${id}'s checkout at ${root}, ${unconfirmed} Check that git works in that checkout, then run this again.`);}@@ -169,29 +190,32 @@ export async function recordSourceOrigin(engine: BrainEngine, ids: string[], optconst locks = opts.apply && await managedPersistenceEnabled(engine)? { ...await import('./persistence/topology-locks.ts'), ...await import('./persistence/topology-transaction.ts') } : null;for (const id of [...new Set(ids)]) {- const lockDown: { state: 'none' | 'started' | 'done' } = { state: 'none' };+ const lockDown: { state: 'none' | 'started' | 'written' | 'done' } = { state: 'none' };try {const [row] = await engine.executeRaw<SourceRowLike>('SELECT id, local_path, config, archived FROM sources WHERE id=$1', [id]);const early = check(row, id, opts.origin);- if (early) { results.push(early); continue; }+ const relock = remote && early?.status === 'already_recorded' && originState(row!) === 'remote' && existsSync(row!.local_path!) && lockDownGaps(row!.local_path!).length > 0;+ if (early && !relock) { results.push(early); continue; }const lfs = remote ? lfsRefusal(id, row!.local_path!) : null;if (lfs) { results.push(lfs); continue; }if (!opts.apply) {- results.push({ source_id: id, status: 'would_record', reason: `Source ${id} would be recorded as added by ${recordedAs}${remote ? '; git-lfs downloads and hooks would be turned off in its checkout now, and every later reclone clones that way' : ''}.` });+ results.push({ source_id: id, status: 'would_record', reason: relock+ ? `Source ${id} is already recorded as added by a remote caller, but its checkout still has git-lfs downloads or hooks on; they would be turned off now.`+ : `Source ${id} would be recorded as added by ${recordedAs}${remote ? '; git-lfs downloads and hooks would be turned off in its checkout now, and every later reclone clones that way' : ''}.` });continue;}const write = async (tx: BrainEngine): Promise<RecordOriginResult> => {const [locked] = await tx.executeRaw<SourceRowLike>('SELECT id, local_path, config, archived FROM sources WHERE id=$1 FOR UPDATE', [id]);const config = locked ? parseSourceConfig(locked.config) : {};- if (check(locked, id, opts.origin) || locked!.local_path !== row!.local_path || config.remote_url !== parseSourceConfig(row!.config).remote_url)+ const again = check(locked, id, opts.origin);+ if ((again && !(relock && again.status === 'already_recorded')) || (!again && relock) || locked!.local_path !== row!.local_path || config.remote_url !== parseSourceConfig(row!.config).remote_url)return refusal(id, 'source_changed', `Source ${id} changed while its origin was being recorded; nothing changed. Run the preview again.`);if (remote) {const lfsNow = lfsRefusal(id, locked!.local_path!);if (lfsNow) return lfsNow;- lockDown.state = 'started';- lockDownCheckout(locked!.local_path!);- lockDown.state = 'done';+ lockDownCheckout(locked!.local_path!, lockDown);}+ if (relock) return { source_id: id, status: 'recorded', reason: `Source ${id} was already recorded as added by a remote caller; its checkout now has git-lfs downloads and hooks off too.` };await tx.executeRaw('UPDATE sources SET config=$2::text::jsonb WHERE id=$1',[id, JSON.stringify(normalizeSourceConfig({ ...config, added_by_remote_caller: remote }))]);return { source_id: id, status: 'recorded', reason: `Source ${id} is recorded as added by ${recordedAs}${remote ? '; its checkout now has git-lfs downloads and hooks off, and every later reclone clones that way' : ''}.` };@@ -202,6 +226,7 @@ export async function recordSourceOrigin(engine: BrainEngine, ids: string[], opt} catch (error) {const message = error instanceof Error ? error.message : String(error);const lockdown = lockDown.state === 'done' ? ' Its checkout already has git-lfs downloads and hooks off.'+ : lockDown.state === 'written' ? " The lock-down settings were already written to its repository's .git/config, but Git does not use them there yet.": lockDown.state === 'started' ? " Its checkout's .git/config may already hold part of the git-lfs and hooks lock-down; running this again writes all of it." : '';const unchanged = lockdown ? `the record was not written.${lockdown}` : 'nothing changed.';const code = (error as { code?: string }).code;diff --git a/test/url-source-origin.test.ts b/test/url-source-origin.test.tsindex c085f6b0ab..81b70eae95 100644--- a/test/url-source-origin.test.ts+++ b/test/url-source-origin.test.ts@@ -413,4 +413,69 @@ for (const flavor of ['pglite', ...(databaseUrl ? ['postgres'] : [])] as const)expect(localConfig(root, 'lfs.fetchexclude')).toBe('*');expect(await config('source-a')).toEqual(URL_CONFIG);}), 60_000);++ const effective = (root: string, key: string) => {+ try { return execFileSync('git', ['-C', root, 'config', '--get', key], { encoding: 'utf8', stdio: 'pipe' }).trim(); } catch { return null; }+ };++ test('a worktree-scoped setting that overrides the lock-down is refused write_failed, naming the key, and the record is not written', () => brain(async () => {+ const root = defaultCloneDir('source-a');+ repoAt(root, { 'example.md': 'Canonical\n' });+ git(['-C', root, 'config', 'extensions.worktreeConfig', 'true']);+ git(['-C', root, 'config', '--worktree', 'core.hooksPath', '/example/hooks']);+ await insert('source-a', root, URL_CONFIG);+ const [result] = (await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: true, remote: false })).results;+ expect(result).toMatchObject({ source_id: 'source-a', status: 'refused', code: 'write_failed' });+ expect(result!.reason).toContain('core.hooksPath');+ expect(result!.reason).toContain('already written');+ expect(effective(root, 'core.hooksPath')).toBe('/example/hooks');+ expect(await config('source-a')).toEqual(URL_CONFIG);+ }), 60_000);++ test('a linked worktree gets the lock-down in the config Git reads for it', () => brain(async home => {+ const main = join(home, 'main-checkout'), root = defaultCloneDir('source-a');+ repoAt(main, { 'example.md': 'Canonical\n' });+ mkdirSync(dirname(root), { recursive: true });+ git(['-C', main, 'worktree', 'add', '-q', '-b', 'source-a', root]);+ await insert('source-a', root, URL_CONFIG);+ expect((await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: true, remote: false })).results[0]).toMatchObject({ status: 'recorded' });+ expect([effective(root, 'lfs.fetchexclude'), effective(root, 'core.hooksPath'), effective(root, 'lfs.locksverify')]).toEqual(['*', '/dev/null', 'false']);+ }), 60_000);++ test('--remote --apply locks down a source recorded as remote before the lock-down existed, then reports already_recorded', () => brain(async () => {+ const root = defaultCloneDir('source-a');+ repoAt(root, { 'example.md': 'Canonical\n' });+ await insert('source-a', root, { ...URL_CONFIG, added_by_remote_caller: true });+ const preview = (await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: false, remote: false })).results[0]!;+ expect(preview).toMatchObject({ status: 'would_record' });+ expect(effective(root, 'core.hooksPath')).toBeNull();+ const applied = (await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: true, remote: false })).results[0]!;+ expect(applied).toMatchObject({ status: 'recorded' });+ expect(applied.reason).toContain('already recorded');+ expect([effective(root, 'lfs.fetchexclude'), effective(root, 'core.hooksPath'), effective(root, 'lfs.locksverify')]).toEqual(['*', '/dev/null', 'false']);+ expect(await config('source-a')).toEqual({ ...URL_CONFIG, added_by_remote_caller: true });+ expect((await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: true, remote: false })).results[0]).toMatchObject({ status: 'already_recorded' });+ }), 60_000);++ test('lfs_unknown tells a corrupt .git and a safe.directory refusal apart from a directory outside its own repository', () => brain(async () => {+ const root = defaultCloneDir('source-a');+ repoAt(root, { 'example.md': 'Canonical\n' });+ await insert('source-a', root, URL_CONFIG);+ writeFileSync(join(root, '.git', 'config'), '[core\n');+ const corrupt = (await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: false, remote: false })).results[0]!;+ expect(corrupt).toMatchObject({ code: 'lfs_unknown' });+ expect(corrupt.reason).toContain('Git could not list or read');+ expect(corrupt.reason).not.toContain('not the top level');+ const original = childProcess.spawnSync;+ const patched = spyOn(childProcess, 'spawnSync').mockImplementation(((file: string, args?: readonly string[], options?: object) => {+ if (file === 'git' && args?.includes('--show-toplevel'))+ return { pid: 0, output: [], stdout: Buffer.from(''), stderr: Buffer.from(`fatal: detected dubious ownership in repository at '${root}'\n`), status: 128, signal: null };+ return original(file, args as string[], options as never);+ }) as typeof childProcess.spawnSync);+ let unsafe: Awaited<ReturnType<typeof recordSourceOrigin>>;+ try { unsafe = await recordSourceOrigin(engine, ['source-a'], { origin: 'remote', apply: false, remote: false }); }+ finally { patched.mockRestore(); }+ expect(unsafe.results[0]).toMatchObject({ code: 'lfs_unknown' });+ expect(unsafe.results[0]!.reason).toContain('another user owns it');+ }), 60_000);});