Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Describe the always-on push secret scan in the key-files guide

garrytanfd60cb907dd1
diff --git a/docs/architecture/key-files/workspace-push-and-backup.md b/docs/architecture/key-files/workspace-push-and-backup.md
index c6b49ceca5..da7a4cd3cd 100644
--- a/docs/architecture/key-files/workspace-push-and-backup.md
+++ b/docs/architecture/key-files/workspace-push-and-backup.md
@@ -13,7 +13,7 @@ Key files:
## Shared-checkout git writer
- `src/core/git/commit-push.ts` + `src/core/git/commit-guard.ts` — writer safety in a shared checkout (#7381). `integrate` replaces the tip 3-way merge when any local commit is patch-equivalent to an upstream one (`rev-list --cherry-mark`): it replays the local commits onto the remote tip in memory (`merge-tree --merge-base` on git 2.40+, a 3-way `read-tree` otherwise), leaving out the equivalent ones and clean merges, so a copy the remote edited never conflicts and one it reverted is never re-applied; the merge message lists what it left out. `supersededPaths` resolves a content conflict to ours when the remote's blob is one the same path held earlier in ours' history (last 200 commits touching it) and a different blob replaced it there; the push reports them as `superseded_remote_paths`, and `integrate_conflict` lists only the rest (`detail.superseded` names them). `commitPaths` refuses a path outside the source roots (`configuredSourceRoots`, git config `gbrain.sourceRoot`, recorded by `sources harden`'s `recordSourceRoot`) and a file another writer staged under a named directory (`commit_prestaged_paths`), stages named tracked paths with `add -u` and visible untracked ones with `add -A` (a tracked deletion under an ignored directory commits, #7984; any other path keeps git's `commit_paths_invalid`), and restores the exact index entries (`update-index --index-info`) when git refuses the commit. After the commit step (and its index restore) `commitPaths` returns the named paths HEAD still lacks (`unstagedPaths`: a modified entry flagged assume-unchanged or skip-worktree, a named file git did not index such as one inside a nested repository, and unstaged status entries under a named directory; a skip-worktree entry absent from disk, a flagged gitlink and ignored files under a named directory are left out) and keeps an unindexed file out of `commit --only`; `commitAndPush` (classic, managed and remoteless) and `commitPathsLocked` then throw `commit_paths_skipped`, or `writer_coordinator_required` for a page in a managed worktree (its `put_page` fix names the managed source), once the rest is committed and, on a classic checkout, pushed (`detail`: `skipped`, `committed`, `pushed`, `branch`, and `publish` when the push failed too). Every commit gbrain makes carries `GBRAIN_COMMIT_VIA=<lock path>`. Every pathspec it passes goes through `literalPathspecs` (`git-env.ts`), never `git --literal-pathspecs`, so the commit's hooks keep Git's normal pathspec rules: a pre-commit guard that selects files by glob still sees and can refuse them (`git_unavailable`, paths unstaged). `commitPathsLocked` (write-through) never commits without the lock: a busy lock or refused commit notes the file in `brain-writethrough-pending.log` (git dir, O_APPEND lines), which the next locked write-through and `commit-push --pending` commit and `settlePending` prunes (exported for a caller holding the commit lock: the derived write-through's rollback). `notePending` is exported: the derived and dream write-through (`cycle/derived-write-through.ts`, #8073) records each file it writes before its batch commit, so a crash in between leaves the file for that sweep. A publish refuses local non-merge commits outside the roots (`commit_outside_source_root`; `--allow-outside-source-root` lifts both refusals, and `sources harden` passes it for its root scaffolding). With git config `gbrain.quarantine` (opt-in) a content conflict publishes the remote's version of each conflicted file plus every other local change, names the local commit under `refs/gbrain/quarantine/<time>-<sha>` and appends it to `brain-quarantine.log` (`readQuarantine` lists entries whose ref still exists); structural conflicts still refuse. `commit-guard.ts` installs the opt-in `prepare-commit-msg` guard (`gbrain git guard`), which admits only commits naming a held commit lock, passes merges and in-progress operations, and honors `GBRAIN_COMMIT_GUARD=off`. The helper's no-gbrain fallback (`renderCommitPushHelper`) marks children with the absolute lock path, passes each path as `:(literal)<path>` (a glob-named path never commits another file, #7792) and releases the commit lock before pushing under `brain-publish.lock`; it runs the same skipped-path check after its commit and reports `commit_paths_skipped` after the push (exit 1, or 4 when the push failed). The CLI (`src/commands/git.ts`) refuses `checkout_not_a_source` before staging when this brain home's managed-root registry records source checkouts and the resolved checkout is none of them and has no `gbrain.sourceRoot` (#8662); a registered checkout must contain the resolved one, or be a subdirectory its HEAD tracks, so a repository that only encloses one is refused, and a registry that cannot be listed refuses `writer_coordinator_required` (GST-1623). `--untrack` (#7753, GST-2519): `untrackPaths` removes the files HEAD holds under the named paths (`untrackTargets`; skip-worktree files refuse) with `git rm --cached` and commits from HEAD in a temporary index, taking back a commit that changed anything else; in a managed canonical worktree `untrackAdmitted` first requires each file to sit under a `storage.db_only` dir that `gbrain.yml` at HEAD declares and that `.gitignore` ignores (`untrack_not_db_only`, index untouched), and the commit stays local like every managed commit; `previewUntrack` is the dry run. The dry run of a commit is `previewPaths` (#7752): `admitPaths` (shared with `commitPaths`: source roots, `--base`, pre-staged files), the real staging (`stagePaths`) as `git add -n` against a copy of the index, and a staged rename's partner, so each refusal carries the real run's code and text, with no lock and no write. A not_staged path that is a hand-staged `git rm --cached` names `--untrack` in the `commit_paths_skipped` fix. Tests: `test/git-writer-safety-7381.serial.test.ts`, `test/git-literal-pathspec-hooks-7759.test.ts`, `test/git-commit-push-not-a-source-8662.test.ts`, `test/git-commit-push-skipped-paths-gst2116.test.ts`, `test/git-commit-push-untrack-gst2519.test.ts`, `test/git-commit-push-door-wave2.test.ts`.
-- `src/core/git/range-secret-scan.ts` — `scanRange(root, from, to, timeoutMs, progress)`, the secret gate for every push gbrain makes: `gbrain git commit-push` in every mode, `--untrack` included where it pushes (the CLI always sets `scanSecrets`; the post-commit hook, `scripts/brain-commit-push.sh` and the child `gbrain sources refresh` runs all go through it) scans `trackingRef..target` under the publish lock, right before a push that never follows tags or recurses into submodules (`--no-follow-tags --recurse-submodules=no`, on the `sources push` and `bootstrap repo` pushes too). Plumbing only (`rev-list`, then `cat-file --batch` and `diff-tree --stdin -p --root --cc` over batches of 256 commits, so the temporary patch file stays one batch in size), so log, diff and attribute settings never change what is scanned. The budget is 300 s unless `--scan-timeout` sets it; `progress` gets `secret scan:` lines (at once for a whole history, then every 10 s), and with `retryArgv` a timed-out scan refuses `publish_scan_unreadable` whose agent fix is that command with `--scan-timeout` four times longer (at least 1200 s). Commit messages, added lines and merge resolutions go through `scanText` with the `.gbrain-scan-allow` allowlist; a finding, added text over the scan cap or a change to `.gbrain-scan-allow`/`.gitattributes` refuses `publish_secret_found` (file, line, rule, commit, never the value), and a range it cannot read in full (git failure, unrecognised output, 300 s budget) refuses `publish_scan_unreadable`. Commits the commit ledger records as page writes are not scanned: door and hosted-repository commits, and Git effect commits recorded `verified` (`effectCommit` in `effect-git.ts` builds the commit with `commit-tree` and moves HEAD with an `update-ref` compare-and-swap, so the recorded sha is gbrain's own; `verified` when each staged blob is the bytes whose sha256 the effect recorded); adopted records, records without `verified` and an unreadable ledger leave the commits scanned. Doctor `git_convergence`'s agent push fixes pass `--scan-secrets` too. `stagedScanPublishRefusal(root, base, target, opts)` is the same refusal for `gbrain sources push` (`workspace-push.ts`: origin's tip read from origin itself with `ls-remote` and a fetch, since the pull may follow another upstream remote, then `<tip>..HEAD` scanned and that exact sha pushed leased on the tip, no push when the local branch lacks the tip; a refusal is the `blocked_history` result, exit 5, printed by `--json` too) and both first pushes of `gbrain bootstrap repo` (`bootstrap/repo.ts`, the whole history; `scanHistory` is the runner-fake seam), with gate changes exempt in commits the ledger records as `workspace_push` or `bootstrap`, whose blobs those writers scanned staged with the same allowlist. The access-policy commit (`access/policy-git.ts`, which reports a scan refusal as `committed_not_pushed` with the refusal in its note) and the durability-scaffolding commit (`brain-repo-durability.ts`) publish through `commitAndPush` with `scanSecrets` (#8902). The helper's no-gbrain fallback commits and never pushes. `gbrain git scan` (`runScan` in `src/commands/git.ts`) runs `scanRange` and `rangeSecretRefusal` over `--from..--to` (default `@{upstream}..HEAD`) and writes nothing: the read-only preview of what a push would refuse, with each finding's fingerprint. Tests: `test/unmanaged-push-history-scan-8902.serial.test.ts`, `test/unmanaged-push-every-site-gst2456.serial.test.ts`.
+- `src/core/git/range-secret-scan.ts` — `scanRange(root, from, to, timeoutMs, progress)`, the secret gate for every push gbrain makes: `gbrain git commit-push` in every mode, `--untrack` included where it pushes (every `commitAndPush` publish scans, with no opt-out: a managed worktree's generated-page write-through commits and a push handed over from a busy publish lock too; a missing tracking ref is fetched first so only what the remote lacks is scanned; the post-commit hook, `scripts/brain-commit-push.sh` and the child `gbrain sources refresh` runs all go through it) scans `trackingRef..target` under the publish lock, right before a push that never follows tags or recurses into submodules (`--no-follow-tags --recurse-submodules=no`, on the `sources push` and `bootstrap repo` pushes too). Plumbing only (`rev-list`, then `cat-file --batch` and `diff-tree --stdin -p --root --cc` over batches of 256 commits, so the temporary patch file stays one batch in size), so log, diff and attribute settings never change what is scanned. The budget is 300 s unless `--scan-timeout` sets it; `progress` gets `secret scan:` lines (at once for a whole history, then every 10 s), and with `retryArgv` a timed-out scan refuses `publish_scan_unreadable` whose agent fix is that command with `--scan-timeout` four times longer (at least 1200 s). Commit messages, added lines and merge resolutions go through `scanText` with the `.gbrain-scan-allow` allowlist; a finding, added text over the scan cap or a change to `.gbrain-scan-allow`/`.gitattributes` refuses `publish_secret_found` (file, line, rule, commit, never the value), and a range it cannot read in full (git failure, unrecognised output, 300 s budget) refuses `publish_scan_unreadable`. Commits the commit ledger records as page writes are not scanned: hosted-repository commits, and strict-source door commits and Git effect commits recorded `verified` (no Git clean filter changed the committed bytes) (`effectCommit` in `effect-git.ts` builds the commit with `commit-tree` and moves HEAD with an `update-ref` compare-and-swap, so the recorded sha is gbrain's own; `verified` when each staged blob is the bytes whose sha256 the effect recorded); adopted records, records without `verified` and an unreadable ledger leave the commits scanned. Doctor `git_convergence`'s agent push fixes pass `--scan-secrets` too. `stagedScanPublishRefusal(root, base, target, opts)` is the same refusal for `gbrain sources push` (`workspace-push.ts`: origin's tip read from origin itself with `ls-remote` and a fetch, since the pull may follow another upstream remote, then `<tip>..HEAD` scanned and that exact sha pushed leased on the tip, no push when the local branch lacks the tip; a refusal is the `blocked_history` result, exit 5, printed by `--json` too) and both first pushes of `gbrain bootstrap repo` (`bootstrap/repo.ts`, the whole history; `scanHistory` is the runner-fake seam), with gate changes exempt in commits the ledger records as `workspace_push` or `bootstrap`, whose blobs those writers scanned staged with the same allowlist. In a managed root below the repository's top level, that root's own `.gbrain-scan-allow` is the loaded allowlist (path globs relative to the root) and a commit changing it is a gate change. The access-policy commit (`access/policy-git.ts`, which reports a scan refusal as `committed_not_pushed` with the refusal in its note) and the durability-scaffolding commit (`brain-repo-durability.ts`) publish through `commitAndPush`, which scans every push (#8902). The helper's no-gbrain fallback commits and never pushes. `gbrain git scan` (`runScan` in `src/commands/git.ts`) runs `scanRange` and `rangeSecretRefusal` over `--from..--to` (default `@{upstream}..HEAD`) and writes nothing: the read-only preview of what a push would refuse, with each finding's fingerprint. Tests: `test/unmanaged-push-history-scan-8902.serial.test.ts`, `test/unmanaged-push-every-site-gst2456.serial.test.ts`.
- `src/commands/doctor/checks/git-writer.ts` — `checkout_lagging`, `dirty_age`, `commits_outside_source_root`, `quarantined_changes` (#7381, brain category, after `git_convergence`): over the unmanaged source checkouts with an upstream (`sourceRoots` from git-convergence, grouped by toplevel with each source's toplevel-relative root), never fetching, every git call bounded at 5 s. Lagging warns when `HEAD..@{u}` holds a commit over an hour old (fix: `gbrain git commit-push --pull --path <top>`, agent). Dirty age warns on pending write-through files older than 30 minutes (`pendingWriteThroughStillDirty`; fix: `commit-push --pending`, agent) and on files staged that long (ask_user: unstage or commit by name). Outside-root warns on unpushed non-merge commits that change files outside the DB source roots plus `gbrain.sourceRoot` (`localCommitsOutsideRoots`; ask_user fix with `--allow-outside-source-root`), and says when the checkout has no root recorded. Test: `test/doctor-git-writer-7381.test.ts`.
- `src/core/git/commit-ledger.ts` — the commit ledger (PR B, docs/designs/REFRESH_WRITE_BARRIER.md part 3): `<git common dir>/gbrain/commit-ledger.jsonl`, one O_APPEND JSON line under 1 KiB per record, under its own flock `commit-ledger.lock` (start line, append, 4 MiB rotation to `commit-ledger.1.jsonl`). `recordCommit`/`recordCommitAsync` take the sha and writer kind (`commit_push`, `door`, `merge`, `replay`, `effect`, `sync_init`, `workspace_push`, `bootstrap`, `hosted_repo`); porcelain sites call `recordIntent` (head_before, `messageDigest` of the message) before `git commit`, plumbing sites record the `commit-tree` sha before the ref moves. A failed append never fails the commit (logged once); a directory outside git records nothing. `readLedger` reads both files; only a missing file reads as empty, any other read error throws (GST-1623). Tests: `test/commit-ledger.test.ts`.
- `src/commands/doctor/checks/foreign-writers.ts` — `foreign_writers` (PR B part 4): every registered source checkout on this host, managed worktrees included, deduplicated by git common dir. Reflogs (`logs/HEAD`, `logs/refs/heads/*`, `worktrees/*/logs/HEAD`, newest 5,000 entries each) are classified by `classifyReflogMessage`: porcelain creations by new sha, other ref moves by the `old..new` range within 10 minutes of the entry, merges/fast-forwards/resets/checkouts/branch/clone excluded. Unledgered commits fail (`plain`, `amend`, `rewrite`, `plumbing`; `guard_bypassed` when the enforce guard predates the commit); intent matches are `unconfirmed` (warn, never written back); entries before the ledger start are uncovered. Dirty page files come from `status --porcelain=v2` under the source roots minus queued/running git effects and the write-through pending ledger. GST-1623: a foreign commit carries `gbrain_trailer` when its trailer paragraph has a `gbrain-*:` line (named as possibly an older gbrain without the ledger; still fails); `partial_reasons` names what the scan skipped (a root inside a checkout whose `rev-parse` fails, roots past `MAX_ROOTS` 50, an unreadable ledger, a failed or timed-out status, given up to 30 s within doctor's budget, the budget itself), and a scan that read no repository but skipped one warns. Repo-less: `not_applicable`. Tests: `test/doctor-foreign-writers.test.ts`.