Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Merge remote-tracking branch 'origin/capy/guarded-refresh-publish-fixcheck-base' into capy/guarded-refresh-publish-fixcheck-base
garrytan34e4144a1fd7
diff --cc CHANGELOG.mdindex a4d377284e,8307448797..0781fdbf33--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -12,18 -12,7 +12,18 @@@ identifiers and attribution are availab## [Unreleased]- **The Git effects of page writes no longer publish a script's secret.** A script that commits into a managed checkout with plain `git commit` never passes the write path's secret scan. The refresh publish refused such a commit with `publish_secret_found`, but the next page write's Git effect pushed it anyway: its plain push published every unpublished commit unscanned, and when the remote had moved its keep-checkout publish ran without the scan. Both paths now scan the exact commits they push. A value that looks like a secret, added text too large to scan, a commit that changes `.gbrain-scan-allow` or a `.gitattributes`, or a range git cannot list (no tracking ref, or a remote without the branch) holds the effects with `publish_secret_found`. Nothing is pushed, and the page writes stay safe in the database and in local commits. `gbrain sync status` and doctor name the user's choice: remove the value from those commits, or allowlist a false positive by fingerprint. Then `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked. Every scanned push is also leased on the remote tip it scanned against. Before, a remote rewound past a pushed secret (an admin removing it with a force-push) left a stale tracking ref that hid the commit from the scan, and the next fast-forward put it back; now the scan runs against the remote's real tip.++**The Git effects of page writes no longer publish a script's secret.** A script that commits into a managed checkout with plain `git commit` never passes the write path's secret scan. The refresh publish refused such a commit with `publish_secret_found`, but the next page write's Git effect pushed it anyway: its plain push published every unpublished commit unscanned, and when the remote had moved its keep-checkout publish ran without the scan. Both paths now scan the exact commits they push. A value that looks like a secret, added text too large to scan, a commit that changes `.gbrain-scan-allow` or a `.gitattributes` holds the effects with `publish_secret_found`; commits the scan cannot read in full, or a remote without the branch, hold them with `publish_scan_unreadable`. Nothing is pushed, and the page writes stay safe in the database and in local commits. `gbrain sync status` and doctor name the user's choice: remove the value from those commits, or allowlist a false positive by fingerprint. Then `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked. Every scanned push is also leased on the remote tip it scanned against. Before, a remote rewound past a pushed secret (an admin removing it with a force-push) left a stale tracking ref that hid the commit from the scan, and the next fast-forward put it back; now the scan runs against the remote's real tip.++### Itemized changes++- `src/core/persistence/effect-git.ts`: `pushGitRoot` reads the remote branch tip with `git ls-remote`, scans `<tip>..<HEAD sha>` and pushes that sha only as a fast-forward of the tip, leased on it; a tip that is not an ancestor, or a remote that moves before the push lands (`stale info`), goes to the keep-checkout publish, which now runs with `--scan-secrets`. A remote without the branch refuses. A refusal from either path holds the effects as `publish_secret_found` with the user's fix.+- `src/core/git/commit-push.ts`: a `scanSecrets` publish scans `<base>..<candidate>` before every push and leases the push on that base, so each merge with a newly fetched remote tip is scanned again.- - `src/core/git/range-secret-scan.ts`: `rangeSecretRefusal`, the one `publish_secret_found` refusal both callers throw.- - `src/core/persistence/git-push-holds.ts`: `publish_secret_found` is a held push code, so doctor, `gbrain sync status` and `retry-effects --parked-pushes` cover it, with an ask-the-user fix that lists the refused commits.- - Docs: `docs/guides/shared-checkout-git.md` (`publish_secret_found` row), `docs/guides/live-sync.md`.++- `src/core/git/range-secret-scan.ts`: `rangeSecretRefusal`, the one refusal both callers throw (`publish_scan_unreadable` for a scan that could not read the range, `publish_secret_found` otherwise).++- `src/core/persistence/git-push-holds.ts`: `publish_secret_found` and `publish_scan_unreadable` are held push codes, so doctor, `gbrain sync status` and `retry-effects --parked-pushes` cover them, with an ask-the-user fix (the refused commits, or `git fsck`).++- Docs: `docs/guides/shared-checkout-git.md` (`publish_secret_found` and `publish_scan_unreadable` rows), `docs/guides/live-sync.md`.+- Tests: `test/effect-git-secret-gate.test.ts` (an ahead checkout, a diverged one, a remote rewound past a pushed secret and a remote without the branch each refuse with nothing pushed; a clean ahead checkout pushes; through the effect worker the page write's effect holds, sync status names the fix, and an allowlisted false positive releases it); `test/guarded-publish.test.ts` (a remote rewound past a pushed secret is not published again); `test/git-literal-pathspec-hooks-7759.test.ts` publishes its probe `.gitattributes` commit first, since the effect's push now refuses one; `test/cli-flag-equals.test.ts` lists `range-secret-scan.ts` as a git child argv reader.+- **A diverged managed checkout now publishes its local commits on refresh by default when Git durability is on, after a secret scan of those commits.** Commits a script made with plain `git commit` in a managed checkout used to strand there whenever the remote moved: every `gbrain sources refresh` refused `refresh_diverged` and asked the user, unless `sources.refresh_publish_diverged` was set to `true`. With the setting unset, a worktree whose Git durability is on (`gbrain sources writer git-durability <source> --enable`, or the legacy hook) now publishes them itself: the local commits merged with the upstream in memory, never a rebase or force-push, then the fast-forward. The publish child scans, under its publish lock and for the exact commits it pushes, what they add: commit messages, added lines (including a value a later commit removed) and merge-commit conflict resolutions. A value that looks like a secret, added text too large to scan, or a commit that changes `.gbrain-scan-allow` or a `.gitattributes` refuses the new `publish_secret_found` naming the file, line, rule and commit, never the value, with nothing pushed. The rewritten-upstream refusal and the `--expect-remote` bound are unchanged. `false` turns it off; `true` still publishes with durability off.+ **A diverged managed checkout now publishes its local commits on refresh by default when Git durability is on, after a secret scan of those commits.** Commits a script made with plain `git commit` in a managed checkout used to strand there whenever the remote moved: every `gbrain sources refresh` refused `refresh_diverged` and asked the user, unless `sources.refresh_publish_diverged` was set to `true`. With the setting unset, a worktree whose Git durability is on (`gbrain sources writer git-durability <source> --enable`, or the legacy hook) now publishes them itself: the local commits merged with the upstream in memory, never a rebase or force-push, then the fast-forward. The publish child scans, under its publish lock and for the exact commits it pushes, what they add: commit messages, added lines (including a value a later commit removed) and merge-commit conflict resolutions. A value that looks like a secret, added text too large to scan, or a commit that changes `.gbrain-scan-allow` or a `.gitattributes` refuses the new `publish_secret_found` naming the file, line, rule and commit, never the value, with nothing pushed; commits the scan cannot read in full refuse `publish_scan_unreadable` instead. The rewritten-upstream refusal and the `--expect-remote` bound are unchanged. `false` turns it off; `true` still publishes with durability off.### Itemized changesdiff --cc docs/guides/shared-checkout-git.mdindex f28954716a,36e8520cb5..4d5345bdd1--- a/docs/guides/shared-checkout-git.md+++ b/docs/guides/shared-checkout-git.md@@@ -185,7 -185,8 +185,8 @@@ Follow the error's `fix.next` as th| `git_index_stale` | 1 | An `index.lock` older than 10 minutes blocks the checkout. Check that no git command is running there, tell the user, and remove the named lock file only with their agreement. || `git_operation_in_progress` | 1 | A rebase, merge, cherry-pick or revert is unfinished in the checkout. gbrain never aborts it for you. Follow [stuck rebase](#stuck-rebase). || `git_detached_head` | 1 | The checkout is on a detached HEAD or an unborn branch. Check out the branch the remote tracks (`git switch <branch>`), or make the first commit, then rerun. |- | <a id="publish_secret_found"></a>`publish_secret_found` | 1 | Only with `--scan-secrets`, which every automatic publish uses: the one `gbrain sources refresh` runs on its own and the push of a managed worktree's Git effects. A commit about to be pushed adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees), or the range cannot be listed (a missing tracking ref, or a remote without the branch); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |-| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | Only with `--scan-secrets` (the publish `gbrain sources refresh` runs on its own): a commit about to be pushed adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. Never publish them with a command that skips the scan. |-| <a id="publish_scan_unreadable"></a>`publish_scan_unreadable` | 1 | Only with `--scan-secrets`: the scan could not read the commits about to be pushed in full (a missing or corrupt object, git output it does not recognise, or its 300 s budget), so it reported no secret and nothing was pushed. Run `git fsck` in the checkout, fix what it names, then retry. Never publish with a command that skips the scan. |++| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | Only with `--scan-secrets`, which every automatic publish uses: the one `gbrain sources refresh` runs on its own and the push of a managed worktree's Git effects. A commit about to be pushed adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |++| <a id="publish_scan_unreadable"></a>`publish_scan_unreadable` | 1 | Only with `--scan-secrets`: the scan could not read the commits about to be pushed in full (a missing or corrupt object, git output it does not recognise, or its 300 s budget), so it reported no secret and nothing was pushed. A managed worktree's Git effects also refuse this when the remote has no such branch to compare against. Run `git fsck` in the checkout, fix what it names, then retry. Never publish with a command that skips the scan. On a managed worktree its Git effects hold meanwhile, as for `publish_secret_found`. || `integrate_conflict` | 1 | Local and remote commits changed the same file; nothing was merged and the checkout is unchanged. Stop, relay the question to the user, then [resolve](#conflicts). || `push_unconfirmed` | 4 | The commit is durable locally but no push destination confirmed it (network, credentials, branch protection, a remote that kept moving, or another push that held the push lock for the whole wait). Fix what the message names, then run `gbrain git commit-push --push-only`. Never force-push. || `writer_coordinator_required` | 1 | This is a managed worktree. Stop using git for pages; save them through `put_page` and the other gbrain write tools. |diff --cc src/core/behavior-change-notice.tsindex 88bc4449ef,3b8a4ac17c..6e05bf75f3--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@@ -1489,7 -1489,6 +1489,7 @@@ export const BEHAVIOR_CHANGES: Readonly{ since: '0.60.618.0', id: 'refresh_replan_own_commits_8828', text: 'A `gbrain sources refresh` whose checkout HEAD moved while it waited for writes admitted before its fence no longer refuses `refresh_source_changed` when every commit since its start is one of gbrain\'s own (in the commit ledger) and the upstream branch holds them: it takes HEAD as its start and fast-forwards to the upstream. A HEAD move that includes a commit gbrain did not make, or gbrain\'s commits the upstream does not hold yet, still refuses as before.' },{ since: '0.60.619.0', id: 'managed_commit_push_unpublished_gst1923', text: 'On a managed checkout, `gbrain git commit-push -m <msg> -- <paths>` now reports how many commits the upstream lacks as last fetched (`unpublished` in `--json`, `null` when there is no upstream) and says nothing was pushed, instead of answering `nothing to commit; <remote>/<branch> at -` and logging `[push] ok-already-on-remote` for a commit that already existed. It writes no push-log line on a managed checkout. It still never pushes there; `gbrain git commit-push --push-only --keep-checkout --path <checkout>` publishes.' },{ since: '0.60.616.0', id: 'refresh_publish_diverged_default_gbra386', text: 'A diverged managed checkout now publishes its local commits on `gbrain sources refresh` by default when the worktree\'s Git durability is on: `sources.refresh_publish_diverged` unset now means on there (it used to mean off), merged with the upstream in memory, never rebased or forced. The refresh publish first scans what the local commits add (messages, added lines, merge resolutions) and refuses the new `publish_secret_found` naming the file, line and rule, never the value, when one looks like a secret; `gbrain git commit-push --push-only --scan-secrets` runs the same gate. Unset with durability not on still refuses, naming `gbrain sources writer git-durability <source> --enable`; `false` turns it off.' },- { since: '0.60.616.0', id: 'effect_git_secret_gate_gst1923', text: 'The Git effects of page writes on a managed worktree now scan the unpublished commits they push, as the refresh publish does: a value that looks like a secret (or text too large to scan, or a change to `.gbrain-scan-allow` or a `.gitattributes`) in a commit a script made with plain `git commit` holds the effects with `publish_secret_found` and nothing is pushed, where they used to push it with the next page write. The push now reads the remote branch tip and scans exactly what it adds, leased on that tip, so a remote rewound since the last fetch never gets a removed commit back; a remote without the branch refuses too. After the user removes the value or allowlists a false positive, `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked.' },++ { since: '0.60.616.0', id: 'effect_git_secret_gate_gst1923', text: 'The Git effects of page writes on a managed worktree now scan the unpublished commits they push, as the refresh publish does: a value that looks like a secret (or text too large to scan, or a change to `.gbrain-scan-allow` or a `.gitattributes`) in a commit a script made with plain `git commit` holds the effects with `publish_secret_found` and nothing is pushed, where they used to push it with the next page write. The push now reads the remote branch tip and scans exactly what it adds, leased on that tip, so a remote rewound since the last fetch never gets a removed commit back; commits the scan cannot read in full, or a remote without the branch, hold the effects with `publish_scan_unreadable`. After the user removes the value or allowlists a false positive, `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --cc src/core/git/commit-push.tsindex 98861fac3b,66ccfa80df..6e0922c174--- a/src/core/git/commit-push.ts+++ b/src/core/git/commit-push.ts@@@ -27,7 -27,7 +27,7 @@@*/import { appendFileSync, existsSync, mkdtempSync, readFileSync, realpathSync, renameSync, rmSync, statSync, writeFileSync } from 'node:fs';import { tmpdir } from 'node:os';- import { findingsInLog, rangeScanArgs, rangeSecretRefusal } from './range-secret-scan.ts';-import { scanRange } from './range-secret-scan.ts';++import { rangeSecretRefusal, scanRange } from './range-secret-scan.ts';import { isAbsolute, join, relative, resolve } from 'node:path';import { execFileSync } from '../spawn.ts';import { gitChildEnv, literalPathspecs } from '../git-env.ts';@@@ -1036,11 -1029,30 +1036,10 @@@ function assertLocalCommitsInsideRoots(docs: 'docs/guides/shared-checkout-git.md#source-roots' } });}-/** The automatic-publish secret gate: scan what the commits in trackingRef..target add (the exact range this publish pushes, under its lock). */+/** The automatic-publish secret gate: scan what the commits in trackingRef..target add (publish passes the base it leases its push on, under its lock). */function assertNoSecretsInRange(ctx: Ctx, target: string): void {- const log = git(ctx.root, rangeScanArgs(ctx.trackingRef, target), { timeoutMs: 300_000, maxBuffer: 1024 * 1024 * 1024 });- const refusal = rangeSecretRefusal(ctx.root, ctx.trackingRef, target, log.code === 0 ? findingsInLog(ctx.root, log.stdout) : null, log.stderr.trim().slice(0, 200) || `exit ${log.code}`);- const scan = scanRange(ctx.root, ctx.trackingRef, target);- if (scan.unreadable !== undefined) {- throw opError('publish_scan_unreadable', `The secret scan could not read the local commits ${ctx.trackingRef}..${target.slice(0, 12)} (${scan.unreadable}). No secret was reported. Nothing was pushed.`,- `The publish pushes only commits it has read in full. Check the checkout with git fsck in ${ctx.root} (a missing or corrupt object, a full disk, or a git too slow for the 300 s budget), fix what it reports, then retry. Never publish with a command that skips this scan.`,- { detail: JSON.stringify({ unreadable: scan.unreadable }), fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,- argv: ['git', '-C', ctx.root, 'fsck'],- why: 'Read-only: names missing or corrupt objects in the unpublished commits, the usual reason the scan cannot read them.',- user_message: `The secret scan could not read the unpublished commits in ${ctx.root}, so nothing was pushed. Check the checkout with git fsck and fix what it reports?`,- docs: 'docs/guides/shared-checkout-git.md#publish_scan_unreadable' } });- }- if (!scan.findings.length && !scan.unscanned.length && !scan.gate_changes.length) return;- const named = [...scan.findings.map(f => `${f.path}:${f.line} (${f.pattern}, ${f.commit.slice(0, 7)})`), ...scan.unscanned.map(u => `${u.path} (too large to scan, ${u.commit.slice(0, 7)})`),- ...scan.gate_changes.map(g => `${g.path} (changes what the scan sees, ${g.commit.slice(0, 7)})`)];- throw opError('publish_secret_found',- `${named.length} item(s) in the local commits ${ctx.trackingRef}..${target.slice(0, 12)} look like secrets or could not be scanned: ${named.slice(0, 10).join(', ')}${named.length > 10 ? ', ...' : ''}. Nothing was pushed.`,- 'Remove the value from those commits (rewriting unpushed local commits is the user\'s call), or add a false positive\'s fingerprint to .gbrain-scan-allow, then retry. Never publish them with a command that skips this scan.',- { detail: JSON.stringify(scan),- fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,- argv: ['git', '-C', ctx.root, 'log', '--stat', `${ctx.trackingRef}..${target}`],- why: 'Lists the unpublished commits the scan refused; read-only. A secret pushed to the remote stays in its history.',- user_message: `Unpublished commits in ${ctx.root} hold ${named.length || 'some'} item(s) that look like secrets or could not be scanned (${named.slice(0, 3).join(', ')}). Nothing was pushed. Remove them from those commits, or confirm a false positive so it can be allowlisted by fingerprint?`,- docs: 'docs/guides/shared-checkout-git.md#publish_secret_found' } });++ const refusal = rangeSecretRefusal(ctx.root, ctx.trackingRef, target, scanRange(ctx.root, ctx.trackingRef, target));+ if (refusal) throw refusal;}/** Advance the shared checkout (caller holds the commit lock) with a real compare-and-swap: fast-forward only, hooks off, never autostash. */diff --cc src/core/git/range-secret-scan.tsindex 03efbe68c2,621eda0024..17b0599938--- a/src/core/git/range-secret-scan.ts+++ b/src/core/git/range-secret-scan.ts@@@ -2,15 -2,26 +2,27 @@@* Secret scan over the commits an automatic publish pushes. A commit gbrain did not write (a script's plain* `git commit`) never passed the write path's scan, so the publish scans, under its publish lock and for the exact* commit it pushes, what each commit in `base..head` adds: its message, the lines it adds file by file (a value a later- * commit removed is still in the pushed history), and for a merge commit the lines it adds that neither parent has+ * commit removed is still in the pushed history), and for a merge commit the lines it adds that no parent has* (a conflict resolution). The workspace allowlist (`.gbrain-scan-allow`: path globs and fingerprints) applies.- * Findings carry the path, line, rule, fingerprint and commit, never the matched value. The diff is forced to text and- * its prefixes and size threshold are pinned, so no attribute or user setting decides what is scanned; added text that- * holds a NUL byte is binary by content and carries no lines. A file whose added text is over the scan cap cannot be scanned and is reported as such, so callers refuse. A commit- * in the range that changes a file deciding what the scan sees (`.gbrain-scan-allow`, a `.gitattributes`) is reported- * too, so callers refuse rather than let the published commits widen their own gate.+ * Findings carry the path, line, rule, fingerprint and commit, never the matched value.+ *+ * Only plumbing reads the commits (`rev-list`, `cat-file commit`, one `diff-tree --stdin --root --cc`), so no log, diff or+ * encoding setting changes what is read, and replace refs and the system attributes file are off. A message that declares+ * an encoding other than UTF-8 is also scanned as its raw bytes, and one the scan cannot decode makes the result unreadable. The diff streams to a private temporary file and is+ * read in chunks, so memory stays bounded. Binary detection is git's own (a NUL in a blob's first 8000 bytes), with+ * attributes read from the empty tree: a file git still calls binary is checked by its own bytes when its old side was+ * not empty or attributes could still apply, and a text file is then scanned whole. Added text over the scan cap is+ * reported as unscanned, and a commit that changes a file deciding what the scan sees (`.gbrain-scan-allow`, a+ * `.gitattributes`) as a gate change, so callers refuse. Output the parser does not recognise, a git failure or a run+ * past the time budget sets `unreadable` instead of returning a partial result.*/+ import { spawnSync } from 'node:child_process';+ import { closeSync, existsSync, mkdtempSync, openSync, readFileSync, readSync, rmSync } from 'node:fs';+ import { tmpdir } from 'node:os';+ import { isAbsolute, join } from 'node:path';+ import { GIT_READ_ENV, gitChildEnv } from '../git-env.ts';+ import { GIT_ENV } from '../git-transport.ts';+import { opError, type OperationError } from '../ops/contract.ts';import { loadWorkspaceAllowlist, pathAllowlisted, SCAN_ALLOW_FILENAME } from '../secret-scan-allowlist.ts';import { scanText, SCAN_MAX_FILE_BYTES } from '../secret-scan.ts';@@@ -26,69 -35,192 +36,222 @@@ export interface RangeScan/** The commit message as `path` in a finding. */export const MESSAGE_PATH = '(commit message)';- export function findingsInLog(root: string, log: string): RangeScan {- const allowlist = loadWorkspaceAllowlist(root);+ const SNIFF_BYTES = 8000;+ const BIG_FILE_THRESHOLD = 512 * 1024 * 1024;+ const CONFIG = ['-c', 'core.quotePath=false', '-c', 'core.attributesFile=/dev/null', '-c', `core.bigFileThreshold=${BIG_FILE_THRESHOLD}`];+ const DIFF = ['diff-tree', '--stdin', '-r', '-p', '--root', '--cc', '--no-renames', '--no-ext-diff', '--no-textconv', '--no-color',+ '--full-index', '--src-prefix=a/', '--dst-prefix=b/', '--unified=0'];++ class Unreadable extends Error {}++ type Run = ReturnType<typeof spawnSync>;+ function run(root: string, args: string[], deadline: number, opts: { input?: string; out?: number; maxBuffer?: number } = {}): Run {+ const timeout = deadline - Date.now();+ if (timeout <= 0) throw new Unreadable('the scan ran past its time budget');+ return spawnSync('git', ['-C', root, ...args], { input: opts.input ?? '', stdio: ['pipe', opts.out ?? 'pipe', 'pipe'], timeout,+ maxBuffer: opts.maxBuffer ?? 64 * 1024 * 1024, env: gitChildEnv({ ...GIT_ENV, ...GIT_READ_ENV, LC_ALL: 'C', GIT_NO_REPLACE_OBJECTS: '1', GIT_ATTR_NOSYSTEM: '1' }) });+ }+ function must(r: Run, what: string): Buffer {+ if (r.error || r.status !== 0) {+ const why = (r.error as NodeJS.ErrnoException | undefined)?.code === 'ETIMEDOUT' ? 'timed out' : String(r.stderr ?? '').trim() || r.error?.message || `exit ${r.status}`;+ throw new Unreadable(`git ${what} failed: ${why.slice(0, 300)}`);+ }+ return r.stdout as Buffer;+ }++ /** Lines of an open file read in 1 MiB chunks; a line longer than `limit` bytes keeps its first `limit` bytes and is marked long. */+ function* fileLines(fd: number, limit: number): Generator<{ text: string; long: boolean }> {+ const chunk = Buffer.alloc(1 << 20);+ let pending: Buffer[] = [], size = 0, position = 0;+ const take = (piece: Buffer) => { if (size < limit) pending.push(Buffer.from(piece.subarray(0, limit - size))); size += piece.length; };+ const line = () => { const out = { text: Buffer.concat(pending).toString('utf8'), long: size > limit }; pending = []; size = 0; return out; };+ for (let n; (n = readSync(fd, chunk, 0, chunk.length, position)) > 0; position += n) {+ const view = chunk.subarray(0, n);+ let start = 0;+ for (let nl; (nl = view.indexOf(10, start)) !== -1; start = nl + 1) { take(view.subarray(start, nl)); yield line(); }+ if (start < n) take(view.subarray(start));+ }+ if (size) yield line();+ }++ const C_ESCAPES: Record<string, number> = { a: 7, b: 8, t: 9, n: 10, v: 11, f: 12, r: 13, '"': 34, '\\': 92 };+ /** A path as git prints it: bare, or C-quoted when it holds a special character. Null when the quoting is malformed. */+ function unquote(text: string): string | null {+ if (!text.startsWith('"')) return text;+ if (text.length < 2 || !text.endsWith('"')) return null;+ const bytes: number[] = [];+ for (let i = 1; i < text.length - 1; i++) {+ if (text[i] !== '\\') { bytes.push(...Buffer.from(text[i]!)); continue; }+ const octal = /^[0-7]{3}/.exec(text.slice(i + 1));+ if (octal) { bytes.push(parseInt(octal[0], 8)); i += 3; continue; }+ const escaped = C_ESCAPES[text[++i] ?? ''];+ if (escaped === undefined) return null;+ bytes.push(escaped);+ }+ return Buffer.from(bytes).toString('utf8');+ }++ /** True when git's attribute lookup, pointed at the empty tree, cannot apply any attribute (git 2.42+, no info/attributes rule). */+ function attributesNeutral(root: string, deadline: number): boolean {+ const version = /git version (\d+)\.(\d+)/.exec(must(run(root, ['version'], deadline), 'version').toString());+ if (!version || Number(version[1]) < 2 || (Number(version[1]) === 2 && Number(version[2]) < 42)) return false;+ const rel = must(run(root, ['rev-parse', '--git-path', 'info/attributes'], deadline), 'rev-parse').toString().trim();+ const file = isAbsolute(rel) ? rel : join(root, rel);+ return !existsSync(file) || readFileSync(file, 'utf8').split('\n').every(line => !line.trim() || line.trim().startsWith('#'));+ }++ export function scanRange(root: string, base: string, head: string, timeoutMs = 300_000): RangeScan {+ const deadline = Date.now() + timeoutMs;const scan: RangeScan = { findings: [], unscanned: [], gate_changes: [] };- let commit = '', path: string | null = null, next = 0, combined = false, inMessage = false, header = false;- let added: string[] = [], at: number[] = [];- const flush = () => {- if (path && added.length && (path === MESSAGE_PATH || !pathAllowlisted(path, allowlist))) {- const text = added.join('\n');- if (path !== MESSAGE_PATH && text.includes('\0')) { /* binary by content */ }- else if (Buffer.byteLength(text) > SCAN_MAX_FILE_BYTES) scan.unscanned.push({ path, commit });- else for (const f of scanText(text, { allowlist })) scan.findings.push({ path, commit, line: at[f.line - 1] ?? f.line, pattern: f.pattern, fingerprint: f.fingerprint });- }- added = []; at = [];+ const allowlist = loadWorkspaceAllowlist(root);+ const scanned = (path: string, commit: string, text: string, lineAt: (line: number) => number = line => line) => {+ if (Buffer.byteLength(text) > SCAN_MAX_FILE_BYTES) { scan.unscanned.push({ path, commit }); return; }+ for (const f of scanText(text, { allowlist })) scan.findings.push({ path, commit, line: lineAt(f.line), pattern: f.pattern, fingerprint: f.fingerprint });};- for (const line of log.split('\n')) {- if (inMessage) {- if (line === '\0') { flush(); inMessage = false; path = null; continue; }- added.push(line); at.push(added.length);- continue;+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-range-scan-'));+ try {+ const commits = must(run(root, ['rev-list', `${base}..${head}`, '--'], deadline), 'rev-list').toString().split('\n').filter(Boolean);+ if (!commits.length) return scan;++ for (const commit of commits) {+ const raw = run(root, ['cat-file', 'commit', commit], deadline, { maxBuffer: SCAN_MAX_FILE_BYTES + 1024 * 1024 });+ if ((raw.error as NodeJS.ErrnoException | undefined)?.code === 'ENOBUFS') { scan.unscanned.push({ path: MESSAGE_PATH, commit }); continue; }+ const object = must(raw, 'cat-file commit');+ const split = object.indexOf('\n\n');+ const encoding = /^encoding (.+)$/m.exec(object.subarray(0, split < 0 ? object.length : split).toString('latin1'))?.[1]?.trim() ?? 'utf-8';+ const bytes = split < 0 ? Buffer.alloc(0) : object.subarray(split + 2);+ let message: string;+ try { message = new TextDecoder(encoding).decode(bytes); } catch {+ throw new Unreadable(`commit ${commit.slice(0, 12)} declares message encoding ${JSON.stringify(encoding.slice(0, 40))}, which the scan cannot decode`);+ }+ const before = scan.findings.length;+ scanned(MESSAGE_PATH, commit, message);+ if (/^utf-?8$/i.test(encoding)) continue;+ const seen = new Set(scan.findings.slice(before).map(f => `${f.line}:${f.fingerprint}`));+ for (const f of scanText(bytes.toString('latin1'), { allowlist })) {+ if (!seen.has(`${f.line}:${f.fingerprint}`)) scan.findings.push({ path: MESSAGE_PATH, commit, line: f.line, pattern: f.pattern, fingerprint: f.fingerprint });+ }+ }++ const neutral = attributesNeutral(root, deadline);+ const attrSource = neutral ? [`--attr-source=${must(run(root, ['hash-object', '-t', 'tree', '--stdin'], deadline), 'hash-object').toString().trim()}`] : [];+ const out = join(dir, 'diff');+ const fd = openSync(out, 'w+', 0o600);+ const binaries: Array<{ commit: string; path: string; blob: string; added: boolean }> = [];+ try {+ must(run(root, [...attrSource, ...CONFIG, ...DIFF], deadline, { input: commits.join('\n') + '\n', out: fd }), 'diff-tree');+ let commit = '', path: string | null = null, header = false, parents = 1, next = 0, bytes = 0, oversized = false;+ let olds: string[] = [], blob = '', added: string[] = [], at: number[] = [];+ const flush = () => {+ if (path && (oversized || added.length) && !pathAllowlisted(path, allowlist)) {+ if (oversized) scan.unscanned.push({ path, commit });+ else scanned(path, commit, added.join('\n'), line => at[line - 1] ?? line);+ }+ added = []; at = []; bytes = 0; oversized = false;+ };+ const unexpected = (line: string) => new Unreadable(`git diff-tree printed a line the scan does not recognise in ${commit.slice(0, 12) || 'the range'}: ${JSON.stringify(line.slice(0, 80))}`);+ for (const { text: line, long } of fileLines(fd, SCAN_MAX_FILE_BYTES + 1)) {+ if (/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/.test(line)) { flush(); commit = line; path = null; header = false; continue; }+ if (line.startsWith('diff --git ') || line.startsWith('diff --cc ') || line.startsWith('diff --combined ')) {+ flush();+ if (!commit || long) throw unexpected(line);+ if (line.startsWith('diff --git ')) {+ const pair = line.slice('diff --git '.length), half = (pair.length - 1) / 2;+ const left = Number.isInteger(half) && pair[half] === ' ' ? unquote(pair.slice(0, half)) : null;+ const right = left === null ? null : unquote(pair.slice(half + 1));+ path = left?.startsWith('a/') && right?.startsWith('b/') && left.slice(2) === right.slice(2) ? right.slice(2) : null;+ } else path = unquote(line.replace(/^diff --(?:cc|combined) /, ''));+ if (path === null) throw unexpected(line);+ if (path === SCAN_ALLOW_FILENAME || path === '.gitattributes' || path.endsWith('/.gitattributes')) scan.gate_changes.push({ path, commit });+ header = true; olds = []; blob = '';+ continue;+ }+ const hunk = /^(@{2,}) ((?:-\d+(?:,\d+)? )+)\+(\d+)(?:,\d+)? \1(?: |$)/.exec(line);+ if (hunk && path && !long) {+ parents = hunk[1]!.length - 1;+ if (hunk[2]!.trim().split(' ').length !== parents) throw unexpected(line);+ header = false; next = Number(hunk[3]);+ continue;+ }+ if (header && path && !long) {+ const index = /^index ((?:[0-9a-f]+,)*[0-9a-f]+)\.\.([0-9a-f]+)(?: [0-7]+)?$/.exec(line);+ if (index) { olds = index[1]!.split(','); blob = index[2]!; continue; }+ if (/^(?:old mode|new mode|deleted file mode|new file mode|mode) [0-7,.]+$/.test(line) || line.startsWith('--- ') || line.startsWith('+++ ')) continue;+ if (line === 'Binary files differ' || /^Binary files .+ differ$/.test(line)) {+ if (blob && !/^0+$/.test(blob)) binaries.push({ commit, path, blob, added: olds.every(old => /^0+$/.test(old)) });+ path = null;+ continue;+ }+ throw unexpected(line);+ }+ if (!path || header) throw unexpected(line);+ if (line.startsWith('\\')) continue;+ const prefix = line.slice(0, parents);+ if (prefix.length < parents || /[^ +-]/.test(prefix)) throw unexpected(line);+ if (prefix.includes('-')) continue;+ if (prefix === '+'.repeat(parents)) {+ bytes += Buffer.byteLength(line) - parents + 1;+ if (long || bytes > SCAN_MAX_FILE_BYTES) { oversized = true; added = []; at = []; }+ else { added.push(line.slice(parents)); at.push(next); }+ }+ next++;+ }+ flush();+ } finally {+ closeSync(fd);}- if (line.startsWith('commit ') && /^commit [0-9a-f]{40,64}$/.test(line)) { flush(); commit = line.slice(7); path = MESSAGE_PATH; inMessage = true; continue; }- if (line.startsWith('diff --git ') || line.startsWith('diff --cc ') || line.startsWith('diff --combined ')) {- flush(); path = null; header = true; combined = !line.startsWith('diff --git ');- const named = (combined ? line.replace(/^diff --(?:cc|combined) /, '') : line.replace(/^diff --git "?a\/.*? "?b\//, '')).replace(/"$/, '');- if (named === SCAN_ALLOW_FILENAME || named === '.gitattributes' || named.endsWith('/.gitattributes')) scan.gate_changes.push({ path: named, commit });- continue;++ const sizes = new Map<string, number>();+ const fresh = binaries.filter(b => neutral && b.added);+ if (fresh.length) {+ const check = must(run(root, ['cat-file', '--batch-check=%(objectname) %(objectsize)'], deadline, { input: fresh.map(b => b.blob).join('\n') + '\n' }), 'cat-file --batch-check');+ for (const row of check.toString().split('\n')) { const [name, size] = row.split(' '); if (name && size) sizes.set(name, Number(size)); }}- const hunk = /^@@@? -\d+(?:,\d+)? (?:-\d+(?:,\d+)? )?\+(\d+)(?:,\d+)? @@@?/.exec(line);- if (hunk) { header = false; next = Number(hunk[1]); continue; }- if (header) {- if (!line.startsWith('+++ ')) continue;- path = line === '+++ /dev/null' ? null : line.slice(4).replace(/^"?b\//, '').replace(/"$/, '');- continue;+ for (const b of binaries) {+ if (pathAllowlisted(b.path, allowlist)) continue;+ if (neutral && b.added && (sizes.get(b.blob) ?? Infinity) <= BIG_FILE_THRESHOLD) continue;+ const sniff = run(root, ['cat-file', 'blob', b.blob], deadline, { maxBuffer: SNIFF_BYTES });+ if ((sniff.error as NodeJS.ErrnoException | undefined)?.code !== 'ENOBUFS') must(sniff, 'cat-file blob');+ if ((sniff.stdout as Buffer).subarray(0, SNIFF_BYTES).includes(0)) continue;+ const whole = run(root, ['cat-file', 'blob', b.blob], deadline, { maxBuffer: SCAN_MAX_FILE_BYTES });+ if ((whole.error as NodeJS.ErrnoException | undefined)?.code === 'ENOBUFS') { scan.unscanned.push({ path: b.path, commit: b.commit }); continue; }+ scanned(b.path, b.commit, must(whole, 'cat-file blob').toString('utf8'));}- if (!path) continue;- if (combined) {- const prefix = line.slice(0, 2);- if (prefix.includes('-') || prefix.length < 2) continue;- if (prefix === '++') { added.push(line.slice(2)); at.push(next); }- next++;- } else if (line.startsWith('+')) { added.push(line.slice(1)); at.push(next++); }+ return scan;+ } catch (e) {+ if (e instanceof Unreadable) return { ...scan, unreadable: e.message };+ throw e;+ } finally {+ rmSync(dir, { recursive: true, force: true });}- flush();- return scan;}++/**- * The automatic-publish refusal for the commits in `base..target` of `root`: null only when the scan read the range and- * found nothing; otherwise `publish_secret_found` naming each item (never its value), or, with no scan, that git could- * not list the range (`failure`: git's stderr or exit), so a range that cannot be read is never published unscanned.++ * The automatic-publish refusal for the commits in `base..target` of `root`, shared by every publish that scans: null++ * when the scan read the whole range and found nothing; `publish_scan_unreadable` when it could not read it all;++ * otherwise `publish_secret_found` naming each finding, unscanned file and gate change (never a value).+ */- export function rangeSecretRefusal(root: string, base: string, target: string, scan: RangeScan | null, failure: string): OperationError | null {- if (scan && !scan.findings.length && !scan.unscanned.length && !scan.gate_changes.length) return null;- const named = scan ? [...scan.findings.map(f => `${f.path}:${f.line} (${f.pattern}, ${f.commit.slice(0, 7)})`), ...scan.unscanned.map(u => `${u.path} (too large to scan, ${u.commit.slice(0, 7)})`),- ...scan.gate_changes.map(g => `${g.path} (changes what the scan sees, ${g.commit.slice(0, 7)})`)] : [];- return opError('publish_secret_found', scan- ? `${named.length} item(s) in the local commits ${base}..${target.slice(0, 12)} look like secrets or could not be scanned: ${named.slice(0, 10).join(', ')}${named.length > 10 ? ', ...' : ''}. Nothing was pushed.`- : `git could not list the local commits ${base}..${target.slice(0, 12)} to scan them for secrets (${failure}). Nothing was pushed.`,++export function rangeSecretRefusal(root: string, base: string, target: string, scan: RangeScan): OperationError | null {++ if (scan.unreadable !== undefined) {++ return opError('publish_scan_unreadable', `The secret scan could not read the local commits ${base}..${target.slice(0, 12)} (${scan.unreadable}). No secret was reported. Nothing was pushed.`,++ `The publish pushes only commits it has read in full. Check the checkout with git fsck in ${root} (a missing or corrupt object, a full disk, or a git too slow for the 300 s budget), fix what it reports, then retry. Never publish with a command that skips this scan.`,++ { detail: JSON.stringify({ unreadable: scan.unreadable }), fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,++ argv: ['git', '-C', root, 'fsck'],++ why: 'Read-only: names missing or corrupt objects in the unpublished commits, the usual reason the scan cannot read them.',++ user_message: `The secret scan could not read the unpublished commits in ${root}, so nothing was pushed. Check the checkout with git fsck and fix what it reports?`,++ docs: 'docs/guides/shared-checkout-git.md#publish_scan_unreadable' } });++ }++ if (!scan.findings.length && !scan.unscanned.length && !scan.gate_changes.length) return null;++ const named = [...scan.findings.map(f => `${f.path}:${f.line} (${f.pattern}, ${f.commit.slice(0, 7)})`), ...scan.unscanned.map(u => `${u.path} (too large to scan, ${u.commit.slice(0, 7)})`),++ ...scan.gate_changes.map(g => `${g.path} (changes what the scan sees, ${g.commit.slice(0, 7)})`)];++ return opError('publish_secret_found',++ `${named.length} item(s) in the local commits ${base}..${target.slice(0, 12)} look like secrets or could not be scanned: ${named.slice(0, 10).join(', ')}${named.length > 10 ? ', ...' : ''}. Nothing was pushed.`,+ 'Remove the value from those commits (rewriting unpushed local commits is the user\'s call), or add a false positive\'s fingerprint to .gbrain-scan-allow, then retry. Never publish them with a command that skips this scan.',- { detail: JSON.stringify(scan ?? { error: 'unreadable' }),++ { detail: JSON.stringify(scan),+ fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,+ argv: ['git', '-C', root, 'log', '--stat', `${base}..${target}`],+ why: 'Lists the unpublished commits the scan refused; read-only. A secret pushed to the remote stays in its history.',+ user_message: `Unpublished commits in ${root} hold ${named.length || 'some'} item(s) that look like secrets or could not be scanned (${named.slice(0, 3).join(', ')}). Nothing was pushed. Remove them from those commits, or confirm a false positive so it can be allowlisted by fingerprint?`,+ docs: 'docs/guides/shared-checkout-git.md#publish_secret_found' } });+}++diff --cc src/core/persistence/effect-git.tsindex f83ec43147,1d7fa8146d..340dd1e110--- a/src/core/persistence/effect-git.ts+++ b/src/core/persistence/effect-git.ts@@@ -14,7 -14,6 +14,7 @@@ import type { NativeLockHandle } from 'import { proveIndexLockAbandoned, reclaimIndexLock, type IndexLockReclaim } from './index-lock-reclaim.ts';import { recordCommitAsync, recordIntentAsync } from '../git/commit-ledger.ts';import { supersededBy } from './import-deferral.ts';- import { findingsInLog, rangeScanArgs, rangeSecretRefusal } from '../git/range-secret-scan.ts';++import { rangeSecretRefusal, scanRange } from '../git/range-secret-scan.ts';/** An index lock younger than this is contention; an older one is reported as stale. */const INDEX_LOCK_GRACE_MS = 10 * 60 * 1000;@@@ -429,27 -419,11 +429,25 @@@ export async function pushGitRoot(rootconst remote = await git(session, ['config', '--get', `branch.${branch.stdout.trim()}.remote`]);const merge = await git(session, ['config', '--get', `branch.${branch.stdout.trim()}.merge`]);if (remote.code !== 0 || merge.code !== 0 || !remote.stdout.trim() || remote.stdout.trim() === '.') return { push: 'skipped', reason: 'no_tracking_remote' } as const;- const push = await git(session, ['push', '--', remote.stdout.trim(), `HEAD:${merge.stdout.trim()}`]);- if (push.code === 0) return { push: 'committed' } as const;- if (!NON_FAST_FORWARD.test(push.stderr)) throw opError('git_push_unavailable', 'The canonical commit is durable locally; its push will retry.',+ const head = await git(session, ['rev-parse', '--verify', 'HEAD']);+ if (head.code !== 0) throw gitFailure('Cannot read the canonical checkout\'s HEAD.', 'git rev-parse HEAD failed, so nothing was scanned or pushed.');+ const sha = head.stdout.trim(), short = merge.stdout.trim().replace(/^refs\/heads\//, ''), ref = `refs/heads/${short}`;+ const unavailable = () => opError('git_push_unavailable', 'The canonical commit is durable locally; its push will retry.',`git push to ${remote.stdout.trim()} failed (network, credentials or a remote that refused it). Nothing is lost locally and the worker pushes again on a later pass; gbrain never pulls or rebases the checkout. If it keeps failing, check the remote and its credentials with git push on the brain host.`,{ fix: effectStatusFix });+ const listed = await git(session, ['ls-remote', '--', remote.stdout.trim(), ref]);+ if (listed.code !== 0) throw unavailable();+ const tip = listed.stdout.split('\n').map(line => line.split('\t')).find(([, name]) => name === ref)?.[0] ?? '';- if (!tip) throw heldSecret(root, rangeSecretRefusal(root, `refs/remotes/${remote.stdout.trim()}/${short}`, sha, null, `${remote.stdout.trim()} has no branch ${short} to compare against`)!);++ if (!tip) throw heldSecret(root, rangeSecretRefusal(root, `refs/remotes/${remote.stdout.trim()}/${short}`, sha,++ { findings: [], unscanned: [], gate_changes: [], unreadable: `${remote.stdout.trim()} has no branch ${short} to compare against` })!);+ if ((await git(session, ['merge-base', '--is-ancestor', tip, sha])).code === 0) {- const scan = await execFileBounded('git', gitArgs(session, rangeScanArgs(tip, sha)),- { ...gitOptions(session), timeout: SECRET_SCAN_TIMEOUT_MS, maxBuffer: 1024 * 1024 * 1024, env: gitChildEnv(sessionEnv(session)) });- signal?.throwIfAborted();- const refusal = rangeSecretRefusal(root, tip, sha, scan.error ? null : findingsInLog(root, scan.stdout),- scan.stderr.trim().slice(0, 200) || `exit ${String(scan.error?.code ?? scan.error?.signal ?? 'unknown')}`);++ const refusal = rangeSecretRefusal(root, tip, sha, scanRange(root, tip, sha));+ if (refusal) throw heldSecret(root, refusal);++ signal?.throwIfAborted();+ const push = await git(session, ['push', `--force-with-lease=${ref}:${tip}`, '--', remote.stdout.trim(), `${sha}:${ref}`]);+ if (push.code === 0) return { push: 'committed' } as const;+ if (!NON_FAST_FORWARD.test(push.stderr) && !STALE_LEASE.test(push.stderr)) throw unavailable();+ }signal?.throwIfAborted();return publishKeepingCheckout(root, signal);});@@@ -484,7 -455,7 +479,7 @@@ async function publishKeepingCheckout(rif (doc && typeof doc.code === 'string' && typeof doc.message === 'string') {const error = opError(doc.code as Parameters<typeof opError>[0], doc.message, typeof doc.suggestion === 'string' ? doc.suggestion : '',{ ...(typeof doc.detail === 'string' ? { detail: doc.detail } : {}), ...(doc.fix ? { fix: doc.fix as Action } : {}), retryable: doc.retryable === true });- throw error.code === 'integrate_conflict' ? heldConflict(root, error) : error.code === 'publish_secret_found' ? heldSecret(root, error) : error;- throw error.code === 'integrate_conflict' ? heldConflict(root, error) : error;++ throw error.code === 'integrate_conflict' ? heldConflict(root, error) : error.code === 'publish_secret_found' || error.code === 'publish_scan_unreadable' ? heldSecret(root, error) : error;}throw opError('push_unconfirmed', 'The canonical commit is durable locally; its push will retry.',`The remote branch moved and the keep-checkout publish did not report a push (${child.error ? `exit ${String(child.error.code ?? child.error.signal ?? 'unknown')}` : `outcome ${String(doc?.outcome ?? 'unreadable')}`}: ${child.stderr.trim().slice(0, 300) || 'no detail'}). Nothing is lost locally; the worker pushes again on a later pass.`,@@@ -507,20 -478,6 +502,21 @@@ function heldConflict(root: string, errreturn held;}+/**- * #8896: values that look like secrets in the unpublished commits hold the checkout's Git effects (they retry, then- * park): no retry can succeed until the user removes them from those commits or allowlists a false positive.++ * #8896: a secret scan that refuses the unpublished commits (`publish_secret_found`) or cannot read them all++ * (`publish_scan_unreadable`) holds the checkout's Git effects (they retry, then park): no retry can succeed until the++ * user removes the values or allowlists a false positive, or repairs what kept the scan from reading the commits.+ */+function heldSecret(root: string, error: OperationError): OperationError {+ const fix = error.fix as Action | undefined;- return opError('publish_secret_found', `Git backups of canonical page writes are held: ${error.message}`,- `${error.suggestion ?? ''} The page writes themselves are committed in the database and in local commits; nothing is lost, and nothing is pushed from ${root} while those commits hold the values. Afterwards, gbrain sources writer retry-effects with --parked-pushes for the source this checkout backs (gbrain sources writer status names it) retries the Git effects that parked meanwhile.`,++ return opError(error.code as 'publish_secret_found' | 'publish_scan_unreadable', `Git backups of canonical page writes are held: ${error.message}`,++ `${error.suggestion ?? ''} The page writes themselves are committed in the database and in local commits; nothing is lost, and nothing is pushed from ${root} until the scan passes. Afterwards, gbrain sources writer retry-effects with --parked-pushes for the source this checkout backs (gbrain sources writer status names it) retries the Git effects that parked meanwhile.`,+ { ...(error.detail ? { detail: error.detail } : {}), fix: { ...(fix ?? { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,- why: 'Rewriting unpublished commits or allowlisting a value is the user\'s call.' }),- user_message: `${fix?.user_message ?? `Unpublished commits in ${root} hold values that look like secrets.`} Until they are removed or allowlisted, Git backups of new page writes from ${root} are held (nothing is lost; the database has every write).`,++ why: 'Rewriting unpublished commits, allowlisting a value or repairing the checkout is the user\'s call.' }),++ user_message: `${fix?.user_message ?? `The secret scan refused the unpublished commits in ${root}.`} Until then, Git backups of new page writes from ${root} are held (nothing is lost; the database has every write).`,+ verify: { argv: ['gbrain', 'sources', 'writer', 'status', '--json'] } } });+}+/** The outcome a Git effect records when its root has no durability hook: no git command ran. */export const DURABILITY_NOT_ENABLED = { git: 'skipped', reason: 'durability_not_enabled', push: 'skipped' } as const;diff --cc src/core/persistence/git-push-holds.tsindex afbf769531,b93b21c01a..afd019a301--- a/src/core/persistence/git-push-holds.ts+++ b/src/core/persistence/git-push-holds.ts@@@ -13,7 -13,7 +13,7 @@@ import { readFix } from '../ops/op-fix.import { readIntegrateConflict, type IntegrateConflictState } from '../git/commit-push.ts';/** The error codes of a Git effect whose commit is local but whose push did not land. */- export const GIT_PUSH_CODES = ['git_push_unavailable', 'push_unconfirmed', 'integrate_conflict', 'publish_secret_found'] as const;-export const GIT_PUSH_CODES = ['git_push_unavailable', 'push_unconfirmed', 'integrate_conflict'] as const;++export const GIT_PUSH_CODES = ['git_push_unavailable', 'push_unconfirmed', 'integrate_conflict', 'publish_secret_found', 'publish_scan_unreadable'] as const;export interface GitPushHold {source_id: string;@@@ -68,13 -68,6 +68,20 @@@ function gitPushFix(hold: Omit<GitPushHuser_message: `This machine and the remote both changed ${hold.conflict.paths.join(', ')} in ${hold.root}, so ${held} Git backup(s) of page writes in source ${hold.source_id} are held (nothing is lost; the database has every write). Which version should win: this machine's (ours) or the remote's (theirs)? Afterwards run gbrain sources writer retry-effects ${hold.source_id} --parked-pushes, then gbrain sources refresh ${hold.source_id}.`,verify: statusVerify(hold.source_id), docs: 'docs/guides/shared-checkout-git.md#conflicts',};+ if (hold.codes.includes('publish_secret_found') && hold.root) return {+ ask_user: true, consent: [], actor: 'user', requires_exclusive: false,+ argv: ['git', '-C', hold.root, 'log', '--stat', '@{upstream}..HEAD'],+ why: 'Lists the unpublished commits the secret scan refused; read-only. A secret pushed to the remote stays in its history.',+ user_message: `Unpublished commits in ${hold.root} hold values that look like secrets (or text too large to scan), so ${held} Git backup(s) of page writes in source ${hold.source_id} are held and nothing was pushed (nothing is lost; the database has every write). Remove the values from those commits, or confirm a false positive so its fingerprint can go in .gbrain-scan-allow? Afterwards run gbrain sources writer retry-effects ${hold.source_id} --parked-pushes.`,+ verify: statusVerify(hold.source_id), docs: 'docs/guides/shared-checkout-git.md#publish_secret_found',+ };++ if (hold.codes.includes('publish_scan_unreadable') && hold.root) return {++ ask_user: true, consent: [], actor: 'user', requires_exclusive: false,++ argv: ['git', '-C', hold.root, 'fsck'],++ why: 'Read-only: names missing or corrupt objects in the unpublished commits, the usual reason the secret scan cannot read them.',++ user_message: `The secret scan could not read the unpublished commits in ${hold.root} (or the remote has no such branch), so ${held} Git backup(s) of page writes in source ${hold.source_id} are held and nothing was pushed (nothing is lost; the database has every write). Check the checkout with git fsck and fix what it reports? Afterwards run gbrain sources writer retry-effects ${hold.source_id} --parked-pushes.`,++ verify: statusVerify(hold.source_id), docs: 'docs/guides/shared-checkout-git.md#publish_scan_unreadable',++ };if (hold.parked && hold.former_worktree) return {argv: ['gbrain', 'sources', 'writer', 'settle-effects', hold.source_id, '--parked-code', hold.codes[0]!, '--dry-run'], consent: [], actor: 'agent', requires_exclusive: false,why: `${hold.parked} Git effect(s) of source ${hold.source_id} failed on a push (${hold.codes.join(', ')}) in a worktree the source is no longer bound to, so no retry reaches them. This previews settling each one whose current bytes the fetched remote already holds; it changes nothing.`,diff --cc test/effect-git-secret-gate.test.tsindex cb0834683f,0000000000..6ee7fe6e33mode 100644,000000..100644--- a/test/effect-git-secret-gate.test.ts+++ b/test/effect-git-secret-gate.test.ts@@@ -1,198 -1,0 +1,239 @@@+/**+ * #8896: the managed writer's Git effects scan what they publish. A script's plain `git commit` in a managed checkout+ * never passed the write path's scan; the Git effect's push used to publish it with the next page write, even after+ * `gbrain sources refresh` refused it with `publish_secret_found`.+ * Protects: a checkout only ahead of its upstream whose unpublished commits add a value that looks like a secret is+ * refused `publish_secret_found` by `pushGitRoot` and nothing is pushed; a diverged checkout (the remote moved) is+ * refused by the keep-checkout publish its child runs, and nothing is pushed; a remote rewound past a commit holding+ * one (the tracking ref still names it) is not re-published; a remote without the branch refuses+ * instead of scanning nothing; a clean ahead checkout still pushes. Through the effect worker, the page write's Git+ * effect holds on `publish_secret_found` (the remote keeps its tip), `gbrain sync status` names the user's fix, and an- * allowlisted false positive releases it.++ * allowlisted false positive releases it; commits the scan cannot read in full hold it as `publish_scan_unreadable`.+ * Seams: real repositories and a bare remote; the keep-checkout publish runs this checkout's own CLI+ * (GBRAIN_JOB_CHILD_CLI points at src/cli.ts); PGLite for the effect worker.+ */+import { afterEach, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { chmodSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { dirname, join, resolve } from 'node:path';++import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { serializePageToMarkdown } from '../src/core/markdown.ts';+import { pushGitRoot } from '../src/core/persistence/effect-git.ts';+import { submissionAuthority } from '../src/core/persistence/authority.ts';+import { localHostId, registerLocalWriter } from '../src/core/persistence/identity.ts';+import { admitWrite, claimNextWrite } from '../src/core/persistence/journal.ts';+import { publishMutation } from '../src/core/persistence/coordinator.ts';+import { claimWorktree, getWorktreeBinding } from '../src/core/persistence/ownership.ts';+import { runPersistenceEffects } from '../src/core/persistence/effects.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { readSyncStatus } from '../src/core/persistence/sync-status.ts';+import { scanText } from '../src/core/secret-scan.ts';+import { isolatedSharedSkillsEngine } from './helpers/shared-skills-engine.ts';+import { git } from './helpers/git-publication.ts';+import { withEnv } from './helpers/with-env.ts';++const token = 'ghp_' + 'Zt4q'.repeat(9);+const config = { engine: 'pglite' as const, embedding_disabled: true };+const dirs: string[] = [];+afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); });++interface Checkout { home: string; root: string; remote: string; other: string; cli: string }++/** A checkout on main pushed to a bare remote, a second clone that can move the remote, and a child CLI wrapper. */+function checkout(): Checkout {+ const home = realpathSync(mkdtempSync(join(tmpdir(), 'gbrain-8896-'))); dirs.push(home);+ const remote = join(home, 'remote.git'), root = join(home, 'brain'), other = join(home, 'other'), cli = join(home, 'gbrain-child');+ writeFileSync(cli, `#!/bin/sh\nexec '${process.execPath}' '${resolve(import.meta.dir, '../src/cli.ts')}' "$@"\n`);+ chmodSync(cli, 0o700);+ git(home, 'init', '-q', '--bare', '-b', 'main', remote);+ mkdirSync(join(root, 'notes'), { recursive: true });+ git(root, 'init', '-q', '-b', 'main');+ git(root, 'config', 'user.name', 'Example Writer'); git(root, 'config', 'user.email', '[email protected]');+ writeFileSync(join(root, 'initial.md'), 'Initial\n');+ git(root, 'add', 'initial.md'); git(root, 'commit', '-q', '-m', 'Initial');+ git(root, 'remote', 'add', 'origin', remote); git(root, 'push', '-q', '-u', 'origin', 'main');+ git(home, 'clone', '-q', remote, other);+ git(other, 'config', 'user.name', 'Other Machine'); git(other, 'config', 'user.email', '[email protected]');+ return { home, root, remote, other, cli };+}++/** A script's plain `git commit`: never through gbrain's write path, so never scanned there. */+function scriptCommit(root: string, path: string, content: string): string {+ mkdirSync(dirname(join(root, path)), { recursive: true });+ writeFileSync(join(root, path), content);+ git(root, 'add', path); git(root, 'commit', '-q', '-m', `script edits ${path}`);+ return git(root, 'rev-parse', 'HEAD').trim();+}+/** Another machine pushes a commit to the shared branch. */+function moveRemote(c: Checkout, path: string): string {+ mkdirSync(dirname(join(c.other, path)), { recursive: true });+ writeFileSync(join(c.other, path), 'From the other machine\n');+ git(c.other, 'add', path); git(c.other, 'commit', '-q', '-m', `Other machine edits ${path}`); git(c.other, 'push', '-q');+ return git(c.other, 'rev-parse', 'HEAD').trim();+}+const remoteTip = (c: Checkout) => git(c.remote, 'rev-parse', 'refs/heads/main').trim();+const childEnv = (c: Checkout) => ({ GBRAIN_HOME: join(c.home, 'home'), GBRAIN_JOB_CHILD_CLI: c.cli, GBRAIN_GIT_ALLOW_FILE_TRANSPORT: '1',+ DATABASE_URL: undefined, GBRAIN_DATABASE_URL: undefined });+async function refusal(run: Promise<unknown>) {+ try { await run; } catch (e) { return e as { code: string; message: string; suggestion?: string; detail?: string; fix?: { user_message?: string; ask_user?: boolean } }; }+ throw new Error('expected publish_secret_found');+}++describe('#8896 pushGitRoot scans the commits it publishes', () => {+ test('a checkout only ahead of its upstream with a script commit holding a token: refused, nothing pushed', async () => {+ const c = checkout();+ const before = remoteTip(c);+ const added = scriptCommit(c.root, 'notes/deploy.md', `Deploy token ${token}\n`);+ scriptCommit(c.root, 'notes/page.md', 'A clean page write on top\n');+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));+ expect(error.code).toBe('publish_secret_found');+ expect(error.message).toContain('notes/deploy.md:1 (');+ expect(error.message).toContain(added.slice(0, 7));+ expect(error.message).toContain('Nothing was pushed');+ expect(error.fix?.ask_user).toBe(true);+ expect(error.fix?.user_message).toContain('held');+ expect(JSON.stringify(error)).not.toContain(token);+ expect(remoteTip(c)).toBe(before);+ }, 60_000);++ test('a diverged checkout (the remote moved): the keep-checkout publish refuses, nothing pushed', async () => {+ const c = checkout();+ scriptCommit(c.root, 'notes/deploy.md', `Deploy token ${token}\n`);+ const theirs = moveRemote(c, 'notes/other-example.md');+ const head = git(c.root, 'rev-parse', 'HEAD').trim();+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));+ expect(error.code).toBe('publish_secret_found');+ expect(error.message).toContain('notes/deploy.md:1 (');+ expect(error.fix?.user_message).toContain('held');+ expect(JSON.stringify(error)).not.toContain(token);+ expect(remoteTip(c)).toBe(theirs);+ expect(git(c.root, 'rev-parse', 'HEAD').trim()).toBe(head);+ }, 120_000);++ test('a remote rewound past a published secret (the tracking ref is stale): it is not published again', async () => {+ const c = checkout();+ const base = remoteTip(c);+ scriptCommit(c.root, 'notes/deploy.md', `Deploy token ${token}\n`);+ git(c.root, 'push', '-q');+ git(c.remote, 'update-ref', 'refs/heads/main', base);+ scriptCommit(c.root, 'notes/page.md', 'A clean page write on top\n');+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));+ expect(error.code).toBe('publish_secret_found');+ expect(error.message).toContain('notes/deploy.md:1 (');+ expect(error.fix?.user_message).toContain('held');+ expect(remoteTip(c)).toBe(base);+ }, 120_000);++ test('a remote without the branch refuses instead of scanning nothing', async () => {+ const c = checkout();+ scriptCommit(c.root, 'notes/page.md', 'A clean page\n');+ git(c.remote, 'update-ref', '-d', 'refs/heads/main');+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));- expect(error.code).toBe('publish_secret_found');++ expect(error.code).toBe('publish_scan_unreadable');+ expect(error.message).toContain('origin has no branch main');+ expect(() => git(c.remote, 'rev-parse', '--verify', 'refs/heads/main')).toThrow();+ }, 60_000);+++ test('commits the scan cannot read in full hold the push as publish_scan_unreadable, nothing pushed', async () => {++ const c = checkout();++ const before = remoteTip(c);++ scriptCommit(c.root, 'notes/page.md', 'plain text\n');++ const blob = git(c.root, 'rev-parse', 'HEAD:notes/page.md').trim();++ git(c.root, 'rm', '-q', 'notes/page.md'); git(c.root, 'commit', '-q', '-m', 'drop page');++ rmSync(join(c.root, '.git', 'objects', blob.slice(0, 2), blob.slice(2)));++ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));++ expect(error.code).toBe('publish_scan_unreadable');++ expect(error.message).not.toContain('look like secrets');++ expect(error.fix?.user_message).toContain('held');++ expect(remoteTip(c)).toBe(before);++ }, 60_000);+++ test('a clean ahead checkout still pushes', async () => {+ const c = checkout();+ scriptCommit(c.root, 'notes/page.md', 'A clean page\n');+ expect(await withEnv(childEnv(c), () => pushGitRoot(c.root))).toEqual({ push: 'committed' });+ expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());+ }, 60_000);+});+- describe('#8896 through the effect worker', () => {- test('a page write\'s Git effect holds on publish_secret_found, sync status names the fix, and an allowlisted false positive releases it', async () => {- const c = checkout();- await withEnv({ ...childEnv(c), GBRAIN_AUDIT_DIR: join(c.home, 'audit'), GBRAIN_BRAIN_ID: 'host', GBRAIN_SOURCE: undefined }, async () => {- const { engine, close } = await isolatedSharedSkillsEngine(undefined);- try {- await registerLocalWriter(engine, 'cli');- const sourceId = `s8896-${randomUUID().slice(0, 8)}`;- await engine.executeRaw('INSERT INTO sources(id,name,local_path) VALUES($1,$1,$2)', [sourceId, c.root]);- const claimed = await claimWorktree(engine, sourceId, c.root, localHostId());- await engine.executeRaw("UPDATE persistence_host_bindings SET git_durability='enabled' WHERE worktree_id=$1::uuid", [claimed.worktree_id]);- const before = remoteTip(c);- scriptCommit(c.root, 'scripts-out/deploy.md', `Deploy token ${token}\n`);-- const slug = 'notes/acme-example';- const page = (body: string) => ({ type: 'note', title: 'Acme example', compiled_truth: body, timeline: '', frontmatter: {} });- const bound = (await getWorktreeBinding(engine, sourceId, localHostId()))!;- await engine.putPage(slug, page('Before'), { sourceId });- const snapshot = (await engine.readPageSnapshot(slug, { sourceId }))!;- const file = join(c.root, `${slug}.md`);- writeFileSync(file, serializePageToMarkdown(snapshot.page, snapshot.tags));- const ctx: OperationContext = { engine, config, remote: false, dryRun: false, sourceId, logger: { info() {}, warn() {}, error() {} } };- const authority = await submissionAuthority(ctx, 'put_page', sourceId, bound.source_incarnation, slug);- await admitWrite(engine, { principal: authority.principal, authority, operation: 'put_page', sourceId,- sourceIncarnation: bound.source_incarnation, slug, pageId: snapshot.page.id, requestId: randomUUID(),- callerIntent: { content: 'After' }, intent: { content: 'After' }, worktreeId: bound.worktree_id, topologyGeneration: bound.topology_generation });- const row = (await claimNextWrite(engine, localHostId()))!;- const result = await publishMutation(engine, row, { observedRevision: snapshot.revision, file: { path: file, root: c.root, content: 'After' },- apply: async tx => { await tx.putPage(slug, page('After'), { sourceId }); return {}; } }, localHostId());- expect(result.state).toBe('committed');-- const runEffects = async () => {++interface ManagedWrite { engine: BrainEngine; sourceId: string; runEffects: () => Promise<void>; effect: () => Promise<{ state: string; error_code: string | null }> }++++/** A managed source on the checkout with git durability on; `script` commits into the checkout before one page write whose Git effect is queued. */++async function managedWrite(c: Checkout, script: () => void, run: (w: ManagedWrite) => Promise<void>): Promise<void> {++ await withEnv({ ...childEnv(c), GBRAIN_AUDIT_DIR: join(c.home, 'audit'), GBRAIN_BRAIN_ID: 'host', GBRAIN_SOURCE: undefined }, async () => {++ const { engine, close } = await isolatedSharedSkillsEngine(undefined);++ try {++ await registerLocalWriter(engine, 'cli');++ const sourceId = `s8896-${randomUUID().slice(0, 8)}`;++ await engine.executeRaw('INSERT INTO sources(id,name,local_path) VALUES($1,$1,$2)', [sourceId, c.root]);++ const claimed = await claimWorktree(engine, sourceId, c.root, localHostId());++ await engine.executeRaw("UPDATE persistence_host_bindings SET git_durability='enabled' WHERE worktree_id=$1::uuid", [claimed.worktree_id]);++ script();++++ const slug = 'notes/acme-example';++ const page = (body: string) => ({ type: 'note', title: 'Acme example', compiled_truth: body, timeline: '', frontmatter: {} });++ const bound = (await getWorktreeBinding(engine, sourceId, localHostId()))!;++ await engine.putPage(slug, page('Before'), { sourceId });++ const snapshot = (await engine.readPageSnapshot(slug, { sourceId }))!;++ const file = join(c.root, `${slug}.md`);++ writeFileSync(file, serializePageToMarkdown(snapshot.page, snapshot.tags));++ const ctx: OperationContext = { engine, config, remote: false, dryRun: false, sourceId, logger: { info() {}, warn() {}, error() {} } };++ const authority = await submissionAuthority(ctx, 'put_page', sourceId, bound.source_incarnation, slug);++ await admitWrite(engine, { principal: authority.principal, authority, operation: 'put_page', sourceId,++ sourceIncarnation: bound.source_incarnation, slug, pageId: snapshot.page.id, requestId: randomUUID(),++ callerIntent: { content: 'After' }, intent: { content: 'After' }, worktreeId: bound.worktree_id, topologyGeneration: bound.topology_generation });++ const row = (await claimNextWrite(engine, localHostId()))!;++ const result = await publishMutation(engine, row, { observedRevision: snapshot.revision, file: { path: file, root: c.root, content: 'After' },++ apply: async tx => { await tx.putPage(slug, page('After'), { sourceId }); return {}; } }, localHostId());++ expect(result.state).toBe('committed');++ await run({ engine, sourceId,++ runEffects: async () => {+ await engine.executeRaw("UPDATE persistence_effects SET next_attempt_at=now() WHERE kind='git'");+ await runPersistenceEffects(engine, config, { hostId: localHostId(), limit: 10 });- };- const effect = async () => (await engine.executeRaw<{ state: string; error_code: string | null }>(- "SELECT state,error_code FROM persistence_effects WHERE request_id=$1::uuid AND kind='git'", [row.id]))[0]!;- await runEffects();- expect(await effect()).toMatchObject({ state: 'queued', error_code: 'publish_secret_found' });- expect(remoteTip(c)).toBe(before);- const hold = (await readSyncStatus(engine, sourceId)).git_push_holds[0]!;- expect(hold).toMatchObject({ retrying: 1, parked: 0, codes: ['publish_secret_found'] });- expect(hold.fix).toMatchObject({ ask_user: true, actor: 'user' });- expect(hold.fix.user_message).toContain('look like secrets');-- writeFileSync(join(c.root, '.gbrain-scan-allow'), `${scanText(`Deploy token ${token}\n`)[0]!.fingerprint}\n`);- await runEffects();- expect(await effect()).toMatchObject({ state: 'committed', error_code: null });- expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());- } finally { await disposePersistenceConsumer(engine); await close(); }++ },++ effect: async () => (await engine.executeRaw<{ state: string; error_code: string | null }>(++ "SELECT state,error_code FROM persistence_effects WHERE request_id=$1::uuid AND kind='git'", [row.id]))[0]! });++ } finally { await disposePersistenceConsumer(engine); await close(); }++ });++}++++describe('#8896 through the effect worker', () => {++ test("a page write's Git effect holds on publish_secret_found, sync status names the fix, and an allowlisted false positive releases it", async () => {++ const c = checkout();++ const before = remoteTip(c);++ await managedWrite(c, () => scriptCommit(c.root, 'scripts-out/deploy.md', `Deploy token ${token}\n`), async w => {++ await w.runEffects();++ expect(await w.effect()).toMatchObject({ state: 'queued', error_code: 'publish_secret_found' });++ expect(remoteTip(c)).toBe(before);++ const hold = (await readSyncStatus(w.engine, w.sourceId)).git_push_holds[0]!;++ expect(hold).toMatchObject({ retrying: 1, parked: 0, codes: ['publish_secret_found'] });++ expect(hold.fix).toMatchObject({ ask_user: true, actor: 'user' });++ expect(hold.fix.user_message).toContain('look like secrets');++++ writeFileSync(join(c.root, '.gbrain-scan-allow'), `${scanText(`Deploy token ${token}\n`)[0]!.fingerprint}\n`);++ await w.runEffects();++ expect(await w.effect()).toMatchObject({ state: 'committed', error_code: null });++ expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());++ });++ }, 180_000);++++ test('commits the scan cannot read hold the Git effect as publish_scan_unreadable, and sync status names git fsck', async () => {++ const c = checkout();++ const before = remoteTip(c);++ await managedWrite(c, () => {++ scriptCommit(c.root, 'scripts-out/report.md', 'plain text\n');++ const blob = git(c.root, 'rev-parse', 'HEAD:scripts-out/report.md').trim();++ git(c.root, 'rm', '-q', 'scripts-out/report.md'); git(c.root, 'commit', '-q', '-m', 'drop report');++ rmSync(join(c.root, '.git', 'objects', blob.slice(0, 2), blob.slice(2)));++ }, async w => {++ await w.runEffects();++ expect(await w.effect()).toMatchObject({ state: 'queued', error_code: 'publish_scan_unreadable' });++ expect(remoteTip(c)).toBe(before);++ const hold = (await readSyncStatus(w.engine, w.sourceId)).git_push_holds[0]!;++ expect(hold).toMatchObject({ retrying: 1, parked: 0, codes: ['publish_scan_unreadable'] });++ expect(hold.fix).toMatchObject({ ask_user: true, actor: 'user', argv: ['git', '-C', c.root, 'fsck'] });+ });+ }, 180_000);+});diff --cc test/guarded-publish.test.tsindex 359d473db8,516527cd7c..9aca524625--- a/test/guarded-publish.test.ts+++ b/test/guarded-publish.test.ts@@@ -147,13 -148,76 +149,87 @@@ test('attributes outside the range andexpect(remoteTip(noprefix)).toBe(noprefix.base);});+test('a remote rewound past a pushed secret is not published again through a stale tracking ref', () => {+ const r = repo('rewound');+ commit(r.root, 'notes.md', `Deploy token ${token}\n`);+ git(r.root, 'push', '-q');+ git(r.remote, 'update-ref', 'refs/heads/main', r.base);+ commit(r.root, 'page.md', 'a clean page\n');+ const error = refused(() => publish(r.root));+ expect(error.message).toContain('notes.md:1 (');+ expect(remoteTip(r)).toBe(r.base);+});+++ test('a NUL byte past the first 8000 bytes of a text file never hides a secret in it', () => {+ const r = repo('late-nul');+ commit(r.root, 'notes.md', `Deploy token ${token}\n${'x'.repeat(10_000)}\0tail\n`);+ const error = refused(() => publish(r.root));+ expect(error.code).toBe('publish_secret_found');+ expect(error.message).toContain('notes.md:1 (');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('a repository log encoding never hides a secret in a commit message', () => {+ const r = repo('encoding');+ git(r.root, 'config', 'i18n.logOutputEncoding', 'UTF-16LE');+ commit(r.root, 'page.md', 'a page\n', `rotate ${token}`);+ expect(refused(() => publish(r.root)).message).toContain('(commit message):1 (');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('a root commit merged in from unrelated history is scanned whatever log.showRoot says', () => {+ const r = repo('showroot');+ git(r.root, 'config', 'log.showRoot', 'false');+ git(r.root, 'checkout', '-q', '--orphan', 'imported');+ git(r.root, 'rm', '-rqf', '.');+ commit(r.root, 'imported.md', `Deploy token ${token}\n`);+ git(r.root, 'checkout', '-q', 'main');+ git(r.root, 'merge', '-q', '--allow-unrelated-histories', '--no-edit', 'imported');+ expect(refused(() => publish(r.root)).message).toContain('imported.md:1 (');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('commits the scan cannot read refuse with their own code, not as secrets to remove', () => {+ const r = repo('unreadable');+ commit(r.root, 'notes.md', 'plain text\n');+ const blob = git(r.root, 'rev-parse', 'HEAD:notes.md');+ git(r.root, 'rm', '-q', 'notes.md'); git(r.root, 'commit', '-qm', 'drop notes');+ rmSync(join(r.root, '.git', 'objects', blob.slice(0, 2), blob.slice(2)));+ const error = refused(() => publish(r.root));+ expect(error.code).toBe('publish_scan_unreadable');+ expect(error.message).toContain('Nothing was pushed.');+ expect(error.message).not.toContain('look like secrets');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('a token in a commit message that declares UTF-16 is still found in its raw bytes', () => {+ const r = repo('utf16-message');+ git(r.root, 'config', 'i18n.commitEncoding', 'UTF-16LE');+ commit(r.root, 'page.md', 'a page\n', `rotate ${token}`);+ expect(refused(() => publish(r.root)).message).toContain('(commit message):1 (');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('a commit message in an encoding the scan cannot decode refuses as unreadable, not as too large', () => {+ const r = repo('utf7-message');+ git(r.root, 'config', 'i18n.commitEncoding', 'UTF-7');+ commit(r.root, 'page.md', 'a page\n', 'plain message');+ const error = refused(() => publish(r.root));+ expect(error.code).toBe('publish_scan_unreadable');+ expect(error.message).not.toContain('too large to scan');+ expect(remoteTip(r)).toBe(r.base);+ });++ test('every git call the scan makes ignores the system gitattributes file', async () => {+ const r = repo('nosystem');+ commit(r.root, 'notes.md', 'plain\n');+ const bin = join(dir, 'git-spy'), log = join(dir, 'git-spy.log');+ const real = execFileSync('sh', ['-c', 'command -v git'], { encoding: 'utf8' }).trim();+ mkdirSync(bin);+ writeFileSync(join(bin, 'git'), `#!/bin/sh\nprintf '%s\\n' "\${GIT_ATTR_NOSYSTEM:-unset}" >> '${log}'\nexec '${real}' "$@"\n`, { mode: 0o755 });+ const scan = await withEnv({ PATH: `${bin}:${process.env.PATH}` }, async () => scanRange(r.root, 'refs/remotes/origin/main', 'HEAD'));+ expect(scan.unreadable).toBeUndefined();+ const calls = readFileSync(log, 'utf8').trim().split('\n');+ expect(calls.length).toBeGreaterThan(3);+ expect(calls.filter(value => value !== '1')).toEqual([]);+ });