Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Close five ways a managed push could publish a secret unscanned

garrytan718964461bce
diff --git a/behavior-changes.d/managed_push_scan_every_publish_gst2650.md b/behavior-changes.d/managed_push_scan_every_publish_gst2650.md
new file mode 100644
index 0000000000..6d74cb3b77
--- /dev/null
+++ b/behavior-changes.d/managed_push_scan_every_publish_gst2650.md
@@ -0,0 +1 @@
+A managed worktree now scans every push it makes for secrets, including the generated-page commits the dream cycle writes through and a push handed over from a busy publish lock, which used to go out unscanned. Only the commits the remote lacks are scanned, never history it already holds. A credential in an unpushed commit, or a false positive there, holds the push with `publish_secret_found` naming the file, line and rule, never the value; remove it from the commit or add its fingerprint to `.gbrain-scan-allow`, then run `gbrain sources writer retry-effects <source> --parked-pushes`. Three narrower gaps are closed too: in a managed root below the repository's top level, a commit changing that root's own `.gbrain-scan-allow` is held like one at the top level, and its path globs match paths relative to that root; a strict (canonical: git) source's commit whose files a Git clean filter changed is scanned instead of trusted (it is still pushed when the scan finds nothing); and Git replace refs no longer change what a page-write commit holds.
diff --git a/docs/guides/shared-checkout-git.md b/docs/guides/shared-checkout-git.md
index a52eb3e9b2..63af6e4db0 100644
--- a/docs/guides/shared-checkout-git.md
+++ b/docs/guides/shared-checkout-git.md
@@ -189,7 +189,11 @@ mode scans either way.
A finding that is not a secret: add its fingerprint (`sha256:` and 16 hex
characters, as the preview prints it) as a line of `.gbrain-scan-allow` at the
checkout root (`#` starts a comment). The scan reads the file from the working
-tree, so the next push uses it without a commit. Keep it out of commits (list it
+tree, so the next push uses it without a commit. That is deliberate: the file is
+the local user's answer to the scan, and writing it takes local write access to
+the checkout, which could push directly anyway. In a managed root below the
+repository's top level the file sits at that root and its path globs match
+paths relative to it. Keep it out of commits (list it
in `.git/info/exclude`): a commit that changes it changes what the scan sees, so
no automatic push publishes it. A fingerprint accepts that exact value wherever
it appears; a line without `sha256:` is a path glob and accepts every finding in
@@ -220,7 +224,7 @@ Follow the error's `fix.next` as the
| `git_index_stale` | 1 | An `index.lock` older than 10 minutes blocks the checkout. Check that no git command is running there, tell the user, and remove the named lock file only with their agreement. |
| `git_operation_in_progress` | 1 | A rebase, merge, cherry-pick or revert is unfinished in the checkout. gbrain never aborts it for you. Follow [stuck rebase](#stuck-rebase). |
| `git_detached_head` | 1 | The checkout is on a detached HEAD or an unborn branch. Check out the branch the remote tracks (`git switch <branch>`), or make the first commit, then rerun. |
-| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | On every push `gbrain git commit-push` makes (commit mode, `--pending` and `--push-only`; `--scan-secrets` is accepted for older scripts), so also the post-commit hook `gbrain sources harden` installs and `scripts/brain-commit-push.sh`, and on every automatic publish: the one `gbrain sources refresh` runs on its own, the push of a managed worktree's Git effects, doctor `git_convergence`'s push fixes, the access-policy and durability-scaffolding commits, `gbrain sources push` and the first push of `gbrain bootstrap repo` (#8902; there the remote has no branch yet, so the whole history is scanned). A commit about to be pushed that gbrain did not make for a page write (its commit ledger records its own page-write commits, whose bodies are stored raw by design and are not scanned; a Git effect's commit counts only by the sha gbrain created for it, when its files held the bytes the effect wrote, so an amended or reworded copy is scanned; one that adopted a file edited outside gbrain, or one older than the retained ledger, is scanned) adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. The exception is the commit `gbrain sources push` or `gbrain bootstrap repo` makes itself from files it has just scanned with that allowlist. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |
+| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | On every push `gbrain git commit-push` makes (commit mode, `--pending` and `--push-only`; `--scan-secrets` is accepted for older scripts), so also the post-commit hook `gbrain sources harden` installs and `scripts/brain-commit-push.sh`, and on every automatic publish: the one `gbrain sources refresh` runs on its own, the push of a managed worktree's Git effects, its generated-page write-through commits, doctor `git_convergence`'s push fixes, the access-policy and durability-scaffolding commits, `gbrain sources push` and the first push of `gbrain bootstrap repo` (#8902; there the remote has no branch yet, so the whole history is scanned). A commit about to be pushed that gbrain did not make for a page write (its commit ledger records its own page-write commits, whose bodies are stored raw by design and are not scanned; a Git effect's commit counts only by the sha gbrain created for it, when its files held the bytes the effect wrote, so an amended or reworded copy is scanned; one that adopted a file edited outside gbrain, or one older than the retained ledger, is scanned) adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. The exception is the commit `gbrain sources push` or `gbrain bootstrap repo` makes itself from files it has just scanned with that allowlist. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |
| <a id="publish_scan_unreadable"></a>`publish_scan_unreadable` | 1 | Wherever `publish_secret_found` can refuse: the scan could not read the commits about to be pushed in full (a missing or corrupt object, git output it does not recognise, or its time budget), so it reported no secret and nothing was pushed. When it ran out of time (a first push of a long history can need more than the 300 s default), the fix is the same command with `--scan-timeout <seconds>`, four times the budget and at least 1200: run it; the scan still reads every commit, and prints its progress on stderr. `gbrain git commit-push`, `gbrain sources push` and `gbrain bootstrap repo` take `--scan-timeout`. Otherwise run `git fsck` in the checkout, fix what it names, then retry. When the remote has no such branch, a managed worktree's Git effect scans the branch's whole history and creates it only when that scan passes; a history too large to scan in the owner process holds the effects with this code and names `git ls-remote --heads <remote>`: check the upstream, then point it at the right branch or review the history and push the branch yourself. Never publish with a command that skips the scan. On a managed worktree its Git effects hold meanwhile, as for `publish_secret_found`. |
| `integrate_conflict` | 1 | Local and remote commits changed the same file; nothing was merged and the checkout is unchanged. Stop, relay the question to the user, then [resolve](#conflicts). |
| `push_unconfirmed` | 4 | The commit is durable locally but no push destination confirmed it (network, credentials, branch protection, a remote that kept moving, or another push that held the push lock for the whole wait). Fix what the message names, then run `gbrain git commit-push --push-only`. Never force-push. |
diff --git a/src/commands/git.ts b/src/commands/git.ts
index 1a185a9196..5646cf2c3f 100644
--- a/src/commands/git.ts
+++ b/src/commands/git.ts
@@ -289,7 +289,6 @@ export async function runGit(args: string[]): Promise<number> {
...(p.pending ? { pending: true } : {}),
...(p.allowOutside ? { allowOutsideSourceRoot: true } : {}),
...(p.keepCheckout ? { keepCheckout: true } : {}),
- scanSecrets: true,
...(p.scanTimeout ? { scanTimeoutMs: Number(p.scanTimeout) * 1000 } : {}),
scanProgress: (line: string) => process.stderr.write(`[gbrain] ${line}\n`),
...(p.untrack ? { untrack: true } : {}),
diff --git a/src/core/access/policy-git.ts b/src/core/access/policy-git.ts
index 1f70f7b060..45a2591f33 100644
--- a/src/core/access/policy-git.ts
+++ b/src/core/access/policy-git.ts
@@ -212,7 +212,7 @@ export async function commitPolicyFile(root: string, relPath: string, edit: (cur
const next = edit(current);
writeFileSync(file, next);
try {
- const r = commitAndPush({ repo: root, cwd: root, paths: [file], message, scanSecrets: true });
+ const r = commitAndPush({ repo: root, cwd: root, paths: [file], message });
return { outcome: r.outcome, commit: r.committed ?? r.sha, path: relPath,
...(r.outcome === 'committed_local' ? { note: 'Committed locally; this checkout has no remote or is managed, so nothing was pushed.' } : {}) };
} catch (e) {
diff --git a/src/core/brain-repo-durability.ts b/src/core/brain-repo-durability.ts
index b1a0abe158..be9e350a05 100644
--- a/src/core/brain-repo-durability.ts
+++ b/src/core/brain-repo-durability.ts
@@ -1298,7 +1298,7 @@ function commitScaffolding(repoPath: string, branch: string, redact: (s: string)
const resolver = findResolverFile(repoPath);
if (resolver) paths.push(relative(repoPath, resolver));
try {
- const r = commitAndPush({ repo: repoPath, paths, cwd: repoPath, message: 'chore(gbrain): install brain durability scaffolding', allowOutsideSourceRoot: true, scanSecrets: true });
+ const r = commitAndPush({ repo: repoPath, paths, cwd: repoPath, message: 'chore(gbrain): install brain durability scaffolding', allowOutsideSourceRoot: true });
if (r.branch !== branch) return { status: 'needs_attention', detail: `checkout is on ${r.branch}, not ${branch}` };
if (r.outcome === 'nothing_to_commit') return { status: 'ok', detail: 'scaffolding already committed' };
return { status: 'fixed', detail: `committed + pushed durability scaffolding${r.checkout === 'lagging' ? ` (checkout lagging: ${r.lagging_reason})` : ''}` };
diff --git a/src/core/git/commit-push.ts b/src/core/git/commit-push.ts
index 8c6054eb39..09ef2c0e48 100644
--- a/src/core/git/commit-push.ts
+++ b/src/core/git/commit-push.ts
@@ -124,8 +124,6 @@ export interface CommitPushOptions {
branch?: string;
/** Push-only: never advance the checkout and skip its pre-push hook (a managed canonical worktree; see the header). */
keepCheckout?: boolean;
- /** Before pushing, scan what the commits the upstream lacks add (messages, added lines, merge resolutions) and refuse `publish_secret_found` on a finding: the gate for automatic publishes of commits gbrain did not write. */
- scanSecrets?: boolean;
/** The secret scan's time budget (`--scan-timeout`; default 300 s) and where its progress lines go. */
scanTimeoutMs?: number;
scanProgress?: (line: string) => void;
@@ -148,7 +146,6 @@ interface Ctx {
mergeRef: string;
trackingRef: string;
keepCheckout?: boolean;
- scanSecrets?: boolean;
scanTimeoutMs?: number;
scanProgress?: (line: string) => void;
}
@@ -1198,13 +1195,13 @@ function conflictError(ctx: Ctx, ours: string, theirs: string, merged: MergeTree
const compare = `Compare the two versions (this machine's: git show ${local}:<path>; the remote's: git show ${remote}:<path>)`;
const fix = merged.structural.length
? `${preface}The local commit ${local} stays local and the checkout is unchanged. ${merged.structural.map(shellWord).join(', ')} ${merged.structural.length === 1 ? 'is' : 'are'} a file/directory or rename conflict that a per-file choice cannot resolve. ${compare}, then move or remove the local path in a new commit (gbrain git commit-push -m "<msg>" -- <paths>) and rerun gbrain git commit-push --push-only. See docs/guides/shared-checkout-git.md#conflicts.`
- : `${preface}The local commit ${local} stays local and the checkout is unchanged. ${compare}, then run gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''}${ctx.scanSecrets ? ' --scan-secrets' : ''} --expect-remote ${remote} ${paths.map(p => `--resolve ${shellWord(p)}=ours|theirs`).join(' ')} (ours = this machine's version, theirs = the remote's, or =file:<path> with a merged copy). Until then every push from this checkout reports this conflict. See docs/guides/shared-checkout-git.md#conflicts.`;
+ : `${preface}The local commit ${local} stays local and the checkout is unchanged. ${compare}, then run gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''} --scan-secrets --expect-remote ${remote} ${paths.map(p => `--resolve ${shellWord(p)}=ours|theirs`).join(' ')} (ours = this machine's version, theirs = the remote's, or =file:<path> with a merged copy). Until then every push from this checkout reports this conflict. See docs/guides/shared-checkout-git.md#conflicts.`;
return opError('integrate_conflict', `Local and remote commits both changed ${paths.length} file(s) on ${ctx.branch}; nothing was merged.`, fix,
{ detail: JSON.stringify({ local: ours, remote: theirs, paths, structural: merged.structural, ...(merged.superseded?.length ? { superseded: merged.superseded } : {}) }),
fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,
why: 'Choosing which version of a file wins discards the other side; ask the user before resolving.',
user_message: `Both this machine and the remote changed ${paths.join(', ')}. Which version should win, or should they be merged by hand?`,
- ...(merged.structural.length ? {} : { argv: ['gbrain', 'git', 'commit-push', '--push-only', ...(ctx.keepCheckout ? ['--keep-checkout'] : []), ...(ctx.scanSecrets ? ['--scan-secrets'] : []), '--path', ctx.root, '--expect-remote', theirs, ...paths.flatMap(p => ['--resolve', `${p}=<ours|theirs>`])] }),
+ ...(merged.structural.length ? {} : { argv: ['gbrain', 'git', 'commit-push', '--push-only', ...(ctx.keepCheckout ? ['--keep-checkout'] : []), '--scan-secrets', '--path', ctx.root, '--expect-remote', theirs, ...paths.flatMap(p => ['--resolve', `${p}=<ours|theirs>`])] }),
docs: 'docs/guides/shared-checkout-git.md#conflicts' } });
}
@@ -1319,11 +1316,13 @@ export function readQuarantine(repo: string): QuarantineEntry[] {
* When the remote already holds the candidate, the published tip (which may be
* a merge built over it earlier) is returned so a lagging checkout catches up.
*/
-function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> | undefined, expectRemote: string | undefined, cwd: string, scanSecrets = false): Published {
+function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> | undefined, expectRemote: string | undefined, cwd: string): Published {
const bound = expectRemote && !resolutions?.size ? gitOk(ctx.root, ['rev-parse', '--verify', `${expectRemote}^{commit}`], 'rev-parse --expect-remote') : undefined;
- // #8896: a scanned publish scans <base>..<candidate> before every push and leases the push on that base, so a remote that
- // moved or was rewound since the tracking ref was fetched (a stale ref hides commits from the scan) is fetched and scanned again.
- let scannedBase = scanSecrets ? pushBase(ctx, git(ctx.root, ['rev-parse', '--verify', '--quiet', ctx.trackingRef]).stdout.trim(), start) : '';
+ // #8896: every publish (GST-2650: the derived write-through and a handed-off request too) scans <base>..<candidate> before
+ // every push and leases the push on that base, so a remote that moved or was rewound since the tracking ref was fetched (a
+ // stale ref hides commits from the scan) is fetched and scanned again; a missing tracking ref is fetched first, so the scan
+ // covers what the remote lacks, never its whole history.
+ let scannedBase = pushBase(ctx, git(ctx.root, ['rev-parse', '--verify', '--quiet', ctx.trackingRef]).stdout.trim() || (fetchTracking(ctx) ?? ''), start);
let candidate = start;
let integrated = 0;
let lastError = '';
@@ -1339,16 +1338,16 @@ function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> |
if (attempt) Atomics.wait(sleeper, 0, 0, Math.min(2000, 50 * 2 ** attempt) + Math.floor(Math.random() * 100));
let remoteTip = bound && !isAncestor(ctx.root, bound, candidate) ? bound : null;
if (!remoteTip) {
- const lease = bound ?? (scanSecrets ? scannedBase : undefined);
- if (scanSecrets) assertNoSecretsInRange({ ...ctx, trackingRef: lease! }, candidate);
+ const lease = bound ?? scannedBase;
+ assertNoSecretsInRange({ ...ctx, trackingRef: lease }, candidate);
const push = lease && !isAncestor(ctx.root, lease, candidate) ? null
- : remoteGit(ctx.root, 'push', ['--no-follow-tags', '--recurse-submodules=no', ...(ctx.keepCheckout ? ['--no-verify'] : []), ...(lease !== undefined ? [`--force-with-lease=${ctx.mergeRef}:${lease}`] : []), '--', ctx.remote, `${candidate}:${ctx.mergeRef}`]);
+ : remoteGit(ctx.root, 'push', ['--no-follow-tags', '--recurse-submodules=no', ...(ctx.keepCheckout ? ['--no-verify'] : []), `--force-with-lease=${ctx.mergeRef}:${lease}`, '--', ctx.remote, `${candidate}:${ctx.mergeRef}`]);
if (push?.code === 0) return { sha: candidate, integrated, confirmed: 'push', ...notes() };
if (push) lastError = push.stderr.trim();
if (remoteContains(ctx, candidate)) return onRemote();
remoteTip = fetchTracking(ctx);
if (!remoteTip) break;
- if (scanSecrets) scannedBase = pushBase(ctx, remoteTip, candidate);
+ scannedBase = pushBase(ctx, remoteTip, candidate);
if (bound && remoteTip !== bound) {
throw opError('push_unconfirmed', `The remote moved to ${remoteTip.slice(0, 12)} after --expect-remote ${bound.slice(0, 12)} was checked, so nothing was pushed: publishing now could merge back commits a rewrite removed.`,
'Check the remote history first (gbrain sources refresh checks it for a force-push), then rerun.', { detail: candidate, retryable: true });
@@ -1358,7 +1357,7 @@ function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> |
continue;
}
}
- if (scanSecrets) assertNoSecretsInRange({ ...ctx, trackingRef: remoteTip }, candidate);
+ assertNoSecretsInRange({ ...ctx, trackingRef: remoteTip }, candidate);
const raw = integrate(ctx.root, candidate, remoteTip);
const kept = supersededPaths(ctx.root, candidate, remoteTip, raw, resolutions);
const merged = kept.length
@@ -1386,7 +1385,7 @@ function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> |
}
if (remoteContains(ctx, candidate)) return onRemote();
throw opError('push_unconfirmed', `The commit is durable locally but no push to ${ctx.remote}/${ctx.mergeRef.replace(/^refs\/heads\//, '')} was confirmed.`,
- `git push failed (network, credentials, branch protection, or a remote that kept moving): ${lastError.slice(0, 500) || 'no detail'}. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''}${ctx.scanSecrets ? ' --scan-secrets' : ''} once the remote is reachable; never git pull --rebase in this checkout.`,
+ `git push failed (network, credentials, branch protection, or a remote that kept moving): ${lastError.slice(0, 500) || 'no detail'}. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''} --scan-secrets once the remote is reachable; never git pull --rebase in this checkout.`,
{ detail: candidate, retryable: true });
}
@@ -1419,7 +1418,7 @@ function assertLocalCommitsInsideRoots(ctx: Ctx, target: string, roots: string[]
`A commit made without gbrain git commit-push (for example a cron's git add plus git commit) can carry files other writers had staged. Show the user git show --stat ${offending[0].sha.slice(0, 12)}; if they want it published as is, rerun with --allow-outside-source-root. See docs/guides/shared-checkout-git.md#source-roots.`,
{ detail: JSON.stringify({ roots, commits: offending }),
fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,
- argv: ['gbrain', 'git', 'commit-push', '--push-only', ...(ctx.keepCheckout ? ['--keep-checkout'] : []), ...(ctx.scanSecrets ? ['--scan-secrets'] : []), '--allow-outside-source-root', '--path', ctx.root],
+ argv: ['gbrain', 'git', 'commit-push', '--push-only', ...(ctx.keepCheckout ? ['--keep-checkout'] : []), '--scan-secrets', '--allow-outside-source-root', '--path', ctx.root],
why: 'These commits change files that are not the brain\'s; publishing them can overwrite other projects\' or writers\' work on the remote.',
user_message: `Local commit(s) ${shas} change files outside the brain's folder (${files.slice(0, 5).join(', ')}${files.length > 5 ? ', ...' : ''}). Publish them anyway?`,
docs: 'docs/guides/shared-checkout-git.md#source-roots' } });
@@ -1577,7 +1576,7 @@ export function commitAndPush(opts: CommitPushOptions): CommitPushResult {
throw opError('commit_paths_invalid', 'A commit message is required.', 'Pass -m "<message>".');
}
const upstream = resolveCtx(root);
- const ctx = { ...withBranch(upstream, opts.branch), ...(opts.keepCheckout ? { keepCheckout: true } : {}), ...(opts.scanSecrets ? { scanSecrets: true } : {}),
+ const ctx = { ...withBranch(upstream, opts.branch), ...(opts.keepCheckout ? { keepCheckout: true } : {}),
...(opts.scanTimeoutMs ? { scanTimeoutMs: opts.scanTimeoutMs } : {}), ...(opts.scanProgress ? { scanProgress: opts.scanProgress } : {}) };
const elsewhere = ctx.mergeRef !== upstream.mergeRef;
assertNoOperationInProgress(root);
@@ -1613,7 +1612,7 @@ export function commitAndPush(opts: CommitPushOptions): CommitPushResult {
return deferred;
}
const busy = opError('push_unconfirmed', `${committed ? 'The commit is durable locally, but another' : 'Another'} publish held this checkout's push lock for ${Math.round(waitMs / 1000)}s.`,
- `The running publish was asked to push HEAD too once it finishes. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''}${ctx.scanSecrets ? ' --scan-secrets' : ''} to confirm; never git pull --rebase in this checkout.`,
+ `The running publish was asked to push HEAD too once it finishes. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''} --scan-secrets to confirm; never git pull --rebase in this checkout.`,
{ detail: committed ?? '', retryable: true });
if (skipped.length) throw skippedPathsError(root, skipped, unpublished, undefined, busy);
throw busy;
@@ -1626,7 +1625,7 @@ export function commitAndPush(opts: CommitPushOptions): CommitPushResult {
const head = gitOk(root, ['rev-parse', 'HEAD'], 'rev-parse HEAD');
const target = start && !isAncestor(root, start, head) ? start : head;
if (!opts.allowOutsideSourceRoot) assertLocalCommitsInsideRoots(ctx, target, roots);
- const published = publish(ctx, target, resolutions, expectRemote, cwd, opts.scanSecrets);
+ const published = publish(ctx, target, resolutions, expectRemote, cwd);
if (!elsewhere) rmSync(absGitPath(root, CONFLICT_STATE_NAME), { force: true });
const moved = published.sha === head || elsewhere || opts.keepCheckout ? { checkout: 'unchanged' as const } : advanceWithLock(root, published.sha, head, onlyWhenClean, opts.lockWaitMs);
return { published, moved };
diff --git a/src/core/git/range-secret-scan.ts b/src/core/git/range-secret-scan.ts
index 13e1b2c75e..671cdaf74a 100644
--- a/src/core/git/range-secret-scan.ts
+++ b/src/core/git/range-secret-scan.ts
@@ -3,7 +3,8 @@
* `git commit`) never passed the write path's scan, so the publish scans, under its publish lock and for the exact
* commit it pushes, what each commit in `base..head` adds: its message, the lines it adds file by file (a value a later
* commit removed is still in the pushed history), and for a merge commit the lines it adds that no parent has
- * (a conflict resolution). The workspace allowlist (`.gbrain-scan-allow`: path globs and fingerprints) applies.
+ * (a conflict resolution). The workspace allowlist (`.gbrain-scan-allow` at the scanned root, below the repository's top
+ * level for a managed root in a subfolder: path globs relative to that root, and fingerprints) applies.
* Findings carry the path, line, rule, fingerprint and commit, never the matched value.
*
* Only plumbing reads the commits (`rev-list`, `cat-file commit`, one `diff-tree --stdin --root --cc`), so no log, diff or
@@ -12,7 +13,7 @@
* read in chunks, so memory stays bounded. Binary detection is git's own (a NUL in a blob's first 8000 bytes), with
* attributes read from the empty tree: a file git still calls binary is checked by its own bytes when its old side was
* not empty or attributes could still apply, and a text file is then scanned whole. Added text over the scan cap is
- * reported as unscanned, and a commit that changes a file deciding what the scan sees (`.gbrain-scan-allow`, a
+ * reported as unscanned, and a commit that changes a file deciding what the scan sees (the loaded `.gbrain-scan-allow` or the top-level one, a
* `.gitattributes`) as a gate change, so callers refuse. Output the parser does not recognise, a git failure or a run
* past the time budget sets `unreadable` instead of returning a partial result. The time budget is checked between git
* calls and while the diff is parsed, so a caller with a short budget (the owner's event loop) is never held much past it.
@@ -56,13 +57,14 @@ const DIFF = ['diff-tree', '--stdin', '-r', '-p', '--root', '--cc', '--no-rename
'--full-index', '--src-prefix=a/', '--dst-prefix=b/', '--unified=0'];
/**
- * Ledger writers whose commits hold only page writes: a Git effect's commit and a hosted repository's door commits. A
- * Git effect's record counts only when `verified` (its sha, parent, paths and blobs were checked against what the effect
- * wrote; records from releases before that check carry none). A record that adopted a file edited outside gbrain
+ * Ledger writers whose commits hold only page writes: a Git effect's commit, a strict source's door commit and a hosted
+ * repository's commits. A Git effect's or a door's record counts only when `verified` (VERIFIED_WRITERS: each committed
+ * blob is the bytes the writer wrote, with no clean filter in between; records from releases before that check carry none). A record that adopted a file edited outside gbrain
* (`adopted`) carries bytes gbrain did not write, so it is scanned, and so is a page-write commit older than the retained
* ledger or made before it existed.
*/
const PAGE_WRITERS: ReadonlySet<string> = new Set(['effect', 'door', 'hosted_repo']);
+const VERIFIED_WRITERS: ReadonlySet<string> = new Set(['effect', 'door']);
class Unreadable extends Error { constructor(message: string, readonly timedOut = false) { super(message); } }
@@ -78,7 +80,7 @@ function pageWriteCommits(root: string, deadline: number): Set<string> {
const common = must(run(root, ['rev-parse', '--path-format=absolute', '--git-common-dir'], deadline), 'rev-parse').toString().trim();
try {
return new Set(readLedger(common).flatMap(record => record.e === 'commit' && PAGE_WRITERS.has(record.writer) && !record.adopted?.length
- && (record.writer !== 'effect' || record.verified) ? [record.sha] : []));
+ && (!VERIFIED_WRITERS.has(record.writer) || record.verified) ? [record.sha] : []));
}
catch { return new Set(); }
}
@@ -147,6 +149,8 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
};
const scan: RangeScan = { findings: [], unscanned: [], gate_changes: [] };
const allowlist = loadWorkspaceAllowlist(root);
+ let allowFile = SCAN_ALLOW_FILENAME, prefix = '';
+ const allowed = (path: string) => path.startsWith(prefix) && pathAllowlisted(path.slice(prefix.length), allowlist);
const scanned = (path: string, commit: string, text: string, lineAt: (line: number) => number = line => line) => {
if (Date.now() > deadline) throw new Unreadable('the scan ran past its time budget', true);
if (Buffer.byteLength(text) > SCAN_MAX_FILE_BYTES) { scan.unscanned.push({ path, commit }); return; }
@@ -154,6 +158,8 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
};
const dir = mkdtempSync(join(tmpdir(), 'gbrain-range-scan-'));
try {
+ prefix = must(run(root, ['rev-parse', '--show-prefix'], deadline), 'rev-parse').toString().replace(/\n$/, '');
+ allowFile = `${prefix}${SCAN_ALLOW_FILENAME}`;
const pageWrites = pageWriteCommits(root, deadline);
const commits = must(run(root, ['rev-list', base ? `${base}..${head}` : head, '--'], deadline), 'rev-list').toString().split('\n')
.filter(commit => commit && !pageWrites.has(commit));
@@ -223,7 +229,7 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
let commit = '', path: string | null = null, header = false, parents = 1, next = 0, bytes = 0, oversized = false;
let olds: string[] = [], blob = '', added: string[] = [], at: number[] = [];
const flush = () => {
- if (path && (oversized || added.length) && !pathAllowlisted(path, allowlist)) {
+ if (path && (oversized || added.length) && !allowed(path)) {
if (oversized) scan.unscanned.push({ path, commit });
else scanned(path, commit, added.join('\n'), line => at[line - 1] ?? line);
}
@@ -247,7 +253,7 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
path = left?.startsWith('a/') && right?.startsWith('b/') && left.slice(2) === right.slice(2) ? right.slice(2) : null;
} else path = unquote(line.replace(/^diff --(?:cc|combined) /, ''));
if (path === null) throw unexpected(line);
- if (path === SCAN_ALLOW_FILENAME || path === '.gitattributes' || path.endsWith('/.gitattributes')) scan.gate_changes.push({ path, commit });
+ if (path === allowFile || path === SCAN_ALLOW_FILENAME || path === '.gitattributes' || path.endsWith('/.gitattributes')) scan.gate_changes.push({ path, commit });
header = true; olds = []; blob = '';
continue;
}
@@ -294,7 +300,7 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
for (const row of check.toString().split('\n')) { const [name, size] = row.split(' '); if (name && size) sizes.set(name, Number(size)); }
}
for (const b of binaries) {
- if (pathAllowlisted(b.path, allowlist)) continue;
+ if (allowed(b.path)) continue;
if (neutral && b.added && (sizes.get(b.blob) ?? Infinity) <= BIG_FILE_THRESHOLD) continue;
const sniff = run(root, ['cat-file', 'blob', b.blob], deadline, { maxBuffer: SNIFF_BYTES });
if ((sniff.error as NodeJS.ErrnoException | undefined)?.code !== 'ENOBUFS') must(sniff, 'cat-file blob');
diff --git a/src/core/persistence/effect-git.ts b/src/core/persistence/effect-git.ts
index 0ac43b364c..3984d65b26 100644
--- a/src/core/persistence/effect-git.ts
+++ b/src/core/persistence/effect-git.ts
@@ -72,7 +72,8 @@ export interface GitSession { root: string; hooks: string; signal?: AbortSignal;
const gitArgs = (session: GitSession, args: string[]) =>
['-C', session.root, '-c', `core.hooksPath=${session.hooks}`, '-c', 'commit.gpgsign=false', ...args];
const gitOptions = (session: GitSession) => ({ timeout: session.timeoutMs, termGraceMs: GIT_TERM_GRACE_MS, maxBuffer: 1024 * 1024, signal: session.signal });
-const sessionEnv = (session: GitSession) => session.indexFile ? { ...GIT_ENV, GIT_INDEX_FILE: session.indexFile } : GIT_ENV;
+/** Replace refs are off, so the commit a Git effect or the door builds is made of the objects the push secret scan reads. */
+const sessionEnv = (session: GitSession) => ({ ...GIT_ENV, GIT_NO_REPLACE_OBJECTS: '1', ...(session.indexFile ? { GIT_INDEX_FILE: session.indexFile } : {}) });
function run(session: GitSession, args: string[]) {
return execFileBounded('git', gitArgs(session, args), { ...gitOptions(session), env: gitChildEnv(sessionEnv(session)) });
@@ -425,7 +426,7 @@ export async function commitGitTargets(root: string, relativePaths: string[], si
const own = moved.filter(path => !foreign.includes(path));
const reset = !own.length || await git(session, ['reset', '-q', '--', ...literalPathspecs(own)]).then(out => out.code === 0, () => false);
const outcome = sha ? { git: 'committed' } : moved.length ? gitFailure('Cannot commit the canonical Git target.',
- `Another commit changed ${moved.join(', ')} while this group of ${commitPaths.length} file(s) was being committed, so nothing was committed and the other commit stays; the effect retries against the new HEAD.`)
+ `Another commit or process changed ${moved.join(', ')} while this group of ${commitPaths.length} file(s) was being committed, so nothing was committed and the other commit stays; the effect retries against the new HEAD.`)
: gitFailure('Cannot commit the canonical Git target.',
`git commit failed for ${commitPaths.length} file(s); a missing Git identity (user.name, user.email) in that checkout is one cause to check.`);
for (const c of changed) {
@@ -505,8 +506,9 @@ export function stripspace(paragraph: string): string {
* is scanned. The record is `verified` only when every path's staged blob is the bytes the effect wrote (`wanted`: the
* sha256 the effect recorded, null for a removal, '' for none known): the file, read once, hashes to it and has the
* staged blob's id; an outside edit that reached the index first fails that, and the commit is scanned. An index that
- * no longer holds what was staged (an entry for a file that is gone, none for one on disk), or a tree that equals
- * HEAD's, fails the commit, so the effect retries rather than reporting a commit of the wrong state.
+ * no longer holds what was staged (an entry for a file that is gone, none for one on disk, or an entry another process
+ * reset to `baseline`'s, which would report that path committed with its old blob), or a tree that equals HEAD's, fails
+ * the commit, so the effect retries rather than reporting a commit of the wrong state.
*
* GST-2552: the staged blobs are laid only over a HEAD that holds every path as `baseline` (the HEAD the group staged
* against) did, with the checkout's index still holding what was staged. A commit that landed since and touched a path,
@@ -538,6 +540,10 @@ async function effectCommit(session: GitSession, root: string, prefix: string, s
const staged = await indexed();
if (format.code !== 0 || !staged) return failed;
if ([...staged.values()].some(entry => entry.stage !== '0') || paths.some(path => staged.has(path) !== existsSync(join(root, path)))) return failed;
+ const base = await committed(baseline);
+ if (!base) return failed;
+ const reset = paths.filter(path => { const record = staged.get(path); return (record && `${record.mode} blob ${record.oid}`) === base.get(path); });
+ if (reset.length) return { sha: null, moved: reset, superseded: [], foreign: reset };
const entry = (record?: { mode: string; oid: string; stage: string }) => record && `${record.mode} ${record.oid} ${record.stage}`;
// Whether the file still holds what the effect staged: its recorded sha256 when known (clean filters change the blob id).
const holds = (path: string) => {
@@ -560,8 +566,8 @@ async function effectCommit(session: GitSession, root: string, prefix: string, s
const before = attempt ? await git(session, ['rev-parse', '--verify', '--quiet', 'HEAD']) : null;
const head = before ? before.code === 0 ? before.stdout.trim() : null : baseline;
if (head !== baseline) {
- const base = await committed(baseline), now = await committed(head), index = await indexed();
- if (!base || !now || !index) return failed;
+ const now = await committed(head), index = await indexed();
+ if (!now || !index) return failed;
// A commit that already holds the staged entry (a commit -a that swept it in) changed nothing this effect would lose.
const committedAway = (path: string) => {
const record = staged.get(path);
@@ -599,7 +605,7 @@ async function effectCommit(session: GitSession, root: string, prefix: string, s
}
/** `<sha256> <git blob id>` of the file's bytes, read once; null when it cannot be read or changes size while read. */
-function fileDigests(path: string, objectFormat: string): string | null {
+export function fileDigests(path: string, objectFormat: string): string | null {
let fd: number;
try { fd = openSync(path, 'r'); } catch { return null; }
try {
diff --git a/src/core/persistence/git-door.ts b/src/core/persistence/git-door.ts
index a830ffa2cd..33e31cfd50 100644
--- a/src/core/persistence/git-door.ts
+++ b/src/core/persistence/git-door.ts
@@ -7,7 +7,8 @@
*
* The commit is made by plumbing, without the checkout's index (GBRA-174: a porcelain `commit --only` rewrites and
* refreshes the whole index, about 1.7 s at 312k files): `hash-object -w` hashes each published file (its clean
- * filters apply, as `git add` would), the trees along the changed paths are rebuilt from HEAD's (`cat-file tree`, then
+ * filters apply, as `git add` would; the ledger records the commit `verified` only when every blob is the file's own bytes,
+ * so the push secret scan reads a filtered one), the trees along the changed paths are rebuilt from HEAD's (`cat-file tree`, then
* `hash-object -w -t tree`, which refuses a malformed tree), `commit-tree` writes the commit with the porcelain path's
* message (`commitMessage`), and `update-ref <branch> <new> <old>` moves the branch only if it still names the HEAD the
* commit was built on. A branch that moved meanwhile (a commit made in the checkout) is read again and the commit
@@ -46,7 +47,7 @@ import { basename, dirname, join, relative, resolve, sep } from 'node:path';
import type { Notice } from '../agent-output.ts';
import { literalPathspecs } from '../git-env.ts';
import { OperationError, opError } from '../ops/contract.ts';
-import { commitMessage, commitMessageArgs, git, gitBytes, gitFailure, gitStepTimeoutMs, ignoredTargetError, reclaimAbandonedIndexLock, stripspace, withSession, type GitSession } from './effect-git.ts';
+import { commitMessage, commitMessageArgs, fileDigests, git, gitBytes, gitFailure, gitStepTimeoutMs, ignoredTargetError, reclaimAbandonedIndexLock, stripspace, withSession, type GitSession } from './effect-git.ts';
import { adoptedBy, type GitCommitNote } from './effect-model.ts';
import { persistenceHome } from './identity.ts';
import { ignoreCandidates } from './ignore-rules.ts';
@@ -230,6 +231,7 @@ async function doorCommit(session: GitSession, root: string, files: DoorFile[],
if (hashed.code !== 0 || oids.length !== present.length) throw gitFailure('Cannot hash the canonical Git target.', `git hash-object failed for ${present.length} file(s).`);
present.forEach((target, i) => { target.oid = oids[i]; target.mode = lstatSync(join(root, target.rel)).mode & 0o111 ? '100755' : '100644'; });
}
+ const verified = present.every(target => fileDigests(join(root, target.rel), format!) === `${target.planned} ${target.oid}`);
let fileModeTrusted: boolean | undefined;
refuseChanged(root, targets.values());
@@ -348,7 +350,7 @@ async function doorCommit(session: GitSession, root: string, files: DoorFile[],
const created = commit.stdout.trim();
const trailers = [...noted.values()].flatMap(note => note.trailers ?? []);
const supersedes = supersededBy(trailers), adopted = adoptedBy(trailers);
- await recordCommitAsync(root, { sha: created, writer: 'door', ref: branch, ...(supersedes.length ? { supersedes } : {}), ...(adopted.length ? { adopted } : {}) });
+ await recordCommitAsync(root, { sha: created, writer: 'door', ref: branch, ...(verified ? { verified: true } : {}), ...(supersedes.length ? { supersedes } : {}), ...(adopted.length ? { adopted } : {}) });
await doorSeams.beforeUpdateRef?.(root, attempt);
const moved = await git(session, ['update-ref', '-m', `commit: ${message[0]!.split('\n')[0]}`, branch, created, head]);
if (moved.code === 0) {
diff --git a/test/guarded-publish.test.ts b/test/guarded-publish.test.ts
index 9aca524625..09eb4619b6 100644
--- a/test/guarded-publish.test.ts
+++ b/test/guarded-publish.test.ts
@@ -1,12 +1,12 @@
/**
- * The secret gate on automatic publishes (`commitAndPush({ pushOnly, keepCheckout, scanSecrets })`, the publish
+ * The secret gate on automatic publishes (`commitAndPush({ pushOnly, keepCheckout })`, the publish
* `gbrain sources refresh` runs in its child): it scans, under the publish lock, what the commits it is about to push add.
* Protects: a value that looks like a secret is found in an added line even when a later commit removed it, in a
* commit message, and in a merge commit's conflict resolution, named by path, line, rule and commit but never by
* value, and nothing is pushed; an added line whose text starts with `++` (printed `+++`, like a file header) is
* still scanned; added text over the scan cap, and a commit in the range that changes .gbrain-scan-allow or a
* .gitattributes, refuse; a fingerprint in the working tree's .gbrain-scan-allow and a binary file are skipped; a clean
- * range publishes; without scanSecrets nothing changes; a remote rewound past a pushed secret (the tracking ref still names
+ * range publishes; a remote rewound past a pushed secret (the tracking ref still names
* it) is fetched and scanned again instead of fast-forwarded over (#8896).
* Seams: none; real repositories and a bare remote.
*/
@@ -39,7 +39,7 @@ const commit = (root: string, file: string, content: string | Buffer, message =
writeFileSync(join(root, file), content); git(root, 'add', file); git(root, 'commit', '-qm', message);
return git(root, 'rev-parse', 'HEAD');
};
-const publish = (root: string, scanSecrets = true) => commitAndPush({ repo: root, cwd: root, paths: [], pushOnly: true, keepCheckout: true, ...(scanSecrets ? { scanSecrets: true } : {}) });
+const publish = (root: string) => commitAndPush({ repo: root, cwd: root, paths: [], pushOnly: true, keepCheckout: true });
const refused = (fn: () => unknown) => {
try { fn(); } catch (e) { return e as { code: string; message: string; detail?: string; fix?: { user_message?: string } }; }
throw new Error('expected publish_secret_found');
@@ -80,7 +80,7 @@ test('a secret typed into a merge commit\'s conflict resolution is refused', ()
expect(remoteTip(r)).toBe(r.base);
});
-test('an allowlisted fingerprint and a binary file are skipped and the clean range publishes; without scanSecrets nothing is scanned', () => {
+test('an allowlisted fingerprint and a binary file are skipped and the clean range publishes', () => {
const r = repo('allow');
commit(r.root, 'data.bin', Buffer.concat([Buffer.from([0, 1, 2]), Buffer.from(token)]));
commit(r.root, 'notes.md', `Deploy token ${token}\n`);
@@ -90,10 +90,6 @@ test('an allowlisted fingerprint and a binary file are skipped and the clean ran
writeFileSync(join(r.root, '.gbrain-scan-allow'), `${fingerprint}\n`);
expect(publish(r.root)).toMatchObject({ outcome: 'pushed', checkout: 'unchanged' });
expect(remoteTip(r)).toBe(git(r.root, 'rev-parse', 'HEAD'));
-
- const plain = repo('plain');
- commit(plain.root, 'notes.md', `Deploy token ${token}\n`);
- expect(publish(plain.root, false)).toMatchObject({ outcome: 'pushed' });
});
diff --git a/test/push-secret-scan-gaps.serial.test.ts b/test/push-secret-scan-gaps.serial.test.ts
new file mode 100644
index 0000000000..5eddeaea06
--- /dev/null
+++ b/test/push-secret-scan-gaps.serial.test.ts
@@ -0,0 +1,237 @@
+/**
+ * Ways a commit could reach the remote past the push secret scan, found by an outside review of the managed-push gate
+ * (#8903). Protects: a commit changing the allowlist a managed root in a subfolder loads is a gate change, and that
+ * allowlist's path globs match paths relative to the root; an uncommitted allowlist is honoured (documented: it is the
+ * user's local answer to the scan); a strict door commit whose blob a clean filter changed is scanned, and one with a
+ * harmless filter still publishes; every commitAndPush scans what it publishes (the derived write-through passes no flag),
+ * only what the remote lacks, a missing tracking ref included; a replace ref on HEAD changes nothing a Git effect or a
+ * door commit holds; a Git effect target whose staged entry another process reset is never reported committed with its
+ * old blob, and the retry commits it.
+ * Seams: a PATH git wrapper for the index race; real repositories, bare remotes and synthetic credentials otherwise.
+ * GBRAIN_HOME, PATH and the file transport switch are process-wide.
+ */
+import { afterEach, describe, expect, test } from 'bun:test';
+import { createHash } from 'node:crypto';
+import { chmodSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { commitGitTargets, pushGitRoot } from '../src/core/persistence/effect-git.ts';
+import { commitThroughDoor } from '../src/core/persistence/git-door.ts';
+import { hashFile } from '../src/core/persistence/large-file.ts';
+import { commitAndPush } from '../src/core/git/commit-push.ts';
+import { rangeSecretRefusal, scanRange } from '../src/core/git/range-secret-scan.ts';
+import { resetLedgerCacheForTests } from '../src/core/git/commit-ledger.ts';
+import { git } from './helpers/git-publication.ts';
+import { withEnv } from './helpers/with-env.ts';
+
+const token = 'ghp_' + 'Wq3e'.repeat(9);
+const dirs: string[] = [];
+afterEach(() => { resetLedgerCacheForTests(); for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); });
+
+interface Checkout { home: string; top: string; remote: string }
+/** A repository on main, with brain/notes/a.md and b.md, pushed to a bare remote. */
+function checkout(): Checkout {
+ const home = realpathSync(mkdtempSync(join(tmpdir(), 'gbrain-scan-gaps-'))); dirs.push(home);
+ const remote = join(home, 'remote.git'), top = join(home, 'repo');
+ git(home, 'init', '-q', '--bare', '-b', 'main', remote);
+ mkdirSync(join(top, 'brain', 'notes'), { recursive: true });
+ git(top, 'init', '-q', '-b', 'main');
+ git(top, 'config', 'user.name', 'Example Writer'); git(top, 'config', 'user.email', '[email protected]');
+ writeFileSync(join(top, 'brain', 'notes', 'a.md'), 'Before a\n');
+ writeFileSync(join(top, 'brain', 'notes', 'b.md'), 'Before b\n');
+ git(top, 'add', '-A'); git(top, 'commit', '-q', '-m', 'Initial');
+ git(top, 'remote', 'add', 'origin', remote); git(top, 'push', '-q', '-u', 'origin', 'main');
+ return { home, top, remote };
+}
+/** A script's plain `git commit`, never through gbrain's write path. */
+function scriptCommit(root: string, files: Record<string, string>, message = 'script edit'): string {
+ for (const [path, body] of Object.entries(files)) { mkdirSync(dirname(join(root, path)), { recursive: true }); writeFileSync(join(root, path), body); git(root, 'add', '--', path); }
+ git(root, 'commit', '-q', '-m', message);
+ return git(root, 'rev-parse', 'HEAD').trim();
+}
+const head = (root: string) => git(root, 'rev-parse', 'HEAD').trim();
+const remoteTip = (c: Checkout) => git(c.remote, 'rev-parse', 'refs/heads/main').trim();
+const env = (c: Checkout, extra: Record<string, string | undefined> = {}) => ({ GBRAIN_HOME: join(c.home, 'gbrain-home'), GBRAIN_GIT_ALLOW_FILE_TRANSPORT: '1',
+ DATABASE_URL: undefined, GBRAIN_DATABASE_URL: undefined, ...extra });
+const outcome = (run: () => unknown) => Promise.resolve().then(run).then(value => ({ value, code: null as string | null }), (e: { code?: string }) => ({ value: null, code: e.code ?? String(e) }));
+const sha256 = (text: string) => createHash('sha256').update(text).digest('hex');
+const refusalOf = (c: Checkout, root: string, base: string, target: string) => withEnv(env(c), async () => rangeSecretRefusal(root, base, target, scanRange(root, base, target))?.code ?? null);
+const tracked = (root: string, commit: string) => git(root, '--no-replace-objects', 'ls-tree', '-r', '--name-only', commit).split('\n').filter(Boolean);
+/** A derived write-through's publish: commitAndPush with no scan option, as src/core/cycle/derived-write-through.ts calls it. */
+const derivedPublish = (c: Checkout, body: string) => {
+ writeFileSync(join(c.top, 'brain', 'notes', 'b.md'), body);
+ return withEnv(env(c), () => outcome(() => commitAndPush({ repo: c.top, cwd: c.top, paths: ['brain/notes/b.md'], message: 'gbrain: derived write-through' })));
+};
+
+describe('a managed root below the repository top level', () => {
+ test('a commit changing the root\'s own .gbrain-scan-allow is a gate change, so the secret it hides is not pushed', async () => {
+ const c = checkout();
+ const before = remoteTip(c);
+ scriptCommit(c.top, { 'brain/.gbrain-scan-allow': '**\n', 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ const result = await withEnv(env(c), () => outcome(() => pushGitRoot(join(c.top, 'brain'))));
+ expect(result.code).toBe('publish_secret_found');
+ expect(remoteTip(c)).toBe(before);
+ }, 60_000);
+
+ test('the same commit at the top level is refused as a gate change', async () => {
+ const c = checkout();
+ const before = remoteTip(c);
+ scriptCommit(c.top, { '.gbrain-scan-allow': '**\n', 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ const result = await withEnv(env(c), () => outcome(() => pushGitRoot(c.top)));
+ expect(result.code).toBe('publish_secret_found');
+ expect(remoteTip(c)).toBe(before);
+ }, 60_000);
+
+ test('a path glob in the root\'s allowlist matches paths relative to that root, and nothing outside it', async () => {
+ const c = checkout();
+ const base = head(c.top);
+ writeFileSync(join(c.top, 'brain', '.gbrain-scan-allow'), 'notes/**\n');
+ const inside = scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ expect(await refusalOf(c, join(c.top, 'brain'), base, inside)).toBeNull();
+ const outside = scriptCommit(c.top, { 'notes/deploy.md': `Deploy token ${token}\n` });
+ expect(await refusalOf(c, join(c.top, 'brain'), inside, outside)).toBe('publish_secret_found');
+ }, 60_000);
+});
+
+describe('an uncommitted .gbrain-scan-allow (documented: the user\'s local answer to the scan)', () => {
+ test('a path glob in an untracked allowlist is honoured and the commit publishes', async () => {
+ const c = checkout();
+ const added = scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ writeFileSync(join(c.top, '.gbrain-scan-allow'), '**\n');
+ const result = await withEnv(env(c), () => outcome(() => pushGitRoot(c.top)));
+ expect(result.code).toBeNull();
+ expect(remoteTip(c)).toBe(added);
+ }, 60_000);
+});
+
+describe('a strict (canonical: git) door commit through a clean filter', () => {
+ function filtered(c: Checkout, clean: string): void {
+ writeFileSync(join(c.top, '.git', 'info', 'attributes'), '*.md filter=example\n');
+ git(c.top, 'config', 'filter.example.clean', clean);
+ }
+ const door = (c: Checkout, body: string) => {
+ const file = join(c.top, 'brain', 'notes', 'a.md');
+ writeFileSync(file, body);
+ return withEnv(env(c), () => commitThroughDoor(c.top, [{ path: file, planned: hashFile(file) }]));
+ };
+
+ test('a blob the filter changed is scanned, so a value it added is refused', async () => {
+ const c = checkout();
+ const base = head(c.top);
+ filtered(c, `sed 's/$/ ${token}/'`);
+ const created = (await door(c, 'After a\n'))!;
+ expect(git(c.top, 'show', `${created}:brain/notes/a.md`)).toContain(token);
+ expect(await refusalOf(c, c.top, base, created)).toBe('publish_secret_found');
+ }, 60_000);
+
+ test('a filter that adds nothing secret still publishes: scanned, not refused', async () => {
+ const c = checkout();
+ const base = head(c.top);
+ filtered(c, `sed 's/$/ reviewed/'`);
+ const created = (await door(c, 'After a\n'))!;
+ expect(git(c.top, 'show', `${created}:brain/notes/a.md`)).toBe('After a reviewed\n');
+ expect(await refusalOf(c, c.top, base, created)).toBeNull();
+ }, 60_000);
+
+ test('an unfiltered door commit stays a page write: its raw body is not scanned', async () => {
+ const c = checkout();
+ const base = head(c.top);
+ const created = (await door(c, `Saved note with ${token}\n`))!;
+ expect(await refusalOf(c, c.top, base, created)).toBeNull();
+ }, 60_000);
+});
+
+describe('every commitAndPush scans what it publishes', () => {
+ test('a publish that passes no scan option (the managed derived write-through) does not publish a script commit holding a token', async () => {
+ const c = checkout();
+ const before = remoteTip(c);
+ scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ const result = await derivedPublish(c, 'Generated page\n');
+ expect(result.code).toBe('publish_secret_found');
+ expect(remoteTip(c)).toBe(before);
+ expect(git(c.top, 'show', 'HEAD:brain/notes/b.md')).toBe('Generated page\n');
+ }, 60_000);
+
+ test('it scans only what the remote lacks: a value already on the remote does not hold the next publish', async () => {
+ const c = checkout();
+ scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ git(c.top, 'push', '-q');
+ const result = await derivedPublish(c, 'Generated page\n');
+ expect(result.code).toBeNull();
+ expect(remoteTip(c)).toBe(head(c.top));
+ }, 60_000);
+
+ test('a missing tracking ref is fetched first: a clean commit pushes, and history the remote holds is not rescanned', async () => {
+ const c = checkout();
+ scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` });
+ git(c.top, 'push', '-q');
+ const created = scriptCommit(c.top, { 'brain/notes/b.md': 'A clean page\n' });
+ git(c.top, 'update-ref', '-d', 'refs/remotes/origin/main');
+ const result = await withEnv(env(c), () => outcome(() => commitAndPush({ repo: c.top, cwd: c.top, paths: [], message: '', pushOnly: true })));
+ expect(result.code).toBeNull();
+ expect(remoteTip(c)).toBe(created);
+ }, 60_000);
+});
+
+describe('replace refs', () => {
+ /** HEAD with a replacement commit that adds a file holding a token. */
+ function replaced(c: Checkout): string {
+ const real = head(c.top);
+ const fake = scriptCommit(c.top, { 'brain/notes/deploy.md': `Deploy token ${token}\n` }, 'replacement');
+ git(c.top, 'reset', '-q', '--hard', real);
+ git(c.top, 'replace', real, fake);
+ return real;
+ }
+
+ test('a Git effect commit is built from HEAD\'s real tree, so it carries nothing the replacement adds', async () => {
+ const c = checkout();
+ const base = replaced(c);
+ writeFileSync(join(c.top, 'brain', 'notes', 'a.md'), 'After a\n');
+ const result = await withEnv(env(c), () => commitGitTargets(c.top, ['brain/notes/a.md'], undefined, undefined, undefined, new Map([['brain/notes/a.md', sha256('After a\n')]])));
+ expect(result.get('brain/notes/a.md')).toEqual({ git: 'committed' });
+ const created = head(c.top);
+ expect(git(c.top, '--no-replace-objects', 'rev-parse', `${created}^`).trim()).toBe(base);
+ expect(tracked(c.top, created).sort()).toEqual(['brain/notes/a.md', 'brain/notes/b.md']);
+ expect(git(c.top, '--no-replace-objects', 'show', `${created}:brain/notes/a.md`)).toBe('After a\n');
+ expect(await refusalOf(c, c.top, base, created)).toBeNull();
+ const pushed = await withEnv(env(c), () => outcome(() => pushGitRoot(c.top)));
+ expect(pushed.code).toBeNull();
+ expect(remoteTip(c)).toBe(created);
+ }, 60_000);
+
+ test('a door commit is built from HEAD\'s real tree too', async () => {
+ const c = checkout();
+ const base = replaced(c);
+ const file = join(c.top, 'brain', 'notes', 'a.md');
+ writeFileSync(file, 'After a\n');
+ const created = (await withEnv(env(c), () => commitThroughDoor(c.top, [{ path: file, planned: hashFile(file) }])))!;
+ expect(git(c.top, '--no-replace-objects', 'rev-parse', `${created}^`).trim()).toBe(base);
+ expect(tracked(c.top, created).sort()).toEqual(['brain/notes/a.md', 'brain/notes/b.md']);
+ expect(await refusalOf(c, c.top, base, created)).toBeNull();
+ }, 60_000);
+});
+
+describe('an index entry reset between staging and commit', () => {
+ test('a target whose staged entry another process reset is not reported committed with its old blob, and the retry commits it', async () => {
+ const c = checkout();
+ const root = c.top, before = head(root);
+ writeFileSync(join(root, 'brain', 'notes', 'a.md'), 'After a\n');
+ writeFileSync(join(root, 'brain', 'notes', 'b.md'), 'After b\n');
+ const path = process.env.PATH ?? '', real = `PATH='${path}' '${Bun.which('git')!}'`;
+ const bin = join(c.home, 'race-bin'), mark = join(c.home, 'raced');
+ mkdirSync(bin);
+ writeFileSync(join(bin, 'git'), ['#!/bin/sh', `${real} "$@"`, 'code=$?',
+ `case " $* " in *" diff --cached "*) if [ $code -eq 0 ] && [ ! -f '${mark}' ]; then touch '${mark}'; ${real} -C '${root}' reset -q -- brain/notes/a.md; fi;; esac`,
+ 'exit $code', ''].join('\n'));
+ chmodSync(join(bin, 'git'), 0o755);
+ const targets = ['brain/notes/a.md', 'brain/notes/b.md'];
+ const raced = await withEnv(env(c, { PATH: `${bin}:${path}` }), () => commitGitTargets(root, targets));
+ expect(raced.get('brain/notes/a.md')).toBeInstanceOf(Error);
+ expect(head(root)).toBe(before);
+ const retried = await withEnv(env(c), () => commitGitTargets(root, targets));
+ expect([retried.get(targets[0]!), retried.get(targets[1]!)]).toEqual([{ git: 'committed' }, { git: 'committed' }]);
+ expect(git(root, 'show', 'HEAD:brain/notes/a.md')).toBe('After a\n');
+ expect(git(root, 'show', 'HEAD:brain/notes/b.md')).toBe('After b\n');
+ expect(readFileSync(join(root, 'brain', 'notes', 'a.md'), 'utf8')).toBe('After a\n');
+ }, 60_000);
+});