Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

persistence: keep --scan-secrets in every publish command a managed-checkout message hands back

garrytan5ec038af67de
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d9339e5936..7ebdda67b5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,7 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/git/commit-push.ts`: a `scanSecrets` publish scans `<base>..<candidate>` before every push and leases the push on that base, so each merge with a newly fetched remote tip is scanned again. It also scans `<remote tip>..<candidate>` before merging with a newly fetched tip, so a remote rewound past a pushed secret that also changed a file this machine changed refuses `publish_secret_found` rather than `integrate_conflict`, and a conflict's resolve command keeps `--scan-secrets`.
- `src/core/git/range-secret-scan.ts`: `rangeSecretRefusal`, the one refusal both callers throw (`publish_scan_unreadable` for a scan that could not read the range, `publish_secret_found` otherwise; a gate-file change says the user pushes it). `scanRange` marks a scan that ran past its budget (`timed_out`) and scans a whole history when the base is empty. It skips commits the commit ledger records as page writes (`effect`, `door`, `hosted_repo`); `gbrain git commit-push` commits, adopted outside edits and commits the ledger no longer holds are still scanned, and an unreadable ledger means everything is scanned. A Git effect now makes its commit by plumbing (`commit-tree` over its staged paths, then `update-ref HEAD <new> <old>`, which refuses when HEAD moved, then `git maintenance run --auto --detach` as `git commit` does, without `--detach` on git before 2.47), so the ledger records a sha gbrain created before any ref names it. The scan skips an effect commit only by that sha, recorded `verified` when every staged file holds the bytes whose sha256 the effect recorded (the page walk passes the hash it checked too). An amend, reset or reword of it is a different commit and is scanned, a page file changed before it was staged leaves it unverified, and records written before this check existed are scanned. Its time budget is also checked while the diff is parsed.
- `src/commands/doctor/checks/git-convergence.ts`, `src/core/persistence/worktree-refresh.ts`, `src/core/persistence/git-push-holds.ts`: every publish fix they name for a managed checkout (stranded, after committing its modified pages, diverged, `refresh_diverged`, and a held Git effect's conflict resolve) runs `gbrain git commit-push --push-only --keep-checkout --scan-secrets`, so a script's commit is scanned there too. This changes what the user sees: a commit that changes `.gbrain-scan-allow` or a `.gitattributes` now refuses `publish_secret_found` when the user runs the diverged fix or the `refresh_diverged` publish, where before their agreement published it.
+- `src/core/git/commit-push.ts`, `src/core/persistence/git-conversion.ts`, `src/commands/git.ts`: the publish commands the other messages hand back keep the scan too: a scanned publish's rerun hints (a push the remote keeps rejecting, a busy publish lock, commits outside the source root), the refresh's publish-by-hand line when it cannot find its CLI, the conversion's pending-push notice and a managed `commit-push -m` that leaves commits unpublished all name `--keep-checkout --scan-secrets`.
- `src/core/persistence/git-push-holds.ts`: `publish_secret_found` and `publish_scan_unreadable` are held push codes, so doctor, `gbrain sync status` and `retry-effects --parked-pushes` cover them, with an ask-the-user fix (the refused commits, or `git fsck`).
- Docs: `docs/guides/shared-checkout-git.md` (`publish_secret_found` and `publish_scan_unreadable` rows), `docs/guides/live-sync.md`.
- Tests: `test/effect-git-secret-gate.test.ts` (an ahead checkout, a diverged one, a remote rewound past a pushed secret, a `.gitattributes` change and a remote without the branch whose history holds a secret each refuse with nothing pushed; a clean ahead checkout pushes, a clean history creates a missing branch, and a scan past the in-process budget still scans in the child; through the effect worker the page write's effect holds, sync status names the fix, and an allowlisted false positive releases it, parked or not; a page write whose body holds a credential-shaped value still backs up; a script commit, amend or reword racing the effect commit, a page file changed after the effect checked it, and an unverified ledger record each hold with nothing pushed; a source-scan walk's page with a credential-shaped body still backs up; a repaired unreadable commit releases its hold); `test/persistence-git-removal-push-7089.test.ts` and `test/persistence-git-subdir-root-7091.test.ts` (doctor's push fixes scan); `test/guarded-publish.test.ts` (a remote rewound past a pushed secret is not published again); `test/git-literal-pathspec-hooks-7759.test.ts` publishes its probe `.gitattributes` commit first, since the effect's push now refuses one; `test/cli-flag-equals.test.ts` lists `range-secret-scan.ts` as a git child argv reader.
diff --git a/src/commands/git.ts b/src/commands/git.ts
index fc6cfc8daf..483cbb1600 100644
--- a/src/commands/git.ts
+++ b/src/commands/git.ts
@@ -250,7 +250,7 @@ export async function runGit(args: string[]): Promise<number> {
const upstream = result.upstream;
const state = !upstream ? `${result.branch} has no upstream`
: result.unpublished === 0 ? `${result.branch} is on ${upstream} as last fetched`
- : `${result.unpublished ?? 'an unknown number of'} commit(s) on ${result.branch} ${result.unpublished === null ? 'may not be' : 'are not'} on ${upstream} as last fetched; a managed checkout publishes through its owner (gbrain git commit-push --push-only --keep-checkout --path ${repo} publishes them)`;
+ : `${result.unpublished ?? 'an unknown number of'} commit(s) on ${result.branch} ${result.unpublished === null ? 'may not be' : 'are not'} on ${upstream} as last fetched; a managed checkout publishes through its owner (gbrain git commit-push --push-only --keep-checkout --scan-secrets --path ${repo} publishes them)`;
emit(json, { ...result, repo }, `${result.committed ? `committed ${result.committed.slice(0, 7)} on ${result.branch}` : 'nothing to commit'}; not pushed: ${state}`);
return 0;
}
diff --git a/src/core/git/commit-push.ts b/src/core/git/commit-push.ts
index 3b65379028..42f84a3bde 100644
--- a/src/core/git/commit-push.ts
+++ b/src/core/git/commit-push.ts
@@ -999,7 +999,7 @@ function publish(ctx: Ctx, start: string, resolutions: Map<string, Resolution> |
}
if (remoteContains(ctx, candidate)) return onRemote();
throw opError('push_unconfirmed', `The commit is durable locally but no push to ${ctx.remote}/${ctx.mergeRef.replace(/^refs\/heads\//, '')} was confirmed.`,
- `git push failed (network, credentials, branch protection, or a remote that kept moving): ${lastError.slice(0, 500) || 'no detail'}. Rerun gbrain git commit-push --push-only once the remote is reachable; never git pull --rebase in this checkout.`,
+ `git push failed (network, credentials, branch protection, or a remote that kept moving): ${lastError.slice(0, 500) || 'no detail'}. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''}${ctx.scanSecrets ? ' --scan-secrets' : ''} once the remote is reachable; never git pull --rebase in this checkout.`,
{ detail: candidate, retryable: true });
}
@@ -1032,7 +1032,7 @@ function assertLocalCommitsInsideRoots(ctx: Ctx, target: string, roots: string[]
`A commit made without gbrain git commit-push (for example a cron's git add plus git commit) can carry files other writers had staged. Show the user git show --stat ${offending[0].sha.slice(0, 12)}; if they want it published as is, rerun with --allow-outside-source-root. See docs/guides/shared-checkout-git.md#source-roots.`,
{ detail: JSON.stringify({ roots, commits: offending }),
fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,
- argv: ['gbrain', 'git', 'commit-push', '--push-only', '--allow-outside-source-root', '--path', ctx.root],
+ argv: ['gbrain', 'git', 'commit-push', '--push-only', ...(ctx.keepCheckout ? ['--keep-checkout'] : []), ...(ctx.scanSecrets ? ['--scan-secrets'] : []), '--allow-outside-source-root', '--path', ctx.root],
why: 'These commits change files that are not the brain\'s; publishing them can overwrite other projects\' or writers\' work on the remote.',
user_message: `Local commit(s) ${shas} change files outside the brain's folder (${files.slice(0, 5).join(', ')}${files.length > 5 ? ', ...' : ''}). Publish them anyway?`,
docs: 'docs/guides/shared-checkout-git.md#source-roots' } });
@@ -1211,7 +1211,7 @@ export function commitAndPush(opts: CommitPushOptions): CommitPushResult {
return { outcome: 'push_deferred', sha: null, committed, branch: ctx.branch, remote: ctx.remote, paths: rel, integrated: 0, checkout: 'unchanged' };
}
throw opError('push_unconfirmed', `${committed ? 'The commit is durable locally, but another' : 'Another'} publish held this checkout's push lock for ${Math.round(waitMs / 1000)}s.`,
- `The running publish was asked to push HEAD too once it finishes. Rerun gbrain git commit-push --push-only to confirm; never git pull --rebase in this checkout.`,
+ `The running publish was asked to push HEAD too once it finishes. Rerun gbrain git commit-push --push-only${ctx.keepCheckout ? ' --keep-checkout' : ''}${ctx.scanSecrets ? ' --scan-secrets' : ''} to confirm; never git pull --rebase in this checkout.`,
{ detail: committed ?? '', retryable: true });
}
diff --git a/src/core/persistence/git-conversion.ts b/src/core/persistence/git-conversion.ts
index e113b1abc1..76e8cd1626 100644
--- a/src/core/persistence/git-conversion.ts
+++ b/src/core/persistence/git-conversion.ts
@@ -901,7 +901,7 @@ function changedSincePlan(sourceId: string, paths: ReadonlySet<string>, mode: 'm
async function pushConversion(engine: BrainEngine, context: ManagedSyncContext): Promise<ConversionPush> {
const { root } = context;
const pending = (code: string, why: string): ConversionPush => ({ state: 'pending', code,
- notice: `The conversion's door commits are committed locally, but their push did not land (${why}). Nothing retries it until the next page write, whose push sends them; to push now, run gbrain git commit-push --push-only --path ${root}.` });
+ notice: `The conversion's door commits are committed locally, but their push did not land (${why}). Nothing retries it until the next page write, whose push sends them; to push now, run gbrain git commit-push --push-only --keep-checkout --scan-secrets --path ${root}.` });
try {
if (!(await gitDurabilityPolicy(context.binding, root)).durable) return { state: 'skipped', reason: DURABILITY_NOT_ENABLED_REASON };
if (!(await gitRootAhead(root))) return { state: 'skipped', reason: 'nothing_to_push' };
diff --git a/src/core/persistence/worktree-refresh.ts b/src/core/persistence/worktree-refresh.ts
index d3f3086365..f690d7b8ad 100644
--- a/src/core/persistence/worktree-refresh.ts
+++ b/src/core/persistence/worktree-refresh.ts
@@ -540,7 +540,7 @@ async function publishInChild(root: string, sourceId: string, checkedUpstream: s
const { CHILD_ENV, resolveChildCliInvocation } = await import('../minions/job-isolation.ts');
const cli = resolveChildCliInvocation(process.env, process.execPath, process.argv[1], () => null);
if (!cli) throw opError('git_push_unavailable', `This process could not locate its own CLI executable to publish the diverged checkout ${root}; nothing was pushed.`,
- `Set ${CHILD_ENV.childCliOverride} to the path of the CLI executable and retry gbrain sources refresh ${sourceId}, or publish by hand: gbrain git commit-push --push-only --keep-checkout --path ${q(root)}.`);
+ `Set ${CHILD_ENV.childCliOverride} to the path of the CLI executable and retry gbrain sources refresh ${sourceId}, or publish by hand: gbrain git commit-push --push-only --keep-checkout --scan-secrets --path ${q(root)}.`);
const child = await execFileBounded(cli.cmd, [...cli.argsPrefix, 'git', 'commit-push', '--push-only', '--keep-checkout', '--scan-secrets', '--path', root, '--expect-remote', checkedUpstream, '--json'], {
timeout: PUBLISH_TIMEOUT_MS, termGraceMs: 10_000, maxBuffer: 1024 * 1024,
env: { ...process.env, GBRAIN_PUBLISH_LOCK_WAIT_SECONDS: '0', GBRAIN_PUSH_LOCK_WAIT_SECONDS: '0' },
diff --git a/test/effect-git-secret-gate.test.ts b/test/effect-git-secret-gate.test.ts
index ead7cd4354..0bb8d1c575 100644
--- a/test/effect-git-secret-gate.test.ts
+++ b/test/effect-git-secret-gate.test.ts
@@ -154,6 +154,18 @@ describe('#8896 pushGitRoot scans the commits it publishes', () => {
expect(remoteTip(c)).toBe(theirs);
}, 120_000);
+ test('a scanned publish whose push the remote keeps rejecting hands back a rerun that still scans', async () => {
+ const c = checkout();
+ const theirs = moveRemote(c, 'notes/other.md');
+ scriptCommit(c.root, 'notes/page.md', 'A clean local page\n');
+ writeFileSync(join(c.remote, 'hooks', 'pre-receive'), '#!/bin/sh\nexit 1\n');
+ chmodSync(join(c.remote, 'hooks', 'pre-receive'), 0o755);
+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));
+ expect(error.code).toBe('push_unconfirmed');
+ expect(`${error.message} ${error.suggestion ?? ''}`).toContain('gbrain git commit-push --push-only --keep-checkout --scan-secrets');
+ expect(remoteTip(c)).toBe(theirs);
+ }, 120_000);
+
test('a remote without the branch (a first push): a clean history creates it', async () => {
const c = checkout();
scriptCommit(c.root, 'notes/page.md', 'A clean page\n');
diff --git a/test/git-canonical-conversion-g8a.test.ts b/test/git-canonical-conversion-g8a.test.ts
index f4166b365d..f63c7cebce 100644
--- a/test/git-canonical-conversion-g8a.test.ts
+++ b/test/git-canonical-conversion-g8a.test.ts
@@ -806,7 +806,7 @@ for (const backend of testBackends()) describe(`[${backend}] G8a: sources canoni
const applied = await text(['default', '--to', 'git', '--apply', '--expect', preview.preview_hash, '--no-embed', '--no-extract']);
expect(applied).toContain('Source default is git-canonical: 0 page(s) committed in 0 door commit(s), 1 private page(s) withheld (1 removal commit(s))');
expect(applied).toContain("Push pending (git_push_unavailable): The conversion's door commits are committed locally");
- expect(applied).toContain(`gbrain git commit-push --push-only --path ${brain.root}`);
+ expect(applied).toContain(`gbrain git commit-push --push-only --keep-checkout --scan-secrets --path ${brain.root}`);
}, { databaseUrl, setup: closedRemoteSetup }), 180_000);
test('work that lands after the strict-equivalent pass leaves the source strict and reports incomplete', () => managedBrain(async brain => {
diff --git a/test/git-commit-push-managed-unpublished.test.ts b/test/git-commit-push-managed-unpublished.test.ts
index ea4eb6c799..845b93bb07 100644
--- a/test/git-commit-push-managed-unpublished.test.ts
+++ b/test/git-commit-push-managed-unpublished.test.ts
@@ -65,6 +65,7 @@ test('a commit that already exists is reported as not on the upstream, not as no
const text = await commitPush(f, false, 'companies/acme-example.md');
expect(text.exitCode).toBe(0);
expect(text.stdout).toContain('nothing to commit; not pushed: 1 commit(s) on main are not on origin/main as last fetched');
+ expect(text.stdout).toContain(`gbrain git commit-push --push-only --keep-checkout --scan-secrets --path ${f.brain} publishes them`);
expect(pushLog(f)).toBe('');
expect(remoteTip(f)).toBe(before);
expect(git(f.brain, 'rev-parse', 'HEAD')).toBe(ours);
diff --git a/test/refresh-publish-diverged-7686.test.ts b/test/refresh-publish-diverged-7686.test.ts
index 145531d464..5d87094a20 100644
--- a/test/refresh-publish-diverged-7686.test.ts
+++ b/test/refresh-publish-diverged-7686.test.ts
@@ -229,6 +229,15 @@ test('setting on: the publish runs in a child gbrain, never in this process, so
expect(await f.refreshRows()).toEqual([]);
}, '/bin/false'), 120_000);
+test('setting on: a refresh that cannot locate its own CLI refuses git_push_unavailable, and the publish by hand it names still scans', () => each(async f => {
+ diverge(f);
+ const remote = remoteHead(f);
+ await f.engine.setConfig(REFRESH_PUBLISH_DIVERGED_KEY, 'true');
+ const error = await refusedWith(refreshWorktree(f.engine, f.alpha), 'git_push_unavailable');
+ expect(error.suggestion).toContain(`gbrain git commit-push --push-only --keep-checkout --scan-secrets --path `);
+ expect(remoteHead(f)).toBe(remote);
+}, ''), 120_000);
+
test('setting on: a dry run publishes nothing and still refuses refresh_diverged', () => each(async f => {
const local = diverge(f);
const remote = remoteHead(f);