Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Keep backing up page writes whose bodies hold credential-shaped text

garrytan688c23a8f166
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4d7259d394..69a47ddd38 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,16 +12,16 @@ identifiers and attribution are available in the pre-removal Git revision
## [0.60.621.0] - 2026-11-09
-**The Git effects of page writes no longer publish a script's secret.** A script that commits into a managed checkout with plain `git commit` never passes the write path's secret scan. The refresh publish refused such a commit with `publish_secret_found`, but the next page write's Git effect pushed it anyway: its plain push published every unpublished commit unscanned, and when the remote had moved its keep-checkout publish ran without the scan. Both paths now scan the exact commits they push. A value that looks like a secret, added text too large to scan, a commit that changes `.gbrain-scan-allow` or a `.gitattributes` holds the effects with `publish_secret_found`; commits the scan cannot read in full hold them with `publish_scan_unreadable`. A first push to a remote that has no such branch yet scans the branch's whole history and creates the branch only when that scan passes. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. Nothing is pushed, and the page writes stay safe in the database and in local commits. `gbrain sync status` and doctor name the user's choice: remove the value from those commits, or allowlist a false positive by fingerprint. Then `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked. Every scanned push is also leased on the remote tip it scanned against. Before, a remote rewound past a pushed secret (an admin removing it with a force-push) left a stale tracking ref that hid the commit from the scan, and the next fast-forward put it back; now the scan runs against the remote's real tip. The in-process scan on the owner's event loop is capped at 10 seconds, inside the writer heartbeat's live window; a longer range goes to the keep-checkout publish, whose child process scans with the full budget.
+**The Git effects of page writes no longer publish a script's secret.** A script that commits into a managed checkout with plain `git commit` never passes the write path's secret scan. The refresh publish refused such a commit with `publish_secret_found`, but the next page write's Git effect pushed it anyway: its plain push published every unpublished commit unscanned, and when the remote had moved its keep-checkout publish ran without the scan. Both paths now scan the exact commits they push, except the ones gbrain made itself for page writes (a Git effect's commit or a hosted repository's door commit, as its commit ledger records them): page bodies are stored raw by design and their Git backups keep going, and the refresh publish now skips them the same way. A value that looks like a secret, added text too large to scan, a commit that changes `.gbrain-scan-allow` or a `.gitattributes` holds the effects with `publish_secret_found`; commits the scan cannot read in full hold them with `publish_scan_unreadable`. A first push to a remote that has no such branch yet scans the branch's whole history and creates the branch only when that scan passes. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. Nothing is pushed, and the page writes stay safe in the database and in local commits. `gbrain sync status` and doctor name the user's choice: remove the value from those commits, or allowlist a false positive by fingerprint. Then `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked. Every scanned push is also leased on the remote tip it scanned against. Before, a remote rewound past a pushed secret (an admin removing it with a force-push) left a stale tracking ref that hid the commit from the scan, and the next fast-forward put it back; now the scan runs against the remote's real tip. The in-process scan on the owner's event loop is capped at 10 seconds, inside the writer heartbeat's live window; a longer range goes to the keep-checkout publish, whose child process scans with the full budget.
### Itemized changes
- `src/core/persistence/effect-git.ts`: `pushGitRoot` reads the remote branch tip with `git ls-remote`, scans `<tip>..<HEAD sha>` and pushes that sha only as a fast-forward of the tip, leased on it; a tip that is not an ancestor, or a remote that moves before the push lands (`stale info`), goes to the keep-checkout publish, which now runs with `--scan-secrets`. The in-process scan gets 10 s; a range it cannot finish goes to the keep-checkout publish. A remote without the branch gets HEAD's whole history scanned and the branch created with a lease that it does not exist; one too large to scan in process holds with `publish_scan_unreadable` naming `git ls-remote --heads`. A refusal from either path holds the effects as `publish_secret_found` with the user's fix.
- `src/core/git/commit-push.ts`: a `scanSecrets` publish scans `<base>..<candidate>` before every push and leases the push on that base, so each merge with a newly fetched remote tip is scanned again.
-- `src/core/git/range-secret-scan.ts`: `rangeSecretRefusal`, the one refusal both callers throw (`publish_scan_unreadable` for a scan that could not read the range, `publish_secret_found` otherwise; a gate-file change says the user pushes it). `scanRange` marks a scan that ran past its budget (`timed_out`) and scans a whole history when the base is empty.
+- `src/core/git/range-secret-scan.ts`: `rangeSecretRefusal`, the one refusal both callers throw (`publish_scan_unreadable` for a scan that could not read the range, `publish_secret_found` otherwise; a gate-file change says the user pushes it). `scanRange` marks a scan that ran past its budget (`timed_out`) and scans a whole history when the base is empty. It skips commits the commit ledger records as page writes (`effect`, `door`, `hosted_repo`); `gbrain git commit-push` commits are still scanned. Its time budget is also checked while the diff is parsed.
- `src/core/persistence/git-push-holds.ts`: `publish_secret_found` and `publish_scan_unreadable` are held push codes, so doctor, `gbrain sync status` and `retry-effects --parked-pushes` cover them, with an ask-the-user fix (the refused commits, or `git fsck`).
- Docs: `docs/guides/shared-checkout-git.md` (`publish_secret_found` and `publish_scan_unreadable` rows), `docs/guides/live-sync.md`.
-- Tests: `test/effect-git-secret-gate.test.ts` (an ahead checkout, a diverged one, a remote rewound past a pushed secret, a `.gitattributes` change and a remote without the branch whose history holds a secret each refuse with nothing pushed; a clean ahead checkout pushes, a clean history creates a missing branch, and a scan past the in-process budget still scans in the child; through the effect worker the page write's effect holds, sync status names the fix, and an allowlisted false positive releases it); `test/guarded-publish.test.ts` (a remote rewound past a pushed secret is not published again); `test/git-literal-pathspec-hooks-7759.test.ts` publishes its probe `.gitattributes` commit first, since the effect's push now refuses one; `test/cli-flag-equals.test.ts` lists `range-secret-scan.ts` as a git child argv reader.
+- Tests: `test/effect-git-secret-gate.test.ts` (an ahead checkout, a diverged one, a remote rewound past a pushed secret, a `.gitattributes` change and a remote without the branch whose history holds a secret each refuse with nothing pushed; a clean ahead checkout pushes, a clean history creates a missing branch, and a scan past the in-process budget still scans in the child; through the effect worker the page write's effect holds, sync status names the fix, and an allowlisted false positive releases it, parked or not; a page write whose body holds a credential-shaped value still backs up; a repaired unreadable commit releases its hold); `test/guarded-publish.test.ts` (a remote rewound past a pushed secret is not published again); `test/git-literal-pathspec-hooks-7759.test.ts` publishes its probe `.gitattributes` commit first, since the effect's push now refuses one; `test/cli-flag-equals.test.ts` lists `range-secret-scan.ts` as a git child argv reader.
## [0.60.620.0] - 2026-11-09
diff --git a/docs/guides/shared-checkout-git.md b/docs/guides/shared-checkout-git.md
index 605dd892d6..3e5525b7b9 100644
--- a/docs/guides/shared-checkout-git.md
+++ b/docs/guides/shared-checkout-git.md
@@ -185,7 +185,7 @@ Follow the error's `fix.next` as the
| `git_index_stale` | 1 | An `index.lock` older than 10 minutes blocks the checkout. Check that no git command is running there, tell the user, and remove the named lock file only with their agreement. |
| `git_operation_in_progress` | 1 | A rebase, merge, cherry-pick or revert is unfinished in the checkout. gbrain never aborts it for you. Follow [stuck rebase](#stuck-rebase). |
| `git_detached_head` | 1 | The checkout is on a detached HEAD or an unborn branch. Check out the branch the remote tracks (`git switch <branch>`), or make the first commit, then rerun. |
-| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | Only with `--scan-secrets`, which every automatic publish uses: the one `gbrain sources refresh` runs on its own and the push of a managed worktree's Git effects. A commit about to be pushed adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |
+| <a id="publish_secret_found"></a>`publish_secret_found` | 1 | Only with `--scan-secrets`, which every automatic publish uses: the one `gbrain sources refresh` runs on its own and the push of a managed worktree's Git effects. A commit about to be pushed that gbrain did not make for a page write (its commit ledger records its own page-write commits, whose bodies are stored raw by design and are not scanned) adds a value that looks like a secret, in its message, an added line (even one a later commit removed) or a merge resolution, adds text too large to scan, or changes `.gbrain-scan-allow` or a `.gitattributes` (files that decide what the scan sees); nothing was pushed. The message names each file, line, rule and commit, never the value. Ask the user: remove the value from those unpushed commits, or add a false positive's fingerprint to `.gbrain-scan-allow`, then retry. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically: the user reviews it and pushes it. Never publish them with a command that skips the scan. On a managed worktree its Git effects hold meanwhile (`gbrain sync status` and doctor list them; the page writes are safe in the database and in local commits); after the fix, `gbrain sources writer retry-effects <source> --parked-pushes` runs the ones that parked. |
| <a id="publish_scan_unreadable"></a>`publish_scan_unreadable` | 1 | Only with `--scan-secrets`: the scan could not read the commits about to be pushed in full (a missing or corrupt object, git output it does not recognise, or its 300 s budget), so it reported no secret and nothing was pushed. Run `git fsck` in the checkout, fix what it names, then retry. When the remote has no such branch, a managed worktree's Git effect scans the branch's whole history and creates it only when that scan passes; a history too large to scan in the owner process holds the effects with this code and names `git ls-remote --heads <remote>`: check the upstream, then point it at the right branch or review the history and push the branch yourself. Never publish with a command that skips the scan. On a managed worktree its Git effects hold meanwhile, as for `publish_secret_found`. |
| `integrate_conflict` | 1 | Local and remote commits changed the same file; nothing was merged and the checkout is unchanged. Stop, relay the question to the user, then [resolve](#conflicts). |
| `push_unconfirmed` | 4 | The commit is durable locally but no push destination confirmed it (network, credentials, branch protection, a remote that kept moving, or another push that held the push lock for the whole wait). Fix what the message names, then run `gbrain git commit-push --push-only`. Never force-push. |
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 08766ae365..90d1e847de 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1489,7 +1489,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.618.0', id: 'refresh_replan_own_commits_8828', text: 'A `gbrain sources refresh` whose checkout HEAD moved while it waited for writes admitted before its fence no longer refuses `refresh_source_changed` when every commit since its start is one of gbrain\'s own (in the commit ledger) and the upstream branch holds them: it takes HEAD as its start and fast-forwards to the upstream. A HEAD move that includes a commit gbrain did not make, or gbrain\'s commits the upstream does not hold yet, still refuses as before.' },
{ since: '0.60.619.0', id: 'managed_commit_push_unpublished_gst1923', text: 'On a managed checkout, `gbrain git commit-push -m <msg> -- <paths>` now reports how many commits the upstream lacks as last fetched (`unpublished` in `--json`, `null` when there is no upstream) and says nothing was pushed, instead of answering `nothing to commit; <remote>/<branch> at -` and logging `[push] ok-already-on-remote` for a commit that already existed. It writes no push-log line on a managed checkout. It still never pushes there; `gbrain git commit-push --push-only --keep-checkout --path <checkout>` publishes.' },
{ since: '0.60.620.0', id: 'refresh_publish_diverged_default_gbra386', text: 'A diverged managed checkout now publishes its local commits on `gbrain sources refresh` by default when the worktree\'s Git durability is on: `sources.refresh_publish_diverged` unset now means on there (it used to mean off), merged with the upstream in memory, never rebased or forced. The refresh publish first scans what the local commits add (messages, added lines, merge resolutions) and refuses the new `publish_secret_found` naming the file, line and rule, never the value, when one looks like a secret; `gbrain git commit-push --push-only --scan-secrets` runs the same gate. Unset with durability not on still refuses, naming `gbrain sources writer git-durability <source> --enable`; `false` turns it off.' },
- { since: '0.60.621.0', id: 'effect_git_secret_gate_gst1923', text: 'The Git effects of page writes on a managed worktree now scan the unpublished commits they push, as the refresh publish does: a value that looks like a secret (or text too large to scan, or a change to `.gbrain-scan-allow` or a `.gitattributes`) in a commit a script made with plain `git commit` holds the effects with `publish_secret_found` and nothing is pushed, where they used to push it with the next page write. The push now reads the remote branch tip and scans exactly what it adds, leased on that tip, so a remote rewound since the last fetch never gets a removed commit back; commits the scan cannot read in full hold the effects with `publish_scan_unreadable`, and a first push to a remote without the branch scans the whole history first. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically; the user reviews it and pushes it. After the user removes the value or allowlists a false positive, `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked.' },
+ { since: '0.60.621.0', id: 'effect_git_secret_gate_gst1923', text: 'The Git effects of page writes on a managed worktree now scan the unpublished commits they push, as the refresh publish does, except gbrain\'s own page-write commits (page bodies are stored raw by design, and both publishes skip them): a value that looks like a secret (or text too large to scan, or a change to `.gbrain-scan-allow` or a `.gitattributes`) in a commit a script made with plain `git commit` holds the effects with `publish_secret_found` and nothing is pushed, where they used to push it with the next page write. The push now reads the remote branch tip and scans exactly what it adds, leased on that tip, so a remote rewound since the last fetch never gets a removed commit back; commits the scan cannot read in full hold the effects with `publish_scan_unreadable`, and a first push to a remote without the branch scans the whole history first. A commit that changes `.gbrain-scan-allow` or a `.gitattributes` is never published automatically; the user reviews it and pushes it. After the user removes the value or allowlists a false positive, `gbrain sources writer retry-effects <source> --parked-pushes` runs the effects that parked.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/git/range-secret-scan.ts b/src/core/git/range-secret-scan.ts
index 766d3c863f..bf74da7641 100644
--- a/src/core/git/range-secret-scan.ts
+++ b/src/core/git/range-secret-scan.ts
@@ -14,12 +14,17 @@
* not empty or attributes could still apply, and a text file is then scanned whole. Added text over the scan cap is
* reported as unscanned, and a commit that changes a file deciding what the scan sees (`.gbrain-scan-allow`, a
* `.gitattributes`) as a gate change, so callers refuse. Output the parser does not recognise, a git failure or a run
- * past the time budget sets `unreadable` instead of returning a partial result.
+ * past the time budget sets `unreadable` instead of returning a partial result. The time budget is checked between git
+ * calls and while the diff is parsed, so a caller with a short budget (the owner's event loop) is never held much past it.
+ * Commits gbrain's commit ledger records as page writes (`PAGE_WRITERS`) are not scanned: page bodies are stored raw by
+ * design (docs/guides/write-refusals.md#what-stays-raw-by-design) and their Git backups have always been pushed, so the
+ * gate covers what gbrain did not write; `gbrain git commit-push` commits, which carry a script's files, are scanned.
*/
import { closeSync, existsSync, mkdtempSync, openSync, readFileSync, readSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { isAbsolute, join } from 'node:path';
import { GIT_READ_ENV, gitChildEnv } from '../git-env.ts';
+import { readLedger } from './commit-ledger.ts';
import { GIT_ENV } from '../git-transport.ts';
import { opError, type OperationError } from '../ops/contract.ts';
import { loadWorkspaceAllowlist, pathAllowlisted, SCAN_ALLOW_FILENAME } from '../secret-scan-allowlist.ts';
@@ -44,6 +49,9 @@ const CONFIG = ['-c', 'core.quotePath=false', '-c', 'core.attributesFile=/dev/nu
const DIFF = ['diff-tree', '--stdin', '-r', '-p', '--root', '--cc', '--no-renames', '--no-ext-diff', '--no-textconv', '--no-color',
'--full-index', '--src-prefix=a/', '--dst-prefix=b/', '--unified=0'];
+/** Ledger writers whose commits hold only page writes: a Git effect's commit and a hosted repository's door commits. */
+const PAGE_WRITERS: ReadonlySet<string> = new Set(['effect', 'door', 'hosted_repo']);
+
class Unreadable extends Error { constructor(message: string, readonly timedOut = false) { super(message); } }
type Run = ReturnType<typeof spawnSync>;
@@ -53,6 +61,12 @@ function run(root: string, args: string[], deadline: number, opts: { input?: str
return spawnSync('git', ['-C', root, ...args], { input: opts.input ?? '', stdio: ['pipe', opts.out ?? 'pipe', 'pipe'], timeout,
maxBuffer: opts.maxBuffer ?? 64 * 1024 * 1024, env: gitChildEnv({ ...GIT_ENV, ...GIT_READ_ENV, LC_ALL: 'C', GIT_NO_REPLACE_OBJECTS: '1', GIT_ATTR_NOSYSTEM: '1' }) });
}
+/** The commits the ledger records as page writes; an unreadable ledger reads as none, so everything is scanned. */
+function pageWriteCommits(root: string, deadline: number): Set<string> {
+ const common = must(run(root, ['rev-parse', '--path-format=absolute', '--git-common-dir'], deadline), 'rev-parse').toString().trim();
+ try { return new Set(readLedger(common).flatMap(record => record.e === 'commit' && PAGE_WRITERS.has(record.writer) ? [record.sha] : [])); }
+ catch { return new Set(); }
+}
function must(r: Run, what: string): Buffer {
if (r.error || r.status !== 0) {
const timedOut = (r.error as NodeJS.ErrnoException | undefined)?.code === 'ETIMEDOUT';
@@ -109,12 +123,15 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
const scan: RangeScan = { findings: [], unscanned: [], gate_changes: [] };
const allowlist = loadWorkspaceAllowlist(root);
const scanned = (path: string, commit: string, text: string, lineAt: (line: number) => number = line => line) => {
+ if (Date.now() > deadline) throw new Unreadable('the scan ran past its time budget', true);
if (Buffer.byteLength(text) > SCAN_MAX_FILE_BYTES) { scan.unscanned.push({ path, commit }); return; }
for (const f of scanText(text, { allowlist })) scan.findings.push({ path, commit, line: lineAt(f.line), pattern: f.pattern, fingerprint: f.fingerprint });
};
const dir = mkdtempSync(join(tmpdir(), 'gbrain-range-scan-'));
try {
- const commits = must(run(root, ['rev-list', base ? `${base}..${head}` : head, '--'], deadline), 'rev-list').toString().split('\n').filter(Boolean);
+ const pageWrites = pageWriteCommits(root, deadline);
+ const commits = must(run(root, ['rev-list', base ? `${base}..${head}` : head, '--'], deadline), 'rev-list').toString().split('\n')
+ .filter(commit => commit && !pageWrites.has(commit));
if (!commits.length) return scan;
for (const commit of commits) {
@@ -154,7 +171,9 @@ export function scanRange(root: string, base: string, head: string, timeoutMs =
added = []; at = []; bytes = 0; oversized = false;
};
const unexpected = (line: string) => new Unreadable(`git diff-tree printed a line the scan does not recognise in ${commit.slice(0, 12) || 'the range'}: ${JSON.stringify(line.slice(0, 80))}`);
+ let read = 0;
for (const { text: line, long } of fileLines(fd, SCAN_MAX_FILE_BYTES + 1)) {
+ if (++read % 4096 === 0 && Date.now() > deadline) throw new Unreadable('the scan ran past its time budget', true);
if (/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/.test(line)) { flush(); commit = line; path = null; header = false; continue; }
if (line.startsWith('diff --git ') || line.startsWith('diff --cc ') || line.startsWith('diff --combined ')) {
flush();
@@ -251,7 +270,7 @@ export function rangeSecretRefusal(root: string, base: string, target: string, s
...scan.gate_changes.map(g => `${g.path} (changes what the scan sees, ${g.commit.slice(0, 7)})`)];
return opError('publish_secret_found',
`${named.length} item(s) in the local commits ${range} look like secrets or could not be scanned: ${named.slice(0, 10).join(', ')}${named.length > 10 ? ', ...' : ''}. Nothing was pushed.`,
- `Remove the value from those commits (rewriting unpushed local commits is the user's call), or add a false positive's fingerprint to .gbrain-scan-allow, then retry. Never publish them with a command that skips this scan.${scan.gate_changes.length ? ` A commit that changes .gbrain-scan-allow or a .gitattributes is never published automatically, because it changes what the scan sees: review it and push it yourself (git -C ${root} push), then retry.` : ''}`,
+ `Remove the value from those commits (rewriting unpushed local commits is the user's call), or add a false positive's fingerprint to .gbrain-scan-allow, then retry. Never publish them with a command that skips this scan.${base ? '' : ' The remote has no such branch, so this covered the branch\'s whole history, which may include commits published elsewhere before: review it, and push the branch yourself once it is safe.'}${scan.gate_changes.length ? ` A commit that changes .gbrain-scan-allow or a .gitattributes is never published automatically, because it changes what the scan sees: review it and push it yourself (git -C ${root} push), then retry.` : ''}`,
{ detail: JSON.stringify(scan),
fix: { ask_user: true, consent: [], actor: 'user', requires_exclusive: false,
argv: ['git', '-C', root, 'log', '--stat', base ? `${base}..${target}` : target],
diff --git a/test/effect-git-secret-gate.test.ts b/test/effect-git-secret-gate.test.ts
index 0e8d6c34d6..0491e06f71 100644
--- a/test/effect-git-secret-gate.test.ts
+++ b/test/effect-git-secret-gate.test.ts
@@ -30,6 +30,8 @@ import { runPersistenceEffects } from '../src/core/persistence/effects.ts';
import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';
import { readSyncStatus } from '../src/core/persistence/sync-status.ts';
import { scanText } from '../src/core/secret-scan.ts';
+import { recordCommit, resetLedgerCacheForTests } from '../src/core/git/commit-ledger.ts';
+import { retryParkedPushes } from '../src/core/persistence/git-durability-administration.ts';
import { isolatedSharedSkillsEngine } from './helpers/shared-skills-engine.ts';
import { git } from './helpers/git-publication.ts';
import { withEnv } from './helpers/with-env.ts';
@@ -37,7 +39,7 @@ import { withEnv } from './helpers/with-env.ts';
const token = 'ghp_' + 'Zt4q'.repeat(9);
const config = { engine: 'pglite' as const, embedding_disabled: true };
const dirs: string[] = [];
-afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); });
+afterEach(() => { resetLedgerCacheForTests(); for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); });
interface Checkout { home: string; root: string; remote: string; other: string; cli: string }
@@ -203,6 +205,21 @@ describe('#8896 pushGitRoot scans the commits it publishes', () => {
expect(remoteTip(c)).toBe(before);
}, 60_000);
+ test('a commit the ledger records as a page write is not scanned; a commit-push commit still is', async () => {
+ const c = checkout();
+ const page = scriptCommit(c.root, 'notes/page.md', `Saved note with ${token}\n`);
+ recordCommit(c.root, { sha: page, writer: 'effect', ref: null });
+ expect(await withEnv(childEnv(c), () => pushGitRoot(c.root))).toEqual({ push: 'committed' });
+ expect(remoteTip(c)).toBe(page);
+ const script = scriptCommit(c.root, 'notes/deploy.md', `Deploy token ${token}\n`);
+ recordCommit(c.root, { sha: script, writer: 'commit_push', ref: null });
+ const error = await withEnv(childEnv(c), () => refusal(pushGitRoot(c.root)));
+ expect(error.code).toBe('publish_secret_found');
+ expect(error.message).toContain('notes/deploy.md:1 (');
+ expect(error.message).not.toContain('notes/page.md');
+ expect(remoteTip(c)).toBe(page);
+ }, 60_000);
+
test('a clean ahead checkout still pushes', async () => {
const c = checkout();
scriptCommit(c.root, 'notes/page.md', 'A clean page\n');
@@ -211,10 +228,10 @@ describe('#8896 pushGitRoot scans the commits it publishes', () => {
}, 60_000);
});
-interface ManagedWrite { engine: BrainEngine; sourceId: string; runEffects: () => Promise<void>; effect: () => Promise<{ state: string; error_code: string | null }> }
+interface ManagedWrite { engine: BrainEngine; sourceId: string; worktreeId: string; runEffects: () => Promise<void>; effect: () => Promise<{ state: string; error_code: string | null }> }
/** A managed source on the checkout with git durability on; `script` commits into the checkout before one page write whose Git effect is queued. */
-async function managedWrite(c: Checkout, script: () => void, run: (w: ManagedWrite) => Promise<void>): Promise<void> {
+async function managedWrite(c: Checkout, script: () => void, run: (w: ManagedWrite) => Promise<void>, body = 'After'): Promise<void> {
await withEnv({ ...childEnv(c), GBRAIN_AUDIT_DIR: join(c.home, 'audit'), GBRAIN_BRAIN_ID: 'host', GBRAIN_SOURCE: undefined }, async () => {
const { engine, close } = await isolatedSharedSkillsEngine(undefined);
try {
@@ -236,12 +253,12 @@ async function managedWrite(c: Checkout, script: () => void, run: (w: ManagedWri
const authority = await submissionAuthority(ctx, 'put_page', sourceId, bound.source_incarnation, slug);
await admitWrite(engine, { principal: authority.principal, authority, operation: 'put_page', sourceId,
sourceIncarnation: bound.source_incarnation, slug, pageId: snapshot.page.id, requestId: randomUUID(),
- callerIntent: { content: 'After' }, intent: { content: 'After' }, worktreeId: bound.worktree_id, topologyGeneration: bound.topology_generation });
+ callerIntent: { content: body }, intent: { content: body }, worktreeId: bound.worktree_id, topologyGeneration: bound.topology_generation });
const row = (await claimNextWrite(engine, localHostId()))!;
- const result = await publishMutation(engine, row, { observedRevision: snapshot.revision, file: { path: file, root: c.root, content: 'After' },
- apply: async tx => { await tx.putPage(slug, page('After'), { sourceId }); return {}; } }, localHostId());
+ const result = await publishMutation(engine, row, { observedRevision: snapshot.revision, file: { path: file, root: c.root, content: body },
+ apply: async tx => { await tx.putPage(slug, page(body), { sourceId }); return {}; } }, localHostId());
expect(result.state).toBe('committed');
- await run({ engine, sourceId,
+ await run({ engine, sourceId, worktreeId: bound.worktree_id,
runEffects: async () => {
await engine.executeRaw("UPDATE persistence_effects SET next_attempt_at=now() WHERE kind='git'");
await runPersistenceEffects(engine, config, { hostId: localHostId(), limit: 10 });
@@ -287,6 +304,37 @@ describe('#8896 through the effect worker', () => {
const hold = (await readSyncStatus(w.engine, w.sourceId)).git_push_holds[0]!;
expect(hold).toMatchObject({ retrying: 1, parked: 0, codes: ['publish_scan_unreadable'] });
expect(hold.fix).toMatchObject({ ask_user: true, actor: 'user', argv: ['git', '-C', c.root, 'fsck'] });
+
+ writeFileSync(join(c.home, 'restored.txt'), 'plain text\n');
+ git(c.root, 'hash-object', '-w', join(c.home, 'restored.txt'));
+ await w.runEffects();
+ expect(await w.effect()).toMatchObject({ state: 'committed', error_code: null });
+ expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());
});
}, 180_000);
+
+ test('a page write whose body holds a credential-shaped value still backs up: page bodies are stored raw by design', async () => {
+ const c = checkout();
+ await managedWrite(c, () => {}, async w => {
+ await w.runEffects();
+ expect(await w.effect()).toMatchObject({ state: 'committed', error_code: null });
+ expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());
+ expect(git(c.remote, 'show', 'main:notes/acme-example.md')).toContain(token);
+ }, `Connection notes: ${token}`);
+ }, 180_000);
+
+ test('a parked publish_secret_found effect is released by retry-effects --parked-pushes once the value is allowlisted', async () => {
+ const c = checkout();
+ const before = remoteTip(c);
+ await managedWrite(c, () => scriptCommit(c.root, 'scripts-out/deploy.md', `Deploy token ${token}\n`), async w => {
+ for (let i = 0; i < 8 && (await w.effect()).state !== 'failed'; i++) await w.runEffects();
+ expect(await w.effect()).toMatchObject({ state: 'failed', error_code: 'targets_parked' });
+ expect(remoteTip(c)).toBe(before);
+ writeFileSync(join(c.root, '.gbrain-scan-allow'), `${scanText(`Deploy token ${token}\n`)[0]!.fingerprint}\n`);
+ expect(await retryParkedPushes(w.engine, w.worktreeId, false)).toMatchObject({ status: 'ok', requests: 1, queued: 1 });
+ await w.runEffects();
+ expect(await w.effect()).toMatchObject({ state: 'committed', error_code: null });
+ expect(remoteTip(c)).toBe(git(c.root, 'rev-parse', 'HEAD').trim());
+ });
+ }, 240_000);
});