Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: run the refresh publish in a bounded child, never on the owner event loop (#7686, adversarial review)
garrytan69c74d6c2baa
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4648a6e24b..d77eed8cb5 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -39,7 +39,7 @@ gbrain config set sources.refresh_publish_diverged true### Itemized changes-- `src/core/persistence/worktree-refresh.ts`: when `sources.refresh_publish_diverged` is on and the run is not a dry run, the precheck runs `commitAndPush` (`pushOnly`, `keepCheckout`) for a diverged checkout (after refusing `refresh_dirty` first when an uncommitted edit touches a file the upstream changed), re-fetches and continues; the result carries `published` (with any quarantined or superseded paths, which the text output names), and a refusal after the publish says the remote already holds the local commits. The `refresh_diverged` suggestion names the setting.+- `src/core/persistence/worktree-refresh.ts`: when `sources.refresh_publish_diverged` is on and the run is not a dry run, the precheck runs the keep-checkout publish (`gbrain git commit-push --push-only --keep-checkout` in a bounded child, never on the PGLite owner's event loop) for a diverged checkout (after refusing `refresh_dirty` first when an uncommitted edit touches a file the upstream changed), re-fetches and continues; the result carries `published` (with any quarantined or superseded paths, which the text output names), and a refusal after the publish says the remote already holds the local commits. The `refresh_diverged` suggestion names the setting.- `src/commands/sources-refresh.ts`: a text line for the publish.- `src/core/config-runtime-keys.ts`: registers `sources.refresh_publish_diverged`, `sources.refresh_fetch_timeout_ms` and `sources.refresh_drain_wait_ms`.- Docs: `docs/guides/live-sync.md`, `docs/guides/write-refusals.md`, `docs/architecture/canonical-writers.md` and its census.diff --git a/docs/guides/live-sync.md b/docs/guides/live-sync.mdindex fa7fee3976..74898855ee 100644--- a/docs/guides/live-sync.md+++ b/docs/guides/live-sync.md@@ -673,8 +673,8 @@ vars — incident-time escape hatches, not everyday knobs.consent with `gbrain config set sources.refresh_publish_diverged true`(default off): a diverged refresh then pushes the local commits merged withthe upstream in memory, the same publish as- `gbrain git commit-push --push-only --keep-checkout`, and fast-forwards to- the merge. It never moves the checkout to merge, a conflicting file refuses+ `gbrain git commit-push --push-only --keep-checkout` run as a child process+ (it skips the checkout's pre-push hook), and fast-forwards to the merge. It never moves the checkout to merge, a conflicting file refuses`integrate_conflict` naming it with nothing pushed (with `gbrain.quarantine`on, the remote's version is published, this machine's is kept under`refs/gbrain/quarantine/`, and the refresh says so), an uncommitted edit to adiff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex 1bbedb8e55..97702e6be6 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -548,7 +548,7 @@ this anchor (`--json` puts them in one object on stdout); the command exits 1.| <a id="refresh_no_upstream"></a>`refresh_no_upstream` | `refresh_no_upstream` | The checkout's branch tracks no upstream, HEAD is detached, or the checkout is not a Git checkout at all (gbrain never initializes Git on its own), so there is nothing to fast-forward to. | `git -C <checkout> branch --set-upstream-to <remote>/<branch>` (the fix fills in the real values), then retry. A checkout that is not Git has nothing to refresh; making it a clone with an upstream is the user's call. || <a id="fetch_failed"></a>`fetch_failed` | `fetch_failed` | `git fetch` failed or did not finish within `sources.refresh_fetch_timeout_ms` (default 120000; `GBRAIN_REFRESH_FETCH_TIMEOUT_MS`; `--fetch-timeout-ms` wins over both). Nothing changed and no refresh record was written. | Retry. For a slow remote, raise the bound with `--fetch-timeout-ms <ms>` or `gbrain config set sources.refresh_fetch_timeout_ms <ms>`. Credentials are the user's: ask them to fix authentication rather than changing the remote. || <a id="refresh_git_unavailable"></a>`git_unavailable` | `git_unavailable` | A git command that reads the checkout's branch or remote (`symbolic-ref`, `remote`, `config --get branch.<branch>.remote` or `.merge`) failed or did not finish, so the checkout's state is unknown. The message names the command and git's first error line. A detached HEAD or an unset key is not this refusal. Nothing changed and no refresh record was written. | Run `git -C <checkout> status` to see why git cannot read the checkout, repair what it reports, then retry `gbrain sources refresh <source>`. |-| <a id="refresh_diverged"></a>`refresh_diverged` | `refresh_diverged` | The checkout has commits its upstream lacks, so a fast-forward is impossible. Nothing changed. | Ask the user, then publish the local commits merged with the upstream: `gbrain git commit-push --push-only --keep-checkout --path <checkout>` merges in memory, pushes the merge, refuses on a conflicting file and never moves the checkout (the refusal's fix fills in the real path and counts the local commits). Then retry `gbrain sources refresh <source>`, which fast-forwards to the merge. `gbrain sources reclone <source>` replaces the checkout instead and drops the local commits. With `sources.refresh_publish_diverged` set to `true` (the user's standing consent, default off), the refresh runs that publish itself and then fast-forwards; a conflicting file then refuses `integrate_conflict` (with `gbrain.quarantine` on, the remote's version is published instead, this machine's is kept under `refs/gbrain/quarantine/`, and the refresh says so), and an uncommitted edit to a file the upstream changed refuses `refresh_dirty`, both with nothing pushed. |+| <a id="refresh_diverged"></a>`refresh_diverged` | `refresh_diverged` | The checkout has commits its upstream lacks, so a fast-forward is impossible. Nothing changed. | Ask the user, then publish the local commits merged with the upstream: `gbrain git commit-push --push-only --keep-checkout --path <checkout>` merges in memory, pushes the merge, refuses on a conflicting file and never moves the checkout (the refusal's fix fills in the real path and counts the local commits). Then retry `gbrain sources refresh <source>`, which fast-forwards to the merge. `gbrain sources reclone <source>` replaces the checkout instead and drops the local commits. With `sources.refresh_publish_diverged` set to `true` (the user's standing consent, default off), the refresh runs that publish itself, in a child process that skips the pre-push hook, and then fast-forwards; a conflicting file then refuses `integrate_conflict` (with `gbrain.quarantine` on, the remote's version is published instead, this machine's is kept under `refs/gbrain/quarantine/`, and the refresh says so), and an uncommitted edit to a file the upstream changed refuses `refresh_dirty`, both with nothing pushed. || <a id="refresh_dirty"></a>`refresh_dirty` | `refresh_dirty` | Uncommitted or untracked files overlap the incoming upstream changes (the first 20 are named), or `git merge --ff-only` refused for the same reason. HEAD did not move. Uncommitted files outside the incoming changes do not refuse; they are kept and listed as `preserved_uncommitted`. | Commit or discard the named paths, then retry. Check `gbrain sources writer status <source> --json` first: a pending Git effect may be about to commit a gbrain-published file. || <a id="sync_in_progress"></a>`sync_in_progress` | `sync_in_progress` | A managed sync of one of the checkout's sources has an unfinished cursor. Moving the checkout now would strand it, and the refresh never waits on a sync. `gbrain repair frontmatter --apply` refuses the same way while an unfinished cursor still names a selected file. | Run the printed `gbrain sync --source <source> --no-pull ...` resume command (it keeps the cursor's options), then retry (for a repair, preview again and approve the new hash). || <a id="refresh_in_progress"></a>`refresh_in_progress` | `refresh_in_progress` | Another refresh of the same checkout is active and its process is still running (the message names its holder's pid and host when the active refresh row is already readable), or a command that changes which sources share the checkout (`sources add`, `remove`, `archive`, `restore`, `reclone`) ran while one was active. A refresh whose process died is not a reason to refuse: the next refresh on the owner host finishes it first (`--dry-run` does not, so it still refuses while the orphan stands, and neither does a refresh that cannot read the checkout's HEAD: it leaves the orphan as it is and refuses until git can read HEAD again). A refresh whose holder is recorded on another host is never taken over either: writer status shows its `liveness` as `unknown`, and `gbrain sources refresh <source> --resume` on the owner host finishes it once that host's refresh is known to have stopped. Pending publication or topology recovery on the checkout refuses `refresh_recovery_required` first. An interrupted refresh that needs a reset decision refuses `refresh_recovery_required` instead. | Wait for the running refresh, or check it with `gbrain sources writer status <source>` (`liveness`, `holder`). `gbrain sources refresh <source> --resume` finishes an orphaned one now; then retry the source command with the same request ID. |diff --git a/llms-full.txt b/llms-full.txtindex e8fe59a260..e8a020b113 100644--- a/llms-full.txt+++ b/llms-full.txt@@ -7409,8 +7409,8 @@ vars — incident-time escape hatches, not everyday knobs.consent with `gbrain config set sources.refresh_publish_diverged true`(default off): a diverged refresh then pushes the local commits merged withthe upstream in memory, the same publish as- `gbrain git commit-push --push-only --keep-checkout`, and fast-forwards to- the merge. It never moves the checkout to merge, a conflicting file refuses+ `gbrain git commit-push --push-only --keep-checkout` run as a child process+ (it skips the checkout's pre-push hook), and fast-forwards to the merge. It never moves the checkout to merge, a conflicting file refuses`integrate_conflict` naming it with nothing pushed (with `gbrain.quarantine`on, the remote's version is published, this machine's is kept under`refs/gbrain/quarantine/`, and the refresh says so), an uncommitted edit to adiff --git a/src/core/persistence/worktree-refresh.ts b/src/core/persistence/worktree-refresh.tsindex 1dbd07783a..4ccd9a234e 100644--- a/src/core/persistence/worktree-refresh.ts+++ b/src/core/persistence/worktree-refresh.ts@@ -42,7 +42,7 @@ import { startPersistenceConsumer } from './service.ts';import { lockTopologyPrincipal, lockTopologyRows, topologyPrincipal, withTopologyLocks } from './topology-locks.ts';import { ACTIVE_REFRESH_STATES_SQL, CHECKOUT_EFFECT_KINDS_SQL, type WorktreeRefreshState } from './worktree-refresh-schema.ts';import { gitChildEnv } from '../git-env.ts';-import { commitAndPush, type QuarantineEntry } from '../git/commit-push.ts';+import type { QuarantineEntry } from '../git/commit-push.ts';import { isTruthy } from '../connectors/config-keys.ts';import { classifyOriginUniqueViolation } from './origin-guard.ts';@@ -287,7 +287,7 @@ async function precheck(engine: BrainEngine, git: Git, sourceId: string, fetchTiconst incoming = await dirtyOverlap(git, root, base, targetHead);if (incoming.overlap.length) throw dirtyRefusal(sourceId, incoming.overlap);const localCommits = Number((await read(['rev-list', '--count', `${targetHead}..${oldHead}`])).stdout.trim()) || 0;- const pushed = commitAndPush({ repo: root, pushOnly: true, keepCheckout: true });+ const pushed = await publishInChild(root, sourceId);const published = { sha: pushed.sha ?? targetHead, local_commits: localCommits, integrated: pushed.integrated,...(pushed.quarantined ? { quarantined: pushed.quarantined } : {}), ...(pushed.superseded_remote_paths ? { superseded_remote_paths: pushed.superseded_remote_paths } : {}) };const plan = await precheck(engine, git, sourceId, fetchTimeoutMs).catch(error => { throw notePublished(error, published); });@@ -302,6 +302,33 @@ async function precheck(engine: BrainEngine, git: Git, sourceId: string, fetchTi* upstream into HEAD in memory and pushes the merge (refusing on a conflict, never touching the checkout), after which HEAD* is an ancestor of the upstream and the refresh fast-forwards. It publishes the local commits, so the user decides.*/+/** #7686: the bound on the keep-checkout publish child (a fetch, an in-memory merge and a push, repeated while the remote keeps moving). */+const PUBLISH_TIMEOUT_MS = 600_000;++/**+ * #7686 (#7010's rule): run the keep-checkout publish in a bounded child `gbrain git commit-push --push-only --keep-checkout`.+ * The shared publish is synchronous (fetches, pushes and lock waits block for minutes), and a refresh delegated to a+ * PGLite owner runs inside the owner, where blocking would stall lease renewals; a busy publish lock fails at once.+ */+async function publishInChild(root: string, sourceId: string): Promise<{ sha: string | null; integrated: number; quarantined?: QuarantineEntry; superseded_remote_paths?: string[] }> {+ const { CHILD_ENV, resolveChildCliInvocation } = await import('../minions/job-isolation.ts');+ const cli = resolveChildCliInvocation(process.env, process.execPath, process.argv[1], () => null);+ if (!cli) throw opError('git_push_unavailable', `This process could not locate its own gbrain executable to publish the diverged checkout ${root}; nothing was pushed.`,+ `Set ${CHILD_ENV.childCliOverride} to the gbrain executable and retry gbrain sources refresh ${sourceId}, or publish by hand: gbrain git commit-push --push-only --keep-checkout --path ${q(root)}.`);+ const child = await execFileBounded(cli.cmd, [...cli.argsPrefix, 'git', 'commit-push', '--push-only', '--keep-checkout', '--path', root, '--json'], {+ timeout: PUBLISH_TIMEOUT_MS, termGraceMs: 10_000, maxBuffer: 1024 * 1024,+ env: { ...process.env, GBRAIN_PUBLISH_LOCK_WAIT_SECONDS: '0', GBRAIN_PUSH_LOCK_WAIT_SECONDS: '0' },+ });+ let doc: Record<string, unknown> | null = null;+ try { doc = JSON.parse(child.stdout.trim()); } catch { /* reported below */ }+ if (!child.error && doc?.outcome === 'pushed') return doc as { sha: string | null; integrated: number; quarantined?: QuarantineEntry; superseded_remote_paths?: string[] };+ if (doc && typeof doc.code === 'string' && typeof doc.message === 'string') {+ throw opError(doc.code as Parameters<typeof opError>[0], doc.message, typeof doc.suggestion === 'string' ? doc.suggestion : '',+ { ...(typeof doc.detail === 'string' ? { detail: doc.detail } : {}), ...(doc.fix ? { fix: doc.fix as Action } : {}), retryable: doc.retryable === true });+ }+ throw opError('push_unconfirmed', `The publish of the diverged checkout ${root} did not report a push (${child.error ? `exit ${String(child.error.code ?? child.error.signal ?? 'unknown')}` : `outcome ${String(doc?.outcome ?? 'unreadable')}`}: ${child.stderr.trim().slice(0, 300) || 'no detail'}); the refresh stopped before fencing anything, and the remote may already hold the merge.`,+ `Check gbrain sources writer status --source ${sourceId} --json, then retry gbrain sources refresh ${sourceId}.`, { retryable: true });+}async function divergedRefusal(git: Git, root: string, sourceId: string, upstreamRef: string, oldHead: string, targetHead: string): Promise<OperationError> {const count = Number((await git(['-C', root, 'rev-list', '--count', `${targetHead}..${oldHead}`])).stdout.trim()) || 0;const local = count ? `${count} local commit(s)` : 'the local commits';diff --git a/test/refresh-publish-diverged-7686.test.ts b/test/refresh-publish-diverged-7686.test.tsindex 5c019ca787..753a57cbdf 100644--- a/test/refresh-publish-diverged-7686.test.ts+++ b/test/refresh-publish-diverged-7686.test.ts@@ -15,11 +15,13 @@* setting, one that moves the checkout or drops the local commits while* publishing, and one that pushes or fences anything when the merge conflicts or* the refresh is about to refuse a dirty overlap.- * Seams: the `hooks.boundary` callback for the refusal after a publish; otherwise+ * Seams: the `hooks.boundary` callback for the refusal after a publish, and+ * GBRAIN_JOB_CHILD_CLI naming the child gbrain that publishes; otherwise* a real bare remote, a pushing clone and the managed checkout.*/import { afterAll, beforeAll, expect, test } from 'bun:test';-import { readFileSync, writeFileSync } from 'node:fs';+import { chmodSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';import { join } from 'node:path';import type { BrainEngine } from '../src/core/engine.ts';import { PGLiteEngine } from '../src/core/pglite-engine.ts';@@ -29,13 +31,19 @@ import { _resetCliExitVerdictForTests } from '../src/core/cli-force-exit.ts';import { REFRESH_PUBLISH_DIVERGED_KEY, refreshWorktree } from '../src/core/persistence/worktree-refresh.ts';import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';import { testBackends } from './helpers/test-backends.ts';+import { withEnv } from './helpers/with-env.ts';import { git, page, withRefreshFixture, type RefreshFixture } from './helpers/worktree-refresh-fixture.ts';const backends = testBackends();const engines: BrainEngine[] = [];let closePostgres: (() => Promise<void>) | undefined;+/** The publish runs in a child `gbrain`; under bun test that is this checkout's CLI through a wrapper. */+const bin = mkdtempSync(join(tmpdir(), 'gbrain-7686-bin-'));+const childCli = join(bin, 'gbrain');beforeAll(async () => {+ writeFileSync(childCli, `#!/bin/sh\nexec '${process.execPath}' '${join(import.meta.dir, '..', 'src', 'cli.ts')}' "$@"\n`);+ chmodSync(childCli, 0o755);if (backends.includes('pglite')) {const engine = new PGLiteEngine();await engine.connect({}); await engine.initSchema(); engines.push(engine);@@ -48,13 +56,14 @@ beforeAll(async () => {afterAll(async () => {for (const engine of engines) if (engine.kind === 'pglite') await engine.disconnect();await closePostgres?.();+ rmSync(bin, { recursive: true, force: true });});-const each = async (fn: (f: RefreshFixture) => Promise<void>) => {+const each = async (fn: (f: RefreshFixture) => Promise<void>, cli = childCli) => {for (const engine of engines) {- await withRefreshFixture(engine, async f => {+ await withEnv({ GBRAIN_JOB_CHILD_CLI: cli }, () => withRefreshFixture(engine, async f => {try { await fn(f); } finally { await engine.unsetConfig(REFRESH_PUBLISH_DIVERGED_KEY); }- });+ }));}};const refusedWith = async (promise: Promise<unknown>, code: string) => {@@ -174,6 +183,17 @@ test('setting on: a refusal after the publish says the local commits were alreadexpect(error.message).toContain(`already published the checkout's 1 local commit(s) merged with the upstream as ${remoteHead(f)}`);}), 120_000);+test('setting on: the publish runs in a child gbrain, never in this process, so an unusable child refuses and nothing is pushed or fenced', () => each(async f => {+ const local = diverge(f);+ const remote = remoteHead(f);+ await f.engine.setConfig(REFRESH_PUBLISH_DIVERGED_KEY, 'true');+ const error = await refusedWith(refreshWorktree(f.engine, f.alpha), 'push_unconfirmed');+ expect(error.message).toContain('did not report a push');+ expect(remoteHead(f)).toBe(remote);+ expect(f.head()).toBe(local);+ expect(await f.refreshRows()).toEqual([]);+}, '/bin/false'), 120_000);+test('setting on: a dry run publishes nothing and still refuses refresh_diverged', () => each(async f => {const local = diverge(f);const remote = remoteHead(f);