Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(refresh): an untyped throw in sources refresh is a typed internal_error refusal with its stack (GBRA-250)

garrytan3d2cbd6f7f6d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index debf81335f..82a31654ad 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,16 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**When `gbrain sources refresh` hits an unexpected error (a database statement error, a code fault), it now prints a typed `internal_error` refusal with the error's stack in `why` and the refresh row it left, instead of a bare `internal_error` with neither. One failure leaves enough to diagnose it, and the next run resumes the row as before.**
+
+### Itemized changes
+
+- `src/core/persistence/worktree-refresh.ts`: `refreshWorktree` turns any non-`OperationError` throw into `internal_error` with the error name, SQLSTATE and message (connection details and URLs redacted), the stack in `why`, and a fix naming the active refresh row and its state. The row is not changed. The already-current path no longer throws a `TypeError` of its own when a member sync rejects with a non-object.
+- `src/commands/sources-refresh.ts`: refusals print `why` (a `why` key under `--json`, a `Why:` line otherwise).
+- Test: `test/worktree-refresh-unexpected-throw.test.ts` (PGLite, and Postgres under `DATABASE_URL`).
+
## [0.60.528.0] - 2026-10-31
**The test-isolation scan is about 25 times faster on Linux and 12 times faster on macOS, so `bun run verify` and the macOS validation job no longer come close to their per-check caps (#8053, #8099).**
diff --git a/docs/architecture/key-files/commands-5.md b/docs/architecture/key-files/commands-5.md
index 77b0984760..2d85b2b151 100644
--- a/docs/architecture/key-files/commands-5.md
+++ b/docs/architecture/key-files/commands-5.md
@@ -45,7 +45,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/commands/sources-strategy.ts` — `gbrain sources set-strategy <id> <markdown|code|auto>` (#4899, PR #5955): writes `sources.config.strategy` through `updateSourceConfig` (single-key merge; every other config key survives), the value `sync --all`, autopilot, the dream cycle and the MCP `sync` op pass to the per-source sync when the caller names no `--strategy`; unset means `markdown`. The one writer of that key (no `sources add --strategy`, no op param). Connector kinds refuse it. [R26]: refuses with `sync_run_unfinished` (exit 1) while the source has an unfinished `managed-sync` cursor, naming the resume and the orphan repair, because the frozen manifest was classified under the old strategy; re-stating the current value is allowed. `sources list --json` and `listSources` report `strategy` (`null` when unset). Pinned by `test/sources-strategy.test.ts`.
- `src/commands/sources-add-next.ts` — GBRA-156: `noGrantFloorNext(engine, id)`, the federate step `sources add` prints when a new non-federated source leaves the no-grant floor (`localFederatedSourceIds(engine, 'default', 'seed_default')`, else `['default']`) holding no live page a remote read may show (private pages excluded when remote reads exclude them); returns `{ command, argv, why }` or null. One read, no write; `sources add` treats a failed read as no step.
-- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit,refresh,set-calendar}`; `set-calendar` dispatches to `src/commands/sources-set-calendar.ts`, and `set-cr-mode` prints `contextualModeHint` after the write. `refresh` dispatches to `src/commands/sources-refresh.ts` (argument parsing, stdout text or `--json` result/refusal objects, exit 1 on refusal or an unfinished sync; protocol in `src/core/persistence/worktree-refresh.ts`). `current [--json]` calls `resolveSourceWithTier()` and prints `source_id`, `tier` (`flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail` (decision table in `skills/conventions/brain-routing.md`). `status [--json]` — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` from `src/commands/sync.ts`; `--json` emits stable `{schema_version: 1, sources, ...}` on stdout; filters input to `local_path IS NOT NULL AND archived IS NOT TRUE`. `audit <id> [--json]` — read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). The live `runStatus` health table gains a `BACKFILL` column between `EMBED` and `FAILS` (`active(N)` beats `queued(N)` beats `idle`, from `SourceMetrics.backfill_active` / `backfill_queued` in `src/core/source-health.ts`) so operators see deferred `embed-backfill` minion work after `sync --all` exits 0; `jobCountsBySource` in `source-health.ts` widens its `minion_jobs` SQL with two `COUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...)` aggregates (best-effort, all-0 on pre-minions brains). `remove` pre-checks OAuth-client referents (guided refusal via `formatClientReferentsBlock` — the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (`unhardenBrainRepo`: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact. `purge` runs the same referents pre-check before its hard DELETE. `add` without `--federated` or `--no-federated` stores the source non-federated and warns on stderr that a remote token or agent connection with no source grant cannot read it, naming `gbrain sources federate <id>` and `gbrain auth rescope --token <name> --sources <id>`; when the no-grant floor (the default source plus federated sources) then holds no live page it also prints `next: gbrain sources federate <id>` on stdout (`noGrantFloorNext` in `sources-add-next.ts`). `add --json` prints one object (`id`, `name`, `local_path`, `remote_url`, `federated`, and `next` with `command`/`argv`/`why` when that step applies) and moves the human lines to stderr (`test/sources-add-federation-warning.test.ts`). Pinned by `test/content-sanity.test.ts`, `test/import-file-content-sanity.test.ts`, `test/source-health.test.ts`.
+- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit,refresh,set-calendar}`; `set-calendar` dispatches to `src/commands/sources-set-calendar.ts`, and `set-cr-mode` prints `contextualModeHint` after the write. `refresh` dispatches to `src/commands/sources-refresh.ts` (argument parsing, stdout text or `--json` result/refusal objects, exit 1 on refusal or an unfinished sync; protocol in `src/core/persistence/worktree-refresh.ts`, where an untyped throw becomes an `internal_error` refusal whose `why` carries the stack and whose fix names the refresh row it left for the next run to resume, GBRA-250). `current [--json]` calls `resolveSourceWithTier()` and prints `source_id`, `tier` (`flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail` (decision table in `skills/conventions/brain-routing.md`). `status [--json]` — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` from `src/commands/sync.ts`; `--json` emits stable `{schema_version: 1, sources, ...}` on stdout; filters input to `local_path IS NOT NULL AND archived IS NOT TRUE`. `audit <id> [--json]` — read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). The live `runStatus` health table gains a `BACKFILL` column between `EMBED` and `FAILS` (`active(N)` beats `queued(N)` beats `idle`, from `SourceMetrics.backfill_active` / `backfill_queued` in `src/core/source-health.ts`) so operators see deferred `embed-backfill` minion work after `sync --all` exits 0; `jobCountsBySource` in `source-health.ts` widens its `minion_jobs` SQL with two `COUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...)` aggregates (best-effort, all-0 on pre-minions brains). `remove` pre-checks OAuth-client referents (guided refusal via `formatClientReferentsBlock` — the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (`unhardenBrainRepo`: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact. `purge` runs the same referents pre-check before its hard DELETE. `add` without `--federated` or `--no-federated` stores the source non-federated and warns on stderr that a remote token or agent connection with no source grant cannot read it, naming `gbrain sources federate <id>` and `gbrain auth rescope --token <name> --sources <id>`; when the no-grant floor (the default source plus federated sources) then holds no live page it also prints `next: gbrain sources federate <id>` on stdout (`noGrantFloorNext` in `sources-add-next.ts`). `add --json` prints one object (`id`, `name`, `local_path`, `remote_url`, `federated`, and `next` with `command`/`argv`/`why` when that step applies) and moves the human lines to stderr (`test/sources-add-federation-warning.test.ts`). Pinned by `test/content-sanity.test.ts`, `test/import-file-content-sanity.test.ts`, `test/source-health.test.ts`.
- `src/commands/status-funnel.ts` — `gbrain status --funnel [--json] [--enable | --delete]` (D8 adoption funnel): engine-free, dispatched from `src/cli/main.ts` before any connect so it works before `gbrain init` and on a thin client; reads and writes only the local record through `src/core/funnel.ts`. Guide: `docs/guides/adoption-funnel.md`. Tests: `test/funnel.test.ts`, `test/funnel-cli.test.ts`.
diff --git a/src/commands/sources-refresh.ts b/src/commands/sources-refresh.ts
index b3d2f3f857..b658f8bb2f 100644
--- a/src/commands/sources-refresh.ts
+++ b/src/commands/sources-refresh.ts
@@ -74,10 +74,10 @@ export function printRefreshOutcome(outcome: { result: WorktreeRefreshResult } |
// #7685: a refusal that carries a structured fix (argv, actor, ask_user) emits it as the v1 `fix`; the prose stays in `suggestion`.
const env = toAgentError(error, { transport: 'cli', command: 'sources refresh', render: cliRenderContext() });
const refusal = { status: 'refused', code: error.code, cause: error.message, fix: error.fix && env.fix ? env.fix : error.suggestion ?? null, docs: error.docs ?? null,
- ...(error.detail ? { detail: error.detail } : {}),
+ ...(error.detail ? { detail: error.detail } : {}), ...(error.why ? { why: error.why } : {}),
error: env.error, message: env.message, suggestion: env.suggestion, docs_cmd: env.docs_cmd, class: env.class, retryable: env.retryable,
contract_version: env.contract_version };
- console.log(json ? JSON.stringify(refusal) : `Refresh refused (${error.code}): ${error.message}\nFix: ${error.suggestion ?? USAGE}${error.docs ? `\nDocs: ${error.docs}` : ''}`);
+ console.log(json ? JSON.stringify(refusal) : `Refresh refused (${error.code}): ${error.message}\nFix: ${error.suggestion ?? USAGE}${error.docs ? `\nDocs: ${error.docs}` : ''}${error.why ? `\nWhy: ${error.why}` : ''}`);
setCliExitVerdict(1);
}
diff --git a/src/core/persistence/worktree-refresh.ts b/src/core/persistence/worktree-refresh.ts
index 93aab6ef82..c83ad12842 100644
--- a/src/core/persistence/worktree-refresh.ts
+++ b/src/core/persistence/worktree-refresh.ts
@@ -42,6 +42,8 @@ import { startPersistenceConsumer } from './service.ts';
import { lockTopologyPrincipal, lockTopologyRows, topologyPrincipal, withTopologyLocks } from './topology-locks.ts';
import { ACTIVE_REFRESH_STATES_SQL, CHECKOUT_EFFECT_KINDS_SQL, type WorktreeRefreshState } from './worktree-refresh-schema.ts';
import { gitChildEnv } from '../git-env.ts';
+import { redactConnectionInfo } from '../audit/redact-connection-info.ts';
+import { redactUrlsInText } from '../url-redact.ts';
import { classifyOriginUniqueViolation } from './origin-guard.ts';
export interface WorktreeRefreshRow {
@@ -482,6 +484,27 @@ function finished(row: WorktreeRefreshRow, sync: { synced: RefreshMemberSync[];
* host only; refusals throw catalogue errors (`code`, cause, literal fix, docs).
*/
export async function refreshWorktree(engine: BrainEngine, sourceId: string, opts: RefreshOptions = {}): Promise<WorktreeRefreshResult> {
+ try { return await runRefresh(engine, sourceId, opts); }
+ catch (error) { throw error instanceof OperationError ? error : await unexpectedRefreshFailure(engine, sourceId, error); }
+}
+
+/**
+ * GBRA-250: an untyped throw (a statement error, a TypeError) leaves the refresh where it stood, so a fenced or syncing row
+ * resumes on the next run. The refusal names that row and carries the stack in `why`, so a failure seen once leaves a trace.
+ */
+async function unexpectedRefreshFailure(engine: BrainEngine, sourceId: string, error: unknown): Promise<OperationError> {
+ const thrown = error instanceof Error ? error : new Error(String(error));
+ const sqlstate = typeof (error as { code?: unknown } | null)?.code === 'string' ? ` (${(error as { code: string }).code})` : '';
+ const redact = (text: string) => redactUrlsInText(redactConnectionInfo(text));
+ const binding = await getWorktreeBinding(engine, sourceId, existingLocalHostId()).catch(() => null);
+ const active = binding ? await readActiveRefresh(engine, binding.worktree_id).catch(() => null) : null;
+ const left = active ? `Refresh ${active.id} stays ${active.state}, and the next gbrain sources refresh ${sourceId} resumes it.` : 'No refresh row was left active.';
+ return opError('internal_error', redact(`The refresh of source ${sourceId} failed unexpectedly: ${thrown.name}${sqlstate}: ${thrown.message}`),
+ `${left} Run gbrain sources refresh ${sourceId} again; if it fails the same way, report this output (why holds the stack) with gbrain doctor --json.`,
+ { why: redact(thrown.stack ?? `${thrown.name}: ${thrown.message}`) });
+}
+
+async function runRefresh(engine: BrainEngine, sourceId: string, opts: RefreshOptions): Promise<WorktreeRefreshResult> {
const syncBudgetMs = await configuredMs(engine, opts.syncBudgetMs, 'GBRAIN_REFRESH_SYNC_BUDGET_MS', 'sources.refresh_sync_budget_ms', SYNC_BUDGET_MS);
if (opts.resume || opts.abandon) return resumeOrAbandon(engine, sourceId, opts, syncBudgetMs);
const fetchTimeoutMs = await configuredMs(engine, opts.fetchTimeoutMs, 'GBRAIN_REFRESH_FETCH_TIMEOUT_MS', 'sources.refresh_fetch_timeout_ms', 120_000, 2 ** 31 - 1);
@@ -512,7 +535,8 @@ async function refreshOnce(engine: BrainEngine, sourceId: string, opts: RefreshO
if (result.blocked) blocked.push(result.blocked);
synced.push({ source_id: member, status: result.status, last_commit: await lastCommit(engine, member) });
} catch (error) {
- blocked.push({ source_id: member, code: (error as OperationError).code ?? 'storage_error', message: (error as Error).message, resume: `gbrain sync --source ${member} --no-pull` });
+ blocked.push({ source_id: member, code: error instanceof OperationError ? error.code : 'storage_error', message: error instanceof Error ? error.message : String(error),
+ resume: `gbrain sync --source ${member} --no-pull` });
}
}
const deferred = synced.some(member => member.status === 'deferred');
diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txt
index 306d1894d5..d07fe34b9a 100644
--- a/test/postgres-unit-arms.txt
+++ b/test/postgres-unit-arms.txt
@@ -548,6 +548,7 @@ test/visibility-repair-scale.test.ts
test/withdrawal-overlay-concurrency.test.ts
test/withdrawal-publication-file.test.ts
test/worktree-refresh-recovery-row-gbra130.test.ts
+test/worktree-refresh-unexpected-throw.test.ts
test/write-roundtrips-gbra156.test.ts
test/write-through-leak-guard-alias-7203.test.ts
test/writer-admin-state-embed-lease.test.ts
diff --git a/test/worktree-refresh-unexpected-throw.test.ts b/test/worktree-refresh-unexpected-throw.test.ts
new file mode 100644
index 0000000000..ad2b98ae44
--- /dev/null
+++ b/test/worktree-refresh-unexpected-throw.test.ts
@@ -0,0 +1,69 @@
+/**
+ * GBRA-250: an untyped throw inside `gbrain sources refresh` (a statement error,
+ * a TypeError) escaped runSourcesRefresh to the CLI fatal seam as a bare
+ * internal_error with no stack and no refresh state, so a one-off failure left
+ * nothing to diagnose. It is now a typed internal_error refusal whose `why`
+ * carries the stack and whose fix names the refresh row it left, which the next
+ * run resumes.
+ */
+import { afterAll, beforeAll, expect, test } from 'bun:test';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { OperationError } from '../src/core/ops/contract.ts';
+import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';
+import { runSourcesRefresh } from '../src/commands/sources-refresh.ts';
+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';
+import { refreshWorktree } from '../src/core/persistence/worktree-refresh.ts';
+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';
+import { page, withRefreshFixture } from './helpers/worktree-refresh-fixture.ts';
+import { withEnv } from './helpers/with-env.ts';
+
+let engine: BrainEngine;
+let close: (() => Promise<void>) | undefined;
+beforeAll(async () => {
+ if (process.env.DATABASE_URL) { const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL); engine = pg.engine; close = pg.close; return; }
+ engine = new PGLiteEngine();
+ await (engine as PGLiteEngine).connect({});
+ await engine.initSchema();
+}, 120_000);
+afterAll(async () => {
+ await disposePersistenceConsumer(engine);
+ if (close) await close(); else await (engine as PGLiteEngine).disconnect();
+});
+
+test('an untyped throw after the merge is a typed refusal with its stack, and the syncing row it leaves resumes', () => withRefreshFixture(engine, async f => {
+ const target = f.push('alpha/two.md', page('Alpha two', 'Upstream side.'));
+ const boom = new TypeError('member bookkeeping exploded');
+ const failure = await refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === 'syncing') throw boom; } } }).then(() => null, error => error);
+ expect(failure).toBeInstanceOf(OperationError);
+ expect(failure.code).toBe('internal_error');
+ expect(failure.message).toContain('TypeError: member bookkeeping exploded');
+ expect(failure.why).toContain(boom.stack!.split('\n')[1]!.trim());
+ const [row] = await f.refreshRows();
+ expect(row!.state).toBe('syncing');
+ expect(failure.suggestion).toContain(`Refresh ${row!.id} stays syncing`);
+ expect(f.head()).toBe(target);
+ const resumed = await refreshWorktree(f.engine, f.alpha);
+ expect(resumed.recovered?.refresh_id).toBe(row!.id);
+ expect(await f.lastCommit(f.alpha)).toBe(target);
+ expect((await f.refreshRows())[0]!.state).toBe('completed');
+}), 120_000);
+
+test('sources refresh prints an untyped throw as a refused document carrying the stack, and exits 1', async () => {
+ const statement = Object.assign(new Error('canceling statement due to user request'), { code: '57014' });
+ const fake = { kind: 'postgres', getConfig: async () => { throw statement; }, executeRaw: async () => { throw statement; } } as unknown as BrainEngine;
+ const lines: string[] = [];
+ const log = console.log;
+ console.log = (line: unknown) => { lines.push(String(line)); };
+ try {
+ await withEnv({ GBRAIN_REFRESH_SYNC_BUDGET_MS: undefined }, () => runSourcesRefresh(fake, ['source-a', '--json']));
+ expect(currentExitCode()).toBe(1);
+ } finally { console.log = log; _resetCliExitVerdictForTests(); process.exitCode = 0; }
+ expect(lines).toHaveLength(1);
+ const doc = JSON.parse(lines[0]!);
+ expect(doc.status).toBe('refused');
+ expect(doc.code).toBe('internal_error');
+ expect(doc.cause).toContain('Error (57014): canceling statement due to user request');
+ expect(doc.why).toContain(statement.stack!.split('\n')[1]!.trim());
+ expect(doc.fix).toContain('gbrain sources refresh source-a');
+});