Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: a webhook capture never falls back to the default source
garrytanbbd6b5ffab00
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 10f316ca3b..e07aa0b06e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -32,6 +32,7 @@ Hermes transcript snapshots no longer fail while Hermes is writing, and `gbrain- A live fact row an older build linked as superseded is cleared by a full reconcile (`gbrain dream --phase extract_facts --source <id>`); the nightly cycle clears it only when the page changes.- Duplicate open loops written before this change and person-to-company links already typed from a machine-written section stay as they are until their thread or page is processed again.- `gbrain takes list --status <x>` used to be ignored silently and now exits 1, pointing to `--expired` and `gbrain takes propose --json`.+- A `POST /ingest` capture whose credential's write source is archived or removed before the capture lands now fails its job with `source_changed` (`detail: webhook_source_unavailable`) and writes nothing, where it used to be written under `default`, a source the credential may not be granted. Restore the source and run `gbrain jobs retry` on the job. Captures that do not come from a webhook keep the `default` fallback.### Itemized changes@@ -86,6 +87,7 @@ Hermes transcript snapshots no longer fail while Hermes is writing, and `gbrain#### Webhook captures+- `src/core/minions/handlers/ingest-capture.ts`: a webhook capture (`untrusted_payload`, `source_kind: webhook`) whose write source is archived, unregistered or removed mid-write refuses `source_changed` (`detail: webhook_source_unavailable`) instead of writing under `default`, on managed and unmanaged brains. Test: `test/ingest-capture-no-default-fallback.test.ts`.- `src/core/minions/handlers/ingest-capture.ts`: the job normalizes its slug once (`validateSlug`) before its snapshot, applied-record and writer lookups, so a mixed-case `X-Gbrain-Slug` naming an existing page on a managed brain replaces that page instead of dead-lettering with `revision_required`. Test: `test/ingest-capture-mixed-case-slug.test.ts`.## [0.60.660.0] - 2026-11-11diff --git a/docs/integrations/qm-harness.md b/docs/integrations/qm-harness.mdindex 7dc73e1fdc..8cec840940 100644--- a/docs/integrations/qm-harness.md+++ b/docs/integrations/qm-harness.md@@ -107,9 +107,10 @@ boundary, and it does not make every side effect prefix-clean:write source is resolved server-side from `oauth_clients.source_id`; thecaller-supplied `X-Gbrain-Source-Id` header does not route anything, so aclient cannot choose its own write partition. A client registered without a- source writes to `default`, and a write falls back to `default` if its- source is unregistered, archived, or deleted mid-write (the job result- reports this via `source_fallback`). If you point a webhook integration at+ source writes to `default`. If its source is unregistered, archived, or+ deleted before the capture lands, the job fails with `source_changed`+ (`detail: webhook_source_unavailable`) and nothing is written; restore the+ source and run `gbrain jobs retry` on the job. If you point a webhook integration atthis route, give its client an explicit `--source` so its captures do notaccumulate in the brain's `default` source. Bound clients are still refusedthe route outright, because the slug fence cannot run on this path.diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex 3b2399052c..6199cb7fb8 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1516,6 +1516,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.655.0', id: 'member_email_link_gst1158', text: 'A member can carry an operator-declared email: `gbrain access member add <id> --email <addr>` or `gbrain access member set <id> --email <addr>|none`, shown by `member list`, stored lower-case and unique per brain, and cleared when the member is removed. `gbrain access member link --email <addr> --qm <id>` (and `gbrain mcp admin link --email`, `member-add --email`, `member-set` over the owner session) links the one member that declares it, exactly as linking by member id does; an unknown email is `not_found` and creates nothing. No OAuth token, attested call or MCP tool can set an email or make a link.' },{ since: '0.60.655.0', id: 'qm_admin_http_gst1094', text: 'The brain owner can now manage the attest right, members and canonical-id links over the running server: `gbrain mcp admin attest <client> --attest-actor qm|none [--attest-ops <list>]`, `member-add`, `member-disable`, `member-enable`, `members`, `link <member> --qm <id>`, `unlink --qm <id>` and `links`, backed by `/admin/api/access/attest`, `/admin/api/access/members` and `/admin/api/access/member-links`. These routes take the owner session only (never an OAuth token or an MCP tool), refuse what the local `gbrain auth rescope` and `gbrain access member` commands refuse, preview with `--dry-run` where those do, and log each applied change as `access_admin`. Removing a member stays a brain-host command.' },{ since: 'TBD', id: 'github_item_webhook_backlog_8799', text: 'A GitHub webhook for an item event (issue, pull request, comment, review, check) now answers HTTP 429 `webhook_backlog_full` with `Retry-After: 30` when 50 item jobs are already waiting for that source and none of them covers the event; it used to answer 202 with another item\'s job while the event was dropped. GitHub does not redeliver a 429 on its own: redeliver it from the webhook\'s Recent Deliveries, or let the next sweep or `gbrain sync --source <id>` catch it.' },+ { since: 'TBD', id: 'webhook_capture_no_default_fallback_gst2113', text: 'A webhook capture (`POST /ingest` and its `ingest_capture` job) whose credential\'s write source is archived, unregistered or removed before the capture lands now fails the job with `source_changed` (`detail: webhook_source_unavailable`) and writes nothing. Before, it was written under `default`, a source the credential may not be granted, and could replace a page there. Restore the source and run `gbrain jobs retry` on the job. Captures that do not come from a webhook keep the `default` fallback.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/minions/handlers/ingest-capture.ts b/src/core/minions/handlers/ingest-capture.tsindex b80e9c9d58..73e543559d 100644--- a/src/core/minions/handlers/ingest-capture.ts+++ b/src/core/minions/handlers/ingest-capture.ts@@ -293,6 +293,16 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {if (applied) return applied;const webhookWrite = untrustedPayload && event.source_kind === 'webhook' && !isTombstone;+ // GST-2113: a webhook capture writes only inside its credential's write source. When that source is archived or+ // removed after the 202, default may be outside the grant, so the job refuses (kept for gbrain jobs retry) instead+ // of writing there.+ const refuseWebhookFallback = (fallback: NonNullable<IngestCaptureResult['source_fallback']>) => {+ if (!webhookWrite) return;+ throw opError('source_changed', `The webhook capture for ${slug} was not written: its write source ${fallback.requested} is ${fallback.reason === 'archived' ? 'archived' : 'no longer registered'}, and a capture never falls back to default, which its credential may not be granted.`,+ `Restore source ${fallback.requested} and run gbrain jobs retry on this job, or send the capture again with a credential whose write source is live.`,+ { detail: 'webhook_source_unavailable' });+ };+ if (sourceFallback) refuseWebhookFallback(sourceFallback);const checkWriter = (sid: string) => webhookWrite ? assertWebhookWriterMayReplace(engine, data.writer, sid, slug, event.content) : Promise.resolve(undefined);let checkedRevision = await checkWriter(sourceId ?? 'default');const door = (operation: 'put_page' | 'delete_page', sid: string, params: Record<string, unknown>) => submitPageMutation(@@ -376,6 +386,7 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {? await engine.executeRaw<{ archived: boolean | null }>('SELECT archived FROM sources WHERE id = $1', [sourceId]) : null;if (!vanished || (vanished.length > 0 && vanished[0]?.archived !== true)) throw err;sourceFallback = { requested: sourceId!, effective: 'default', reason: vanished.length ? 'archived' : 'fk_violation' };+ refuseWebhookFallback(sourceFallback);await checkWriter('default');response = await put('default', requestId && `${requestId}:default`);}@@ -411,6 +422,7 @@ export function makeIngestCaptureHandler(engine: BrainEngine) {// The sources row can vanish (ON DELETE CASCADE) between the pre-check and this write.if (sourceId !== undefined && isPagesSourceFkViolation(err)) {sourceFallback = { requested: sourceId, effective: 'default', reason: 'fk_violation' };+ refuseWebhookFallback(sourceFallback);checkedRevision = await checkWriter('default');result = await importAs(undefined);} else {diff --git a/test/ingest-capture-applied-once.test.ts b/test/ingest-capture-applied-once.test.tsindex bd652585c6..72da15f4f0 100644--- a/test/ingest-capture-applied-once.test.ts+++ b/test/ingest-capture-applied-once.test.ts@@ -241,7 +241,9 @@ for (const backend of backends) {});const slug = `notes/fk-race-${backend}`;const key = `ingest:webhook:client-example:${racingSource}:fk-race`;- const result = await makeIngestCaptureHandler(racing)(jobCtx(5, captureData(note('Racing capture', 'Racing body.'), slug, racingSource), key));+ // A webhook capture refuses instead of falling back (GST-2113); a capture of another kind keeps the fallback.+ const data = captureData(note('Racing capture', 'Racing body.'), slug, racingSource);+ const result = await makeIngestCaptureHandler(racing)(jobCtx(5, { ...data, event: { ...data.event, source_kind: 'url-fetch' } }, key));expect(attempts).toBe(2);expect(result.status).toBe('imported');expect(result.source_fallback).toEqual({ requested: racingSource, effective: 'default', reason: 'fk_violation' });diff --git a/test/ingest-capture-no-default-fallback.test.ts b/test/ingest-capture-no-default-fallback.test.tsnew file mode 100644index 0000000000..34fb8ef809--- /dev/null+++ b/test/ingest-capture-no-default-fallback.test.ts@@ -0,0 +1,137 @@+/**+ * GST-2113: a webhook capture (POST /ingest, then its ingest_capture job) writes only inside its credential's write source.+ *+ * When the granted source was archived or removed after the 202, the job wrote the capture under default, where the+ * credential may have no grant, replacing an unrestricted default page at the same slug.+ * Protects (real token verifier and Express route for the 202, the real handler, PGLite):+ * - a webhook job whose granted source was archived after the 202 refuses source_changed and leaves default untouched;+ * - a webhook job whose source row vanished mid-write (pages source FK) refuses the same way;+ * - on a managed brain, a source archived between the job's check and the door's put refuses and leaves default untouched.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import express from 'express';+import { mkdirSync } from 'node:fs';+import type { Server } from 'node:http';+import type { AddressInfo } from 'node:net';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { importFromContent } from '../src/core/import-file.ts';+import { mintLegacyToken } from '../src/core/token-mint.ts';+import { mountWebhooks } from '../src/commands/serve-http-webhooks.ts';+import type { ServeHttpContext } from '../src/commands/serve-http.ts';+import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';+import { sqlQueryForEngine } from '../src/core/sql-query.ts';+import { computeContentHash } from '../src/core/ingestion/types.ts';+import { makeIngestCaptureHandler } from '../src/core/minions/handlers/ingest-capture.ts';+import type { MinionJobContext } from '../src/core/minions/types.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { claimWorktree } from '../src/core/persistence/ownership.ts';+import { managedBrain } from './helpers/managed-brain.ts';++const SRC = 'source-a';+const note = (title: string, line: string) => `---\ntype: note\ntitle: ${title}\n---\n# ${title}\n\n${line}\n`;+const jobCtx = (id: number, data: Record<string, unknown>): MinionJobContext => ({ id, name: 'ingest_capture', data,+ attempts_made: 0, idempotency_key: null, signal: new AbortController().signal, deadlineAtMs: null, shutdownSignal: new AbortController().signal,+ updateProgress: async () => {}, updateTokens: async () => {}, log: async () => {}, isActive: async () => true, readInbox: async () => [] });+const captureData = (content: string, slug: string, sourceId: string) => ({ slug, sourceId, event: { source_id: 'webhook-client-example', source_kind: 'webhook',+ source_uri: 'mcp-webhook:client-example:1', received_at: new Date().toISOString(), content_type: 'text/markdown', content,+ content_hash: computeContentHash(content), untrusted_payload: true } });+const text = async (engine: BrainEngine, slug: string, sourceId: string) => (await engine.readPageSnapshot(slug, { sourceId }))?.page.compiled_truth ?? null;+const refused = { code: 'source_changed', detail: 'webhook_source_unavailable' };++describe('a webhook capture never falls back to default (GST-2113)', () => {+ let engine: PGLiteEngine;+ let server: Server;+ let base: string;+ let token: string;++ beforeAll(async () => {+ engine = new PGLiteEngine();+ await engine.connect({});+ await engine.initSchema();+ token = (await mintLegacyToken(engine, { name: 'granted-writer', scopes: ['read', 'write'], takesHolders: ['world'], sourceGrant: [SRC] })).token;+ const provider = new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine), transaction: fn => engine.transaction(tx => fn(sqlQueryForEngine(tx))) });+ const app = express();+ const pass: express.RequestHandler = (_req, _res, next) => next();+ mountWebhooks(app, { engine, resourceVerifier: provider, resourceMetadataUrl: 'http://127.0.0.1/.well-known/oauth-protected-resource',+ ingestRateLimiter: pass, githubWebhookLimiter: pass, broadcastEvent: () => {} } as unknown as ServeHttpContext);+ server = await new Promise<Server>(resolve => { const s = app.listen(0, '127.0.0.1', () => resolve(s)); });+ base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;+ }, 60_000);+ afterAll(async () => {+ await new Promise<void>(resolve => server.close(() => resolve()));+ await engine.disconnect();+ }, 30_000);++ test('a job whose granted source was archived after the 202 refuses and leaves the default page as it was', async () => {+ await engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT (id) DO NOTHING', [SRC]);+ const slug = 'notes/shared-example';+ await importFromContent(engine, slug, note('Shared example', 'Default body.'), { noEmbed: true });+ const res = await fetch(`${base}/ingest`, { method: 'POST',+ headers: { authorization: `Bearer ${token}`, 'content-type': 'text/markdown', 'x-gbrain-slug': slug },+ body: note('Shared example', 'Captured body.') });+ expect(res.status).toBe(202);+ const jobId = (await res.json() as { job_id: number }).job_id;+ await engine.executeRaw('UPDATE sources SET archived = true WHERE id = $1', [SRC]);+ const [row] = await engine.executeRaw<{ data: unknown }>('SELECT data FROM minion_jobs WHERE id = $1', [jobId]);+ const data = (typeof row!.data === 'string' ? JSON.parse(row!.data) : row!.data) as Record<string, unknown>;+ expect(data.sourceId).toBe(SRC);++ await expect(makeIngestCaptureHandler(engine)(jobCtx(jobId, data))).rejects.toMatchObject(refused);+ expect(await text(engine, slug, 'default')).toContain('Default body.');+ expect(await text(engine, slug, 'default')).not.toContain('Captured body.');+ await engine.executeRaw('UPDATE sources SET archived = false WHERE id = $1', [SRC]);+ });++ test('a job whose source row vanished mid-write refuses and leaves the default page as it was', async () => {+ const racingSource = 'racing-example';+ await engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT (id) DO NOTHING', [racingSource]);+ const slug = 'notes/raced-example';+ await importFromContent(engine, slug, note('Raced example', 'Default body.'), { noEmbed: true });+ let attempts = 0;+ const racing = new Proxy(engine as BrainEngine, { get(target, prop) {+ if (prop === 'transaction') return async <T>(fn: (tx: BrainEngine) => Promise<T>): Promise<T> => {+ if (++attempts === 1) throw Object.assign(new Error('insert or update on table "pages" violates foreign key constraint "pages_source_id_fk"'), { code: '23503' });+ return target.transaction(fn);+ };+ const value = Reflect.get(target, prop, target);+ return typeof value === 'function' ? value.bind(target) : value;+ } });++ await expect(makeIngestCaptureHandler(racing)(jobCtx(2, captureData(note('Raced example', 'Captured body.'), slug, racingSource)))).rejects.toMatchObject(refused);+ expect(attempts).toBe(1);+ expect(await text(engine, slug, 'default')).toContain('Default body.');+ });+});++describe('a managed webhook capture never falls back to default (GST-2113)', () => {+ test('a source archived before the door\'s put refuses and leaves the default page as it was', () => managedBrain(async ({ engine, ctx }) => {+ const slug = 'notes/managed-shared-example';+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug, content: note('Managed shared', 'Default body.') } });+ let archived = false;+ const archiving = new Proxy(engine, { get(target, prop) {+ if (prop === 'readPageSnapshot') return async (...args: Parameters<BrainEngine['readPageSnapshot']>) => {+ if (!archived && args[1]?.sourceId === SRC) {+ archived = true;+ await target.transaction(async tx => {+ await tx.executeRaw("SELECT set_config('gbrain.topology_change', 'on', true)");+ await tx.executeRaw('UPDATE sources SET archived = true, archived_at = now() WHERE id = $1', [SRC]);+ });+ }+ return target.readPageSnapshot(...args);+ };+ const value = Reflect.get(target, prop, target);+ return typeof value === 'function' ? value.bind(target) : value;+ } });++ await expect(makeIngestCaptureHandler(archiving)(jobCtx(3, captureData(note('Managed shared', 'Captured body.'), slug, SRC)))).rejects.toMatchObject(refused);+ expect(archived).toBe(true);+ expect(await text(engine, slug, 'default')).toContain('Default body.');+ expect(await text(engine, slug, 'default')).not.toContain('Captured body.');+ }, { setup: async ({ engine, root }) => {+ const sourceRoot = join(root, '..', 'source-a'); mkdirSync(sourceRoot);+ await engine.executeRaw('INSERT INTO sources (id, name, local_path) VALUES ($1, $1, $2) ON CONFLICT (id) DO NOTHING', [SRC, sourceRoot]);+ await claimWorktree(engine, SRC, sourceRoot);+ } }), 120_000);+});diff --git a/test/ingestion/ingest-capture.test.ts b/test/ingestion/ingest-capture.test.tsindex 87ecf86a46..016bfbac2c 100644--- a/test/ingestion/ingest-capture.test.ts+++ b/test/ingestion/ingest-capture.test.ts@@ -295,12 +295,12 @@ describe('ingest_capture handler — write-source attribution', () => {expect(result.source_fallback).toBeUndefined();});- test('job.data.sourceId naming a missing source falls back to default', async () => {+ test('job.data.sourceId naming a missing source falls back to default for a non-webhook capture', async () => {const handler = makeIngestCaptureHandler(engine);const ev = makeEvent({content: '# missing scoped source',source_id: 'webhook-client-x',- untrusted_payload: true,+ source_kind: 'url-fetch', untrusted_payload: true,});const result = await handler(makeJob({@@ -317,7 +317,7 @@ describe('ingest_capture handler — write-source attribution', () => {});});- test('job.data.sourceId naming an archived source falls back to default', async () => {+ test('job.data.sourceId naming an archived source falls back to default for a non-webhook capture', async () => {await engine.executeRaw(`INSERT INTO sources (id, name, archived) VALUES ('archived-source', 'archived-source', true)`,);@@ -325,7 +325,7 @@ describe('ingest_capture handler — write-source attribution', () => {const ev = makeEvent({content: '# archived scoped source',source_id: 'webhook-client-x',- untrusted_payload: true,+ source_kind: 'url-fetch', untrusted_payload: true,});const result = await handler(makeJob({@@ -342,7 +342,7 @@ describe('ingest_capture handler — write-source attribution', () => {});});- test('foreign-key violation retries once under default', async () => {+ test('foreign-key violation retries a non-webhook capture once under default', async () => {await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ('racing-source', 'racing-source')`,);@@ -366,7 +366,7 @@ describe('ingest_capture handler — write-source attribution', () => {const ev = makeEvent({content: '# source deleted during write',source_id: 'webhook-client-x',- untrusted_payload: true,+ source_kind: 'url-fetch', untrusted_payload: true,});const result = await handler(makeJob({@@ -451,7 +451,7 @@ describe('ingest_capture handler — write-source attribution', () => {content: '# provenance across retry',source_id: 'webhook-client-x',source_uri: 'https://example.com/retry-provenance',- untrusted_payload: true,+ source_kind: 'url-fetch', untrusted_payload: true,});const result = await handler(makeJob({@@ -602,7 +602,7 @@ describe('ingest_capture handler — write-source attribution', () => {const ev = makeEvent({content: '# structured fk fields',source_id: 'webhook-client-x',- untrusted_payload: true,+ source_kind: 'url-fetch', untrusted_payload: true,});const result = await handler(makeJob({@@ -658,7 +658,7 @@ describe('ingest_capture handler — write-source attribution', () => {const sharedContent = '# same content two sources';await handler(makeJob({- event: makeEvent({ content: sharedContent, untrusted_payload: true }),+ event: makeEvent({ content: sharedContent, source_kind: 'url-fetch', untrusted_payload: true }),slug,sourceId: 'shadow-source',}));@@ -666,7 +666,7 @@ describe('ingest_capture handler — write-source attribution', () => {// Same slug + same content, but this client falls back to default.const result = await handler(makeJob({- event: makeEvent({ content: sharedContent, untrusted_payload: true }),+ event: makeEvent({ content: sharedContent, source_kind: 'url-fetch', untrusted_payload: true }),slug,sourceId: 'never-registered-source',}));@@ -680,7 +680,7 @@ describe('ingest_capture handler — write-source attribution', () => {// And the differing-content variant must not throw out of createVersion.const updated = await handler(makeJob({- event: makeEvent({ content: '# different content entirely', untrusted_payload: true }),+ event: makeEvent({ content: '# different content entirely', source_kind: 'url-fetch', untrusted_payload: true }),slug,sourceId: 'never-registered-source',}));