Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Review fixes: ingest log fails closed on mixed and page-ref rows, entity lookups never read past an unreadable page
garrytan3653e326dbaa
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 90bb0a0083..c4a67c2622 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,13 +16,13 @@ identifiers and attribution are available in the pre-removal Git revision### Itemized changes-- `src/core/persistence/page-visibility.ts`: `remoteTargetHidden` tests the caller's whole read policy (member access, then privacy with its grant and hatch), the policy `get_page` applies, instead of the member-access part alone; `remoteHiddenPages` and `remoteReadPolicySql` give the same test to a batch of slugs. `requireWritablePage` (so `put_raw_data`, `add_link` on either endpoint and the managed `add_timeline_entry` preflight) and `remove_link` use it; a delegated subagent write keeps its own privacy refusal.-- `src/core/ops/extraction.ts`, `src/core/enrichment-service.ts`, `src/core/trust/taint.ts`: `extract_entities` treats a source page its remote caller cannot read as missing (no backlink, no trust read from it), refuses with `page_not_found` to update an entity page the caller cannot read (as a managed brain's timeline write already did), and counts mentions only on pages the caller can read.+- `src/core/persistence/page-visibility.ts`: `remoteTargetHidden` tests the caller's whole read policy (member access, then privacy with its grant and hatch), the policy `get_page` applies, instead of the member-access part alone; `remotePageAccess` and `remoteReadPolicySql` give the same test to a batch of slugs. `requireWritablePage` (so `put_raw_data`, `add_link` on either endpoint and the managed `add_timeline_entry` preflight) and `remove_link` use it; a delegated subagent write keeps its own privacy refusal.+- `src/core/ops/extraction.ts`, `src/core/enrichment-service.ts`, `src/core/trust/taint.ts`: `extract_entities` treats a source page its remote caller cannot read as missing (no backlink, no trust read from it), skips an entity whose page the caller cannot read (`skipped_reason: page_not_found`, under the slug its name maps to, nothing written) and carries on with the rest of the batch, follows a retired entity page's redirect only through pages the caller can read, and counts mentions only on pages the caller can read.- `src/core/ops/chronicle.ts`: `ontology_propose` from a remote caller requires a readable entity page when one exists (a hidden one answers like `put_page`'s create on it) and tightens the observation's visibility to that page, as `remember` and `extract_facts` do.-- `src/core/ops/ingest-log.ts`: for remote callers, `get_ingest_log` drops hidden slugs from `pages_updated`, drops a row about hidden pages only, and replaces host paths in `source_ref` with `<local path>`.+- `src/core/ops/ingest-log.ts`: for remote callers, `get_ingest_log` shows a row only when every page it names (in `pages_updated`, or in `source_ref` for rows that name one page there) is a live page the caller can read, and shows `source_ref` without host paths (a sync row keeps only its commit, an import row reads `<local path>`).- `src/core/ops/admin.ts`: `get_health`'s `most_connected` lists, for a remote caller, only pages its `get_page` would show.- `src/mcp/validate-params.ts`, `src/mcp/dispatch.ts`: `narrowingUnknownParams`; a read rejects an unknown scope or filter parameter in warn mode too (`mcp.strict_params` keeps its default), and `source`, `sourceId`, `sources` and `sourceIds` suggest `source_id` / `source_ids`.-- `src/commands/serve-http-mcp.ts`: POST `/mcp` runs the IP bucket before the token lookup and the token bucket after it, each only while its env var is set, and passes `GBRAIN_HTTP_MAX_BODY_BYTES` (default the SDK's 4 MiB) to the transport. `SECURITY.md`, `docs/guides/memory-boundaries.md`, `docs/guides/remote-mcp.md` state the real defaults.+- `src/commands/serve-http-mcp.ts`: POST `/mcp` runs the IP bucket before the token lookup and the token bucket after it, each only while its env var is set, and passes `GBRAIN_HTTP_MAX_BODY_BYTES` (default the SDK's 4 MiB) to the transport; a value that is not a positive whole number is ignored with a warning. `SECURITY.md`, `docs/guides/memory-boundaries.md`, `docs/guides/remote-mcp.md` state the real defaults.- Tests: `test/access-hidden-page-writes.test.ts` (PGLite and Postgres; repo-less and managed), `test/serve-http-mcp-limits.test.ts` (a live server), `test/validate-params.test.ts`.## [0.60.625.0] - 2026-11-09diff --git a/src/commands/serve-http-mcp.ts b/src/commands/serve-http-mcp.tsindex 7d096bbaf8..87568eecea 100644--- a/src/commands/serve-http-mcp.ts+++ b/src/commands/serve-http-mcp.ts@@ -672,8 +672,12 @@ async function recordMcpToolResult(/** A positive integer env var, or undefined when unset or not one. */function positiveEnvInt(name: string): number | undefined {- const n = Number.parseInt(process.env[name] ?? '', 10);- return Number.isFinite(n) && n > 0 ? n : undefined;+ const raw = process.env[name]?.trim();+ if (!raw) return undefined;+ const n = /^\d+$/.test(raw) ? Number(raw) : NaN;+ if (Number.isSafeInteger(n) && n > 0) return n;+ console.error(`[serve-http] ${name}=${JSON.stringify(raw)} is not a positive whole number; ignoring it.`);+ return undefined;}/** GBRA-431: the POST /mcp body cap, GBRAIN_HTTP_MAX_BODY_BYTES when the operator sets it, else the SDK's 4 MiB. Over it the SDK answers 413 before the body is read. */diff --git a/src/core/enrichment-service.ts b/src/core/enrichment-service.tsindex 22df9dbe04..f49f96932b 100644--- a/src/core/enrichment-service.ts+++ b/src/core/enrichment-service.ts@@ -34,7 +34,7 @@ import { isJunkEntityName } from './entity-name-quality.ts';import { followRetirement } from './entities/resolve.ts';import { maintenanceTransaction } from './persistence/attribution.ts';import { managedPersistenceEnabled } from './persistence/ownership.ts';-import { remoteTargetHidden } from './persistence/page-visibility.ts';+import { remoteReadPolicySql, remoteTargetHidden } from './persistence/page-visibility.ts';import { resolveExcludePrivatePages } from './search/private-visibility.ts';import { assertUnmanagedCanonicalWriter } from './persistence/maintenance.ts';import { deriveTrust, derivedMaintenanceTransaction, lowerDerivedPage, lowerDerivedPageIn, lowerToDerivedTier } from './trust/taint.ts';@@ -246,7 +246,13 @@ async function planEntity(const tierEscalated = suggestedTier < (request.tier || 3); // lower tier number = higher importance// 3. Check if entity page exists+ // GBRA-431: a remote caller never writes onto, or learns anything from, an entity page it cannot read: the first+ // such page met while resolving the slug skips this entity as not found, under the slug its name maps to.+ const hidden = async (at: string, page: unknown) => page !== null && ctx !== undefined && await remoteTargetHidden(engine, ctx.remote, sourceId, at);+ const notFound = (): EnrichmentResult => ({ slug: candidateSlug, action: 'skipped', skipped_reason: 'page_not_found', tier, backlinkCreated: false,+ timelineAdded: false, mentionCount, mentionSources, suggestedTier, tierEscalated });let existingPage = await engine.getPage(slug, scope);+ if (await hidden(slug, existingPage)) return notFound();// F8: a page whose title maps to this slug base names this entity; any other page at the slug may name another one.const namesEntity = (title: string | undefined) => entitySlugBase(title ?? '') === base;if (!existingPage || !namesEntity(existingPage.title)) {@@ -256,6 +262,7 @@ async function planEntity(if (legacy !== candidateSlug && !/^(people|companies)\/$/.test(legacy)) {const legacySlug = await engine.resolveSlugWithAlias(legacy, sourceId);const legacyPage = await engine.getPage(legacySlug, { sourceId });+ if (await hidden(legacySlug, legacyPage)) return notFound();// The legacy slug is lossy (José -> people/jos), so only reuse a page that names this entity.const same = (t: string) => t.normalize('NFC').trim().toLowerCase();if (legacyPage && same(legacyPage.title ?? '') === same(request.entityName)) { slug = legacySlug; existingPage = legacyPage; legacyFound = true; }@@ -265,6 +272,7 @@ async function planEntity(if (!legacyFound && !existingPage && untagged !== candidateSlug) {const untaggedSlug = await engine.resolveSlugWithAlias(untagged, sourceId);const untaggedPage = await engine.getPage(untaggedSlug, scope);+ if (await hidden(untaggedSlug, untaggedPage)) return notFound();if (untaggedPage && namesEntity(untaggedPage.title)) { slug = untaggedSlug; existingPage = untaggedPage; legacyFound = true; }}// An escaped slug (Aux -> companies/aux-company) can be another entity's natural slug, and a natural slug can be@@ -277,6 +285,7 @@ async function planEntity(const tagged = `${prefix}/${entitySegment(base, request.entityType, true)}`;const taggedSlug = tagged === candidateSlug ? null : await engine.resolveSlugWithAlias(tagged, sourceId);const taggedPage = taggedSlug === null ? null : await engine.getPage(taggedSlug, scope);+ if (taggedSlug !== null && await hidden(taggedSlug, taggedPage)) return notFound();if (taggedSlug === null || (taggedPage && !namesEntity(taggedPage.title))) {return { slug, action: 'skipped', skipped_reason: 'entity_slug_taken', tier, backlinkCreated: false, timelineAdded: false,mentionCount, mentionSources, suggestedTier, tierEscalated };@@ -287,7 +296,7 @@ async function planEntity(}// B-9: a retired page (canonical: false, superseded_by) answers through its canonical page, as fact resolution does (#6898).if (existingPage) {- const live = await followRetirement(engine, sourceId, slug);+ const live = await followRetirement(engine, sourceId, slug, ctx && ctx.remote !== false ? `AND ${await remoteReadPolicySql(engine, 'p')}` : '');if (!live) {return { slug, action: 'skipped', skipped_reason: 'retired_page_unfollowable', tier, backlinkCreated: false, timelineAdded: false,mentionCount, mentionSources, suggestedTier, tierEscalated };@@ -297,12 +306,6 @@ async function planEntity(let action: 'created' | 'updated' | 'skipped';if (existingPage) {- // GBRA-431: a remote caller never writes onto an entity page it cannot read; that entity is skipped as not found- // and the rest of the batch still runs.- if (ctx && await remoteTargetHidden(engine, ctx.remote, sourceId, slug)) {- return { slug, action: 'skipped', skipped_reason: 'page_not_found', tier, backlinkCreated: false, timelineAdded: false,- mentionCount, mentionSources, suggestedTier, tierEscalated };- }// UPDATE path — add timeline entryaction = 'updated';} else {diff --git a/src/core/ops/ingest-log.ts b/src/core/ops/ingest-log.tsindex abbd269629..05cb904696 100644--- a/src/core/ops/ingest-log.ts+++ b/src/core/ops/ingest-log.ts@@ -9,7 +9,10 @@import type { Operation } from './contract.ts';import { clampSearchLimit } from '../engine.ts';import { linkReadScopeOpts } from './context.ts';-import { remoteReadablePages } from '../persistence/page-visibility.ts';+import { remotePageAccess } from '../persistence/page-visibility.ts';++/** Row kinds whose source_ref and summary name one page while pages_updated stays empty. */+const PAGE_REF_TYPES = new Set(['facts:absorb', 'write-verify-guard']);/** A file URL or an absolute, home-relative, relative or Windows path token in a free-form source_ref. */const HOST_PATH_RE = /(^|\s)(?:file:\/\/|\.{1,2}[\\/]|~?\/|[A-Za-z]:[\\/])\S*/g;@@ -75,14 +78,18 @@ const get_ingest_log: Operation = {...(scope.sourceIds ? { sourceIds: scope.sourceIds } : scope.sourceId ? { sourceIds: [scope.sourceId] } : {}),});if (ctx.remote === false) return rows;- // GBRA-431: a remote reader sees no slug of a page it cannot read, no row about such pages alone (its summary is- // about them) and no host filesystem path (a sync row's source_ref names the brain checkout).- const readable = await remoteReadablePages(ctx.engine, rows.flatMap(r => (r.pages_updated ?? []).map(slug => ({ source_id: r.source_id, slug }))));- return rows.flatMap(r => {+ // GBRA-431: a remote reader sees a row only when every page it names is a live page the reader can read (its+ // summary may describe any of them), with no host filesystem path (a sync row's source_ref names the checkout).+ // A row that names its page in source_ref alone is checked on that page; one whose ref names a page the reader+ // cannot read is dropped too.+ const { known, readable } = await remotePageAccess(ctx.engine, rows.flatMap(r => [...(r.pages_updated ?? []), r.source_ref].map(slug => ({ source_id: r.source_id, slug }))));+ const key = (r: { source_id: string }, slug: string) => `${r.source_id}\u0000${slug}`;+ return rows.filter(r => {const pages = r.pages_updated ?? [];- const visible = pages.filter(slug => readable.has(`${r.source_id}\u0000${slug}`));- return pages.length > 0 && visible.length === 0 ? [] : [{ ...r, source_ref: remoteSourceRef(r.source_type, r.source_ref), pages_updated: visible }];- });+ if (!pages.every(slug => readable.has(key(r, slug)))) return false;+ if (pages.length === 0 && PAGE_REF_TYPES.has(r.source_type)) return readable.has(key(r, r.source_ref));+ return !known.has(key(r, r.source_ref)) || readable.has(key(r, r.source_ref));+ }).map(r => ({ ...r, source_ref: remoteSourceRef(r.source_type, r.source_ref) }));},scope: 'read',};diff --git a/src/core/persistence/page-visibility.ts b/src/core/persistence/page-visibility.tsindex 23668a447f..257ad75ab2 100644--- a/src/core/persistence/page-visibility.ts+++ b/src/core/persistence/page-visibility.ts@@ -105,17 +105,20 @@ export async function remoteTargetHidden(engine: BrainEngine, remote: boolean |}/**- * GBRA-431: the (source_id, slug) pairs among `pages` that name a live page the current remote caller can read, keyed- * `${source_id}\u0000${slug}`. A slug with no live page row (never written, deleted, purged or renamed) is not- * readable, so a caller of this keeps only what it returns (fail closed). Call for remote callers only.+ * GBRA-431: for the (source_id, slug) pairs among `pages`, keyed `${source_id}\u0000${slug}`, which name any page row+ * (`known`, soft-deleted rows included) and which name a live page the current remote caller can read (`readable`).+ * A slug with no live page row (never written, deleted, purged or renamed) is not readable, so a caller that must+ * fail closed keeps only `readable`. Call for remote callers only.*/-export async function remoteReadablePages(engine: BrainEngine, pages: readonly { source_id: string; slug: string }[]): Promise<Set<string>> {- if (pages.length === 0) return new Set();+export async function remotePageAccess(engine: BrainEngine, pages: readonly { source_id: string; slug: string }[]): Promise<{ known: Set<string>; readable: Set<string> }> {+ if (pages.length === 0) return { known: new Set(), readable: new Set() };const policy = await remoteReadPolicySql(engine, 'p');- const rows = await engine.executeRaw<{ source_id: string; slug: string }>(`SELECT p.source_id, p.slug FROM pages p+ const rows = await engine.executeRaw<{ source_id: string; slug: string; readable: boolean }>(`SELECT p.source_id, p.slug,+ bool_or(p.deleted_at IS NULL AND ${policy}) AS readable FROM pages pJOIN unnest($1::text[], $2::text[]) AS t(source_id, slug) ON p.source_id = t.source_id AND p.slug = t.slug- WHERE p.deleted_at IS NULL AND ${policy}`, [pages.map(p => p.source_id), pages.map(p => p.slug)]);- return new Set(rows.map(r => `${r.source_id}\u0000${r.slug}`));+ GROUP BY p.source_id, p.slug`, [pages.map(p => p.source_id), pages.map(p => p.slug)]);+ const key = (r: { source_id: string; slug: string }) => `${r.source_id}\u0000${r.slug}`;+ return { known: new Set(rows.map(key)), readable: new Set(rows.filter(r => r.readable).map(key)) };}/**diff --git a/test/access-hidden-page-writes.test.ts b/test/access-hidden-page-writes.test.tsindex 5bbd7550d2..8a3dbbfa2a 100644--- a/test/access-hidden-page-writes.test.ts+++ b/test/access-hidden-page-writes.test.ts@@ -105,6 +105,8 @@ for (const kind of testBackends()) {await importFromContent(engine, HUB, note('Shared roadmap', 'The shared roadmap.', ['type: note']), { sourceId: SRC, noEmbed: true });await importFromContent(engine, OPEN, note('Open plan', 'Open planning notes.', ['type: note']), { sourceId: SRC, noEmbed: true });await importFromContent(engine, RESTRICTED, note('Vesper Example', 'Person notes.', ['type: person', 'readable_by: [alice-example]']), { sourceId: SRC, noEmbed: true });+ await importFromContent(engine, 'people/hidden-successor-example', note('Hidden Successor', 'Person notes.', ['type: person', 'visibility: private']), { sourceId: SRC, noEmbed: true });+ await importFromContent(engine, 'people/olden-example', note('Olden Example', 'Retired.', ['type: person', 'superseded_by: people/hidden-successor-example']), { sourceId: SRC, noEmbed: true });await importFromContent(engine, PRIVATE, note('Quiet plan', 'Quiet planning notes.', ['type: note', 'visibility: private']), { sourceId: SRC, noEmbed: true });for (const slug of [OPEN, RESTRICTED, PRIVATE]) {await engine.addLink(slug, HUB, 'roadmap', 'mentions', 'manual', undefined, undefined, { fromSourceId: SRC, toSourceId: SRC }); // gbrain-allow-direct-insert: test fixture edges@@ -160,6 +162,10 @@ for (const kind of testBackends()) {.toEqual({ who, isError: false, hidden: expect.objectContaining({ action: 'skipped', skipped_reason: 'page_not_found', backlinkCreated: false }) });expect(entities.some(e => e.slug !== RESTRICTED && e.action !== 'skipped')).toBe(true);}+ for (const who of ['bob', 'writer']) {+ const out = await call(who, 'extract_entities', { text: 'Olden Example joined the planning call.', source_slug: OPEN });+ expect({ who, isError: out.isError, leaked: out.text.includes('hidden-successor') }).toEqual({ who, isError: false, leaked: false });+ }expect(await state(RESTRICTED)).toEqual(restricted);const alice = await call('alice', 'extract_entities', { text: 'Vesper Example joined the planning call.', source_slug: OPEN });expect((alice.json.entities as Array<{ slug: string; action: string }>).find(e => e.slug === RESTRICTED)?.action).toBe('updated');@@ -172,24 +178,26 @@ for (const kind of testBackends()) {}, 120_000);test('get_ingest_log names no page a remote reader cannot read and no host path; local callers see the row whole', async () => {- await engine.logIngest({ source_id: SRC, source_type: 'git_sync', source_ref: '/srv/brains/acme-example @ abcdef12', pages_updated: [OPEN, RESTRICTED, PRIVATE, MISSING], summary: 'Sync: +4 ~0 -0' });+ await engine.logIngest({ source_id: SRC, source_type: 'git_sync', source_ref: '/srv/brains/acme-example @ abcdef12', pages_updated: [OPEN], summary: 'Sync: +1 ~0 -0' });+ await engine.logIngest({ source_id: SRC, source_type: 'fixture', source_ref: 'ref-mixed', pages_updated: [OPEN, RESTRICTED, PRIVATE], summary: 'Ingested canaryhpmixed' });+ await engine.logIngest({ source_id: SRC, source_type: 'fixture', source_ref: 'ref-gone', pages_updated: [MISSING], summary: 'Ingested canaryhpgone' });await engine.logIngest({ source_id: SRC, source_type: 'fixture', source_ref: 'ref-hidden-only', pages_updated: [RESTRICTED, PRIVATE], summary: 'Ingested canaryhphidden' });+ await engine.logIngest({ source_id: SRC, source_type: 'fixture', source_ref: 'ref-alice', pages_updated: [RESTRICTED], summary: 'Ingested canaryhpalice' });+ await engine.logIngest({ source_id: SRC, source_type: 'write-verify-guard', source_ref: PRIVATE, pages_updated: [], summary: 'read-back failed canaryhpguard' });+ await engine.logIngest({ source_id: SRC, source_type: 'note', source_ref: RESTRICTED, pages_updated: [], summary: 'canaryhpref' });await engine.logIngest({ source_id: SRC, source_type: 'git_sync', source_ref: '/home/alice-example/Mobile Documents/brain @ 0123abcd', pages_updated: [OPEN], summary: 'Sync: +1 ~0 -0' });await engine.logIngest({ source_id: SRC, source_type: 'directory', source_ref: './brain-example', pages_updated: [OPEN], summary: 'Import' });- const remote = await call('reader', 'get_ingest_log', { limit: 5 });+ const remote = await call('reader', 'get_ingest_log', { limit: 20 });expect(remote.isError).toBe(false);- const row = (JSON.parse(remote.text.split('\n_META_HOT')[0]) as Array<{ source_ref: string; pages_updated: string[] }>).find(r => r.source_ref.endsWith('@ abcdef12'))!;- expect({ ref: row.source_ref, pages: row.pages_updated }).toEqual({ ref: '<local path> @ abcdef12', pages: [OPEN] });- const refs = (JSON.parse(remote.text.split('\n_META_HOT')[0]) as Array<{ source_ref: string }>).map(r => r.source_ref);- expect(refs).toEqual(expect.arrayContaining(['<local path> @ 0123abcd', '<local path>']));- expect(remote.text).not.toContain('Mobile Documents');- expect(remote.text).not.toContain('brain-example');- expect(remote.text).not.toContain('canaryhphidden');- expect((await call('alice', 'get_ingest_log', { limit: 5 })).text).toContain(RESTRICTED);- const whole = await local('get_ingest_log', { limit: 5 });- expect(whole.text).toContain('/srv/brains/acme-example');- expect(whole.text).toContain(PRIVATE);- expect(whole.text).toContain('canaryhphidden');+ const refs = (JSON.parse(remote.text.split('\n_META_HOT')[0]) as Array<{ source_ref: string; pages_updated: string[] }>).map(r => r.source_ref);+ expect(refs).toEqual(expect.arrayContaining(['<local path> @ abcdef12', '<local path> @ 0123abcd', '<local path>']));+ for (const hidden of [RESTRICTED, PRIVATE, MISSING, 'canaryhpmixed', 'canaryhpgone', 'canaryhphidden', 'canaryhpalice', 'canaryhpguard', 'canaryhpref', 'Mobile Documents', 'brain-example', '/srv/brains']) {+ expect({ hidden, shown: remote.text.includes(hidden) }).toEqual({ hidden, shown: false });+ }+ const alice = (await call('alice', 'get_ingest_log', { limit: 20 })).text;+ expect({ alice: alice.includes('canaryhpalice'), ref: alice.includes('canaryhpref'), mixed: alice.includes('canaryhpmixed') }).toEqual({ alice: true, ref: true, mixed: false });+ const whole = await local('get_ingest_log', { limit: 20 });+ for (const shown of ['/srv/brains/acme-example', PRIVATE, 'canaryhphidden', 'canaryhpmixed', 'canaryhpguard']) expect({ shown, ok: whole.text.includes(shown) }).toEqual({ shown, ok: true });}, 120_000);test('ontology_propose refuses an entity page its caller cannot read, and keeps an observation no wider than the page', async () => {diff --git a/test/serve-http-mcp-limits.test.ts b/test/serve-http-mcp-limits.test.tsindex 0b2d7e4300..7857b45cae 100644--- a/test/serve-http-mcp-limits.test.ts+++ b/test/serve-http-mcp-limits.test.ts@@ -67,6 +67,16 @@ describe('serve --http enforces the /mcp limits the operator sets', () => {});}, 120_000);+ test('a body cap that is not a whole number of bytes is ignored, not read as a tiny cap', async () => {+ await withEnv({ ...UNSET, GBRAIN_HTTP_MAX_BODY_BYTES: '4MiB' }, async () => {+ const app = await startServeHttp(engine);+ try {+ const token = await legacyToken(engine, ['read']);+ expect((await rpc(app.base, token, 'tools/list')).status).toBe(200);+ } finally { await app.close(); }+ });+ }, 120_000);+test('with no limit set, serve --http keeps its unlimited rate and 4 MiB body default', async () => {await withEnv(UNSET, async () => {const app = await startServeHttp(engine);