Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

recall and list_pages honour source_ids (GST-359 S)

garrytane8df7b7252ea
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a1166e1e0a..0d817d6c58 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,18 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**`recall` and `list_pages` now honour `source_ids`, the per-request source narrowing `search` and `query` already had. Before, both ops ignored the list, so an agent narrowing each call to the asking member's sources still got other sources' facts and pages from these two tools.** A harness-evals run measured it: under per-request narrowing, `recall` returned a page from outside the requester's sources twice and `list_pages` 57 pages over three calls. Each id is checked like an explicit `source_id`, so the list can only narrow what the token's grant allows; an id outside a remote grant refuses the whole request, and an empty intersection with `source_id` returns nothing with an `empty_retrieval` notice.
+
+### Itemized changes
+
+- `src/core/ops/facts.ts`: `recall` takes `source_ids` and reads every fact arm and its page-search arm inside the narrowed scope.
+- `src/core/ops/pages.ts`: `list_pages` takes `source_ids`.
+- `src/core/ops/search-source-scope.ts`: the shared parser and resolver serve all four ops.
+- `docs/guides/multi-source-brains.md`: "Narrow one request with `source_ids`" names all four ops.
+- `test/recall-list-pages-source-ids.test.ts` (PGLite and Postgres): each recall arm and list_pages under narrowing, the grant refusal and the empty intersection; red on the old code.
+
## [0.60.519.0] - 2026-10-30
**The unit lane now runs 28 shards instead of 22, on shard weights refreshed from the Oct 30 nightly. With the fresh weights, every one of 22 shards would be planned at 624.4 s against the 600 s per-shard budget; at 28 shards the largest is 490.6 s.**
diff --git a/docs/guides/multi-source-brains.md b/docs/guides/multi-source-brains.md
index c7a486b6eb..02d8951e41 100644
--- a/docs/guides/multi-source-brains.md
+++ b/docs/guides/multi-source-brains.md
@@ -169,12 +169,14 @@ grant.
### Narrow one request with `source_ids`
-`search` and `query` take `source_ids`, a list of source ids (CLI:
-`--source-ids source-a,source-b`). The request reads only those sources:
+`search`, `query`, `recall` and `list_pages` take `source_ids`, a list of
+source ids (CLI: `--source-ids source-a,source-b`). The request reads only
+those sources:
- Each id is checked exactly like an explicit `source_id`. A remote agent
- naming a source outside its grant gets `permission_denied` for the whole
- request, and a removed or archived source gets `unknown_source`. So the
+ naming a source outside its grant gets `permission_denied` (`scope_denied`
+ from `recall`) for the whole request, and a removed or archived source gets
+ `unknown_source` (`not_found` from `recall`). So the
list can narrow a request but never widen it.
- Omitting `source_ids`, or passing `[]` or a blank string, reads every
source the caller can read, as before.
@@ -183,10 +185,14 @@ grant.
searched: the result is empty with an `empty_retrieval` notice, never a
fallback to the wider scope.
- `__all__` and malformed ids in the list are `invalid_params`.
+- On the starter surface, `recall` and `list_pages` accept `source_ids` but
+ leave it out of their listed schema to keep the tool list small; harness
+ code sets it, and `request_tools` returns the full schema.
Every part of the request reads the narrowed sources: keyword, vector and
title matches, other names, entity anchoring, saved facts, the CRAG re-run,
-think escalation and evidence delivery.
+think escalation and evidence delivery; on `recall`, every fact arm and the
+page-search arm.
**Say to your agent:** *"Search my brain for the acme-example renewal, but
only in source-a."*
diff --git a/src/core/operation-manifest.generated.ts b/src/core/operation-manifest.generated.ts
index 493164be62..cca15f2d97 100644
--- a/src/core/operation-manifest.generated.ts
+++ b/src/core/operation-manifest.generated.ts
@@ -609,6 +609,14 @@ export const OPERATION_MANIFEST: OperationMeta[] = [
"source_id": {
"type": "string",
"description": "One source, or '__all__'."
+ },
+ "source_ids": {
+ "type": "array",
+ "items": {
+ "type": "string"
+ },
+ "description": "Narrow to these sources (omit or [] for all you can read).",
+ "fullSurfaceOnly": true
}
},
"scope": "read",
@@ -5700,6 +5708,14 @@ export const OPERATION_MANIFEST: OperationMeta[] = [
"type": "string",
"description": "One readable source."
},
+ "source_ids": {
+ "type": "array",
+ "items": {
+ "type": "string"
+ },
+ "description": "Narrow to these sources (omit or [] for all you can read).",
+ "fullSurfaceOnly": true
+ },
"since": {
"type": "string",
"description": "ISO 8601 or \"8 hours ago\"."
diff --git a/src/core/ops/facts.ts b/src/core/ops/facts.ts
index dcceb507c1..4d1152a61a 100644
--- a/src/core/ops/facts.ts
+++ b/src/core/ops/facts.ts
@@ -26,7 +26,8 @@ import type { Operation, OperationContext } from './contract.ts';
import type { Notice } from '../agent-output.ts';
import { OperationError, opError, verbError } from './contract.ts';
import { invalidParam, paramUse } from './op-fix.ts';
-import { assertExplicitSourceLive, federatedSearchScope, parseSourceIdParam, sourceScopeOpts, stampEvidenceSafe } from './context.ts';
+import { parseSourceIdParam, sourceScopeOpts, stampEvidenceSafe } from './context.ts';
+import { parseSourceIdsParam, requestSourceScope, SOURCE_IDS_PARAM } from './search-source-scope.ts';
import { markKeywordHits } from '../search/evidence.ts';
import { hybridSearchCached, stampContentFlags, type HybridSearchOpts } from '../search/hybrid.ts';
import { dedupResults } from '../search/dedup.ts';
@@ -280,6 +281,7 @@ const recall: Operation = {
budget_tokens: { type: 'number', description: 'Token budget; facts pack first.' },
budget_policy: { type: 'string', enum: ['facts_first', 'query_first'], description: 'Default facts_first.', fullSurfaceOnly: true },
source_id: { type: 'string', description: 'One readable source.' },
+ source_ids: { ...SOURCE_IDS_PARAM, fullSurfaceOnly: true },
since: { type: 'string', description: 'ISO 8601 or "8 hours ago".' },
session_id: { type: 'string', description: "This session's facts." },
include_expired: { type: 'boolean', description: 'Also expired; question ranks them.', fullSurfaceOnly: true },
@@ -313,10 +315,9 @@ const recall: Operation = {
sourceIdParam = parseSourceIdParam(p.source_id, 'recall');
// The facts arms widen an unqualified no-grant caller across the transport-computed
// federated set, exactly like the page-search arm below and every sibling read op.
- scope = federatedSearchScope(ctx, sourceIdParam);
- await assertExplicitSourceLive(ctx, sourceIdParam);
+ scope = await requestSourceScope(ctx, sourceIdParam, parseSourceIdsParam(ctx, 'recall', p.source_ids));
} catch (error) {
- if (!(error instanceof OperationError) || p.source_id === undefined || p.source_id === null) throw error;
+ if (!(error instanceof OperationError) || ((p.source_id === undefined || p.source_id === null) && (p.source_ids === undefined || p.source_ids === null))) throw error;
const code = error.code === 'permission_denied' ? 'scope_denied'
: error.code === 'unknown_source' ? 'not_found'
: error.code === 'invalid_params' ? 'invalid_params' : null;
@@ -327,7 +328,8 @@ const recall: Operation = {
}
// Set-dedupe: a grant carrying a repeated id (or the scalar source again)
// must not fan out the same source twice into the merge.
- const factSources: string[] = [...new Set(
+ const readsNoSource = scope.sourceIds?.length === 0;
+ const factSources: string[] = readsNoSource ? [] : [...new Set(
scope.sourceIds && scope.sourceIds.length > 0 ? scope.sourceIds
: scope.sourceId ? [scope.sourceId]
: [sourceId],
@@ -403,7 +405,7 @@ const recall: Operation = {
excludeAuditRows: true, eligibility,
};
- if (question !== null) {
+ if (question !== null && !readsNoSource) {
ranking = await rankFactsByQuestion(ctx, { question, params: p, factSources, limit, entity: entityParam, sessionId: sessionParam, since, grep, includeExpired,
scope: { sourceIds: factSources, remote: ctx.remote !== false, excludePrivate, minTrust: eligibility.floor, suppressFlagged: eligibility.suppressFlagged } });
({ rows, ambiguousEntity } = ranking);
@@ -514,13 +516,13 @@ const recall: Operation = {
let delivery: DeliveryMeta | undefined;
const evidencePlan = queryText ? await recallEvidencePlan(ctx, p) : null;
const rankCutOpts: Pick<HybridSearchOpts, 'pageRankCut'> = evidencePlan?.rankCut ? { pageRankCut: evidencePlan.rankCut } : {};
- if (queryText) {
+ if (queryText && !readsNoSource) {
// #3242 parity (#4707): the page-search arm widens an unqualified
// no-grant caller across the transport-computed federated set, exactly
// like search/query/get_page/list_pages/resolve_slugs. sourceScopeOpts
// alone pinned this arm to the scalar source, so a `federated: true`
// source was invisible to recall while visible to every sibling read op.
- const searchScope = { ...federatedSearchScope(ctx, sourceIdParam), ...(eligibility.floor ? { minTrust: eligibility.floor } : {}) };
+ const searchScope = { ...scope, ...(eligibility.floor ? { minTrust: eligibility.floor } : {}) };
// #4352 — recall's page-search arm enforces `visibility: private` for
// untrusted callers with the excludePrivate resolved above.
const keywordOnly = await recallKeywordOnlyReason(ctx);
diff --git a/src/core/ops/pages.ts b/src/core/ops/pages.ts
index 3bf892a5e2..7caba235a2 100644
--- a/src/core/ops/pages.ts
+++ b/src/core/ops/pages.ts
@@ -34,6 +34,7 @@ import { listPagesPagination, listingTruncatedNotice } from './list-pages-pagina
import { OperationError, opError, type Operation, type OperationContext } from './contract.ts';
import { invalidParam, readFix } from './op-fix.ts';
import { WHERE_PARAM, whereOpParam } from './where-param.ts';
+import { parseSourceIdsParam, requestSourceScope, SOURCE_IDS_PARAM } from './search-source-scope.ts';
import { isDatetimeInputError } from '../utils.ts';
import { rethrowNamingRevisionSource } from './put-page-revision-source.ts';
import {
@@ -536,6 +537,7 @@ const list_pages: Operation = {
type: 'string',
description: "One source, or '__all__'.",
},
+ source_ids: { ...SOURCE_IDS_PARAM, fullSurfaceOnly: true },
},
handler: async (ctx, p) => {
// Whitelist the sort enum at the handler before passing to the engine.
@@ -568,9 +570,7 @@ const list_pages: Operation = {
// them silently return [] or, for the CLI's `--source-id ""`, widen to
// every source.
const sourceIdParam = parseSourceIdParam(p.source_id, 'list_pages', { allowAll: true });
- const scope = federatedSearchScope(ctx, sourceIdParam);
- // #4620: an explicit source_id must name a live source (after the grant check).
- await assertExplicitSourceLive(ctx, sourceIdParam);
+ const scope = await requestSourceScope(ctx, sourceIdParam, parseSourceIdsParam(ctx, 'list_pages', p.source_ids));
// #4352 remediation: untrusted listing never enumerates
// `visibility: private` pages (slugs + titles are the leak surface here).
// Composes with the #4400 per-call source_id and the v0.34.1 grant scope
@@ -608,7 +608,7 @@ const list_pages: Operation = {
// list and never learns rows were dropped, and with the default
// updated_desc sort the dropped rows are always the OLDEST, i.e. exactly
// the pages such consumers exist to find.
- const rows = await ctx.engine.listPages({
+ const rows = scope.sourceIds?.length === 0 ? [] : await ctx.engine.listPages({
type: p.type as any,
tag: p.tag as string,
limit: limit + 1,
diff --git a/src/core/ops/search-source-scope.ts b/src/core/ops/search-source-scope.ts
index f75045e111..e30b94c980 100644
--- a/src/core/ops/search-source-scope.ts
+++ b/src/core/ops/search-source-scope.ts
@@ -26,7 +26,7 @@ export const SOURCE_IDS_PARAM = {
};
/** `source_ids` as distinct, trimmed source ids; undefined when absent. */
-export function parseSourceIdsParam(ctx: OperationContext, tool: 'search' | 'query', raw: unknown): string[] | undefined {
+export function parseSourceIdsParam(ctx: OperationContext, tool: 'search' | 'query' | 'recall' | 'list_pages', raw: unknown): string[] | undefined {
if (raw === undefined || raw === null) return undefined;
// #5390: OpenAI-family MCP clients fill every optional param with an empty value; [] and a blank string mean absent.
if (Array.isArray(raw) && raw.length === 0) return undefined;
@@ -47,7 +47,7 @@ export function parseSourceIdsParam(ctx: OperationContext, tool: 'search' | 'que
return [...ids];
}
-/** The sources one search/query request reads; `sourceIds: []` means it reads none. */
+/** The sources one search, query, recall or list_pages request reads; `sourceIds: []` means it reads none. */
export async function requestSourceScope(ctx: OperationContext, sourceIdParam: string | undefined, sourceIds: string[] | undefined): Promise<{ sourceId?: string; sourceIds?: string[] }> {
const scope = federatedSearchScope(ctx, sourceIdParam);
if (sourceIds === undefined) {
diff --git a/test/e2e/recall-list-pages-source-ids-postgres.test.ts b/test/e2e/recall-list-pages-source-ids-postgres.test.ts
new file mode 100644
index 0000000000..f4f5009cc8
--- /dev/null
+++ b/test/e2e/recall-list-pages-source-ids-postgres.test.ts
@@ -0,0 +1,3 @@
+import { registerPostgresTests } from '../helpers/test-backends.ts';
+
+await registerPostgresTests(() => import('../recall-list-pages-source-ids.test.ts'));
diff --git a/test/fixtures/goldens/cli/tools-json.json b/test/fixtures/goldens/cli/tools-json.json
index 258c9c0361..bdf0a71950 100644
--- a/test/fixtures/goldens/cli/tools-json.json
+++ b/test/fixtures/goldens/cli/tools-json.json
@@ -2,8 +2,8 @@
"golden": {
"exitCode": 0,
"stderr": "",
- "stdoutBytes": 264563,
- "stdoutSha256": "0d71d7f1367627c5f9c27d3d7a9b09513131117000012eed0d35ab9ede69095f",
+ "stdoutBytes": 265045,
+ "stdoutSha256": "4358e6505ccac2f00bcbe3a01fb54daf3d13c8a7c526315a8c53aaf223a01dc8",
"toolNames": [
"remember",
"entity",
@@ -755,6 +755,7 @@
"slug_prefix": "string?",
"sort": "string?",
"source_id": "string?",
+ "source_ids": "array?",
"tag": "string?",
"type": "string?",
"updated_after": "string?",
@@ -794,6 +795,13 @@
"description": "One source, or '__all__'.",
"type": "string"
},
+ "source_ids": {
+ "description": "Narrow to these sources (omit or [] for all you can read).",
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
"tag": {
"description": "Tag.",
"type": "string"
@@ -6656,6 +6664,7 @@
"session_id": "string?",
"since": "string?",
"source_id": "string?",
+ "source_ids": "array?",
"supersessions": "boolean?"
},
"required_scopes": [],
@@ -6740,6 +6749,13 @@
"description": "One readable source.",
"type": "string"
},
+ "source_ids": {
+ "description": "Narrow to these sources (omit or [] for all you can read).",
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
"supersessions": {
"description": "Supersession log only.",
"type": "boolean"
diff --git a/test/recall-list-pages-source-ids.test.ts b/test/recall-list-pages-source-ids.test.ts
new file mode 100644
index 0000000000..b01110f98b
--- /dev/null
+++ b/test/recall-list-pages-source-ids.test.ts
@@ -0,0 +1,129 @@
+/**
+ * GST-359 item S: per-request `source_ids` on `recall` and `list_pages`.
+ *
+ * 1. Protects: an agent that narrows each call to the sources the asking
+ * member may read (#7481's `source_ids`) gets that narrowing on recall and
+ * list_pages too. The HARN-12 v0.3 pilot's told-scoped cells measured the
+ * gap: under a per-request scope, recall returned another member's DM page
+ * (2 events) and list_pages returned 57 pages outside the grant (3 events),
+ * because both ops ignored the list.
+ * 2. Fails when: either op drops `source_ids`, any recall arm (question
+ * ranking, since, entity, session, the page-search arm) reads past the
+ * narrowed scope, an id outside a remote grant is not refused, or an empty
+ * intersection with `source_id` falls back to the ambient or full scope.
+ * 3. search and query are pinned by test/search/search-source-ids.test.ts.
+ * 4. No production seam: keyless brains (keyword arms only), as the harness runs.
+ */
+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { importFromContent } from '../src/core/import-file.ts';
+import { operationsByName, type OperationContext } from '../src/core/operations.ts';
+import { testBackends } from './helpers/test-backends.ts';
+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';
+
+const CANARY = 'canarybravo';
+const GRANTED_A = { token: 'gbrain_at_test', clientId: 'gbrain_cl_test', scopes: ['read'], allowedSources: ['source-a'] };
+const GRANTED_ABC = { ...GRANTED_A, allowedSources: ['source-a', 'source-b', 'source-c'] };
+
+for (const kind of testBackends()) {
+ describe(`recall/list_pages source_ids (${kind})`, () => {
+ let engine: BrainEngine;
+ let close: () => Promise<void>;
+
+ async function run(op: 'recall' | 'list_pages', params: Record<string, unknown>, ctxOver: Partial<OperationContext> = {}) {
+ const notices: Array<{ code: string; why: string }> = [];
+ const ctx = {
+ engine, config: { engine: kind, embedding_disabled: true }, dryRun: false, remote: true, sourceId: 'source-a',
+ auth: GRANTED_ABC,
+ logger: { info() {}, warn() {}, error() {} },
+ emitResponseMeta: () => {},
+ emitNotice: (n: { code: string; why: string }) => { notices.push(n); },
+ ...ctxOver,
+ } as unknown as OperationContext;
+ const out = await operationsByName[op].handler(ctx, params);
+ return { out, wire: JSON.stringify(out), notices };
+ }
+
+ async function refusal(promise: Promise<unknown>): Promise<string> {
+ try { await promise; } catch (e) { return (e as { code?: string }).code ?? String(e); }
+ return 'no_error';
+ }
+
+ beforeAll(async () => {
+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));
+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }
+ for (const id of ['source-a', 'source-b', 'source-c']) {
+ await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT (id) DO NOTHING`, [id]);
+ }
+ const pages: Array<[string, string, string]> = [
+ ['source-a', 'chats/orchard-alpha', '# Zephyrblatt orchard\nZephyrblatt orchard harvest notes for the alpha team.'],
+ ['source-b', 'chats/orchard-bravo', `# Zephyrblatt orchard\nZephyrblatt orchard harvest notes ${CANARY}.`],
+ ['source-c', 'chats/orchard-charlie', '# Zephyrblatt orchard\nZephyrblatt orchard harvest notes for the charlie team.'],
+ ];
+ for (const [sourceId, slug, body] of pages) await importFromContent(engine, slug, body, { sourceId });
+ const fact = (text: string, sourceId: string) => engine.insertFact({ fact: text, kind: 'fact', entity_slug: 'zephyrblatt-orchard', source: 'notes', visibility: 'world', valid_from: new Date('2026-07-01T00:00:00Z') }, { source_id: sourceId });
+ await fact('Zephyrblatt orchard harvest moved to May for the alpha team.', 'source-a');
+ await fact(`Zephyrblatt orchard harvest moved to June ${CANARY}.`, 'source-b');
+ }, 180_000);
+
+ afterAll(async () => { await close?.(); }, 60_000);
+
+ test('the op contracts carry source_ids', () => {
+ for (const op of ['recall', 'list_pages'] as const) {
+ expect(operationsByName[op].params.source_ids).toMatchObject({ type: 'array', items: { type: 'string' } });
+ }
+ });
+
+ test('fixture control: an unnarrowed read reaches source-b on both ops', async () => {
+ expect((await run('recall', { question: 'zephyrblatt orchard harvest', query: 'zephyrblatt orchard harvest' })).wire).toContain(CANARY);
+ expect((await run('list_pages', {})).wire).toContain('chats/orchard-bravo');
+ });
+
+ const recallShapes: Array<[string, Record<string, unknown>]> = [
+ ['question ranking and the page arm', { question: 'zephyrblatt orchard harvest', query: 'zephyrblatt orchard harvest' }],
+ ['since', { since: '2020-01-01T00:00:00Z' }],
+ ['entity', { entity: 'zephyrblatt-orchard' }],
+ ['page arm alone', { query: 'zephyrblatt orchard harvest' }],
+ ];
+ for (const [label, params] of recallShapes) {
+ test(`recall (${label}): narrowing to source-a returns nothing from source-b or source-c`, async () => {
+ const { wire } = await run('recall', { ...params, source_ids: ['source-a'] });
+ expect(wire).not.toContain(CANARY);
+ expect(wire).not.toContain('"source_id":"source-b"');
+ expect(wire).not.toContain('charlie');
+ });
+ }
+
+ test('recall: narrowing still returns source-a rows (the list narrows, it does not blank)', async () => {
+ const { wire } = await run('recall', { question: 'zephyrblatt orchard harvest', query: 'zephyrblatt orchard harvest', source_ids: ['source-a'] });
+ expect(wire).toContain('alpha team');
+ });
+
+ test('list_pages: narrowing to source-a and source-c lists exactly those sources', async () => {
+ const { out, wire } = await run('list_pages', { source_ids: ['source-a', 'source-c'] });
+ const rows = (Array.isArray(out) ? out : (out as { pages: unknown[] }).pages) as Array<{ slug: string; source_id: string }>;
+ expect(rows.map(r => r.slug).sort()).toEqual(['chats/orchard-alpha', 'chats/orchard-charlie']);
+ expect(wire).not.toContain('orchard-bravo');
+ });
+
+ test('an id outside a remote grant is refused on both ops, before any row is read', async () => {
+ expect(await refusal(run('list_pages', { source_ids: ['source-b'] }, { auth: GRANTED_A } as never))).toBe('permission_denied');
+ expect(await refusal(run('recall', { query: 'zephyrblatt', source_ids: ['source-b'] }, { auth: GRANTED_A } as never))).toBe('scope_denied');
+ });
+
+ test('an empty intersection with source_id reads no source on both ops', async () => {
+ const list = await run('list_pages', { source_id: 'source-a', source_ids: ['source-c'] });
+ expect(list.wire).not.toContain('orchard-');
+ expect(list.notices.map(n => n.code)).toContain('empty_retrieval');
+ const recall = await run('recall', { question: 'zephyrblatt orchard harvest', query: 'zephyrblatt orchard harvest', source_id: 'source-a', source_ids: ['source-c'] });
+ expect(recall.wire).not.toContain('Zephyrblatt');
+ expect(recall.notices.map(n => n.code)).toContain('empty_retrieval');
+ });
+
+ test('an empty array behaves exactly like omitting it', async () => {
+ const omitted = (await run('list_pages', {})).wire;
+ expect((await run('list_pages', { source_ids: [] })).wire).toBe(omitted);
+ });
+ });
+}