Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
perf(access): plan-cheap readers and lineage predicates (GBRA-218)
garrytana9ce828f9251
diff --git a/docs/architecture/key-files/core-search-1.md b/docs/architecture/key-files/core-search-1.mdindex 792fded123..6a76dd6a7d 100644--- a/docs/architecture/key-files/core-search-1.md+++ b/docs/architecture/key-files/core-search-1.md@@ -66,7 +66,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/search/modes-report.ts` — `buildModesReport(engine)` → `SearchModesReport` (`schema_version: 2`), the read-only dashboard behind `gbrain search modes` and the `search_modes` MCP op: `active_mode` + validity, the per-knob `resolved` attribution (`attributeKnob` over `KNOB_DESCRIPTIONS`, including the five `reranker_*` knobs), the three frozen `bundles`, `config_keys`, and `reranker_readiness` — `{model, enabled, ready, required_key, key_present, self_hosted, fix}` from `rerankerReadinessForEngine` + `describeRerankerFix` (a thrown readiness check still yields a `ready: false` verdict whose `fix` says to run `gbrain doctor`; the block never vanishes silently). `redactReadinessForRemote(report)` is what the `search_modes` op returns when `ctx.remote !== false`: `required_key`, `key_present` and `fix` are dropped and `self_hosted` is forced false — which env vars exist on the host is fingerprinting data and the fix names them; `ready` stays because it is observable anyway (reranked results carry `rerank_score`). `src/commands/search.ts:formatModesText` renders the runtime `Reranker:` line (`off (resolved) — …` / `<model> (enabled) — <KEY> present` / `<model> (enabled but NOT running) — <fix>`) and a per-bundle `reranker=… topNIn=… autocut=…` line. Pinned by `test/modes-report-reranker.test.ts`. The CLI dashboard leads with command-specific expansion: `query` follows the mode's expansion default (off in all modes) unless the call passes `expand`, while `search` is off. It discloses provider prerequisites/text flow and distinguishes retained bundle configuration from effective command behavior.- `src/core/search/private-visibility.ts` — shared page-visibility resolver and SQL predicates. Explicit local trust disables page filtering; operator env/config opt-outs keep their semantics; failed config reads enforce privacy. `privatePagesFilterFragment` authorizes concrete page rows and `isPrivatePage` is its row-side twin: explicit `private` hides a page, absent visibility is world for ordinary pages and private for extracted atoms (`type = 'atom'`) and synthesized concepts (`type = 'concept'` with `synthesized_by`), because their origin may be a private page or transcript. The page predicate also follows derived origins: an atom whose `source_slug` page is explicitly private, and a synthesized concept with a private input atom (or an input whose origin is private) over `concept-provenance` edges, are private whatever their own field says, so a later private flip of an origin reaches derived pages before any repair. Chronicle events follow `event.depth` (fail closed). Declared lineage matches `derived_from` slugs with `= ANY(ARRAY(...))`, not `IN (SELECT ... jsonb_array_elements_text(...))`: same rows, but the planner priced the IN form as a 100-row semi-join per page and pushed every read carrying the rule past `jit_above_cost` (`test/e2e/private-visibility-plan-postgres.test.ts`). `privateSnapshotFilterFragment` is the same rule on a row's own frontmatter for `page_versions`. `effectiveVisibility(origin)` (transcript or missing origin private, page origin per the read rule) and `strictestVisibility` are used by both `extract_atoms` paths, concept synthesis and `gbrain repair visibility`. Remote publication rechecks (`persistence/page-visibility.ts`) and the alias-inject recheck in `hybrid.ts` use the same rule. Tests: `test/private-visibility.test.ts`, `test/derived-page-visibility.test.ts` (PGLite and Postgres); link origins and timeline projections use independent ID-based predicates. `privateProvenanceFilterFragment` hides a fact whose provenance page (`s... (line truncated to 2000 chars)-- `src/core/access/` (GBRA-218) — per-member read access inside one source. `principals.ts` normalizes frontmatter `readable_by` (bare id = `member:<id>`, `group:<g>`, ASCII case fold, space trim; anything malformed is `['!nobody']`); it is the row twin of the SQL `gbrain_access_readers()` (`test/access-principals.test.ts` holds them equal). A BEFORE trigger keeps `pages.page_readers` equal to that function (direct writes to the column are recomputed), and predicates test the array, never the function, which costs ~0.2 ms per row. A page that names readers with no derived value yet (written before the migration) reads as nobody until `gbrain repair access` (`repair/access.ts`). `members.ts` (members, groups, tombstoned ids never reused, bindings), `token-binding.ts` (the verifiers' join; a credential bound to a disabled, tombstoned or missing member is refused with `access_member_disabled`) and `caller.ts` (`remoteCallerAccess`: the identity dispatch puts on the caller-access record; stdio binds through `mcp.stdio_member`; publication re-resolves from the stored principal). In `search/private-visibility.ts` every page fragment is `accessAllowedSql` AND the privacy part: plain fragments use the empty identity (over-hide, never leak), `*ForCaller` fragments the caller's, and `GBRAIN_REMOTE_PRIVATE_PAGES`, `search.remote_private_pages` and `granted_sources` relax only the privacy part (`privacyForCaller`). A derivative of a page that names readers is hidden from every remote caller (`originRestrictedSql`). Config `access.restrictions_present` (set by a trigger, sticky) keeps every pre-A1 path on a brain with no restriction. `check:remote-privacy-reads` keeps the two relaxations read only in `private-visibility.ts`. Tests: `test/member-read-access.test.ts` (remote matrix, both engines), `test/access-*.test.ts`.+- `src/core/access/` (GBRA-218) — per-member read access inside one source. `principals.ts` normalizes frontmatter `readable_by` (bare id = `member:<id>`, `group:<g>`, ASCII case fold, space trim; anything malformed is `['!nobody']`); it is the row twin of the SQL `gbrain_access_readers()` (`test/access-principals.test.ts` holds them equal). A BEFORE trigger keeps `pages.page_readers` equal to that function (direct writes to the column are recomputed), and predicates test the array, never the function, which costs ~0.2 ms per row. A page that names readers with no derived value yet (written before the migration) reads as nobody until `gbrain repair access` (`repair/access.ts`). `members.ts` (members, groups, tombstoned ids never reused, bindings), `token-binding.ts` (the verifiers' join; a credential bound to a disabled, tombstoned or missing member is refused with `access_member_disabled`) and `caller.ts` (`remoteCallerAccess`: the identity dispatch puts on the caller-access record; stdio binds through `mcp.stdio_member`; publication re-resolves from the stored principal). In `search/private-visibility.ts` every page fragment is `accessAllowedSql` AND the privacy part: plain fragments use the empty identity (over-hide, never leak), `*ForCaller` fragments the caller's, and `GBRAIN_REMOTE_PRIVATE_PAGES`, `search.remote_private_pages` and `granted_sources` relax only the privacy part (`privacyForCaller`). A derivative of a page that names readers is hidden from every remote caller (`originRestrictedSql`, which calls the SQL function `gbrain_access_origin_restricted()` only for atom, question, concept, event or `derived_from` rows, so each read plans one call instead of four subplans). The readers tests use `frontmatter ? 'readable_by'`, never `->'readable_by' IS NULL`, whose row estimate (1) sent restricted vector searches into nested-loop sequential scans. Config `access.restrictions_present` (set by a trigger, sticky) keeps every pre-A1 path on a brain with no restriction. `check:remote-privacy-reads` keeps the two relaxations read only in `private-visibility.ts`. Tests: `test/member-read-access.test.ts` (remote matrix, both engines), `test/access-*.test.ts`.- `src/core/search/projection-readiness.ts` — one statement probes visible unsealed pages: ready, projection_pending, by_design (protected/archived only) or unknown; LIMIT 1 arms walk the v161 pending index, not pages. Cached per scope by DB generation (page clock, pages filenode, archive flags, no open tx); `GBRAIN_PROJECTION_READINESS_CACHE=0` disables.diff --git a/src/core/schema-migrations/v274-member-read-access.ts b/src/core/schema-migrations/v274-member-read-access.tsindex 27f5d40f4e..198b4251b4 100644--- a/src/core/schema-migrations/v274-member-read-access.ts+++ b/src/core/schema-migrations/v274-member-read-access.ts@@ -13,6 +13,8 @@ import type { Migration } from './types.ts';// is rewritten on upgrade. A trigger sets config access.restrictions_present// once any page names readers or any member exists (sticky), so a brain with no// restriction keeps every pre-GBRA-218 read path and escape hatch exactly.+// gbrain_access_origin_restricted() is the read-time lineage test (a derived+// page whose origin names readers); reads call it behind a cheap guard.// No schema.sql mirror: like v273, these exist only// through migrations. Provisional number: renumbered at the merge slot.export const v274: Migration = {@@ -66,6 +68,33 @@ CREATE OR REPLACE FUNCTION gbrain_access_readers(fm JSONB) RETURNS TEXT[]THEN gbrain_access_norm_readers(ARRAY(SELECT jsonb_array_elements_text(fm->'readable_by')))ELSE ARRAY['!nobody'] END$fn$;+CREATE OR REPLACE FUNCTION gbrain_access_origin_restricted(p_id INTEGER, p_source_id TEXT, p_type TEXT, fm JSONB) RETURNS boolean+ LANGUAGE plpgsql STABLE PARALLEL SAFE SET search_path = pg_catalog, public AS $fn$+BEGIN+ IF p_type = 'atom' THEN+ IF EXISTS (SELECT 1 FROM pages o WHERE o.source_id = p_source_id AND o.slug = fm->>'source_slug' AND o.frontmatter ? 'readable_by') THEN+ RETURN true;+ END IF;+ ELSIF p_type = 'question' OR (p_type = 'concept' AND fm->>'synthesized_by' IS NOT NULL) THEN+ IF EXISTS (SELECT 1 FROM links l JOIN pages i ON i.id = l.to_page_id+ WHERE l.from_page_id = p_id AND l.link_source = 'concept-provenance' AND l.link_type = 'synthesized_from'+ AND (i.frontmatter ? 'readable_by' OR EXISTS (SELECT 1 FROM pages io WHERE io.source_id = i.source_id+ AND io.slug = i.frontmatter->>'source_slug' AND io.frontmatter ? 'readable_by'))) THEN+ RETURN true;+ END IF;+ ELSIF p_type = 'event' AND fm->>'captured_via' LIKE 'life-chronicle:%' THEN+ IF EXISTS (SELECT 1 FROM pages c WHERE c.source_id = p_source_id AND c.slug = fm->'event'->>'depth' AND c.frontmatter ? 'readable_by') THEN+ RETURN true;+ END IF;+ END IF;+ IF jsonb_typeof(fm->'derived_from') IN ('array', 'string') THEN+ RETURN EXISTS (SELECT 1 FROM pages d WHERE d.source_id = p_source_id AND d.frontmatter ? 'readable_by'+ AND d.slug = ANY(ARRAY(SELECT regexp_replace(declared.slug, '\\.md$', '') FROM jsonb_array_elements_text(+ CASE WHEN jsonb_typeof(fm->'derived_from') = 'array' THEN fm->'derived_from' ELSE jsonb_build_array(fm->'derived_from') END) AS declared(slug))));+ END IF;+ RETURN false;+END+$fn$;CREATE OR REPLACE FUNCTION gbrain_access_mark_restricted() RETURNS triggerLANGUAGE plpgsql SET search_path = pg_catalog, public AS $fn$BEGINdiff --git a/src/core/search/private-visibility.ts b/src/core/search/private-visibility.tsindex 46e9d4a0d7..582cd6db13 100644--- a/src/core/search/private-visibility.ts+++ b/src/core/search/private-visibility.ts@@ -84,12 +84,12 @@ export function readersAllowedSql(alias: string, identity: readonly string[], roconst ids = identity.length && validIdentity(identity) ? identity : [];const readers = row === 'page' ? `${alias}.page_readers` : `gbrain_access_readers(${alias}.frontmatter)`;const listed = ids.length ? ` OR ${readers} && ARRAY[${ids.map(id => `'${id}'`).join(', ')}]::text[]` : '';- return `(${alias}.frontmatter->'readable_by' IS NULL${listed})`;+ return `(NOT (${alias}.frontmatter ? 'readable_by')${listed})`;}/** A page names readers (valid or not); SQL twin of hasReaders. */function namesReadersSql(alias: string): string {- return `(${alias}.frontmatter->'readable_by' IS NOT NULL)`;+ return `(${alias}.frontmatter ? 'readable_by')`;}/**@@ -99,25 +99,13 @@ function namesReadersSql(alias: string): string {* atom's origin), a Life Chronicle event extracted from one, or a page whose* declared `derived_from` names one. Read-time, so derivatives written before* the origin gained readers are covered; new derivations are stamped private- * by isDerivationPrivate.+ * by isDerivationPrivate. The lineage test lives in the SQL function+ * gbrain_access_origin_restricted (migration v274) behind a cheap type and+ * key guard, so a read plans one function call instead of four subplans.*/function originRestrictedSql(p: string): string {- const restrictedOrigin = (atom: string, alias: string) => `EXISTS (SELECT 1 FROM pages ${alias} WHERE ${alias}.source_id = ${atom}.source_id- AND ${alias}.slug = ${atom}.frontmatter->>'source_slug' AND ${namesReadersSql(alias)})`;- return `((CASE WHEN ${p}.type = 'atom' THEN ${restrictedOrigin(p, 'acl_origin')}- WHEN ${derivedPageSql(p)} THEN EXISTS (SELECT 1 FROM links acl_input_link- JOIN pages acl_input ON acl_input.id = acl_input_link.to_page_id- WHERE acl_input_link.from_page_id = ${p}.id AND acl_input_link.link_source = 'concept-provenance'- AND acl_input_link.link_type = 'synthesized_from'- AND (${namesReadersSql('acl_input')} OR ${restrictedOrigin('acl_input', 'acl_input_origin')}))- WHEN ${p}.type = 'event' AND ${p}.frontmatter->>'captured_via' LIKE 'life-chronicle:%' THEN EXISTS (SELECT 1 FROM pages acl_chronicle- WHERE acl_chronicle.source_id = ${p}.source_id AND acl_chronicle.slug = ${p}.frontmatter->'event'->>'depth' AND ${namesReadersSql('acl_chronicle')})- ELSE false END)- OR (CASE WHEN jsonb_typeof(${p}.frontmatter->'derived_from') IN ('array', 'string') THEN EXISTS (- SELECT 1 FROM pages acl_declared WHERE acl_declared.source_id = ${p}.source_id AND ${namesReadersSql('acl_declared')}- AND acl_declared.slug = ANY(ARRAY(SELECT regexp_replace(declared.slug, '\\.md$', '') FROM jsonb_array_elements_text(- CASE WHEN jsonb_typeof(${p}.frontmatter->'derived_from') = 'array' THEN ${p}.frontmatter->'derived_from'- ELSE jsonb_build_array(${p}.frontmatter->'derived_from') END) AS declared(slug)))) ELSE false END))`;+ return `(CASE WHEN ${p}.type IN ('atom', 'question', 'concept', 'event') OR ${p}.frontmatter ? 'derived_from'+ THEN gbrain_access_origin_restricted(${p}.id, ${p}.source_id, ${p}.type, ${p}.frontmatter) ELSE false END)`;}/**diff --git a/test/repair-command.test.ts b/test/repair-command.test.tsindex b4aea612f9..e7593afe58 100644--- a/test/repair-command.test.ts+++ b/test/repair-command.test.ts@@ -214,7 +214,7 @@ describe('gbrain repair timeline', () => {} finally { console.log = log; }const json = JSON.parse(out[0]);expect(json.scope.source_ids).toEqual([source]);- expect(json.results.map((r: { kind: string }) => r.kind)).toEqual(['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'fences', 'slug-conflicts']);+ expect(json.results.map((r: { kind: string }) => r.kind)).toEqual(['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'fences', 'slug-conflicts', 'access']);expect(json.results[0]).toMatchObject({ mode: 'dry_run', affected: 1 });expect(out[1]).toContain(`Scope: brain `);expect(out[1]).toContain(`sources ${source}`);