Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(access): core memory, entity resolution, reflex pointers and pull briefs read with the caller's policy
garrytandd126635c406
diff --git a/docs/architecture/key-files/core-services-1-continued.md b/docs/architecture/key-files/core-services-1-continued.mdindex cb2792afcc..82c8d81fe6 100644--- a/docs/architecture/key-files/core-services-1-continued.md+++ b/docs/architecture/key-files/core-services-1-continued.md@@ -9,7 +9,7 @@ The content-repair judgment tier under `src/core/content-repair/`, then the Retr- `src/core/content-repair/judgment.ts` — #6377 (Lane B3) the content-repair lane's model tier for a slug conflict: `judgeSlugConflict` builds the prompt from both files' frontmatter, headings and first 60 lines of body plus every line that mentions the other's slug anywhere in either body (evidence past line 60 counts), both slugs and paths, the hold's reason and the allowed answers, and parses exactly one JSON object `{"action":"remove_slug"}`, `{"action":"merge_into","canonical":"<one of the two slugs>"}` or `{"action":"needs_human"}` with an optional `why` sentence that reaches the run's output only, never hold metadata. The same failure classes, `thinkingOffControl` and output ceiling as the fence tier; a `canonical` outside the two slugs is `llm_malformed`. Memo key: sha256 of both participants' content + both ids + model + prompt version + rules version. The model is `models.content_repair` (an explicit model always runs), else `models.fence_repair`, else `CONTENT_REPAIR_MEASURED_MODELS` (the eval's qualifiers, best first, ties to the cheaper), else the fence list. Tests: `test/repair-slug-conflicts.test.ts`.- `src/core/content-repair/llm.ts` — #6377 (Lane B3) the paid call behind `judgment.ts`: one gateway chat under the fence repair's caps and ledger (`FENCE_REPAIR_LEDGER`, so one `fences.repair.max_usd_per_day` covers the whole lane and today's fence spend counts; the per-page cap applies per candidate), `fences.repair.llm=false` and `--no-llm` turn it off, the attempt memo keeps rejected bytes from costing twice, and a transient `llm_unavailable` does not consume the memo. Spend is reported as the item's `llm_usd` and carried into the `content_repair` receipt (`model`, `cost`); prompts never include the rest of either page.-- `src/core/context/` — Retrieval Reflex (Layer 1). `entity-salience.ts`: pure, zero-LLM, precision-biased `extractCandidates(text)` (capitalized runs + `@handles`, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emits `weak: true` candidates — lowercase words ≥3 chars on a separate `MAX_WEAK_CANDIDATES=32` budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) + `extractCandidatesFromWindow(turns)` (merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first). `retrieval-reflex.ts`: `resolveEntitiesToPointers(engine, sourceId, candidates, opts)` — alias arm (`resolveAliases`, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespaced `people/x` but `slugify` drops the prefix) + two lexical identity arms behind `opts.lexicalArms` (kill switch: config `retrieval_reflex_lexical_arms` / env `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via a `lower(title) LIKE '% <token>'` predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carry `source_id`/`arm`/`confidence`/`matchedNorm` (`ARM_CONFIDENCE` alias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts: `sourceIds?` federated scope (alias arm loops per source, arm 2 uses `source_id = ANY`), `suppression?` ('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver — `logDeliveredReflexPointers(engine, pointers)` fires only once a block is actually handed to the consumer (serve's resolve-IPC `onDelivered` hook post-write; `buildReflexAddition` post-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs through `stripTakesFence`/`stripFactsFence` (the same privacy boundary `get_page` applies) so private facts never reach the prompt; `visibility: private` (and derived-private) pages never resolve on any arm unless the caller passes `excludePrivate: false` (default true, the remote-search predicate `privatePagesFilterFragment`; only `volunteer_context` widens, via `resolveExcludePrivatePages` for a trusted local caller — gbrain-evals N8-1/N8-2, pinned by `test/reflex-private-visibility.test.ts`); capped at `MAX_POINTERS`. `reflex.ts`: the orchestrator + engine-aware resolver ladder (host `resolveEntities` → PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check, `reflexEnabled(cfg)` (file/env gate, default ON; DB-plane does NOT gate — `assemble()` is sync); windowed extraction when `windowTurns` present and `retrieval_reflex_window_turns` (default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap). `resolve-ipc.ts`: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connection `gbrain serve` holds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports via `src/mcp/resolve-ipc-binding.ts` (engine-uniform; socket + secret keyed off `hash12(database_url)` under `~/.gbrain/run`, cleaned up on shutdown). A lean read-only Postgres serve binds the `read` socket kind instead (#7612); `hookResolveSocketForConfig` falls back to it after the source socket, and a `delegation: true` caller never targets it. Doctor surface: `retrieval_reflex_health` in `src/commands/doctor/checks/verbs-reflex.ts` (reads the heartbeat for truthful runtime status; categorized in `doctor-categories.ts`) + `volunteer_channels` (engine-aware sibling: groups `context_volunteer_events` by channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned by `test/doctor-volunteer-channels.test.ts`). Config: `retrieval_reflex` + `retrieval_reflex_max_pointers` + `retrieval_reflex_window_turns` + `retrieval_reflex_lexical_arms` in `src/core/config.ts` (env `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`, `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`). `volunteer.ts`: `parseWindow` (lenient `user:`/`assistant:` prefixes, unprefixed → one user turn), `volunteerContext` (extract → resolve → +0.05 multi-turn/newest-turn boost → `min_confidence` 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression), `volunteerUsageStats` (per-arm/channel precision from the `pages.last_retrieved_at > volunteered_at` join — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives). `volunteer-events.ts`: `insertVolunteerEvents` (ONE multi-row parameterized INSERT), `logVolunteerEventsFireAndForget` + bounded drain registered as the `volunteer-events` background-work sink (order 4), `purgeStaleVolunteerEvents` (90-day GC, called from the dream cycle's purge phase). Policy layer ships as the `retrieval-reflex` recipe (`recipes/retrieval-reflex/`). Pinned by `test/context/entity-salience.test.ts`, `test/retrieval-reflex.test.ts`, `test/retrieval-reflex-pre-v110.test.ts`, `test/context/resolve-ipc.test.ts`, `test/doctor-retrieval-reflex.test.ts`, `test/volunteer-context.test.ts`, `test/e2e/volunteer-context-postgres.test.ts`. Checkpoint-compaction modules in this dir: `corpus-segments.ts` — engine-free content-addressed compaction segments (`<session>.seg-<hash24>.txt`, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal), `sliceBoundaryWindow`/`splitByBoundaries`, the redacted `renderSegmentText` (a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-set `coverageComplete`/`decideCorpusMode` (count equality can be fooled by a duplicated boundary), the compact-time `bankCompactSegment` step (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (`readOpenclawBoundaryTail`, delegates mapping to the adapter's exported `mapOpenclawLine`; also returns turn-stamped `toolCalls`/`toolCallTurnIndexes` for the Memorable receipt — name-only v1, `input: null` by design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned by `test/corpus-segments.test.ts`. `checkpoint-harvest.ts` — the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates, `signal.aborted` POST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scoped `getPage` verification — a link that resolves to nothing is never banked), `.ingested` last, explicit `shutdownCheckpointHarvest()` called by serve BEFORE `engine.disconnect()` (the background-work drain is CLI-exit-only by contract), a writeback turn whose facts preflight finds the canonical writer busy (`writer_lock_unavailable`, before any model call) re-queued after 5, 15 and 45 s (`writer_busy_requeued`; the retry skips admission, a pending retry dedups a re-schedule as `already_queued`, shutdown clears pending retries) while a compact segment's busy refusal stays an error the sweep resumes, and error reasons that keep the error's code or `reason` within the 48-char reason-code shape (`operationerror:writer_lock_unavailable`), the first failure of each reason logged to stderr once per serve run — pinned by `test/checkpoint-harvest.serial.test.ts`. `hook-heartbeat.ts` — the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpoint `segment`/`inserted`/`duplicate`/`links` count-only fields. `session-state.ts` also carries the v132 `checkpoint_manifest` helpers (`getCheckpointManifest`/`appendCheckpointManifest`: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned by `test/checkpoint-manifest.test.ts`. `sensitivity-scan.ts` + `compile-view.ts` — the compile-context stack: composed detector (secret-scan + ordered `PII_PATTERNS` + path/blocklist families + operator pattern file; uniform `.gbrain-scan-allow` fingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidate `updated_at`, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs and pass `excludePrivate: true` on every arm, so private pages never reach the compiled file; whole-file `packToBudget` math that never passes a <=0 budget) — pinned by `test/sensitivity-scan.test.ts` + `test/compile-view.test.ts` + `test/e2e/compile-context-pglite.test.ts`; the CLI shell is `src/commands/compile-context.ts` (targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes, `--check` recompile-and-compare exit codes; the guide is `docs/guides/checkpoint-compaction.md`). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (`session-receipts.jsonl`, `appendSessionReceipt` — dedup by post-redaction `content_hash` so a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and a `skipCompaction` seam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptive `readJsonlTail` reader shared by `lastReceiptMatches`, `lastRelayResult` and `readSessionReceiptsTail` (a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty"); `maybeTrimRelayResults` (BOTH capture lanes trim the child-appended `memorable-relay.jsonl` — an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (`memorable-consent.json`, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 + `MEMORABLE_CAPTURE_HARNESSES`, so widening the capture surface invalidates old stamps); `memorableGateAllowed(cfg)` ({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor); `memorableConsentEvidence()` (the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam); `redactedToolCallsJson` (the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via \" escaping — with a post-serialization re-scan as belt-and-braces); `recordAndRelayReceipt` (per-receipt harness scope-binding first — a harness outside `MEMORABLE_CAPTURE_HARNESSES` is refused with `memorable_harness_undisclosed` → receipt → prior-run outcome surfacing with the child's reason clamped via the shared `clampRelayCause` (32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence → `resolveMemorableBin` → detached spawn; never throws; `spawnFn` test seam); and `priorRelayFailure`, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy. `resolveMemorableBin` does the pre-spawn PATH/`MEMORABLE_BIN` resolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT. #6195: `entity-salience.extractCandidates` also emits lowercase 2-3-word weak n-grams (`multiToken`, own budget `MAX_WEAK_NGRAM_CANDIDATES` = 16, never across punctuation, never all stopwords/common words) that ride the exact weak-alias fold, and `retrieval-reflex` arm 2.6 (`weak-title`, 0.72, behind `lexicalArms`, private pages hidden by default) matches them to the exact title of a person/company/organization page whose title is globally unique; single lowercase words stay alias-only.+- `src/core/context/` — Retrieval Reflex (Layer 1). `entity-salience.ts`: pure, zero-LLM, precision-biased `extractCandidates(text)` (capitalized runs + `@handles`, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emits `weak: true` candidates — lowercase words ≥3 chars on a separate `MAX_WEAK_CANDIDATES=32` budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) + `extractCandidatesFromWindow(turns)` (merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first). `retrieval-reflex.ts`: `resolveEntitiesToPointers(engine, sourceId, candidates, opts)` — alias arm (`resolveAliases`, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespaced `people/x` but `slugify` drops the prefix) + two lexical identity arms behind `opts.lexicalArms` (kill switch: config `retrieval_reflex_lexical_arms` / env `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via a `lower(title) LIKE '% <token>'` predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carry `source_id`/`arm`/`confidence`/`matchedNorm` (`ARM_CONFIDENCE` alias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts: `sourceIds?` federated scope (alias arm loops per source, arm 2 uses `source_id = ANY`), `suppression?` ('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver — `logDeliveredReflexPointers(engine, pointers)` fires only once a block is actually handed to the consumer (serve's resolve-IPC `onDelivered` hook post-write; `buildReflexAddition` post-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs through `stripTakesFence`/`stripFactsFence` (the same privacy boundary `get_page` applies) so private facts never reach the prompt; `visibility: private` (and derived-private) pages never resolve on any arm unless the caller passes `excludePrivate: false` (default true, the caller's read policy `privatePagesFilterForCaller`, the predicate get_page uses, so a bound member resolves the pages its readers allow; only `volunteer_context` widens, via `resolveExcludePrivatePages` for a trusted local caller — gbrain-evals N8-1/N8-2, pinned by `test/reflex-private-visibility.test.ts`); capped at `MAX_POINTERS`. `reflex.ts`: the orchestrator + engine-aware resolver ladder (host `resolveEntities` → PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check, `reflexEnabled(cfg)` (file/env gate, default ON; DB-plane does NOT gate — `assemble()` is sync); windowed extraction when `windowTurns` present and `retrieval_reflex_window_turns` (default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap). `resolve-ipc.ts`: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connection `gbrain serve` holds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports via `src/mcp/resolve-ipc-binding.ts` (engine-uniform; socket + secret keyed off `hash12(database_url)` under `~/.gbrain/run`, cleaned up on shutdown). A lean read-only Postgres serve binds the `read` socket kind instead (#7612); `hookResolveSocketForConfig` falls back to it after the source socket, and a `delegation: true` caller never targets it. Doctor surface: `retrieval_reflex_health` in `src/commands/doctor/checks/verbs-reflex.ts` (reads the heartbeat for truthful runtime status; categorized in `doctor-categories.ts`) + `volunteer_channels` (engine-aware sibling: groups `context_volunteer_events` by channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned by `test/doctor-volunteer-channels.test.ts`). Config: `retrieval_reflex` + `retrieval_reflex_max_pointers` + `retrieval_reflex_window_turns` + `retrieval_reflex_lexical_arms` in `src/core/config.ts` (env `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`, `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`). `volunteer.ts`: `parseWindow` (lenient `user:`/`assistant:` prefixes, unprefixed → one user turn), `volunteerContext` (extract → resolve → +0.05 multi-turn/newest-turn boost → `min_confidence` 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression), `volunteerUsageStats` (per-arm/channel precision from the `pages.last_retrieved_at > volunteered_at` join — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives). `volunteer-events.ts`: `insertVolunteerEvents` (ONE multi-row parameterized INSERT), `logVolunteerEventsFireAndForget` + bounded drain registered as the `volunteer-events` background-work sink (order 4), `purgeStaleVolunteerEvents` (90-day GC, called from the dream cycle's purge phase). Policy layer ships as the `retrieval-reflex` recipe (`recipes/retrieval-reflex/`). Pinned by `test/context/entity-salience.test.ts`, `test/retrieval-reflex.test.ts`, `test/retrieval-reflex-pre-v110.test.ts`, `test/context/resolve-ipc.test.ts`, `test/doctor-retrieval-reflex.test.ts`, `test/volunteer-context.test.ts`, `test/e2e/volunteer-context-postgres.test.ts`. Checkpoint-compaction modules in this dir: `corpus-segments.ts` — engine-free content-addressed compaction segments (`<session>.seg-<hash24>.txt`, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal), `sliceBoundaryWindow`/`splitByBoundaries`, the redacted `renderSegmentText` (a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-set `coverageComplete`/`decideCorpusMode` (count equality can be fooled by a duplicated boundary), the compact-time `bankCompactSegment` step (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (`readOpenclawBoundaryTail`, delegates mapping to the adapter's exported `mapOpenclawLine`; also returns turn-stamped `toolCalls`/`toolCallTurnIndexes` for the Memorable receipt — name-only v1, `input: null` by design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned by `test/corpus-segments.test.ts`. `checkpoint-harvest.ts` — the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates, `signal.aborted` POST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scoped `getPage` verification — a link that resolves to nothing is never banked), `.ingested` last, explicit `shutdownCheckpointHarvest()` called by serve BEFORE `engine.disconnect()` (the background-work drain is CLI-exit-only by contract), a writeback turn whose facts preflight finds the canonical writer busy (`writer_lock_unavailable`, before any model call) re-queued after 5, 15 and 45 s (`writer_busy_requeued`; the retry skips admission, a pending retry dedups a re-schedule as `already_queued`, shutdown clears pending retries) while a compact segment's busy refusal stays an error the sweep resumes, and error reasons that keep the error's code or `reason` within the 48-char reason-code shape (`operationerror:writer_lock_unavailable`), the first failure of each reason logged to stderr once per serve run — pinned by `test/checkpoint-harvest.serial.test.ts`. `hook-heartbeat.ts` — the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpoint `segment`/`inserted`/`duplicate`/`links` count-only fields. `session-state.ts` also carries the v132 `checkpoint_manifest` helpers (`getCheckpointManifest`/`appendCheckpointManifest`: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned by `test/checkpoint-manifest.test.ts`. `sensitivity-scan.ts` + `compile-view.ts` — the compile-context stack: composed detector (secret-scan + ordered `PII_PATTERNS` + path/blocklist families + operator pattern file; uniform `.gbrain-scan-allow` fingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidate `updated_at`, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs and pass `excludePrivate: true` on every arm, so private pages never reach the compiled file; whole-file `packToBudget` math that never passes a <=0 budget) — pinned by `test/sensitivity-scan.test.ts` + `test/compile-view.test.ts` + `test/e2e/compile-context-pglite.test.ts`; the CLI shell is `src/commands/compile-context.ts` (targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes, `--check` recompile-and-compare exit codes; the guide is `docs/guides/checkpoint-compaction.md`). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (`session-receipts.jsonl`, `appendSessionReceipt` — dedup by post-redaction `content_hash` so a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and a `skipCompaction` seam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptive `readJsonlTail` reader shared by `lastReceiptMatches`, `lastRelayResult` and `readSessionReceiptsTail` (a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty"); `maybeTrimRelayResults` (BOTH capture lanes trim the child-appended `memorable-relay.jsonl` — an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (`memorable-consent.json`, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 + `MEMORABLE_CAPTURE_HARNESSES`, so widening the capture surface invalidates old stamps); `memorableGateAllowed(cfg)` ({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor); `memorableConsentEvidence()` (the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam); `redactedToolCallsJson` (the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via \" escaping — with a post-serialization re-scan as belt-and-braces); `recordAndRelayReceipt` (per-receipt harness scope-binding first — a harness outside `MEMORABLE_CAPTURE_HARNESSES` is refused with `memorable_harness_undisclosed` → receipt → prior-run outcome surfacing with the child's reason clamped via the shared `clampRelayCause` (32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence → `resolveMemorableBin` → detached spawn; never throws; `spawnFn` test seam); and `priorRelayFailure`, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy. `resolveMemorableBin` does the pre-spawn PATH/`MEMORABLE_BIN` resolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT. #6195: `entity-salience.extractCandidates` also emits lowercase 2-3-word weak n-grams (`multiToken`, own budget `MAX_WEAK_NGRAM_CANDIDATES` = 16, never across punctuation, never all stopwords/common words) that ride the exact weak-alias fold, and `retrieval-reflex` arm 2.6 (`weak-title`, 0.72, behind `lexicalArms`, private pages hidden by default) matches them to the exact title of a person/company/organization page whose title is globally unique; single lowercase words stay alias-only.- `src/core/context/hook-heartbeat.ts` — the hooks telemetry plane under `<gbrain home>/integrations/hooks/` (`heartbeat.jsonl`, `status.json`, session receipts, the memorable relay log; directories 0700, appends 0600). Entries hold counters, durations and codes only: `HEARTBEAT_ALLOWED_KEYS` is the full key allowlist CI greps the fixture against, and includes `pattern` (scanner pattern name), `fingerprint` (`sha256:<16 hex>`) and `hint` (fixed text) for refusals. `near_duplicate` is the shadow count on the `writeback_dedup` event that `facts/capture-dedup.ts` appends (lane in `reason`, exact drops in `duplicate`). `relayRefusalHeartbeat(hit)` builds the line `context-engine.ts` writes when its high-entropy re-scan refuses a memorable relay (`event: relay`, `outcome: degraded`, `reason: secret_scan_refused`); the receipt and relay are skipped for that compaction window and the checkpoint is kept. `memorableGateAllowed` gates the relay lane; span redaction runs `redactFindings` with `highEntropy: true`. Tests: `test/context-engine-checkpoint.serial.test.ts`.- `src/core/context/seat.ts` — seat identity for captured sessions (#4618), engine-free. `resolveSeat` returns the lowercased `GBRAIN_SEAT` label when it matches `SEAT_LABEL_RE` (`^[a-z0-9][a-z0-9._-]{0,63}$`), else `home-<8 hex>` over the realpath of the harness home (Claude Code: the directory holding `projects/` in the transcript path, then `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `~/.claude`; codex: `CODEX_HOME` or `~/.codex`; OpenClaw: the agent directory holding `sessions/`), so no raw path reaches frontmatter; `GBRAIN_SEAT=off` (the opt-out) resolves to null and nothing is recorded. `writeSeatSidecar` writes `<sessionId>.seat.json` (`{version: 1, seat, seat_source, hook_lane, harness, first_seen}`, 0600, tmp+rename) unless one exists, reporting `seat_conflict` when the existing seat differs; `readSeatSidecar` accepts only a valid v1 record. `SEAT_REASON_HINTS` holds the fixed heartbeat hints. The sidecar name and its GC live in `corpus-segments.ts` (`seatFileName`, `SEAT_SIDECAR_SUFFIX`).- `src/core/context/session-speaker.ts` — who wrote a captured session's user turns (#8064), engine-free. `resolveSessionSpeaker` reads `GBRAIN_SESSION_AGENT` (a label: agent; `off`: skip banking; `owner`: no detection; invalid: agent `agent`), else `detectAgentLaunched` over `readProcessAncestry` (Linux /proc, else `ps`; 12 levels, 300 ms): the nearest claude/codex process runs headless (`-p`/`--print`, `exec`) and another claude/codex process is an ancestor, principal `agent-launched`. `bankAsSpeaker` writes `<sessionId>.speaker.json` (`{version: 1, principal, source, harness, first_seen}`, 0600, first write wins) before the hook's compact, stop and session-end banking; a write failure banks nothing (`speaker_write_failed`), `off` records `capture_opt_out`. `corpusFileSpeaker`/`speakerOf` give the capture lanes (checkpoint-harvest, sweep) the `speaker` they pass to `runFactsPipeline`, which saves user and unattributed claims as `other` (`asAgentSpoken`), adds `speakerContextNote` to the context (`facts/speaker-context.ts`, also used by the managed facts batch) and names the agent in the extractor prompt. Name and GC: `speakerFileName`, `SPEAKER_SIDECAR_SUFFIX` in `corpus-segments.ts`. Pinned by `test/agent-session-speaker-8064.serial.test.ts`.diff --git a/src/core/context/proactive-brief.ts b/src/core/context/proactive-brief.tsindex e662df3aca..fc4e5e5e02 100644--- a/src/core/context/proactive-brief.ts+++ b/src/core/context/proactive-brief.ts@@ -19,7 +19,7 @@ import { isConfigTruthy } from '../config.ts';import { HALFLIFE_DAYS } from '../facts/decay.ts';import { listOpenLoops, type OpenLoopRow } from '../loops/loops-store.ts';import { normalizeAlias } from '../search/alias-normalize.ts';-import { privatePagesFilterFragment, privateProvenanceFilterFragment, privateTimelineEventFilterFragment } from '../search/private-visibility.ts';+import { privatePagesFilterForCaller, privatePagesFilterFragment, privateProvenanceFilterFragment, privateTimelineEventFilterForCaller } from '../search/private-visibility.ts';import { pageEligibleSql, projectionEligibleSql } from '../eligibility/sql.ts';import type { ReadEligibility } from '../eligibility/policy.ts';import { renderTrustedInline, trustFields } from '../eligibility/labels.ts';@@ -202,7 +202,7 @@ async function entityPages(const rows = await engine.executeRaw<{ slug: string; title: string | null; type: string | null }>(`SELECT p.slug, p.title, p.type FROM pages pWHERE p.source_id = $1 AND p.slug = ANY($2::text[]) AND p.deleted_at IS NULL- ${opts.excludePrivate ? `AND ${privatePagesFilterFragment('p')}` : ''}+ ${opts.excludePrivate ? `AND ${privatePagesFilterForCaller('p')}` : ''}AND ${pageEligibleSql('p', opts.eligibility)}`,[sourceId, slugs],);@@ -483,7 +483,7 @@ async function editItem(engine: BrainEngine, cp: BriefCounterparty, clock: Briefp.timeline, p.trust_tier, p.write_originFROM pages pWHERE p.source_id = $1 AND p.slug = $2 AND p.deleted_at IS NULL- AND ${opts.includePrivate ? 'TRUE' : privatePagesFilterFragment('p')}+ AND ${opts.includePrivate ? 'TRUE' : privatePagesFilterForCaller('p')}AND ${pageEligibleSql('p', opts.eligibility)}`,[cp.source_id, cp.slug, EDIT_MAX_PAGE_BYTES],);@@ -493,7 +493,7 @@ async function editItem(engine: BrainEngine, cp: BriefCounterparty, clock: Briefengine.executeRaw<{ id: number; date: string; summary: string; detail: string; created_at: string | Date }>(`SELECT te.id, te.date::text AS date, te.summary, te.detail, te.created_at FROM timeline_entries teWHERE te.page_id = $1 AND te.created_at >= $2::timestamptz AND te.date >= $2::timestamptz::date- AND ${opts.includePrivate ? 'TRUE' : privateTimelineEventFilterFragment('te')}+ AND ${opts.includePrivate ? 'TRUE' : privateTimelineEventFilterForCaller('te')}AND ${projectionEligibleSql('timeline_entries', 'te', opts.eligibility)}ORDER BY te.created_at DESC, te.date DESC, te.id DESC LIMIT 1`,[pg.id, since],diff --git a/src/core/context/retrieval-reflex.ts b/src/core/context/retrieval-reflex.tsindex 481b9e2eef..815a6fbe21 100644--- a/src/core/context/retrieval-reflex.ts+++ b/src/core/context/retrieval-reflex.ts@@ -39,7 +39,7 @@ import { slugify } from '../entities/resolve.ts';import { stripTakesFence } from '../takes-fence.ts';import { stripFactsFence } from '../facts-fence.ts';import { redactFindings } from '../secret-scan.ts';-import { privatePagesFilterFragment } from '../search/private-visibility.ts';+import { privatePagesFilterForCaller } from '../search/private-visibility.ts';import type { EntityCandidate } from './entity-salience.ts';import { reflexPointerRationale } from './reflex-rationale.ts';import { preferEntityTitleMatches } from './title-matches.ts';@@ -237,7 +237,7 @@ export async function resolveEntitiesToPointers(// v0.46.15: weak-alias + surname arms share one kill switch (default ON).const lexicalArms = opts.lexicalArms !== false;- const privacySql = opts.excludePrivate === false ? '' : `AND ${privatePagesFilterFragment('p')}`;+ const privacySql = opts.excludePrivate === false ? '' : `AND ${privatePagesFilterForCaller('p')}`;// display lookup keyed by normalized query, so resolved slugs can recover a// human surface form for the pointer label.diff --git a/src/core/core-memory.ts b/src/core/core-memory.tsindex 55e0ab2df9..f7e77d45e4 100644--- a/src/core/core-memory.ts+++ b/src/core/core-memory.ts@@ -17,7 +17,7 @@import { createHash } from 'node:crypto';import type { BrainEngine } from './engine.ts';import { sanitizeRemoteBody } from './remote-body.ts';-import { privatePagesFilterFragment } from './search/private-visibility.ts';+import { privatePagesFilterForCaller } from './search/private-visibility.ts';import { quarantineFilterFragment } from './quarantine.ts';import { compareTrust, OWNER_TIER_FLOOR, type TrustTier } from './trust/tier.ts';import type { ReadEligibility } from './eligibility/policy.ts';@@ -252,7 +252,7 @@ export async function listCorePages(exec: RawExec, opts: ListCorePagesOpts = {})params.push(opts.exclude.sourceId, opts.exclude.slug);where.push(`NOT (p.source_id = $${params.length - 1} AND p.slug = $${params.length})`);}- if (opts.excludePrivate) where.push(privatePagesFilterFragment('p'));+ if (opts.excludePrivate) where.push(privatePagesFilterForCaller('p'));const policy = opts.eligibility;if (policy) where.push(quarantineFilterFragment('p'), pageEligibleSql('p', { floor: policy.floor }));const trustCols = policydiff --git a/src/core/entities/resolve.ts b/src/core/entities/resolve.tsindex a6bb9e484f..ca97ea8375 100644--- a/src/core/entities/resolve.ts+++ b/src/core/entities/resolve.ts@@ -30,7 +30,7 @@ import { slugifySegment } from '../sync.ts';const siblingCanonical: typeof import('../mentions/siblings.ts').siblingCanonical = async (...args) =>(await import('../mentions/siblings.ts')).siblingCanonical(...args);import { isMissingSchemaObject } from '../utils.ts';-import { privatePagesFilterFragment } from '../search/private-visibility.ts';+import { privatePagesFilterForCaller } from '../search/private-visibility.ts';import { rankingLinkSql } from '../link-ranking-policy.ts';/**@@ -490,7 +490,7 @@ export async function resolveStrictEntityReference(): Promise<StrictResolution> {const trimmed = raw.trim();if (!trimmed) return { slug: null, miss: 'no_page' };- const privacy = opts.excludePrivate ? `AND ${privatePagesFilterFragment('p')}` : '';+ const privacy = opts.excludePrivate ? `AND ${privatePagesFilterForCaller('p')}` : '';const live = async (slugs: string[]) => slugs.length === 0 ? [] : engine.executeRaw<{ slug: string; type: string | null }>(`SELECT p.slug, p.type FROM pages pWHERE p.source_id = $1 AND p.deleted_at IS NULL AND p.slug = ANY($2::text[]) ${privacy}`,@@ -572,7 +572,7 @@ export async function entityReferenceCandidates(const trimmed = raw.trim();const token = slugify(trimmed);if (!token) return [];- const privacy = opts.excludePrivate ? `AND ${privatePagesFilterFragment('p')}` : '';+ const privacy = opts.excludePrivate ? `AND ${privatePagesFilterForCaller('p')}` : '';const aliases = await aliasHitSlugs(engine, source_id, trimmed).catch(() => [] as string[]);const prefixed = (await findPrefixCandidates(engine, source_id, token)).map(c => c.slug);const named = await engine.executeRaw<{ slug: string }>(diff --git a/test/caller-aux-page-reads.test.ts b/test/caller-aux-page-reads.test.tsnew file mode 100644index 0000000000..0d35dbc886--- /dev/null+++ b/test/caller-aux-page-reads.test.ts@@ -0,0 +1,96 @@+/**+ * #8395: the auxiliary page reads (core memory, strict entity resolution,+ * retrieval pointers, pull-brief counterparties and edits) apply the same+ * caller policy as get_page.+ *+ * 1. Protects: a member-bound caller gets the pages its readers allow and+ * nothing more, and the operator's privacy opt-out applies to these reads+ * the way it applies to get_page.+ * 2. Fails when: one of these reads uses the identity-less privacy fragment,+ * so a bound member loses its own readable_by pages, or when the swap lets+ * another member's restricted page through.+ * 3. Existing coverage: member-read-access pins the engine read paths.+ * 4. Real seams: PGLite, real pages, the caller-access record a bound remote+ * request runs under; repo-less brain.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { listCorePages } from '../src/core/core-memory.ts';+import { entityReferenceCandidates, resolveStrictEntityReference } from '../src/core/entities/resolve.ts';+import { resolveEntitiesToPointers } from '../src/core/context/retrieval-reflex.ts';+import { assembleBrief, resolvePullCounterparty } from '../src/core/context/proactive-brief.ts';+import { withCallerAccess, __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { addMember } from '../src/core/access/members.ts';++const SRC = 'source-a';+const ALICE = 'people/alice-example';+const BOB = 'people/bob-example';+const SECRET = 'people/carol-example';++let engine: PGLiteEngine;++function as<T>(member: 'alice' | 'bob' | null, hidePrivate: boolean, fn: () => Promise<T>): Promise<T> {+ return withCallerAccess({ identity: member ? [`member:${member}-example`] : [], hidePrivate, applyRemoteReadPolicy: true, restricted: true }, fn);+}++beforeAll(async () => {+ engine = new PGLiteEngine();+ await engine.connect({ engine: 'pglite' } as never);+ await engine.initSchema();+ await engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1)', [SRC]);+ await addMember(engine, 'alice-example');+ await addMember(engine, 'bob-example');+ const today = new Date().toISOString().slice(0, 10);+ for (const [slug, title, fm] of [+ [ALICE, 'Alice Example', { readable_by: ['alice-example'] }],+ [BOB, 'Bob Example', { readable_by: ['bob-example'] }],+ [SECRET, 'Carol Example', { visibility: 'private' }],+ ] as const) {+ await engine.putPage(slug, { type: 'person', title, compiled_truth: `Standing guidance for ${title}.`, frontmatter: { always_load: true, ...fm } }, { sourceId: SRC });+ await engine.addTimelineEntry(slug, { date: today, summary: `Timeline note for ${title}` }, { sourceId: SRC });+ }+}, 60_000);++afterAll(async () => {+ __resetPrivateVisibilityCacheForTests();+ await engine.disconnect();+});++const core = () => listCorePages(engine, { sourceIds: [SRC], excludePrivate: true }).then(pages => pages.map(p => p.slug).sort());++describe('#8395 auxiliary page reads follow the caller', () => {+ test('core memory lists the pages the bound member may read', async () => {+ expect(await as('alice', true, core)).toEqual([ALICE]);+ expect(await as('bob', true, core)).toEqual([BOB]);+ expect(await as(null, true, core)).toEqual([]);+ expect(await as('alice', false, core)).toEqual([ALICE, SECRET]);+ });++ test('strict entity resolution and refusal candidates', async () => {+ await as('alice', true, async () => {+ expect(await resolveStrictEntityReference(engine, SRC, ALICE, { excludePrivate: true })).toEqual({ slug: ALICE, arm: 'exact_page' });+ expect(await entityReferenceCandidates(engine, SRC, 'Alice Example', { excludePrivate: true })).toContain(ALICE);+ expect((await resolveStrictEntityReference(engine, SRC, BOB, { excludePrivate: true })).slug).toBeNull();+ expect((await resolveStrictEntityReference(engine, SRC, SECRET, { excludePrivate: true })).slug).toBeNull();+ });+ await as('alice', false, async () => {+ expect((await resolveStrictEntityReference(engine, SRC, SECRET, { excludePrivate: true })).slug).toBe(SECRET);+ });+ await as('bob', true, async () => {+ expect((await resolveStrictEntityReference(engine, SRC, ALICE, { excludePrivate: true })).slug).toBeNull();+ expect(await entityReferenceCandidates(engine, SRC, 'Alice Example', { excludePrivate: true })).not.toContain(ALICE);+ });+ });++ test('retrieval pointers, pull counterparties and brief edits', async () => {+ for (const who of ['alice', 'bob'] as const) {+ await as(who, true, async () => {+ const block = await resolveEntitiesToPointers(engine, SRC, [{ query: 'Alice Example', display: 'Alice Example' }] as never, { excludePrivate: true });+ expect(block?.pointers.some(p => p.slug === ALICE) ?? false).toBe(who === 'alice');+ expect((await resolvePullCounterparty(engine, SRC, ALICE, { excludePrivate: true }))?.slug ?? null).toBe(who === 'alice' ? ALICE : null);+ const brief = await assembleBrief(engine, { trigger: 'pull', local: false, counterparties: [{ slug: ALICE, source_id: SRC, display: 'Alice Example', confidence: 1 }], clock: { now: Date.now(), stopAt: Date.now() + 30_000 } });+ expect(brief?.text.includes('Timeline note for Alice Example') ?? false).toBe(who === 'alice');+ });+ }+ }, 30_000);+});