Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

A late volunteer-log write no longer restores a purged page

garrytan386fb19583a5
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 166137a4e6..d99c41c0ab 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -27,20 +27,22 @@ identifiers and attribution are available in the pre-removal Git revision
### Removing a source, a connector delete and a late eval capture no longer leave a deleted page's slug behind
-**Three more ways a deleted page's slug outlived the page are closed, and the federation warning stops counting private pages.** Before, only a page purge scrubbed the stores that name a page without a foreign key to it.
+**Four more ways a deleted page's slug outlived the page are closed, and the federation warning stops counting private pages.** Before, only a page purge scrubbed the stores that name a page without a foreign key to it.
| What you did | Before | Now |
|---|---|---|
| A query or search with eval capture on, whose capture runs after the response, racing a page purge | the capture could write the purged slug back into `eval_candidates.retrieved_slugs` | the insert keeps only slugs whose page still exists, checked under the same row lock the feedback writers take |
+| A volunteered-context answer whose background volunteer-log write lands after a page purge (or while it is open) | the write put the purged slug and its name-quoting rationale back into `context_volunteer_events` | the write keeps only rows whose page still exists, under the same row lock |
| A connector, GitHub or Google sync deletes a page (`deletePages`), or a transcript re-ingest deletes a stale part | retrieval logs, ratings, weights, the volunteer log, the gazetteer and captured eval slugs kept naming it | scrubbed in the delete's transaction |
| `gbrain sources remove`, `sources purge`, the expired-archive sweep, or a managed `remove`/`purge` | the source's retrieval logs, ratings, volunteer rows and captured eval slugs stayed | deleted in the statement that deletes the source, and swept again right after for a write that raced it, unless a source with the same id was created in between |
| `gbrain sources federate` or `sources restore` on a source with a remote-written `visibility: private` page and no readers | the warning named that page, though no-grant connections still cannot read it | the warning applies the read paths' privacy rules, unless `search.remote_private_pages` or `GBRAIN_REMOTE_PRIVATE_PAGES` relaxes them |
- `src/core/feedback/store.ts`: `insertEvalCandidate` is the one eval capture insert both engines run. A facts-arm result (`facts/<id>`) names a fact, not a page, so it stays while that fact exists.
+- `src/core/context/volunteer-events.ts`: `insertVolunteerEvents`, behind the fire-and-forget volunteer-log sink, inserts only rows whose page exists, probed the same way.
- `src/core/engine-sql/pages.ts`: `deletePages` runs `scrubPurgedSlugs` and `scrubPurgedNameStores` after the delete, in one transaction. The transcript importer's stale-part delete now calls `deletePages`. `deletePage` is unchanged: the page purge, its main caller, scrubs and records the stores in its receipt.
- `src/core/source-delete.ts`: `removedSourceScrubCtes` runs inside `deleteSourceRow` and the managed lifecycle's source delete, then once more after it. `deleteSourceRow` skips that second pass when a source with the same id exists again, so a re-created source keeps its rows. `retrieval_weights` and `mention_gazetteer_entries` already cascaded from `sources`.
- `src/core/access/federation-exposure.ts`: the exposure read adds `privacyPartSql` unless `remotePrivacyRelaxed`.
-- Tests: `test/purge-scrub-followups-9068.test.ts` and `test/federation-exposure-private-9068.test.ts` (PGLite, and Postgres through `test/postgres-unit-arms.txt`). `test/eval-candidates.test.ts` now seeds the pages its round-trip captures, since a capture keeps only live slugs.
+- Tests: `test/purge-scrub-followups-9068.test.ts` and `test/federation-exposure-private-9068.test.ts` (PGLite, and Postgres through `test/postgres-unit-arms.txt`). `test/eval-candidates.test.ts`, the volunteer round-trip in `test/migrate.test.ts` and the fact-purge case in `test/purge-name-stores-9068.test.ts` now create the pages they log, since a capture or volunteer write keeps only live pages.
## [0.60.650.0] - 2026-11-10
diff --git a/docs/architecture/key-files/core-services-1-continued.md b/docs/architecture/key-files/core-services-1-continued.md
index 82c8d81fe6..03debf2fe9 100644
--- a/docs/architecture/key-files/core-services-1-continued.md
+++ b/docs/architecture/key-files/core-services-1-continued.md
@@ -9,7 +9,7 @@ The content-repair judgment tier under `src/core/content-repair/`, then the Retr
- `src/core/content-repair/judgment.ts` — #6377 (Lane B3) the content-repair lane's model tier for a slug conflict: `judgeSlugConflict` builds the prompt from both files' frontmatter, headings and first 60 lines of body plus every line that mentions the other's slug anywhere in either body (evidence past line 60 counts), both slugs and paths, the hold's reason and the allowed answers, and parses exactly one JSON object `{"action":"remove_slug"}`, `{"action":"merge_into","canonical":"<one of the two slugs>"}` or `{"action":"needs_human"}` with an optional `why` sentence that reaches the run's output only, never hold metadata. The same failure classes, `thinkingOffControl` and output ceiling as the fence tier; a `canonical` outside the two slugs is `llm_malformed`. Memo key: sha256 of both participants' content + both ids + model + prompt version + rules version. The model is `models.content_repair` (an explicit model always runs), else `models.fence_repair`, else `CONTENT_REPAIR_MEASURED_MODELS` (the eval's qualifiers, best first, ties to the cheaper), else the fence list. Tests: `test/repair-slug-conflicts.test.ts`.
- `src/core/content-repair/llm.ts` — #6377 (Lane B3) the paid call behind `judgment.ts`: one gateway chat under the fence repair's caps and ledger (`FENCE_REPAIR_LEDGER`, so one `fences.repair.max_usd_per_day` covers the whole lane and today's fence spend counts; the per-page cap applies per candidate), `fences.repair.llm=false` and `--no-llm` turn it off, the attempt memo keeps rejected bytes from costing twice, and a transient `llm_unavailable` does not consume the memo. Spend is reported as the item's `llm_usd` and carried into the `content_repair` receipt (`model`, `cost`); prompts never include the rest of either page.
-- `src/core/context/` — Retrieval Reflex (Layer 1). `entity-salience.ts`: pure, zero-LLM, precision-biased `extractCandidates(text)` (capitalized runs + `@handles`, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emits `weak: true` candidates — lowercase words ≥3 chars on a separate `MAX_WEAK_CANDIDATES=32` budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) + `extractCandidatesFromWindow(turns)` (merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first). `retrieval-reflex.ts`: `resolveEntitiesToPointers(engine, sourceId, candidates, opts)` — alias arm (`resolveAliases`, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespaced `people/x` but `slugify` drops the prefix) + two lexical identity arms behind `opts.lexicalArms` (kill switch: config `retrieval_reflex_lexical_arms` / env `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via a `lower(title) LIKE '% <token>'` predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carry `source_id`/`arm`/`confidence`/`matchedNorm` (`ARM_CONFIDENCE` alias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts: `sourceIds?` federated scope (alias arm loops per source, arm 2 uses `source_id = ANY`), `suppression?` ('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver — `logDeliveredReflexPointers(engine, pointers)` fires only once a block is actually handed to the consumer (serve's resolve-IPC `onDelivered` hook post-write; `buildReflexAddition` post-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs through `stripTakesFence`/`stripFactsFence` (the same privacy boundary `get_page` applies) so private facts never reach the prompt; `visibility: private` (and derived-private) pages never resolve on any arm unless the caller passes `excludePrivate: false` (default true, the caller's read policy `privatePagesFilterForCaller`, the predicate get_page uses, so a bound member resolves the pages its readers allow; only `volunteer_context` widens, via `resolveExcludePrivatePages` for a trusted local caller — gbrain-evals N8-1/N8-2, pinned by `test/reflex-private-visibility.test.ts`); capped at `MAX_POINTERS`. `reflex.ts`: the orchestrator + engine-aware resolver ladder (host `resolveEntities` → PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check, `reflexEnabled(cfg)` (file/env gate, default ON; DB-plane does NOT gate — `assemble()` is sync); windowed extraction when `windowTurns` present and `retrieval_reflex_window_turns` (default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap). `resolve-ipc.ts`: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connection `gbrain serve` holds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports via `src/mcp/resolve-ipc-binding.ts` (engine-uniform; socket + secret keyed off `hash12(database_url)` under `~/.gbrain/run`, cleaned up on shutdown). A lean read-only Postgres serve binds the `read` socket kind instead (#7612); `hookResolveSocketForConfig` falls back to it after the source socket, and a `delegation: true` caller never targets it. Doctor surface: `retrieval_reflex_health` in `src/commands/doctor/checks/verbs-reflex.ts` (reads the heartbeat for truthful runtime status; categorized in `doctor-categories.ts`) + `volunteer_channels` (engine-aware sibling: groups `context_volunteer_events` by channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned by `test/doctor-volunteer-channels.test.ts`). Config: `retrieval_reflex` + `retrieval_reflex_max_pointers` + `retrieval_reflex_window_turns` + `retrieval_reflex_lexical_arms` in `src/core/config.ts` (env `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`, `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`). `volunteer.ts`: `parseWindow` (lenient `user:`/`assistant:` prefixes, unprefixed → one user turn), `volunteerContext` (extract → resolve → +0.05 multi-turn/newest-turn boost → `min_confidence` 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression), `volunteerUsageStats` (per-arm/channel precision from the `pages.last_retrieved_at > volunteered_at` join — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives). `volunteer-events.ts`: `insertVolunteerEvents` (ONE multi-row parameterized INSERT), `logVolunteerEventsFireAndForget` + bounded drain registered as the `volunteer-events` background-work sink (order 4), `purgeStaleVolunteerEvents` (90-day GC, called from the dream cycle's purge phase). Policy layer ships as the `retrieval-reflex` recipe (`recipes/retrieval-reflex/`). Pinned by `test/context/entity-salience.test.ts`, `test/retrieval-reflex.test.ts`, `test/retrieval-reflex-pre-v110.test.ts`, `test/context/resolve-ipc.test.ts`, `test/doctor-retrieval-reflex.test.ts`, `test/volunteer-context.test.ts`, `test/e2e/volunteer-context-postgres.test.ts`. Checkpoint-compaction modules in this dir: `corpus-segments.ts` — engine-free content-addressed compaction segments (`<session>.seg-<hash24>.txt`, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal), `sliceBoundaryWindow`/`splitByBoundaries`, the redacted `renderSegmentText` (a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-set `coverageComplete`/`decideCorpusMode` (count equality can be fooled by a duplicated boundary), the compact-time `bankCompactSegment` step (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (`readOpenclawBoundaryTail`, delegates mapping to the adapter's exported `mapOpenclawLine`; also returns turn-stamped `toolCalls`/`toolCallTurnIndexes` for the Memorable receipt — name-only v1, `input: null` by design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned by `test/corpus-segments.test.ts`. `checkpoint-harvest.ts` — the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates, `signal.aborted` POST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scoped `getPage` verification — a link that resolves to nothing is never banked), `.ingested` last, explicit `shutdownCheckpointHarvest()` called by serve BEFORE `engine.disconnect()` (the background-work drain is CLI-exit-only by contract), a writeback turn whose facts preflight finds the canonical writer busy (`writer_lock_unavailable`, before any model call) re-queued after 5, 15 and 45 s (`writer_busy_requeued`; the retry skips admission, a pending retry dedups a re-schedule as `already_queued`, shutdown clears pending retries) while a compact segment's busy refusal stays an error the sweep resumes, and error reasons that keep the error's code or `reason` within the 48-char reason-code shape (`operationerror:writer_lock_unavailable`), the first failure of each reason logged to stderr once per serve run — pinned by `test/checkpoint-harvest.serial.test.ts`. `hook-heartbeat.ts` — the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpoint `segment`/`inserted`/`duplicate`/`links` count-only fields. `session-state.ts` also carries the v132 `checkpoint_manifest` helpers (`getCheckpointManifest`/`appendCheckpointManifest`: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned by `test/checkpoint-manifest.test.ts`. `sensitivity-scan.ts` + `compile-view.ts` — the compile-context stack: composed detector (secret-scan + ordered `PII_PATTERNS` + path/blocklist families + operator pattern file; uniform `.gbrain-scan-allow` fingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidate `updated_at`, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs and pass `excludePrivate: true` on every arm, so private pages never reach the compiled file; whole-file `packToBudget` math that never passes a <=0 budget) — pinned by `test/sensitivity-scan.test.ts` + `test/compile-view.test.ts` + `test/e2e/compile-context-pglite.test.ts`; the CLI shell is `src/commands/compile-context.ts` (targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes, `--check` recompile-and-compare exit codes; the guide is `docs/guides/checkpoint-compaction.md`). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (`session-receipts.jsonl`, `appendSessionReceipt` — dedup by post-redaction `content_hash` so a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and a `skipCompaction` seam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptive `readJsonlTail` reader shared by `lastReceiptMatches`, `lastRelayResult` and `readSessionReceiptsTail` (a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty"); `maybeTrimRelayResults` (BOTH capture lanes trim the child-appended `memorable-relay.jsonl` — an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (`memorable-consent.json`, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 + `MEMORABLE_CAPTURE_HARNESSES`, so widening the capture surface invalidates old stamps); `memorableGateAllowed(cfg)` ({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor); `memorableConsentEvidence()` (the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam); `redactedToolCallsJson` (the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via \" escaping — with a post-serialization re-scan as belt-and-braces); `recordAndRelayReceipt` (per-receipt harness scope-binding first — a harness outside `MEMORABLE_CAPTURE_HARNESSES` is refused with `memorable_harness_undisclosed` → receipt → prior-run outcome surfacing with the child's reason clamped via the shared `clampRelayCause` (32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence → `resolveMemorableBin` → detached spawn; never throws; `spawnFn` test seam); and `priorRelayFailure`, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy. `resolveMemorableBin` does the pre-spawn PATH/`MEMORABLE_BIN` resolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT. #6195: `entity-salience.extractCandidates` also emits lowercase 2-3-word weak n-grams (`multiToken`, own budget `MAX_WEAK_NGRAM_CANDIDATES` = 16, never across punctuation, never all stopwords/common words) that ride the exact weak-alias fold, and `retrieval-reflex` arm 2.6 (`weak-title`, 0.72, behind `lexicalArms`, private pages hidden by default) matches them to the exact title of a person/company/organization page whose title is globally unique; single lowercase words stay alias-only.
+- `src/core/context/` — Retrieval Reflex (Layer 1). `entity-salience.ts`: pure, zero-LLM, precision-biased `extractCandidates(text)` (capitalized runs + `@handles`, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emits `weak: true` candidates — lowercase words ≥3 chars on a separate `MAX_WEAK_CANDIDATES=32` budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) + `extractCandidatesFromWindow(turns)` (merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first). `retrieval-reflex.ts`: `resolveEntitiesToPointers(engine, sourceId, candidates, opts)` — alias arm (`resolveAliases`, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespaced `people/x` but `slugify` drops the prefix) + two lexical identity arms behind `opts.lexicalArms` (kill switch: config `retrieval_reflex_lexical_arms` / env `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via a `lower(title) LIKE '% <token>'` predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carry `source_id`/`arm`/`confidence`/`matchedNorm` (`ARM_CONFIDENCE` alias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts: `sourceIds?` federated scope (alias arm loops per source, arm 2 uses `source_id = ANY`), `suppression?` ('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver — `logDeliveredReflexPointers(engine, pointers)` fires only once a block is actually handed to the consumer (serve's resolve-IPC `onDelivered` hook post-write; `buildReflexAddition` post-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs through `stripTakesFence`/`stripFactsFence` (the same privacy boundary `get_page` applies) so private facts never reach the prompt; `visibility: private` (and derived-private) pages never resolve on any arm unless the caller passes `excludePrivate: false` (default true, the caller's read policy `privatePagesFilterForCaller`, the predicate get_page uses, so a bound member resolves the pages its readers allow; only `volunteer_context` widens, via `resolveExcludePrivatePages` for a trusted local caller — gbrain-evals N8-1/N8-2, pinned by `test/reflex-private-visibility.test.ts`); capped at `MAX_POINTERS`. `reflex.ts`: the orchestrator + engine-aware resolver ladder (host `resolveEntities` → PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check, `reflexEnabled(cfg)` (file/env gate, default ON; DB-plane does NOT gate — `assemble()` is sync); windowed extraction when `windowTurns` present and `retrieval_reflex_window_turns` (default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap). `resolve-ipc.ts`: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connection `gbrain serve` holds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports via `src/mcp/resolve-ipc-binding.ts` (engine-uniform; socket + secret keyed off `hash12(database_url)` under `~/.gbrain/run`, cleaned up on shutdown). A lean read-only Postgres serve binds the `read` socket kind instead (#7612); `hookResolveSocketForConfig` falls back to it after the source socket, and a `delegation: true` caller never targets it. Doctor surface: `retrieval_reflex_health` in `src/commands/doctor/checks/verbs-reflex.ts` (reads the heartbeat for truthful runtime status; categorized in `doctor-categories.ts`) + `volunteer_channels` (engine-aware sibling: groups `context_volunteer_events` by channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned by `test/doctor-volunteer-channels.test.ts`). Config: `retrieval_reflex` + `retrieval_reflex_max_pointers` + `retrieval_reflex_window_turns` + `retrieval_reflex_lexical_arms` in `src/core/config.ts` (env `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`, `GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS`). `volunteer.ts`: `parseWindow` (lenient `user:`/`assistant:` prefixes, unprefixed → one user turn), `volunteerContext` (extract → resolve → +0.05 multi-turn/newest-turn boost → `min_confidence` 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression), `volunteerUsageStats` (per-arm/channel precision from the `pages.last_retrieved_at > volunteered_at` join — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives). `volunteer-events.ts`: `insertVolunteerEvents` (ONE multi-row parameterized INSERT that keeps only rows whose page still exists, probed under the page row's key-share lock, so a late write never restores a purged page; #9068), `logVolunteerEventsFireAndForget` + bounded drain registered as the `volunteer-events` background-work sink (order 4), `purgeStaleVolunteerEvents` (90-day GC, called from the dream cycle's purge phase). Policy layer ships as the `retrieval-reflex` recipe (`recipes/retrieval-reflex/`). Pinned by `test/context/entity-salience.test.ts`, `test/retrieval-reflex.test.ts`, `test/retrieval-reflex-pre-v110.test.ts`, `test/context/resolve-ipc.test.ts`, `test/doctor-retrieval-reflex.test.ts`, `test/volunteer-context.test.ts`, `test/e2e/volunteer-context-postgres.test.ts`. Checkpoint-compaction modules in this dir: `corpus-segments.ts` — engine-free content-addressed compaction segments (`<session>.seg-<hash24>.txt`, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal), `sliceBoundaryWindow`/`splitByBoundaries`, the redacted `renderSegmentText` (a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-set `coverageComplete`/`decideCorpusMode` (count equality can be fooled by a duplicated boundary), the compact-time `bankCompactSegment` step (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (`readOpenclawBoundaryTail`, delegates mapping to the adapter's exported `mapOpenclawLine`; also returns turn-stamped `toolCalls`/`toolCallTurnIndexes` for the Memorable receipt — name-only v1, `input: null` by design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned by `test/corpus-segments.test.ts`. `checkpoint-harvest.ts` — the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates, `signal.aborted` POST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scoped `getPage` verification — a link that resolves to nothing is never banked), `.ingested` last, explicit `shutdownCheckpointHarvest()` called by serve BEFORE `engine.disconnect()` (the background-work drain is CLI-exit-only by contract), a writeback turn whose facts preflight finds the canonical writer busy (`writer_lock_unavailable`, before any model call) re-queued after 5, 15 and 45 s (`writer_busy_requeued`; the retry skips admission, a pending retry dedups a re-schedule as `already_queued`, shutdown clears pending retries) while a compact segment's busy refusal stays an error the sweep resumes, and error reasons that keep the error's code or `reason` within the 48-char reason-code shape (`operationerror:writer_lock_unavailable`), the first failure of each reason logged to stderr once per serve run — pinned by `test/checkpoint-harvest.serial.test.ts`. `hook-heartbeat.ts` — the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpoint `segment`/`inserted`/`duplicate`/`links` count-only fields. `session-state.ts` also carries the v132 `checkpoint_manifest` helpers (`getCheckpointManifest`/`appendCheckpointManifest`: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned by `test/checkpoint-manifest.test.ts`. `sensitivity-scan.ts` + `compile-view.ts` — the compile-context stack: composed detector (secret-scan + ordered `PII_PATTERNS` + path/blocklist families + operator pattern file; uniform `.gbrain-scan-allow` fingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidate `updated_at`, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs and pass `excludePrivate: true` on every arm, so private pages never reach the compiled file; whole-file `packToBudget` math that never passes a <=0 budget) — pinned by `test/sensitivity-scan.test.ts` + `test/compile-view.test.ts` + `test/e2e/compile-context-pglite.test.ts`; the CLI shell is `src/commands/compile-context.ts` (targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes, `--check` recompile-and-compare exit codes; the guide is `docs/guides/checkpoint-compaction.md`). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (`session-receipts.jsonl`, `appendSessionReceipt` — dedup by post-redaction `content_hash` so a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and a `skipCompaction` seam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptive `readJsonlTail` reader shared by `lastReceiptMatches`, `lastRelayResult` and `readSessionReceiptsTail` (a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty"); `maybeTrimRelayResults` (BOTH capture lanes trim the child-appended `memorable-relay.jsonl` — an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (`memorable-consent.json`, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 + `MEMORABLE_CAPTURE_HARNESSES`, so widening the capture surface invalidates old stamps); `memorableGateAllowed(cfg)` ({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor); `memorableConsentEvidence()` (the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam); `redactedToolCallsJson` (the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via \" escaping — with a post-serialization re-scan as belt-and-braces); `recordAndRelayReceipt` (per-receipt harness scope-binding first — a harness outside `MEMORABLE_CAPTURE_HARNESSES` is refused with `memorable_harness_undisclosed` → receipt → prior-run outcome surfacing with the child's reason clamped via the shared `clampRelayCause` (32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence → `resolveMemorableBin` → detached spawn; never throws; `spawnFn` test seam); and `priorRelayFailure`, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy. `resolveMemorableBin` does the pre-spawn PATH/`MEMORABLE_BIN` resolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT. #6195: `entity-salience.extractCandidates` also emits lowercase 2-3-word weak n-grams (`multiToken`, own budget `MAX_WEAK_NGRAM_CANDIDATES` = 16, never across punctuation, never all stopwords/common words) that ride the exact weak-alias fold, and `retrieval-reflex` arm 2.6 (`weak-title`, 0.72, behind `lexicalArms`, private pages hidden by default) matches them to the exact title of a person/company/organization page whose title is globally unique; single lowercase words stay alias-only.
- `src/core/context/hook-heartbeat.ts` — the hooks telemetry plane under `<gbrain home>/integrations/hooks/` (`heartbeat.jsonl`, `status.json`, session receipts, the memorable relay log; directories 0700, appends 0600). Entries hold counters, durations and codes only: `HEARTBEAT_ALLOWED_KEYS` is the full key allowlist CI greps the fixture against, and includes `pattern` (scanner pattern name), `fingerprint` (`sha256:<16 hex>`) and `hint` (fixed text) for refusals. `near_duplicate` is the shadow count on the `writeback_dedup` event that `facts/capture-dedup.ts` appends (lane in `reason`, exact drops in `duplicate`). `relayRefusalHeartbeat(hit)` builds the line `context-engine.ts` writes when its high-entropy re-scan refuses a memorable relay (`event: relay`, `outcome: degraded`, `reason: secret_scan_refused`); the receipt and relay are skipped for that compaction window and the checkpoint is kept. `memorableGateAllowed` gates the relay lane; span redaction runs `redactFindings` with `highEntropy: true`. Tests: `test/context-engine-checkpoint.serial.test.ts`.
- `src/core/context/seat.ts` — seat identity for captured sessions (#4618), engine-free. `resolveSeat` returns the lowercased `GBRAIN_SEAT` label when it matches `SEAT_LABEL_RE` (`^[a-z0-9][a-z0-9._-]{0,63}$`), else `home-<8 hex>` over the realpath of the harness home (Claude Code: the directory holding `projects/` in the transcript path, then `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `~/.claude`; codex: `CODEX_HOME` or `~/.codex`; OpenClaw: the agent directory holding `sessions/`), so no raw path reaches frontmatter; `GBRAIN_SEAT=off` (the opt-out) resolves to null and nothing is recorded. `writeSeatSidecar` writes `<sessionId>.seat.json` (`{version: 1, seat, seat_source, hook_lane, harness, first_seen}`, 0600, tmp+rename) unless one exists, reporting `seat_conflict` when the existing seat differs; `readSeatSidecar` accepts only a valid v1 record. `SEAT_REASON_HINTS` holds the fixed heartbeat hints. The sidecar name and its GC live in `corpus-segments.ts` (`seatFileName`, `SEAT_SIDECAR_SUFFIX`).
- `src/core/context/session-speaker.ts` — who wrote a captured session's user turns (#8064), engine-free. `resolveSessionSpeaker` reads `GBRAIN_SESSION_AGENT` (a label: agent; `off`: skip banking; `owner`: no detection; invalid: agent `agent`), else `detectAgentLaunched` over `readProcessAncestry` (Linux /proc, else `ps`; 12 levels, 300 ms): the nearest claude/codex process runs headless (`-p`/`--print`, `exec`) and another claude/codex process is an ancestor, principal `agent-launched`. `bankAsSpeaker` writes `<sessionId>.speaker.json` (`{version: 1, principal, source, harness, first_seen}`, 0600, first write wins) before the hook's compact, stop and session-end banking; a write failure banks nothing (`speaker_write_failed`), `off` records `capture_opt_out`. `corpusFileSpeaker`/`speakerOf` give the capture lanes (checkpoint-harvest, sweep) the `speaker` they pass to `runFactsPipeline`, which saves user and unattributed claims as `other` (`asAgentSpoken`), adds `speakerContextNote` to the context (`facts/speaker-context.ts`, also used by the managed facts batch) and names the agent in the extractor prompt. Name and GC: `speakerFileName`, `SPEAKER_SIDECAR_SUFFIX` in `corpus-segments.ts`. Pinned by `test/agent-session-speaker-8064.serial.test.ts`.
diff --git a/src/core/context/volunteer-events.ts b/src/core/context/volunteer-events.ts
index 5bfe78157a..5ab6c8523d 100644
--- a/src/core/context/volunteer-events.ts
+++ b/src/core/context/volunteer-events.ts
@@ -168,6 +168,12 @@ export interface VolunteerEventRow {
* RTTs ≈ 355ms on a cross-region deployment; eng-review D4). Throws on
* failure; callers run it through the volunteer-events background-work sink
* with try/catch so logging can never fail the op.
+ *
+ * #9068: it writes only rows whose page still exists, probed under the
+ * feedback writers' key-share lock on the page row, so a write built before a
+ * page purge and landing after it (or while the purge is open) drops that
+ * page's slug and rationale instead of restoring them (insertEvalCandidate
+ * does the same for eval captures).
*/
export async function insertVolunteerEvents(
engine: BrainEngine,
@@ -187,13 +193,14 @@ export async function insertVolunteerEvents(
r.session_id ?? null,
r.turn ?? null,
);
- const ph = Array.from({ length: 8 }, (_, i) => `$${base + i + 1}`);
- return `(${ph.join(', ')})`;
+ const types = ['text', 'text', 'float8', 'text', 'text', 'text', 'text', 'int'];
+ return `(${types.map((t, i) => `$${base + i + 1}::${t}`).join(', ')})`;
});
await engine.executeRaw(
`INSERT INTO context_volunteer_events
(source_id, slug, confidence, match_arm, rationale, channel, session_id, turn)
- VALUES ${tuples.join(', ')}`,
+ SELECT v.* FROM (VALUES ${tuples.join(', ')}) AS v(source_id, slug, confidence, match_arm, rationale, channel, session_id, turn)
+ WHERE EXISTS (SELECT 1 FROM pages pg WHERE pg.source_id = v.source_id AND pg.slug = v.slug FOR KEY SHARE)`,
params,
);
}
diff --git a/test/migrate.test.ts b/test/migrate.test.ts
index 937105705d..b1faf4f9e2 100644
--- a/test/migrate.test.ts
+++ b/test/migrate.test.ts
@@ -2420,6 +2420,8 @@ describe('v117 — context_volunteer_events_table', () => {
test('insert + 90-day purge round-trip (purgeStaleVolunteerEvents)', async () => {
const { insertVolunteerEvents, purgeStaleVolunteerEvents } = await import('../src/core/context/volunteer-events.ts');
+ await engine.putPage('people/alice-example', { type: 'person', title: 'Alice Example', compiled_truth: '', timeline: '' });
+ await engine.putPage('projects/acme-example', { type: 'project', title: 'Acme Example', compiled_truth: '', timeline: '' });
await insertVolunteerEvents(engine, [
{ source_id: 'default', slug: 'people/alice-example', confidence: 0.9, match_arm: 'alias', rationale: 'alias match', channel: 'op' },
{ source_id: 'default', slug: 'projects/acme-example', confidence: 0.8, match_arm: 'title', rationale: 'exact title', channel: 'reflex', session_id: 's1', turn: 3 },
diff --git a/test/purge-name-stores-9068.test.ts b/test/purge-name-stores-9068.test.ts
index c164a80214..b258575a74 100644
--- a/test/purge-name-stores-9068.test.ts
+++ b/test/purge-name-stores-9068.test.ts
@@ -138,6 +138,7 @@ describe('#9068 a page purge removes the slug and name from the volunteer log an
<!--- gbrain:facts:end -->\n`, { noEmbed: true, sourceId: 'default' });
await runExtractFacts(engine, { slugs: [slug] });
const [fact] = await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE fact = $1 AND entity_slug = $2', [claim, slug]);
+ await importFromContent(engine, 'notes/door-code-example', `---\ntitle: Door code example\ntype: note\n---\n# Door code example\n\nBody text.\n`, { noEmbed: true, sourceId: 'default' });
await nameRows(engine, 'default', 'notes/door-code-example', claim);
const purge = (params: Record<string, unknown>) => op('purge_fact').handler(ctx(engine), params) as Promise<Record<string, any>>;
const dry = await purge({ id: Number(fact.id), dry_run: true });
diff --git a/test/purge-scrub-followups-9068.test.ts b/test/purge-scrub-followups-9068.test.ts
index e5bf5e4ef8..36f104f3e0 100644
--- a/test/purge-scrub-followups-9068.test.ts
+++ b/test/purge-scrub-followups-9068.test.ts
@@ -10,9 +10,11 @@
* (c) removing a source (unmanaged removeSource -> deleteSourceRow, and the managed lifecycle remove) drops every
* such row of the source, and strips its slugs from eval candidates no other covered source still answers;
* (d) a capture keeps a facts-arm result (facts/<id>) while its fact exists; (e) a source re-created with the same
- * id between deleteSourceRow's delete and its second scrub keeps its rows.
+ * id between deleteSourceRow's delete and its second scrub keeps its rows; (f) a fire-and-forget volunteer-log write
+ * built before a purge and landing after it (or, on Postgres, while the purge is open) drops the purged page's slug
+ * and rationale and keeps live pages.
* 2. Fails when: logEvalCandidate inserts slugs without probing the page row under a key-share lock or drops a live
- * fact's result, a hard delete or a source removal leaves any of those rows behind, or a scrub reaches another
+ * fact's result, insertVolunteerEvents writes a row for a page that no longer exists, a hard delete or a source removal leaves any of those rows behind, or a scrub reaches another
* source's rows, a re-created source's rows or a slug another source the candidate covers still holds.
* 3. Existing coverage: purge-retrieval-feedback-8821 and purge-name-stores-9068 pin delete_page purge and the
* tombstone purge only; nothing pinned the capture, the connector and stale-part deletes, or source removal.
@@ -32,7 +34,8 @@ import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { operations } from '../src/core/operations.ts';
import { importFromContent } from '../src/core/import-file.ts';
import { applyRatings, insertRetrievalEvents, scrubPurgedSlugs } from '../src/core/feedback/store.ts';
-import { insertVolunteerEvents } from '../src/core/context/volunteer-events.ts';
+import { awaitPendingVolunteerEventWrites, insertVolunteerEvents, logVolunteerEventsFireAndForget, type VolunteerEventRow } from '../src/core/context/volunteer-events.ts';
+import { scrubPurgedNameStores } from '../src/core/persistence/purge-name-stores.ts';
import { captureEvalCandidate } from '../src/core/eval-capture.ts';
import { withResponseDeferral } from '../src/core/background-work.ts';
import { deleteConnectorPage } from '../src/core/connectors/api-source.ts';
@@ -179,6 +182,48 @@ describe('#9068 an eval capture does not write a purged slug back', () => {
}, 120_000);
});
+const volunteered = (sourceId: string, slug: string, name: string): VolunteerEventRow =>
+ ({ source_id: sourceId, slug, confidence: 0.9, match_arm: 'alias', rationale: `alias match "${name}"`, channel: 'op' });
+const volunteerRows = async (engine: BrainEngine, slug: string, name: string) =>
+ (await engine.executeRaw<{ slug: string }>('SELECT slug FROM context_volunteer_events WHERE slug = $1 OR strpos(rationale, $2) > 0 ORDER BY id', [slug, name])).map(r => r.slug);
+
+describe('#9068 a late volunteer-log write does not restore a purged page', () => {
+ test('a volunteer write built before delete_page purge:true and landing after it keeps only live pages', () => withEnv({ GBRAIN_HOME: home }, async () => {
+ for (const engine of engines) {
+ const slug = 'people/alice-late-volunteer', name = 'Alice Late Volunteer';
+ const live = `notes/live-volunteer-${engine.kind}`;
+ await page(engine, 'default', slug);
+ await page(engine, 'default', live);
+ const rows = [volunteered('default', slug, name), volunteered('default', live, 'Live Volunteer')];
+ const snapshot = await engine.readPageSnapshot(slug, { sourceId: 'default' });
+ const result = await op('delete_page').handler(ctx(engine), { slug, purge: true, request_id: randomUUID(), expected_revision: snapshot!.revision }) as Record<string, any>;
+ expect(result.status).toBe('purged');
+ logVolunteerEventsFireAndForget(engine, rows);
+ expect(await awaitPendingVolunteerEventWrites()).toEqual({ unfinished: 0 });
+ expect({ engine: engine.kind, purged: await volunteerRows(engine, slug, name), live: await volunteerRows(engine, live, 'Live Volunteer') })
+ .toEqual({ engine: engine.kind, purged: [], live: [live] });
+ }
+ }), 120_000);
+
+ const postgresTest = testBackends().includes('postgres') ? test : test.skip;
+ postgresTest('on Postgres a volunteer write during an open purge waits for it and drops the purged page', async () => {
+ const engine = engines.find(e => e.kind === 'postgres')!;
+ const slug = 'people/alice-volunteer-race', name = 'Alice Volunteer Race';
+ await page(engine, 'default', slug);
+ await page(engine, 'default', OTHER);
+ let late: Promise<void> | null = null;
+ await engine.transaction(async tx => {
+ await tx.executeRaw(`DELETE FROM pages WHERE source_id = 'default' AND slug = $1`, [slug]);
+ await scrubPurgedNameStores(tx, 'default', [slug]);
+ late = insertVolunteerEvents(engine, [volunteered('default', slug, name), volunteered('default', OTHER, 'Acme Race')]);
+ await new Promise(r => setTimeout(r, 500));
+ });
+ await late;
+ expect(await volunteerRows(engine, slug, name)).toEqual([]);
+ expect((await volunteerRows(engine, OTHER, 'Acme Race')).length).toBeGreaterThan(0);
+ }, 120_000);
+});
+
describe('#9068 the batch hard delete scrubs what names the page', () => {
test('the connector delete (deletePages) scrubs the source\'s rows and leaves other sources and pages', () => withEnv({ GBRAIN_HOME: home }, async () => {
for (const engine of engines) {