Local-first π§ memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly π΄ dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
Plan - retrieval-precision-quality-loop
Branch: feat/retrieval-precision-quality-loop
Slug: retrieval-precision-quality-loop
Combined design: docs/brainstorm/retrieval-precision-quality-loop/design.md
Drive order: A β C β D β E β B (blast-radius order; each later card extends an already-merged
seam). Cards are driven ONE AT A TIME on the shared branch. Each worker builds on the commits the
previously-driven in-scope cards already landed and must not duplicate or conflict with sibling
tasks.
Global conventions for every card:
- TDD: write the failing test first, then implement until it passes. Run
bun run validate(typecheck + lint + test) before claiming the card done. - Default-off switch per card (one config key), off-branch byte-identical to today.
- Telemetry via
emitGatedTelemetry, fail-open. - Kernel calls no LLM. The cross-encoder (A) is an opt-in external service call.
- No TypeScript cast crutches; build correct shapes with conditional spreads / narrowing.
- English-only strings in code; logic is language-agnostic (no hardcoded NL word lists).
In-scope cards shipping together in this one release:
| id | card | one-line title |
|---|---|---|
t_110867f5 |
A | Optional pluggable cross-encoder rerank stage for the search reader |
t_e163feb2 |
C | Epistemic provenance markers on recalled/packed items |
t_b8f66fec |
D | Recall adequacy verdict with explicit low-adequacy actions |
t_97091fff |
E | Cross-query demand log surfacing queries asked often but recall poorly |
t_62363378 |
B | Outcome-tagged, recency-scored lessons loop unifying dead-ends and preferences with corroboration tiering |
A (t_110867f5) - Optional pluggable cross-encoder rerank stage
Files
src/core/search/rerank/cross-encoder.ts- the rerank provider (OpenAI-compatible endpoint: base_url + model + env-resolved api key, re-scores top-K query/doc pairs).src/core/search/rerank/registry.ts- named-provider registry persisted toBrain/search/rerank-providers.json(env-key NAMES only, fail-soft load), mirroringembeddings/registry.ts.src/core/search/rerank/provider.ts- provider interface + built-in/default resolution.src/core/search/rerank/index.ts-applyCrossEncoderRerank(results, query, config)final reader step; returns input unchanged when disabled or on endpoint error.- config keys threaded through
resolveSearchConfig:search_rerank_enabled(default false),search_rerank_model,search_rerank_base_url,search_rerank_env_key,search_rerank_top_k,search_rerank_min_score, with env parity + validation. - one guarded call site in the reader tail (after
enrich.tsheuristic reranks). - MCP/CLI exposure for provider registration (mirror the embedding registry CLI verbs).
- tests under the module's sibling
.test.ts(follow repo convention).
Acceptance (a passing test)
- Disabled (default): the reader tail output is byte-identical to the pre-feature baseline for the same query/results (snapshot/diff equality asserted in a test).
- Enabled + endpoint unconfigured: validation fails closed with a clear error (mirrors
openai-compat.tswhen semantic is enabled without base_url/model). - Enabled + endpoint returns scores: top-K are re-ordered by the cross-encoder scores; a doc the heuristic ranker placed 3rd but the cross-encoder scored highest lands 1st.
- Enabled + endpoint errors: stage degrades to the heuristic-ordered input (no throw into the hot path) and emits one fail-open telemetry record.
Depends on
- None. Lands first (pure search-side, no vault writes, lowest blast radius, seeds the
provider-registry pattern). Must not touch
context-pack.ts,context-receipts.ts, or any brain/ persistence.
C (t_e163feb2) - Epistemic provenance markers
Files
src/core/brain/provenance/epistemic-status.ts- pure derivation:deriveEpistemicStatus(meta)->EpistemicStatus(observed | derived | hypothesis | plan | unknown) +evidence_refs, from existing graph metadata (source-backed vs inferred vs preference/hypothesis vs intended vs gap). Language agnostic, no manual tagging.- optional fields
epistemic_statusandevidence_refsadded toContextPackItemand the recall result type (absent by default). - derivation wired into
context-pack.tsand recall result rendering behind a feature flag. src/mcp/brain/*(or the pack/recall surface) so the field surfaces inbrain_context_pack/brain_context_receiptsoutput when enabled.
Acceptance (a passing test)
- Flag off:
brain_context_packand recall output is byte-identical to the pre-feature baseline (the new fields are absent, not present-with-empty). - Flag on: a source-backed note item is tagged
observedwith the source wikilink asevidence_refs; an inferred/derived-fact item isderivedwith its basis linked; a preference-tier item ishypothesis; an intended/not-yet-true item isplan; an acknowledged gap isunknown. - Derivation is deterministic across two runs on the same input (Syncthing-peer safe).
Depends on
- A landed (so the branch is stable at the reader tail). Establishes the item-shape seam D
extends. Must not collide with
context-receipts.tssummary shape beyond adding optional fields.
D (t_b8f66fec) - Recall adequacy verdict with explicit low-adequacy actions
Files
src/core/brain/recall/adequacy.ts- pure verdict:classifyAdequacy(gateScores, coverageReport, epistemicMix, thresholds)->{ verdict: sufficient|weak|insufficient, action: proceed|re_recall|abstain|escalate }. Reusesgate-telemetryrelevance scores,coverage.tsIDF-weighted coverage, and C's epistemic mix; thresholds configurable.- verdict + action exposed in
brain_recall_gateoutput andcontext-receiptssummary. weakaction: trigger an alternate-strategy re-recall (broaden scope / different path) via existing recall machinery, behind the feature flag.insufficientaction: abstain signal ("insufficient grounding") and/or escalate flag, behind the feature flag.
Acceptance (a passing test)
- Flag off:
brain_recall_gateoutput andcontext-receiptssummary are byte-identical to the pre-feature baseline. - Flag on, high-coverage + mostly-
observedrecall: verdictsufficient, actionproceed. - Flag on, partial coverage / mixed
hypothesis-heavy recall: verdictweak, actionre_recall(one alternate-strategy re-recall fires, deterministic). - Flag on, near-zero coverage / mostly
unknown: verdictinsufficient, actionabstain(explicit "insufficient grounding" signal returned, not a low-confidence answer) and/orescalate. - Verdict is deterministic over the same scores.
Depends on
- C landed (D extends C's merged item shape with the verdict; D reads C's epistemic mix). Reuses
existing
gate-telemetry.tsandcoverage.tsscores - must NOT introduce a new score or a separate attention organ (card guardrail).
E (t_97091fff) - Cross-query demand log surfacing weak-recall queries
Files
src/core/brain/recall/demand-log.ts- append + rotate/cap + redact: each recall appends (normalized terms or redacted query form + timestamp + result count + the coverage score already computed per query incoverage.ts) to a rolling, size-cappedBrain/recall/demand-log.jsonl. Query normalized/redacted before append; log capped/rotated.- aggregation reader (a
knowledge_gaps-style read tool / CLI) bucketing by normalized query and ranking by frequency x low-satisfaction, reusing the existing coverage score as the "answered poorly" axis. - append wired at the recall seam D opened, behind a feature flag.
Acceptance (a passing test)
- Flag off: no file is written under
Brain/, recall path byte-identical to baseline. - Flag on: three recalls of the same normalized query that each return low coverage produce one bucket ranked by frequency x low-satisfaction; a different query asked once with good coverage ranks far lower.
- Privacy: the on-disk log contains the normalized/redacted form, never the raw sensitive query (asserted by inspecting the written file in the test).
- Size cap: after N appends the oldest entries are rotated out; the file never grows unbounded.
- Aggregation is deterministic across two runs on the same log (Syncthing-peer safe).
Depends on
- D landed (E appends at the recall seam D opened and reuses the coverage score D surfaces).
Independent persistence directory
Brain/recall/- must not share a writer with B'sBrain/lessons/.
B (t_62363378) - Outcome-tagged, recency-scored lessons loop with corroboration tiering
Files
src/core/brain/lessons/outcome-corpus.ts- unifydead-ends.ts(negative) andpreference.ts/apply-evidence.ts(positive) into one outcome-tagged corpus (useful | dead_end | corrected).src/core/brain/lessons/score.ts- signed recency-decayed score reusingusage-signal.tsdecayWeight()(configurable half-life) as the promotion signal.src/core/brain/lessons/digest.ts- corroboration-gated aggregation: a node isPREFERREDonly once corroborated by >=N distinct results (configurable N), elseTENTATIVE; mixed-signal nodes renderCONTESTED(recency-wins). Folds the corpus into a singleBrain/lessons/LESSONS.md.active.ts/dream-refresh.tsextension for auto-load (SessionStart) + auto-refresh (dream cron) of the digest.- dream-pass extension (
dream.ts/dream-stage.ts) adding signed + corroboration-gated tiers, behind a feature flag.
Acceptance (a passing test)
- Flag off: dream pass and
active.mdbyte-identical to baseline; noBrain/lessons/written. - Flag on: a node corroborated by <N distinct results is
TENTATIVE; once >=N distinct results corroborate it, it promotes toPREFERRED; a freshdead_endoutcome outweighs a staleusefulon the same node (recency + sign), rendering itCONTESTEDwhen mixed. - A one-off outcome does NOT promote to
PREFERRED(corroboration threshold prevents it). - The
LESSONS.mddigest is deterministic across two runs on the same corpus (stable sort keys, stable slugs; Syncthing-peer safe). - Auto-load: the digest surface is reachable via the existing
active.ts/SessionStart path; no git-hook dependency.
Depends on
- A, C, D, E landed (B is the capstone assembly over a fully-merged branch; it must not force a rebase of the recall/telemetry surfaces). Extends the dream pass rather than running a parallel aggregation; does not rewrite or delete the source dead-ends/preference notes.