Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
Embedding Provider Suite - shared signature kernel, local embedder, provider registry, RRF fusion
Status: draft Author: claude-dev-agent (via feature-release-playbook) Audience: implementation
Problem statement
Open Second Brain's semantic layer has one real embedding provider (openai-compat, remote HTTP) and one null provider. There is no offline/local-first embedder, no runtime way to register providers without editing config, no cost visibility or spend gate before a large embedding run, and exactly one rank-fusion mode (weighted linear sum of min-max-normalised BM25 and cosine). Four triage tasks (t_9d24c9fd p4, t_d1fbfeee p3, t_f42f1b69 p2, t_4796dff3 p2) all extend this one subsystem and share provider-identity logic.
Scope
- Unit 1 - Hybrid RRF fusion (t_9d24c9fd): add Reciprocal Rank Fusion as an alternative to linear fusion of the sparse (BM25) and dense (cosine) lanes. Opt-in via
search_fusion_mode; defaultlinearkeeps ranking bit-identical. No cloud vector DB - storage stays SQLite + sqlite-vec + FTS5. - Unit 2 - Local embedder (t_d1fbfeee): add a
localprovider that produces deterministic embeddings with no cloud call, no API key, and no model download - a character n-gram feature-hashing embedder. Predictable dimension, stable model signature, clear diagnostics. - Unit 3 - Provider registry (t_f42f1b69): CLI verbs (
o2b search provider add/list/show/remove) that persist named provider profiles (base-url, default-model, env-key) to a vault-local registry file. A registered name resolves to anopenai-compatconfig at config-resolution time; built-ins are never shadowed. - Unit 4 - Cost model + signatures (t_4796dff3): one signature kernel that canonicalises
{provider, model, dimension}, a model-aware price-per-million-tokens table, a token/cost estimator, signature-based stale reporting in status/check, and a cost gate that blocks an embedding run whose estimated spend exceeds a configurable threshold.
Out of scope
- Cloud/hosted vector databases (Qdrant/Milvus/Weaviate/TCVDB) - explicitly deferred.
- A heavy bundled transformer model. The local embedder is dependency-free; opting into a larger local model is future work.
- Per-token exact billing reconciliation against a provider invoice - we estimate, we do not reconcile.
- Replacing the closed provider union with a descriptor/plugin spine (consultant Variant 3) - rejected as over-built.
Chosen approach
Consultant Variant 2 - shared signature/provenance kernel + registry module. Two focused new domain modules carry the logic Units 2/3/4 share, and a small fusion strategy module carries Unit 1:
embeddings/signature.ts- canonicalise{provider, model, dimension}to a stable signature string; hold the per-model pricing table; estimate tokens and cost; decide staleness by comparing an active signature with a stored one. The single source of provider-identity truth, consumed by the factory, indexer, store, and CLI rather than re-derived per call site.embeddings/registry.ts- load/persist provider profiles fromBrain/search/embedding-providers.json; expand a registered name intoopenai-compatconfig fields duringresolveSearchConfig, after built-ins so a configured built-in key is never shadowed. Fail-soft: an absent or malformed registry resolves to no extra providers.embeddings/local-provider.ts-LocalProvider implements EmbeddingProvider: deterministic character n-gram feature hashing into a fixed-dimension unit-normalised vector. No network, no key, no download.search/fusion.ts- purelinearFusionandrrfFusion(lanes, k)helpers split out ofranker.ts; the ranker calls the selected strategy and keeps every existing boost (link/tag/recency/entity/tier/session-focus) unchanged.
Design decisions
- Closed union stays closed. The only new resolved provider member is
local. Registry entries are config-time indirection that resolve toprovider: "openai-compat", soResolvedEmbeddingConfig.providerremains"openai-compat" | "disabled" | "local"andparseProviderkeeps validating a closed set. Registry expansion happens beforeparseProviderso a registered name never reaches the validator. - Local embedder = feature hashing, not a model. A character n-gram (n=3, plus token unigrams) hashing trick gives an offline, deterministic, zero-dependency baseline that works the instant the package installs - matching the upstream "tiny default embedder, no model download" intent. Vectors are unit-normalised so the existing cosine-from-L2 ranker math is unchanged. Quality limits are documented; the design leaves room for a future opt-in transformer.
- Signature is the one fingerprint.
embeddingSignature(provider, model, dimension)is reused bycorpusGeneration()(cache invalidation), per-run stale reporting, and the cost estimate, so the four units cannot drift on what "the same embedding configuration" means. The store keeps its existing clear-on-model/dimension-change safety; the signature adds finer-grained reporting and gating on top, it does not replace the clear. - Cost gate is off by default.
embedding_cost_gate_usddefaults to0(disabled). When positive and a run's estimated spend exceeds it, the indexer throwsEMBEDDING_COST_GATEcarrying the estimate; an explicit override (--force-cost/ config) proceeds. The local provider prices at0, so it is never gated. - RRF is off by default.
search_fusion_modedefaults tolinear; with it the ranker output is byte-for-byte the current behaviour.rrfmode fuses lane ranks with1 / (rrf_k + rank)(search_rrf_kdefault 60, the canonical RRF constant) and adds anrrf:reason; boosts still compose on top. - Every knob follows the house pattern: a YAML key, an
OPEN_SECOND_BRAIN_*env override, aDEFAULTSentry, and range validation. - Fail-soft everywhere: a bad registry, an unavailable local runtime, or an unknown pricing entry degrades to current behaviour or a clear diagnostic, never an exception in the hot path.
File changes
New source:
src/core/search/embeddings/signature.tssrc/core/search/embeddings/registry.tssrc/core/search/embeddings/local-provider.tssrc/core/search/fusion.ts
Modified source:
src/core/search/embeddings/provider.ts-makeProviderlocalbranch.src/core/search/types.ts- provider union+ "local";fusionMode,rrfK,costGateUsdon resolved config; signature + estimated-refresh-cost fields on the status snapshot.src/core/search/index.ts-parseProviderlocal; registry expansion; fusion/rrf/cost-gate config parsing;DEFAULTS.src/core/search/indexer.ts- cost gate + estimate inpopulateEmbeddings; signature + refresh-cost inindexStatus.src/core/search/store.ts- signature helpers / signature-based stale count;corpusGenerationreuses the kernel.src/core/search/ranker.ts- delegate relevance fusion tofusion.ts; rrf path;rrfreason.src/cli/search.ts-providersubverbs; cost-gate flag on reindex.src/cli/command-manifest.ts,src/cli/help-text.ts,src/cli/completions.ts- register and document the new verbs.
New tests (one focused suite per unit + integration):
tests/embeddings.local-provider.test.ts,tests/embeddings.signature.test.ts,tests/embeddings.registry.test.ts,tests/search.fusion-rrf.test.ts,tests/indexer.cost-gate.test.ts,tests/search-provider.cli.test.ts,tests/search-config.embedding-suite.test.ts, plus an e2e search slice exercising rrf + local provider.
Docs:
README.md,CHANGELOG.md,docs/cli-reference.md,docs/how-it-works.md.
Risks and open questions
- Local embedder quality. Feature hashing is lexical, not semantic; near-synonyms it cannot match. Mitigation: document it as a no-cloud baseline, keep
openai-compatthe recommended path for semantic depth, and leave the dimension configurable. - RRF re-baselining. RRF changes scores when enabled, so e2e ranking fixtures must assert
linear(default) stays bit-identical and add separaterrffixtures. Mitigation: gate behind the off-by-default switch and test both modes. - Pricing table staleness. Hardcoded per-model prices drift as providers change rates. Mitigation: treat unknown models as price
0(never falsely gate), document the table as best-effort, and allow a per-model override via config. - Registry file and Syncthing. The registry stores env-key names, never secret values, so it is safe to sync across devices. Confirmed no secret material lands on disk.