Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
feat: honest embedding spend and a semantic belief order (v1.72.0) (#234)
Serg Eroshe2b424d9a7eaa
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.jsonindex 87d3ee42..479b316a 100644--- a/.claude-plugin/plugin.json+++ b/.claude-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.71.0",+ "version": "1.72.0","description": "Plugin-first second brain package for AI agents and humans.","author": {"name": "Open Second Brain contributors"diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.jsonindex 237c24c4..489f84c3 100644--- a/.codex-plugin/plugin.json+++ b/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.71.0",+ "version": "1.72.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 867dead8..bbb24e12 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,41 @@ All notable changes to this project will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [1.72.0] - 2026-10-05++Open Second Brain now prices embedding spend honestly: every estimate names where its price came from (`builtin`, `operator` or `unknown`), an operator can declare the price of a model the built-in table does not list, and one spend plan feeds the estimate, the cost gate, the spend receipt and `search status`, so an unknown price is reported as unknown instead of as $0 and a positive cost gate refuses it by name; edits re-embed only the chunks that changed, `o2b search vector-backfill` can be scoped to vault paths, `o2b search check` names an unpriced model, a stale price declaration and the places it looked for an embedding key, and `brain_context_pack` gains a `semantic` query mode that orders the curated belief notes by the stored vectors of their chunks.++### Added++- **Price quotes that name their source.** Every embedding estimate carries a price source: `builtin` (the frozen price table, and the local embedder, which is free), `operator` (declared by the operator) or `unknown`. A price is not part of the embedding identity, so it never triggers a reindex.+- **An operator price pair.** `embedding_price_model` and `embedding_price_usd_per_mtok` (USD per million tokens, a plain non-negative decimal number no larger than 1,000,000, `0` declares the model free) bind a price to one model name, compared case-insensitively, and win over the table for that model. Set both or neither; a blank value counts as unset, so a blank half is refused as a half pair, never read as a free price. A blank `embedding_cost_gate_usd` (or its env twin) is refused with `INVALID_INPUT`, never read as a gate of 0, and a blank `search_rerank_min_score` (or its env twin) is refused the same way, never read as a floor of 0. A half pair, a negative rate, a non-decimal rate (`0x10`, `1e3`, `Infinity`) or a rate above 1,000,000 USD per million tokens fails config resolution with `INVALID_INPUT` naming the key or env variable that supplied it. The env twins `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL` and `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK` win as a pair: when either is set, both halves come from env, so an env model never pairs with a config rate. A loopback `embedding_base_url` is not assumed to be free.+- **One embedding spend plan.** The pending census (optionally scoped by path prefix), the model resolution, the token estimate, the price quote and the cost-gate verdict are one computation. The reindex task's embedding phase, the maintenance preview banner, the backfill dry run and the `search status` refresh estimate all read it, so what is announced, refused, spent and receipted agrees.+- **`EMBEDDING_COST_UNPRICED`.** Under a positive `embedding_cost_gate_usd`, an embedding run with pending chunks on a model with no known price is refused before the provider is contacted, because an unknown price cannot be checked against a cap. The message names the model, both price keys and `--force-cost`. `--force-cost` (MCP `force_cost`) passes the refusal, and the receipt then records `forced: true`, `price_source: unknown` and a null estimate. The code joins the stable MCP error codes.+- **Vector carry-over on edit.** Replacing a document's chunks keeps the stored vector of every new chunk whose content hash matches an old chunk of the same document (duplicates match as a multiset), as long as the old vector was written by the model and dimension the index records; nothing is carried while that identity is unrecorded. A paragraph moved within a note is carried, one moved into another note is re-embedded, and only the vectors not carried are purged. The index run reports the count as `embeddingsReused`.+- **A repeatable `--path` on `o2b search vector-backfill`.** Each `--path <prefix>` scopes the run to chunks under a vault-relative prefix, and the census, the estimate, the cost gate and the spend receipt read the same scoped census, so `--path Brain/preferences/` prices and embeds only belief notes. The prefix is a raw string prefix (end a directory with `/`); a leading `./` is dropped, `\` becomes `/`, an empty prefix is refused with `INVALID_INPUT`, and an unsafe prefix is refused by name. The text report adds a `scope:` line and its next step keeps the scope; `--json` adds `path_prefixes`, and `unmatched_path_prefixes` with a stderr warning (`scope <prefix> matches no indexed document`) for a prefix that matches nothing. `--json` also names the gate verdict and the spend receipt: `gate_blocked` and `gate_reason` when the configured gate would refuse the run unforced, and `spend` (`model`, `tokens`, `estimated_usd`, `price_source`, `forced`) when an `--apply` run reached the provider, each absent otherwise. A dry run the gate would refuse prints `cost gate: would refuse (<reason>)` with the remedy; the remedy line names `--force-cost`, and the advised next step stays unforced. On Windows a scope prefix that would need quoting is left out of the advice: `next_command` is omitted and the text says `next: rerun this command with --apply and the same --path flags` instead of naming a command.+- **Price recommendations in `o2b search check`.** The recommendations name an active embedding model with no known price, with the price pair to declare and what a positive gate does with it, and a price pair that names a model other than the active one.+- **Where `o2b search check` looked for an embedding key.** When no embedding key resolves, `key_sources_checked:` lists the sources in probe order (`OPEN_SECOND_BRAIN_EMBEDDING_KEY`, `embedding_api_key`, then the env-key names of the registered profile `embedding_provider` selects), and `key_present_under:` lists the other registered profiles whose env key is set, or says `none`. `--json` carries the same as `credential_sources` `{consulted, present_elsewhere}`. Only names the operator declared are consulted, no value is printed, nothing goes over the network, and a configured setup's output is unchanged.+- **A semantic belief order on `brain_context_pack`.** `query_mode: "semantic"` (MCP only) embeds `query` once and scores each curated belief note by the maximum cosine over its stored chunk vectors, counting only vectors written by the configured model at the query's dimension; nothing new is embedded for the notes themselves. Tier still decides first; within a tier a note without a usable vector sorts after the scored ones, then by recency. The response gains a `semantic` object with `model`, `price_source`, `query_tokens`, `estimated_usd` (null for an unknown price), `scored` and `unembedded`, counted after the reach filter. The mode refuses by stable `error.data.code` when the semantic tier is blocked (`EMBEDDING_DISABLED`, `EMBEDDING_KEY_MISSING`), when sqlite-vec is not loadable (`VEC_EXTENSION_UNAVAILABLE`), and with `BELIEF_VECTORS_MISSING` when no kept candidate has a usable vector, naming `o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply`; a refusal that is certain in advance is decided before the query is embedded. The query embed is one paid call per request and is disclosed in the response; for a remote caller it is refused with `EMBEDDING_COST_UNPRICED` when the model has no known price and `embedding_cost_gate_usd` is positive, while a local caller keeps the disclosed, ungated embed, as for `search`. A query vector refused after the embed (`EMBEDDING_INVALID_VECTOR`) names that spend, as a late `BELIEF_VECTORS_MISSING` does.++### Changed++- **An unknown price reads as unknown, never as $0.** The maintenance banner, the backfill dry run and `search status` print `price unknown` where they used to print `$0.0000` for an unlisted model, and the JSON estimates (`estimated_cost_usd`, `estimated_usd`, the status refresh estimate) are `null`. The backfill's `estimated_cost_usd` used to be omitted for an unknown price and is now always present. A fully embedded index has nothing pending, so `search status` then prints no `refresh_cost_est` line and its JSON estimate is `0`.+- **A positive cost gate now refuses an unpriced model.** Before this release a model missing from the price table was priced at $0, so a positive `embedding_cost_gate_usd` never stopped it; local servers (Ollama, LM Studio, llama.cpp) and any unlisted model are now refused with `EMBEDDING_COST_UNPRICED` until their price is declared or the run is forced. With the gate at 0, the default, nothing is refused.+- **Spend receipts carry the price source.** Receipts, the `maintenance_spend` metric, the backfill JSON and the status JSON (`refresh_price_source`) gain `price_source`, and `o2b brain maintenance status` lists each receipt with it; journal rows written before this release read `unrecorded`, and a journaled receipt without a numeric estimate reads `price unknown` instead of failing the whole listing. `brain_maintenance` spend receipts carry a null `estimated_usd` for an unknown price.+- **`brain_context_pack` caps `query` at 2000 characters.** The cap is declared in the schema and enforced in every query mode, counted in Unicode code points as the schema's `maxLength` counts them, and a blank `query` is refused with `INVALID_PARAMS` in `semantic` mode. `brain_search` now counts its 2000-character cap the same way, so a query of characters outside the Basic Multilingual Plane is no longer refused at half the advertised length. `brain_recall_feedback`, which advertised the same `maxLength` without checking it, now refuses a longer `query` with the same `INVALID_PARAMS` error before it records anything.+- **A refused path prefix names its argument and value.** A search `path_prefix` (and a backfill `--path`) that escapes the vault is refused with the argument name and the JSON-quoted value instead of a bare `path_prefix escapes vault`.+- **Docs:** `docs/cli-reference.md` gains "Embedding prices", "Vector carry-over", the `--path` option and the credential-source lines of `search check`, `docs/how-it-works.md` the embedding spend, vector carry-over and semantic belief order entries, `docs/mcp.md` the `semantic` query mode and `EMBEDDING_COST_UNPRICED`, and `docs/updating.md` "Upgrading to 1.72.0"; the `embeddings-setup` skill gains the price declaration step and the scoped backfill. The README names this release.++### Fixed++- **The managed-file upgrade worker never recreates a removed vault.** The automatic upgrade worker checks again that the vault still exists right before it rewrites managed files or records a failure, and writes nothing when the vault was removed or moved while it ran.+- **The Hermes anti-drift test no longer fails intermittently on Windows.** It waits for the live `o2b mcp` server and its launcher to exit, and kills the whole process tree if they do not, before removing its temporary directory, so a log file the server still holds open no longer breaks the cleanup.++### Notes++- Account-side model discovery and automatic model switching are deliberately not part of this release: both would reach the network or start a paid re-embed the operator did not request.+- The hygiene dedup detector and the entity alias detector still embed short texts live, outside the spend plan, with no gate and no receipt, as before this release.+## [1.71.0] - 2026-10-04Prompt-time recall now remembers what a session was already shown: the recall-inject hook skips a note span it injected earlier in the session or that the SessionStart digest already delivered, an operator can lay the brief out as named slices declared in `Brain/_brain.yaml`, and its four caps are config keys with env overrides; and an opt-in chunked re-grounding splits an oversized SessionStart payload into parts that a new `reground-deliver` hook hands out exactly once each on the following tool calls and prompts in Claude Code and Codex, so a post-compaction payload is no longer cut to a preview.@@ -8162,6 +8197,7 @@ plugin config (vault field)`, and exits with a clear- Sandbox vault and plugin manifest fixtures for tests.- GitHub release workflow for tag-based and manually dispatched releases.+[1.72.0]: https://github.com/itechmeat/open-second-brain/compare/v1.71.0...v1.72.0[1.71.0]: https://github.com/itechmeat/open-second-brain/compare/v1.70.0...v1.71.0[1.70.0]: https://github.com/itechmeat/open-second-brain/compare/v1.69.0...v1.70.0[1.69.0]: https://github.com/itechmeat/open-second-brain/compare/v1.68.0...v1.69.0diff --git a/README.md b/README.mdindex d2cb9666..9c04821f 100644--- a/README.md+++ b/README.md@@ -113,7 +113,7 @@ The full router with readiness criteria is [`install.md`](install.md); native Wi## What is new-1.71.0 stops prompt-time recall from repeating itself. With `recall_inject_enabled` on, the recall-inject hook now skips a note span it already showed in the session or that the SessionStart digest already delivered (`recall_inject_dedupe`, on by default), its note, character, time and confidence caps are config keys with env overrides, and an operator can lay the brief out as named slices declared in `Brain/_brain.yaml`. An opt-in chunked re-grounding (`reground_parts_enabled`, off by default) splits an oversized SessionStart payload into parts that the new `reground-deliver` hook hands out exactly once each on the following tool calls and prompts in Claude Code and Codex, so a post-compaction payload is no longer cut to a preview. Every release is described in the [CHANGELOG](CHANGELOG.md).+1.72.0 makes embedding spend honest. Every estimate names its price source (`builtin`, `operator` or `unknown`), `embedding_price_model` and `embedding_price_usd_per_mtok` declare the price of a model the built-in table does not list, and an unknown price is reported as `price unknown` instead of $0; under a positive `embedding_cost_gate_usd` such a model is refused with `EMBEDDING_COST_UNPRICED` until its price is declared or the run is forced. Edited notes keep the vectors of their unchanged chunks, `o2b search vector-backfill` accepts a repeatable `--path`, `o2b search check` names an unpriced model and where it looked for an embedding key, and `brain_context_pack` gains `query_mode: "semantic"`, which orders the curated belief notes by the stored vectors of their chunks. Every release is described in the [CHANGELOG](CHANGELOG.md).## Documentationdiff --git a/docs/brainstorm/honest-embedding-spend/cli-output/claude.md b/docs/brainstorm/honest-embedding-spend/cli-output/claude.mdnew file mode 100644index 00000000..ec3cc98a--- /dev/null+++ b/docs/brainstorm/honest-embedding-spend/cli-output/claude.md@@ -0,0 +1,36 @@+### Variant 1: Nullable price quote threaded through the existing sites+- **Approach**: Replace the bare number returned by `pricePerMillionTokens` with a `PriceQuote {usdPerMtok: number | null, source}` where `source` is a registered closed vocabulary (`builtin | operator | unknown`), resolved from the frozen table and then an operator pair in flat config (`embedding_price_model` + `embedding_price_usd_per_mtok`, both-or-neither like the sunset pair). The four pricing sites keep their shape but carry `estimatedUsd: number | null`; `evaluateCostGate` refuses on `null` under a positive cap with a new `SearchError` code that names the config pair and `--force-cost`, the banner prints "price unknown", and the receipt and journal gain an optional `price_source`. Card B ships the credential-source naming in `search check` and content-hash vector reuse inside `replaceChunks`; Card C ships only a `--path` scope on `findChunksWithoutEmbeddings` that flows into estimate, gate and receipt, plus an additive `vector_coverage` field on the context pack response so a caller can see whether belief vectors exist at all.+- **Trade-offs**:+ - Pro: smallest diff, every change is local to a site that already exists, byte-identical for tabled models and gate-off.+ - Pro: no new MCP enum values, no registry schema change, low census-test churn.+ - Con: one config pair prices one model; a second registry profile stays unknown, so operators with failover profiles hit the refusal often.+ - Con: four sites must each be audited for `null` handling by hand; nothing structural stops a fifth site from reintroducing `0`.+ - Con: Card C's retrieval ask is not met; beliefs can be paid for in isolation but not ranked semantically.+- **Complexity**: medium+- **Risk**: low++### Variant 2: One spend plan feeds estimate, gate, receipt and a belief-scoped semantic mode+- **Approach**: Introduce a single `planEmbeddingSpend({scope})` that owns the census, token count, `PriceQuote`, estimate and gate verdict; `runEmbeddingPhase`, the maintenance preview, `planVectorBackfill` and the `search status` estimate all consume the plan, and the receipt is the plan plus actual tokens, so the announced and spent figures cannot diverge. Prices resolve from the builtin table, then an optional `price_usd_per_mtok` field declared on registry profiles through `o2b search provider add`, then the flat-config pair as override; scope is a path prefix that `vector-backfill --path` and the plan carry end to end. Context pack gains `query_mode: semantic`: the async MCP and CLI boundary embeds the query once through the same resolver and reports that spend in the response, runs a KNN pre-filtered to belief chunk rowids via the existing `chunk_vec_map`, and hands a `relevanceByPath` map into the still-synchronous `packContext` through an optional injected scorer; when the embedder is unavailable or belief coverage is incomplete the tool refuses and names `vector-backfill --path Brain/preferences`. `replaceChunks` reuses vector rows whose content hash, model and dimension match, and `search check` names the credential sources consulted and which other profiles have a key present.+- **Trade-offs**:+ - Pro: the plan object is the structural guarantee behind "same census feeds estimate, gate and receipt"; new spend sites must go through it.+ - Pro: prices live where the operator declares models (profiles), so failover profiles are priced too; the config pair remains for the built-in `zeroentropy` case.+ - Pro: all three cards are met; `packContext` stays synchronous for the planner and anticipatory cache.+ - Con: largest planner refactor across four call sites and three CLI verbs; CLI help parity and MCP parity tests all move.+ - Con: a new paid path (query embedding) must be declared on the wire, which adds an additive response field and one more enum member.+ - Con: pre-filtered KNN over a rowid list is a new query shape in the vector store and needs a size guard.+- **Complexity**: large+- **Risk**: medium++### Variant 3: Price provenance persisted beside the vectors, with a guarded migration verb+- **Approach**: Make the price a stored fact of the index: `index_state` gains optional `price_usd_per_mtok`, `price_source` and `price_declared_at` for the active signature, every receipt row and the `maintenance_spend` metric copy them, and the operator declares the price through a `price_model` + `price_usd_per_mtok` pair in the `_brain.yaml` embeddings block with `o2b search price show | set` verbs. On top of that plan, build the deferred `o2b search migrate-embeddings --to <model>` verb (dry run, `--apply`, `--force-cost`) that prices a full re-embed at the target model's declared price and refuses when unknown. Card C is served by fixing `search(pathPrefix)` itself: the prefix becomes a KNN pre-filter instead of a post-filter, and the context pack's semantic mode routes through ordinary `search`; scoped backfill reuses the same plan.+- **Trade-offs**:+ - Pro: the receipt reads the price that was actually assumed at spend time, not a later reconstruction; old rows read as unrecorded.+ - Pro: fixes the prefix under-return defect for every `search` caller, not only beliefs; closes Card B(e).+ - Con: price is per account, but `_brain.yaml` is per vault; two vaults on one key must declare it twice and can drift.+ - Con: a migration verb is explicitly deferred by `restamp.ts`; adding it widens the release past the three cards and adds the most new surface.+ - Con: changing the KNN filter order touches the hot search path and its ranking tests; schema additions to `index_state` need a migration and census registration.+- **Complexity**: large+- **Risk**: high++### Recommended: Variant 2+**Rationale**: The release's argument is that one census produces one number everywhere, and only Variant 2 makes that a structural property rather than a four-site audit; Variant 1 leaves Card C's retrieval ask unmet and prices only one model, while Variant 3 pulls in a deferred migration verb and a hot-path KNN change that the cards do not require. Declaring prices on registry profiles follows the existing rule that only operator-declared names count, the plan object keeps spend behind the dry-run and apply idiom, and the injected scorer lets `packContext` stay synchronous for every caller it has today.diff --git a/docs/brainstorm/honest-embedding-spend/cli-output/prompt.md b/docs/brainstorm/honest-embedding-spend/cli-output/prompt.mdnew file mode 100644index 00000000..9767a6d4--- /dev/null+++ b/docs/brainstorm/honest-embedding-spend/cli-output/prompt.md@@ -0,0 +1,72 @@+You are brainstorming architectural variants for the following task. Do not write code. Do not write a final design. Only produce variants and a recommendation.++# Task++Ship one release of Open Second Brain whose argument is "honest embedding spend": every path that spends money on embeddings states what it costs, where the price came from and what it covers, and no path assumes a benign answer it cannot prove. Three tracker cards feed it. Reconnaissance against the live source has already been done and is summarised below as fact, not conjecture; do not re-litigate whether these findings are true.++**Card A (t_0bed79f3) - operator-configurable embedding pricing with fail-closed cost caps.**+`src/core/search/embeddings/signature.ts` holds a frozen four-entry `EMBEDDING_PRICING` table (local hashing embedder at 0, three hosted OpenAI models). `pricePerMillionTokens(model)` returns 0 for every model the table does not list, and the docblock states the deliberate polarity: "the cost gate must never falsely block on a model whose rate we do not know". `evaluateCostGate` blocks only when `gateUsd > 0 && !forced && estimatedUsd > gateUsd`, so a paid model outside the table (the built-in paid `zeroentropy` provider, every `o2b search provider add` registry profile, every curated preset - none of them carries a price field) is estimated at 0 and never blocked, even under an explicit positive `embedding_cost_gate_usd`. The gate defaults to 0 (off). The enforcement point is `runEmbeddingPhase` in `src/core/search/indexer.ts`; three other places price spend and must agree with it: `estimatePendingEmbeddingSpend` (maintenance preview), `planVectorBackfill` (dry run, reports `costKnown: false` already), and the `search status` refresh estimate. The maintenance banner prints `estimated $0.0000` for an unpriced model, and the spend receipt (`MaintenanceSpendReceipt {model, tokens, estimatedUsd, forced}`, persisted to a JSONL journal and a `maintenance_spend` metric) cannot distinguish "free" from "price unknown". Price is NOT part of `embeddingSignature` (provider:model:dimension), so a price change must never mark vectors stale. Config is a flat `key: value` file split on the FIRST colon (env overrides config overrides default); model ids such as `nomic-embed-text:latest` therefore cannot appear in a key, only in a value. A precedent for an operator price key exists in the decision-model subsystem (`decision_model_input_price_usd_per_mtok`, with a `cost_source: unknown` record and a diagnostics warning). The `_brain.yaml` embeddings block deliberately binds a fact about one model as a named pair (`sunset_model` + `sunset_at`, both or neither) so that switching models never silently retargets the fact. There is no "implicit default cap" in this codebase: the two real states are "gate off" and "gate explicitly positive". The upstream card also mentions conversation-facts extraction; that path does not exist on the embedding kernel here (decision models have their own per-day gate), so scope is embeddings only. The operator picked this release scope knowing it inverts the unknown-is-free polarity under an explicit cap.++**Card B (t_82c3b275) - automated model refresh and recall tooling.**+The card's strongest upstream evidence is a design with (a) key-aware tier defaults, (b) latest-model discovery from the operator's account (`GET /v1/models`, priced-only, 24 h cache, fail-open), (c) keyless honesty, (d) vectors that survive a re-index only when content hash and model provenance match, (e) a guarded embedding migration. Verified here:+- (b) contradicts a documented stance in `src/core/search/embeddings/sunset.ts`: "This project takes no unrequested outbound call for a fact like this ... an unreachable one is a third state to design for"; every outbound sender is also declared in an egress census test. It is refused, as is automatic switching of the configured model (a model switch invalidates every vector, so an automatic upgrade is an unrequested paid re-embed).+- (a) `resolveSemanticCapability` is documented as derived "from the resolved config alone - never from process.env", so key presence cannot enter the tier. Registry profiles already carry an ordered env-key list (first present wins, rest kept as failover). When the tier is `credential-missing`, `o2b search check` emits one generic label; it does not name which credential sources were consulted (`OPEN_SECOND_BRAIN_EMBEDDING_KEY` / `embedding_api_key`, or the active profile's env-key names) nor which other registered profiles have a key present.+- (c) keyless tiering itself already ships.+- (d) Vectors do not survive an edit: a changed document goes through `replaceChunks` (`src/core/search/store/chunks.ts`), which purges every vector row of the document and deletes every chunk before reinserting, so unchanged chunks of an edited note are re-embedded and re-paid on the next embedding pass. `chunks.content_hash` is `sha256(content)` and the embedded text is exactly `content`; per-row `embeddings.model` and `dimension` are stored; query and passage prefixes are store-level (`index_state`), and any prefix, model or dimension change already clears the whole vector table through `ensureEmbeddingModel`. Vector rows live in a `vec0` virtual table reached through `chunk_vec_map(chunk_id -> vec_rowid)`.+- (e) A guarded migration verb does not exist; `restamp.ts` names it as deferred. Recall tooling (benchmark, rerank-eval, check, doctor) already exists.++**Card C (t_0431f64d) - semantic retrieval over beliefs.**+The card says beliefs (preference notes under `Brain/preferences/` and `Brain/retired/`) have no embeddings. That is refuted: they are ordinary indexed documents and their chunks get vectors on any embedding pass; `surprisal.ts` already reads stored vectors of a note through `getDocumentIdByPath -> chunksForDocument -> embeddingForChunk`. What is true: the belief-specific reader `brain_context_pack` (`src/core/brain/context-pack.ts`, synchronous `packContext`, called from the MCP tool, a CLI verb, the bench, the retrieval planner and the anticipatory cache) offers `query_mode` `substring | ranked`, where `ranked` is Jaccard token overlap and is documented as "no model, no embedding". Its pipeline is: collect candidates, prefer supersession-chain tips, apply the caller's reach filter (`visible`), then sort by tier, focus, relevance, density, recency. Routing beliefs through `search(pathPrefix)` inherits a post-KNN prefix filter (global top `limit*4` neighbours, then drop rows outside the prefix), which silently under-returns in large vaults. Separately, the only way to pay for missing vectors is a whole-vault `o2b search vector-backfill` (dry run by default, `--apply` spends behind the cost gate); its pending census `findChunksWithoutEmbeddings` takes no path scope, so "embed my beliefs" cannot be previewed or paid on its own. The same census must feed the estimate, the gate and the receipt, or the announced and the spent figures disagree.++# Project context++Open Second Brain: a local-first agent memory system. TypeScript on Bun, about 900 modules under `src/`, 1100+ test files, plus a Python plugin under `plugins/hermes/`. Surfaces: an MCP server, an `o2b` CLI, and an OpenClaw bundle.++Recent commits on the default branch:++```+2353cb1e feat: recall injection that arrives once and survives compaction (v1.71.0) (#232)+39c2f225 feat: scoped operator rules per project, harness and host, per-profile Hermes settings on a multiplexed gateway and readers that answer at the caller's reach (v1.70.0) (#229)+d49c4697 feat: ingest that reads CSV, TSV and HTML into table and parts sections, names the formats it skips and previews with o2b brain extract (v1.69.0) (#228)+f5c46615 feat: dependency facts from project manifests, Terraform seeds in the pre-extractor and readers that answer at the caller's reach (v1.68.0) (#227)+400b20ea feat: distillation pages that prove what they quote and say how much of their source the vault holds (v1.67.0)+3ec2334a feat: stable error codes on the MCP wire, typed rerank degradation and a rerank doctor check (v1.66.0) (#225)+a1ecfe7b feat: unattended maintenance recipes, install-owned lane tasks and an MCP fault guard (v1.65.0) (#224)+```++Related files: `src/core/search/embeddings/signature.ts`, `src/core/search/indexer.ts`, `src/core/search/index.ts`, `src/core/search/types.ts`, `src/core/search/search-error.ts`, `src/core/search/vector-backfill.ts`, `src/core/search/store/chunks.ts`, `src/core/search/store/vectors.ts`, `src/core/search/capability-tier.ts`, `src/core/search/embeddings/registry.ts`, `src/core/brain/maintenance/journal.ts`, `src/core/brain/maintenance/reindex-task.ts`, `src/cli/brain/verbs/maintenance.ts`, `src/cli/search/verbs/check.ts`, `src/cli/search/verbs/vector-backfill.ts`, `src/core/brain/context-pack.ts`, `src/mcp/brain/pack-tools.ts`.++Conventions that constrain any design:++- Closed vocabularies are a four-piece idiom enforced by a census test: a frozen object with camelCase keys and snake_case values, a derived union type, a members array, and a type guard over `unknown`. Values that leave TypeScript (MCP schema enums, JSON payloads, persisted records) must be registered.+- Spend runs only behind an explicit apply step: dry run by default, `--apply` to spend, `--force-cost` to pass a refused gate, and a receipt of what was spent.+- Errors are `SearchError` with a registered code; a refusal names the key or flag that resolves it.+- Architecture census tests enforce acyclic imports, the egress census, progress-stage and terminal-state vocabularies, CLI help-surface parity, MCP surface parity and version sync across manifests.+- Persisted records only gain optional fields; old rows must read as "unrecorded", never as a known value.++Constraints:++- No stubs and no placeholder implementations.+- No fallback that silently does nothing or produces a misleading success. A verdict that cannot be reached must say so rather than default to the benign answer.+- No hardcoded natural-language word lists in any language, and no hardcoded vendor list for key detection: only operator-declared names (config keys, registry profiles) count.+- No unrequested network call; no new outbound sender.+- Do not change existing public MCP tool names; additive schema changes only.+- No new external runtime dependencies.+- Byte-identical output when a new feature is absent or inactive.++# Required output format++Produce exactly 3 distinct architectural variants. For each variant:++### Variant N: <short name>+- **Approach**: 2-3 sentences describing the variant.+- **Trade-offs**: bullet list of pros and cons.+- **Complexity**: small | medium | large+- **Risk**: low | medium | high++After the three variants, add exactly one recommendation:++### Recommended: Variant N+**Rationale**: 2-3 sentences explaining why this variant over the others, considering the project context and constraints above.++Output nothing outside of these sections.diff --git a/docs/brainstorm/honest-embedding-spend/design.md b/docs/brainstorm/honest-embedding-spend/design.mdnew file mode 100644index 00000000..a5e8acca--- /dev/null+++ b/docs/brainstorm/honest-embedding-spend/design.md@@ -0,0 +1,137 @@+# Honest Embedding Spend - priced, gated and scoped embedding work, vectors that survive an edit and a semantic belief reader++**Status:** draft+**Author:** Open Second Brain maintainers (via feature-release-playbook)+**Audience:** implementation++## Problem statement++The embedding cost kernel prices any model outside a four-entry frozen table at 0. Under an explicit positive `embedding_cost_gate_usd`, a backfill on a paid model the table does not list (the built-in `zeroentropy` provider, every registry profile, every curated preset) is estimated at $0 and never refused. The maintenance banner then prints `estimated $0.0000`, and the spend receipt cannot tell "free" from "price unknown". Four places price spend (the phase gate, the maintenance preview, the backfill dry run and the `search status` estimate). They resolve the model in two different ways, so the backfill reports a local model as "cost unknown". An edited note re-embeds and re-pays every chunk, including the chunks that did not change. Paying for vectors is all-or-nothing across the vault, and the belief reader `brain_context_pack` cannot rank by meaning even though belief notes already carry vectors.++## Premise verdicts (authoritative for scope)++Every claim was checked against the live source before the brainstorm. Line numbers in the cards had drifted. The table records the verdict that bounds what is built.++| Card | Verdict | What is built |+|---|---|---|+| t_0bed79f3 pricing overrides with fail-closed caps | confirmed, corrected in three places | Built: an operator price pair, a price-source vocabulary, and a fail-closed refusal for an unpriced model under an explicit positive gate. Every pricing site renders an unknown price as unknown. Corrections: (1) visibility is partly present already (`planVectorBackfill` reports `costKnown`), but the banner, receipt and status hide or misstate it. (2) There is no "implicit default cap" in Open Second Brain. The gate is either off (0, the default) or explicitly positive, so the upstream "warn under the implicit cap" leg maps to "gate off: never block, report unknown". (3) No cost-gated conversation-facts extractor exists on the embedding kernel, and decision models have their own per-day gate, so the scope is embeddings only. The card's accounting anchors (`recall-telemetry.ts`, `memory-cost-meter.ts`) are the wrong files: they meter memory reads, not spend. |+| t_82c3b275 model refresh and recall tooling | refuted in part, corrected in part | Refuted and not built: (1) latest-model discovery from the operator's account (`GET /v1/models`, 24 h cache, fail-open). It contradicts the documented no-unrequested-network stance in `src/core/search/embeddings/sunset.ts` and the egress census. (2) Automatic switching to a newer model, because a model switch invalidates every vector, so an automatic upgrade is an unrequested paid re-embed. (3) Folding env-key presence into `resolveSemanticCapability`, which is documented as derived from the resolved config alone. Corrected: keyless tiering already ships, recall tooling already ships (benchmark, rerank-eval, check, doctor), and an automatic model-change trigger exists (`ensureEmbeddingModel` clears on a change). Built: (a) `search check` names the credential sources it consulted and the registered profiles whose key is present (names only), which is the offline form of "key-aware". (b) Vectors survive an edit when content hash and model provenance match, which is the upstream provenance-gated survival rule applied at chunk level. |+| t_0431f64d semantic retrieval over beliefs | refuted premise, residue buildable | Refuted: "beliefs have no embeddings of their own". Belief notes are indexed documents, and their chunks get vectors on any embedding pass. `surprisal.ts` already reads them. No new belief vector table and no new embed pass are built. Corrected: the belief-specific reader `brain_context_pack` is lexical (`substring`, and `ranked` documented as "no model, no embedding"), and routing beliefs through `search(pathPrefix)` under-returns because the prefix filter runs after a global KNN. Built: `query_mode: "semantic"` on the context pack, which uses exact cosine over the stored vectors of the post-reach candidates, plus a path-scoped vector backfill so belief vectors can be previewed and paid for on their own. |++## Scope++- **Price sources (t_0bed79f3).** A registered closed vocabulary `EMBEDDING_PRICE_SOURCE` with the values `builtin`, `operator` and `unknown`, and a `PriceQuote { usdPerMtok: number | null, source }` resolved by one function. The resolution order is: the operator pair when its model matches, then the frozen table, then unknown. The local hashing embedder is `builtin` at 0, so it is known-free and never unpriced.+- **Operator price pair (t_0bed79f3).** Two flat config keys: `embedding_price_model` and `embedding_price_usd_per_mtok`, with env overrides `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL` and `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK`. They must be set both or neither. The rate is a non-negative finite number, and 0 declares a model free. The pair binds the price to one model name, so switching models never silently re-targets the price. The override wins over the table, so a vendor price change can be declared.+- **One spend plan (t_0bed79f3, t_0431f64d).** `planEmbeddingSpend` owns the pending census (optionally path-scoped), the model resolution, the token estimate, the price quote, the estimate and the gate verdict. The phase gate, the maintenance preview, the backfill dry run and the `search status` refresh estimate all consume it, so what is announced, refused, spent and receipted is one computation.+- **Fail-closed under an explicit cap (t_0bed79f3).** The plan refuses as unpriced when all three hold: the gate is positive, the run is not forced, and the price source is `unknown` with a non-empty pending set. The phase throws the new `SearchError` code `EMBEDDING_COST_UNPRICED`, naming the model, the two price keys and `--force-cost`. With the gate at 0 nothing new blocks.+- **Unknown rendered as unknown (t_0bed79f3).** The estimate is `number | null` across the plan, the preview, the backfill result and the status snapshot, with null meaning the price is unknown. The maintenance banner prints `price unknown` instead of `$0.0000`. The spend receipt, the `maintenance_spend` metric payload and the JSON surfaces gain an optional `price_source`, and a new receipt for an unknown price stores `estimatedUsd: null`. The backfill's `costKnown` reads the quote source, so a local model is known-free.+- **Check recommendations (t_0bed79f3).** `search check` emits one recommendation when the active model's price is unknown. It names the price pair, says what the gate does with it, and is silent for priced and local models. It emits another when the declared price pair names a model that is not the active one (a stale declaration).+- **Credential-source report (t_82c3b275).** When the capability tier is `credential-missing`, `search check` names the credential sources it consulted: the `embedding_api_key` config key and `OPEN_SECOND_BRAIN_EMBEDDING_KEY`, or the active registry profile's env-key names in probe order. It also lists the other registered profiles whose env key is present. Only names are reported. Values are tested for presence and never read further. Only operator-declared names (config keys and registry profiles) are consulted, never a vendor list.+- **Vector carry-over on edit (t_82c3b275).** `replaceChunks` keeps a stored vector for a new chunk whose `content_hash` equals an old chunk's hash in the same document, provided the old row's `model` and `dimension` equal the identity recorded in `index_state`. It purges only the vectors it does not carry. `IndexStats` gains `embeddingsReused`.+- **Path-scoped backfill (t_0431f64d).** `o2b search vector-backfill --path <prefix>` is repeatable and validated by the existing `assertSafePathPrefix`. It scopes the pending census, the estimate, the gate and the receipt to chunks under those prefixes.+- **Semantic belief order (t_0431f64d).** `brain_context_pack` accepts `query_mode: "semantic"`. This is the MCP surface only, as `ranked` is today: the `o2b brain context-pack` verb has no `--query-mode` flag, and adding one is a separate parity change. The async boundary embeds the query once, reads the stored vectors of every belief note, and passes a `relevanceByPath` map into the still-synchronous `packContext`. That function uses the map in the relevance slot of its existing comparator, and reports `semantic` facts computed after the reach filter:+ - the model, the price source, the query tokens and the estimated query cost;+ - the number of scored candidates;+ - the ids of kept candidates that have no vector.++ The mode refuses by name in three cases: the tier is not `configured`, sqlite-vec is not loadable, or no kept candidate has a usable vector. The first two are decided by the loader before any query embed. The third is decided by the tool from the post-reach `semantic.scored` count, because only `packContext` knows the kept set. The last refusal names `o2b search vector-backfill --path Brain/preferences/ --apply`. A usable vector is a stored row whose `embeddings.model` and `dimension` equal the configured model and the query vector's dimension. Older rows count as unembedded, never as scored. A contradicted embedding identity is carried as the same drift warning the search lane attaches (`formatEmbeddingAbiDrift`).++## Out of scope++- Account-side model discovery, a model registry over the network, and automatic model switching (refuted, see the verdict table).+- Env-key presence inside `resolveSemanticCapability` (refused, see the verdict table).+- A guarded `migrate-embeddings` verb and retaining stale-model vectors across a model change. A dimension change cannot coexist in one `vec0` table, so the model-change lifecycle needs its own design. `restamp.ts` keeps naming it as deferred.+- A price field on registry profiles or presets. One active model at a time is priced by the pair. A per-profile price would add a second source to keep consistent with the pair, for no case the pair cannot express.+- Making the `search(pathPrefix)` KNN a pre-filter for every caller. The semantic belief mode avoids KNN entirely, and changing the hot search path is a separate change.+- A belief-specific vector table or embed-at-write pass (refuted premise).+- The hygiene dedup detector's live embedding of up to 200 preference principles (`src/core/brain/hygiene/detectors/dedup.ts`). It calls the provider outside the spend plan, with no gate and no receipt. It is a known residual of this release, and a follow-up card moves it onto stored vectors through the reader this release adds.+- Pricing with the two-sided token ceiling (`tokenEstimateCeiling`) instead of the chars/4 floor.+- A `--query-mode` flag on `o2b brain context-pack`. Neither `ranked` nor `semantic` is on the CLI, and CLI parity for both is a separate change.+- Gating the single query embed of the semantic belief mode. It is disclosed on the wire, as below, and is not gated, matching `search`.++## Chosen approach++This is consultant Variant 2, "one spend plan feeds estimate, gate, receipt and a belief-scoped semantic mode", accepted with three overrides.++The cost kernel gains a price-source vocabulary and a quote resolver (`src/core/search/embeddings/pricing.ts`). A new `src/core/search/embedding-spend.ts` holds `planEmbeddingSpend(store, config, { scope, forced })`. It returns the pending set, the model, the tokens, the quote, the estimate and a gate verdict `{ blocked, reason: "over_cap" | "unpriced" | null }`. `runEmbeddingPhase` is the only spender and throws on a blocked verdict. `estimatePendingEmbeddingSpend`, `planVectorBackfill` and `indexStatus` read the same plan, so a fifth pricing site has nothing to call but the plan. The store's pending census grows an optional path scope that the plan threads end to end.++On the read side, a new `src/core/brain/belief-semantic.ts` holds a pure scorer and a store-backed loader. The MCP tool calls the loader before `packContext`, so the CLI verb, the planner, the bench and the anticipatory cache keep calling `packContext` synchronously and byte-identically.++**Override 1: no pre-filtered KNN.** The consultant proposed a KNN restricted to belief chunk rowids through `chunk_vec_map`. We score exact cosine over the stored vectors of the belief notes instead, reading them the way `surprisal.ts` already does (`getDocumentIdByPath`, then `chunksForDocument`, then `embeddingForChunk`). The belief set is bounded by the vault's preferences, so exact scoring is cheap. It adds no new `vec0` query shape and no size guard, and it cannot under-return.++**Override 2: no price on registry profiles.** The consultant proposed an optional profile price as a second price source. We keep the single model-bound pair. Exactly one model is active at a time, and the pair already binds the price to its name. A second source would need a precedence rule and a consistency check for a case the pair already covers. When the operator switches to an unpriced model, the honest answer is the named refusal, not a second table.++**Override 3: partial coverage reports, it does not refuse.** The consultant proposed refusing whenever belief coverage is incomplete. That would make the mode unusable after every new preference until the next embedding pass. The mode refuses only when no kept candidate has a usable vector, which is when a "semantic" order would be pure recency. Below that, it names every kept candidate without a vector in `semantic.unembedded`. Those candidates read relevance 0 and are sorted by the existing tier and recency keys, which is stated in the response rather than implied.++## Design decisions++- **Vocabulary values.** `EMBEDDING_PRICE_SOURCE = { builtin: "builtin", operator: "operator", unknown: "unknown" }` follows the four-piece idiom: frozen object, union, members array and an `isEmbeddingPriceSource(value: unknown)` guard. The value leaves TypeScript in receipts, metrics, JSON and MCP output, so it is registered with the vocabulary census. Alternative considered: `table | override | unknown`. Rejected, because "builtin" and "operator" name who stated the price, which is what a receipt reader needs.+- **Pair, not a map.** A model-keyed key (`pricing.overrides.<model>`) cannot carry colon-bearing model ids (`nomic-embed-text:latest`), because the flat parser splits on the first colon, and the `_brain.yaml` key index refuses model-keyed maps by construction. The pair mirrors `embeddings.sunset_model` and `sunset_at`. The model is compared after `canonicalToken` (NFC, trim, lowercase), the same normalisation the table uses. A half pair, a negative rate or a non-numeric rate fails config resolution with `SearchError("INVALID_INPUT")` naming the key, the error `parseNonNegativeFloat` already raises.+- **Where the pair lives.** It goes in the flat plugin config beside `embedding_cost_gate_usd`, so the price resolves where the gate resolves (`resolveSearchConfig`). It is added to `ResolvedEmbeddingConfig` as an optional `priceOverride?: { model, usdPerMtok }`, so the dozen literal fixtures that build the config stay untouched. Alternative considered: the vault's `_brain.yaml` `embeddings:` block. Rejected, because price is a fact about the account and the provider, which the flat config already carries, while the vault file is replicated vault policy.+- **Price stays out of identity.** `embeddingSignature`, `isStaleSignature` and the corpus-generation fingerprint are not touched. An explicit regression test asserts that the signature of a model is identical with and without an override, so editing a price can never trigger a paid reindex.+- **Fail-closed only under an explicit cap.** The upstream card's "implicit default cap" does not exist here. Gate 0 keeps today's behaviour for blocking. The inverted polarity applies only to an operator who asked for a ceiling, and for that operator an unknown price is not a price below the ceiling. The docblocks that state the old polarity (`signature.ts` above the table, `presets.ts`, and the cross-references in `sunset.ts`) are rewritten to the new rule. `--force-cost` passes an unpriced refusal as it passes an over-cap one. The receipt then says `forced: true` with `price_source: unknown` and `estimatedUsd: null`. The operator scope choice for this release accepted the polarity change.+- **What counts as free.** Two things count as free: the `builtin` local model, and an `operator` declaration of 0. A loopback `embedding_base_url` (Ollama, llama.cpp) is not inferred to be free. It stays unknown, which only matters to an operator who set a positive gate, and the refusal tells them how to declare it. Alternative considered: treating loopback as free. Rejected, because a loopback URL can be a paid proxy, and inference is the silent assumption this release removes.+- **Null estimates, not zero.** `estimatedUsd` becomes `number | null` in `CostGateResult`, the plan, `EmbeddingSpendPreview`, the backfill result (`estimatedCostUsd`), the status snapshot (`estimatedRefreshCostUsd`) and new receipts. The type change reaches ten source sites, all inside this diff. Old journal rows have no `price_source` and read as unrecorded. Their numeric `estimatedUsd` stays as written, and the renderer does not reinterpret history. Alternative considered: keeping 0 plus a source field. Rejected, because a JSON consumer reading `estimated_usd` alone would read 0 as "free", which is the misleading value this release removes.+- **One model resolution.** The plan resolves the model as the phase does (`config.semantic.model ?? provider.model`). The backfill estimate (today `config.semantic.model` alone) and the status estimate (`activeEmbeddingModel`) switch to it. If implementation finds that `activeEmbeddingModel` is the same rule, the plan calls it and the duplicate goes.+- **Scope on the census.** `findChunksWithoutEmbeddings` and its count form take an optional `pathPrefixes` list. They join `documents` and match `path` with the `substr(d.path, 1, length(?)) = ?` form the vector prefix filter in `store/vectors.ts` already uses. That form needs no wildcard escaping. An empty list means vault-wide, so today's SQL is unchanged when the flag is absent. The phase receives the scope only through the plan, so the gate and the receipt cannot read a different census from the estimate.+- **Carry-over key.** The key is `content_hash` alone within one document. The embedded text is exactly `chunks.content` (`indexer.ts`: `texts = batch.map((p) => p.content)`), and `content_hash = sha256(content)`. Query and passage prefixes are store-level, and any change to them already clears every vector through `ensureEmbeddingModel`. Duplicate hashes match as a multiset in `chunk_index` order. The guard is that the old row's `embeddings.model` and `dimension` equal the `index_state` model and dimension. When the identity is unrecorded nothing is carried, and the guard says so in a code comment and a test. Mechanics, inside the existing transaction:+ 1. Read old `(content_hash, chunk_id, vec_rowid, model, dimension, embedding_hash)`.+ 2. Purge only the vec rows that will not be carried.+ 3. Delete the old chunks. The cascades remove the old map and embeddings rows.+ 4. Insert the new chunks.+ 5. Insert map and embeddings rows for the carried pairs.++ Alternative considered: keying on `(content_hash, heading_path)`. Rejected, because the heading path is not embedded, so it would re-pay for identical vectors.+- **Credential report placement.** A pure `describeCredentialSources(semantic, registry, env)` in `src/core/search/embeddings/credential-report.ts` returns `{ consulted: string[], presentElsewhere: string[] }`. The `search check` path passes the env and the loaded registry explicitly. The tier function is not touched. The report is attached to the check result as an additive field and rendered as two lines under the existing keyless warning.+- **Semantic score per note.** The score is the maximum cosine over the note's chunk vectors. The frontmatter chunk and the body chunk both carry the principle, so a sum would double-count, and the maximum is invariant to chunk count. Cosine is computed in JS over the `Float32Array` vectors read from the store. Ties fall through to the comparator's existing keys, ending in the id, so output is deterministic. The query is embedded with the `query` prefix through the configured provider, the same call `search` makes.+- **Scored set and reach.** The loader scores every belief note (`Brain/preferences/`, `Brain/retired/`, through the existing path constants that `collectPreferencePages` uses). `packContext` reads the map only for candidates that survived the reach filter. `scored` and `unembedded` are computed after that filter, so a withheld page can influence neither the order nor the report.+- **Refusal codes.** The semantic mode reuses the registered capability codes for a blocked tier (`EMBEDDING_DISABLED`, `EMBEDDING_KEY_MISSING`) and `VEC_EXTENSION_UNAVAILABLE`, and adds one code, `BELIEF_VECTORS_MISSING`. A contradicted embedding identity warns rather than refuses, matching the search lane under the shipped `warn` default. The per-row model check above is what keeps a stale vector out of the score. `EMBEDDING_COST_UNPRICED` joins `SEARCH_ERROR_CODES` and the MCP stable-error-code registry. The `semantic` mode joins `CONTEXT_PACK_QUERY_MODES` and the MCP schema enum.+- **Query spend disclosure.** The `semantic` report carries `query_tokens`, `estimated_usd` (null when the price is unknown) and `price_source`. This is the release's rule applied to the one spend path that stays ungated.+- **Byte identity.** The following cases are pinned by tests:+ - the `substring` and `ranked` pack modes;+ - every pricing surface for a table model with gate 0;+ - `findChunksWithoutEmbeddings` without a scope;+ - `replaceChunks` when vec is not loaded;+ - `search check` on a configured tier;+ - the `embeddingSignature` of every model.++## File changes++Cost kernel and plan:+- `src/core/search/embeddings/pricing.ts` (new): `EMBEDDING_PRICE_SOURCE`, `EmbeddingPriceSource`, `EMBEDDING_PRICE_SOURCES`, `isEmbeddingPriceSource`, `PriceQuote`, `resolveEmbeddingPrice(model, override)`.+- `src/core/search/embeddings/signature.ts` (modified): the table docblock is rewritten. `estimateCostUsd` and `evaluateCostGate` take a `PriceQuote` and return a nullable estimate plus a reason. `pricePerMillionTokens` is removed once its callers move, and its test moves to the resolver.+- `src/core/search/embeddings/presets.ts`, `src/core/search/embeddings/sunset.ts` (modified): polarity cross-references.+- `src/core/search/embedding-spend.ts` (new): `planEmbeddingSpend`, `EmbeddingSpendPlan`, `EMBEDDING_GATE_REASON` (closed vocabulary `over_cap`, `unpriced`).+- `src/core/search/index.ts` (modified): parse and validate the price pair. `src/core/search/types.ts` (modified): `priceOverride`, the nullable estimates, `priceSource` on the preview, the spend and the status, and `embeddingsReused` on `IndexStats`.+- `src/core/search/search-error.ts` (modified): `EMBEDDING_COST_UNPRICED`, `BELIEF_VECTORS_MISSING`.+- `src/core/search/indexer.ts` (modified): `runEmbeddingPhase`, `estimatePendingEmbeddingSpend` and `indexStatus` consume the plan. `buildRecommendations` gains the unpriced-model and stale-pair hints, and the check result gains the credential report.+- `src/core/search/vector-backfill.ts` (modified): `pathPrefixes`, the plan, `costKnown` from the quote.+- `src/core/search/store/chunks.ts`, `src/core/search/store.ts` (modified): the scoped census and the count form, and vector carry-over in `replaceChunks`.+- `src/core/search/embeddings/credential-report.ts` (new): `describeCredentialSources`.++Spend surfaces:+- `src/core/brain/maintenance/journal.ts`, `src/core/brain/maintenance/reindex-task.ts` (modified): receipt and metric `priceSource`, nullable estimate.+- `src/cli/brain/verbs/maintenance.ts` (modified): banner `price unknown`.+- `src/cli/search/verbs/vector-backfill.ts`, `src/cli/search/verbs/status.ts`, `src/cli/search/verbs/check.ts`, `src/core/search/serialize.ts`, `src/mcp/search-tools.ts` (modified): rendering and JSON fields.+- `src/cli/command-manifest.ts`, `src/cli/search.ts` (modified): `--path` on the backfill in the manifest and the search help.++Belief reader:+- `src/core/brain/belief-semantic.ts` (new): `scoreBeliefsByVector` (pure) and `loadBeliefSemanticRelevance` (store-backed, async).+- `src/core/brain/context-pack.ts` (modified): the `semantic` mode, `semanticRelevance` option and `semantic` report.+- `src/mcp/brain/pack-tools.ts` (modified): schema enum, async load, refusal mapping and output.+- `src/core/search/visibility-surface-registry.ts` (modified only if the pack surface's reason text changes).++Docs:+- `docs/cli-reference.md`, `docs/how-it-works.md`, `docs/updating.md` (migration note: gated vaults on unpriced models now refuse), `docs/mcp.md`.+- `skills/embeddings-setup/SKILL.md`, followed by `bun run sync-plugin-mirrors`.+- The CHANGELOG entry and version bump happen in PR preparation.++## Risks and open questions++- **Gated vaults start refusing.** An operator with a positive gate on an unpriced model (ZeroEntropy, any registry profile) sees maintenance reindex and backfills refused after upgrading, and a nightly lane reports refused. The mitigation is the named error, which carries the exact keys to set, plus the check recommendation, the `updating.md` entry and the CHANGELOG. This is the intended behaviour change, accepted at scope choice.+- **Nullable estimate reach.** Any consumer of `estimatedUsd` outside this repository (scripts reading the journal JSONL) sees `null` for unknown prices on new rows. This is disclosed in CHANGELOG. Old rows are unchanged.+- **Carry-over correctness.** A carried vector that does not match its content would be silent garbage. The guards are the model and dimension check against `index_state`, the exact-content key, and the fact that every identity change already clears all vectors. Tests cover an edited note, a moved paragraph, duplicated paragraphs, a model mismatch, unrecorded identity and vec not loaded. The embedder record audit (`store/embedder-audit.ts`) must still pass on carried rows. Implementation verifies that it reads only the model and dimension.+- **Foreign-key cascade.** The carry-over relies on `ON DELETE CASCADE` for `chunk_vec_map` and `embeddings`. If the store does not enable `foreign_keys`, the old rows are deleted explicitly, which the existing `replaceChunks` and purge code already have to do. Implementation reads the pragma state and writes the test against it.+- **Census churn.** New error codes, a new vocabulary, a new query mode, a new CLI flag and new JSON fields move the vocabulary, MCP surface, help-parity and instruction-body fixtures. Each is registered, not excluded.+- **Local embedder labelled semantic.** Over the hashing n-gram embedder, `semantic` order is lexical in practice. The report names the model, so the caller can see it. No silent relabelling is done.+- **Open question for implementation.** Whether `estimatePendingEmbeddingSpend` keeps its null-return contract unchanged when the plan reports an unpriced refusal. Expected answer: yes. It returns a preview with `blocked: true`, `reason: "unpriced"` and `estimatedUsd: null`, and the banner renders it. A test pins it.diff --git a/docs/brainstorm/honest-embedding-spend/plan.md b/docs/brainstorm/honest-embedding-spend/plan.mdnew file mode 100644index 00000000..776263f2--- /dev/null+++ b/docs/brainstorm/honest-embedding-spend/plan.md@@ -0,0 +1,152 @@+# Honest Embedding Spend - implementation plan++Every task is test-first: write the failing test, run it and see it fail for the expected reason, implement, then run the formatter and linter before the commit. Every new test file opens with a docblock that states the defect it pins and what it deliberately does not cover. Task numbers give a valid order. Tasks 8, 9 and 11 do not depend on the pricing chain and can be done in any order.++## Tasks++### Task 1: Price-source vocabulary and quote resolver+- **Files**: `src/core/search/embeddings/pricing.ts` (new), `src/core/search/embeddings/signature.ts`, `tests/core/search/embeddings.pricing.test.ts` (new), `tests/core/search/embeddings.signature.test.ts`, the vocabulary census registration.+- **Acceptance**:+ - `resolveEmbeddingPrice` returns `{0.02, builtin}` for `text-embedding-3-small` and `{0, builtin}` for the local model.+ - It returns `{null, unknown}` for an unlisted model and for null.+ - It returns `{rate, operator}` when the override model matches after `canonicalToken`. That includes re-pricing a table model and declaring a rate of 0.+ - A non-matching override leaves the table answer unchanged.+ - `EMBEDDING_PRICE_SOURCE` passes the four-piece vocabulary census.+ - `estimateCostUsd` returns null for an unknown quote.+ - The `pricePerMillionTokens` assertions move to the resolver, and the function is deleted once task 5 removes its last caller.+ - The table docblock states the new rule.+- **Depends on**: none++### Task 2: Operator price pair in config+- **Files**: `src/core/search/index.ts`, `src/core/search/types.ts`, `tests/core/search/embeddings.cost-gate.test.ts`.+- **Acceptance**:+ - `embedding_price_model` and `embedding_price_usd_per_mtok` resolve from config, and the env overrides `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL` and `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK` win over it.+ - A colon-bearing model id in the value (`nomic-embed-text:latest`) survives the flat parser.+ - A half pair is refused with `SearchError("INVALID_INPUT")`, naming the missing key. So are a negative rate and a non-numeric rate.+ - When both keys are absent, `priceOverride` is absent, and every literal `ResolvedEmbeddingConfig` fixture compiles unchanged.+ - `embeddingSignature` of a model is identical with and without an override (regression test).+- **Depends on**: Task 1++### Task 3: Shared spend plan and scoped pending census+- **Files**: `src/core/search/embedding-spend.ts` (new), `src/core/search/store/chunks.ts`, `src/core/search/store.ts`, `src/core/search/embeddings/signature.ts` (`evaluateCostGate` takes the quote and returns a reason), `tests/core/search/embedding-spend.test.ts` (new), `tests/core/search/pending-vectors.test.ts`.+- **Acceptance**:+ - `planEmbeddingSpend` returns the pending set, model, tokens, quote, nullable estimate and the gate verdict `{blocked, reason}`, with `EMBEDDING_GATE_REASON` registered.+ - Its gate verdicts:+ - gate 0 never blocks;+ - gate > 0 with a known price blocks only over the cap, with reason `over_cap`;+ - gate > 0 with an unknown price and pending chunks blocks with reason `unpriced`;+ - forced never blocks;+ - an empty pending set never blocks;+ - the local model never reads `unpriced`.+ - `findChunksWithoutEmbeddings({ pathPrefixes })` and the count form return only chunks under the prefixes and agree with each other.+ - Without a scope, the SQL and the results are unchanged.+- **Depends on**: Task 2++### Task 4: Phase gate consumes the plan and refuses unpriced spend+- **Files**: `src/core/search/indexer.ts` (`runEmbeddingPhase`), `src/core/search/search-error.ts`, the MCP stable error-code registry, `tests/core/search/embedding-spend-preview.test.ts`, `tests/core/search/indexer.embeddings.test.ts`.+- **Acceptance**:+ - With gate > 0 and an unknown price, `runEmbeddingPhase` throws `EMBEDDING_COST_UNPRICED`. The message names the model, both price keys and `--force-cost`, and the provider is never called (injected throwing provider).+ - With `forceCost` the phase proceeds, and `stats.spend` carries `forced: true`, `priceSource: unknown` and `estimatedUsd: null`.+ - Over-cap behaviour and the `EMBEDDING_COST_GATE` message are unchanged.+ - The new code is registered on the MCP wire.+- **Depends on**: Task 3++### Task 5: Preview, backfill and status read the plan+- **Files**: `src/core/search/indexer.ts` (`estimatePendingEmbeddingSpend`, `indexStatus`), `src/core/search/vector-backfill.ts`, `src/core/search/types.ts`, `tests/core/search/embedding-spend-preview.test.ts`, `tests/cli/search-vector-backfill.test.ts`, the status test that covers `estimatedRefreshCostUsd`.+- **Acceptance**:+ - The preview, the backfill dry run and the status estimate report the same tokens, estimate and source as the phase for one fixture.+ - An unknown price yields `estimatedUsd: null` and `blocked: true` with reason `unpriced` under a positive gate.+ - The backfill reports `costKnown: true` for the local model and for an operator-priced model, and `false` only for `unknown`.+ - The backfill and the status estimate resolve the model as the phase does.+- **Depends on**: Task 4++### Task 6: Receipts, banner and JSON surfaces carry the price source+- **Files**: `src/core/brain/maintenance/journal.ts`, `src/core/brain/maintenance/reindex-task.ts`, `src/cli/brain/verbs/maintenance.ts`, `src/cli/search/verbs/vector-backfill.ts`, `src/cli/search/verbs/status.ts`, `src/core/search/serialize.ts`, `src/mcp/search-tools.ts`, `tests/cli/brain-maintenance.test.ts`, `tests/cli/search-vector-backfill.test.ts`.+- **Acceptance**:+ - The maintenance banner prints `price unknown` for an unknown quote and the dollar figure otherwise.+ - A new receipt and the `maintenance_spend` metric carry `price_source`.+ - A journal row without `price_source` renders as unrecorded, and its numeric estimate is unchanged.+ - The backfill and status text and JSON show `price unknown` or null and never `$0.0000` for an unknown price.+ - Output for a table model with gate 0 is byte-identical to the base.+- **Depends on**: Task 5++### Task 7: Check recommendations for unpriced and stale price declarations+- **Files**: `src/core/search/indexer.ts` (`buildRecommendations`), `src/cli/search/verbs/check.ts`, `tests/cli/search-check-provider.test.ts`.+- **Acceptance**:+ - With semantic configured, the price unknown and the gate at 0, there is one recommendation that names both price keys and states that a positive gate would refuse.+ - With a positive gate, the recommendation states that backfills refuse.+ - A declared pair for a model other than the active one yields one stale-declaration recommendation.+ - Priced and local models produce no new line.+- **Depends on**: Task 5++### Task 8: Credential-source report in search check+- **Files**: `src/core/search/embeddings/credential-report.ts` (new), `src/core/search/indexer.ts` (check result field), `src/cli/search/verbs/check.ts`, `tests/core/search/credential-report.test.ts` (new), `tests/cli/search-check-provider.test.ts`.+- **Acceptance**:+ - For `credential-missing` with explicit config, `consulted` is `embedding_api_key` and `OPEN_SECOND_BRAIN_EMBEDDING_KEY`. For a registry profile, it is the profile's env-key names in probe order.+ - `presentElsewhere` lists the other registered profiles whose env key is non-empty.+ - A test env with secret values proves that no value appears in the text or JSON output.+ - Nothing is reported for a `configured` tier, so that output is byte-identical.+ - `resolveSemanticCapability` is unchanged, and the egress census is unchanged.+- **Depends on**: none++### Task 9: Vector carry-over on edit+- **Files**: `src/core/search/store/chunks.ts` (`replaceChunks`), `src/core/search/store/vectors.ts` (a read helper for the old rows, if needed), `src/core/search/indexer.ts` (`embeddingsReused` tally), `src/core/search/types.ts`, `tests/core/search/vector-carry-over.test.ts` (new).+- **Acceptance**:+ - After editing one paragraph of a three-chunk note, only the changed chunk is pending, the unchanged chunks keep their vectors (same `embedding_hash`), and `embeddingsReused` is 2.+ - A moved paragraph is carried. Duplicated paragraphs carry as a multiset.+ - A row whose model or dimension differs from `index_state` is not carried, and neither is any row when the identity is unrecorded.+ - With vec not loaded, behaviour is unchanged.+ - The embedder record audit passes on carried rows.+ - A semantic query after the edit returns the carried chunk.+- **Depends on**: none++### Task 10: Path-scoped vector backfill+- **Files**: `src/core/search/vector-backfill.ts`, `src/core/search/indexer.ts` (the phase takes its scope through the plan), `src/cli/search/verbs/vector-backfill.ts`, `src/cli/command-manifest.ts`, `src/cli/search.ts`, `tests/cli/search-vector-backfill.test.ts`.+- **Acceptance**:+ - The `--path` flag is repeatable and validated by `assertSafePathPrefix`. An unsafe prefix is refused by name.+ - The dry run with `--path Brain/preferences/` prices only belief chunks.+ - `--apply` embeds only them.+ - A refusal over the cap or for an unpriced model reads the scoped census, and the receipt agrees with the dry run.+ - The progress and terminal-state censuses still pass, and help-surface parity covers the flag.+- **Depends on**: Task 5++### Task 11: Belief vector scorer and loader+- **Files**: `src/core/brain/belief-semantic.ts` (new), `tests/core/brain/belief-semantic.test.ts` (new).+- **Acceptance**:+ - The pure `scoreBeliefsByVector` returns max-cosine per path over the injected vectors, a stable order, `unembedded` paths and the count scored.+ - `loadBeliefSemanticRelevance` reads stored vectors of `Brain/preferences/` and `Brain/retired/` notes from a fixture index.+ - Rows from another model or dimension count as unembedded.+ - The query is embedded once with the `query` prefix (counting provider).+ - A blocked tier and missing vec each refuse with their named code before any query embed (throwing provider). Zero usable vectors returns an empty map, not a refusal: the refusal is decided after the reach filter in task 13.+ - The report carries the model, price source, query tokens and nullable query cost.+- **Depends on**: Task 1++### Task 12: Semantic mode in packContext+- **Files**: `src/core/brain/context-pack.ts`, `tests/core/brain/context-pack-semantic.test.ts` (new), `tests/core/brain/context-pack-ranked-query.test.ts`.+- **Acceptance**:+ - In `semantic` mode with a relevance map, a paraphrased query with zero token overlap orders the closest belief first within its tier.+ - `scored` and `unembedded` are computed after the reach filter, so a withheld page appears in neither.+ - A `semantic` call without a map throws a named error.+ - `substring` and `ranked` output is byte-identical to the base.+ - The bench, planner and anticipatory cache callers compile unchanged.+- **Depends on**: Task 11++### Task 13: MCP surface for the semantic mode+- **Files**: `src/mcp/brain/pack-tools.ts`, `tests/mcp/context-pack-tool.test.ts`, the instruction-body and tool-census fixtures.+- **Acceptance**:+ - The schema enum accepts `semantic` and requires `query`.+ - A blocked tier and missing vec surface their stable codes on the wire.+ - When `semantic.scored` is 0 after the reach filter, the tool refuses with `BELIEF_VECTORS_MISSING`, naming `o2b search vector-backfill --path Brain/preferences/ --apply`. A reach-withheld page never changes that verdict.+ - The response carries the `semantic` report.+ - The surface parity and fixtures are updated by registration, not exclusion.+- **Depends on**: Task 12++### Task 14: Docs and skill+- **Files**: `docs/cli-reference.md`, `docs/how-it-works.md`, `docs/updating.md`, `docs/mcp.md`, `skills/embeddings-setup/SKILL.md`, the mirror under `plugins/codex/` via `bun run sync-plugin-mirrors`.+- **Acceptance**:+ - The docs describe the price pair, the fail-closed rule, `price unknown`, the credential report, vector carry-over, `--path` and the MCP `semantic` query mode.+ - `updating.md` names the behaviour change for gated vaults on unpriced models.+ - The stale "unlisted-model price is 0" sentence is gone.+ - `bun run sync-plugin-mirrors:check` passes.+ - The CHANGELOG entry and version bump are done in PR preparation (minor bump).+- **Depends on**: Tasks 1-13diff --git a/docs/brainstorm/honest-embedding-spend/variants.md b/docs/brainstorm/honest-embedding-spend/variants.mdnew file mode 100644index 00000000..233300e6--- /dev/null+++ b/docs/brainstorm/honest-embedding-spend/variants.md@@ -0,0 +1,62 @@+# Variants considered - honest embedding spend++The primary consultant CLI ran once against `cli-output/prompt.md` (output in `cli-output/claude.md`). It returned three parseable variants, so the fallback consultant was not invoked.++The consultant's output is reproduced verbatim below. The decision follows it, including the three places where it departs from the recommendation.++---++### Variant 1: Nullable price quote threaded through the existing sites+- **Approach**: Replace the bare number returned by `pricePerMillionTokens` with a `PriceQuote {usdPerMtok: number | null, source}` where `source` is a registered closed vocabulary (`builtin | operator | unknown`), resolved from the frozen table and then an operator pair in flat config (`embedding_price_model` + `embedding_price_usd_per_mtok`, both-or-neither like the sunset pair). The four pricing sites keep their shape but carry `estimatedUsd: number | null`; `evaluateCostGate` refuses on `null` under a positive cap with a new `SearchError` code that names the config pair and `--force-cost`, the banner prints "price unknown", and the receipt and journal gain an optional `price_source`. Card B ships the credential-source naming in `search check` and content-hash vector reuse inside `replaceChunks`; Card C ships only a `--path` scope on `findChunksWithoutEmbeddings` that flows into estimate, gate and receipt, plus an additive `vector_coverage` field on the context pack response so a caller can see whether belief vectors exist at all.+- **Trade-offs**:+ - Pro: smallest diff, every change is local to a site that already exists, byte-identical for tabled models and gate-off.+ - Pro: no new MCP enum values, no registry schema change, low census-test churn.+ - Con: one config pair prices one model; a second registry profile stays unknown, so operators with failover profiles hit the refusal often.+ - Con: four sites must each be audited for `null` handling by hand; nothing structural stops a fifth site from reintroducing `0`.+ - Con: Card C's retrieval ask is not met; beliefs can be paid for in isolation but not ranked semantically.+- **Complexity**: medium+- **Risk**: low++### Variant 2: One spend plan feeds estimate, gate, receipt and a belief-scoped semantic mode+- **Approach**: Introduce a single `planEmbeddingSpend({scope})` that owns the census, token count, `PriceQuote`, estimate and gate verdict; `runEmbeddingPhase`, the maintenance preview, `planVectorBackfill` and the `search status` estimate all consume the plan, and the receipt is the plan plus actual tokens, so the announced and spent figures cannot diverge. Prices resolve from the builtin table, then an optional `price_usd_per_mtok` field declared on registry profiles through `o2b search provider add`, then the flat-config pair as override; scope is a path prefix that `vector-backfill --path` and the plan carry end to end. Context pack gains `query_mode: semantic`: the async MCP and CLI boundary embeds the query once through the same resolver and reports that spend in the response, runs a KNN pre-filtered to belief chunk rowids via the existing `chunk_vec_map`, and hands a `relevanceByPath` map into the still-synchronous `packContext` through an optional injected scorer; when the embedder is unavailable or belief coverage is incomplete the tool refuses and names `vector-backfill --path Brain/preferences`. `replaceChunks` reuses vector rows whose content hash, model and dimension match, and `search check` names the credential sources consulted and which other profiles have a key present.+- **Trade-offs**:+ - Pro: the plan object is the structural guarantee behind "same census feeds estimate, gate and receipt"; new spend sites must go through it.+ - Pro: prices live where the operator declares models (profiles), so failover profiles are priced too; the config pair remains for the built-in `zeroentropy` case.+ - Pro: all three cards are met; `packContext` stays synchronous for the planner and anticipatory cache.+ - Con: largest planner refactor across four call sites and three CLI verbs; CLI help parity and MCP parity tests all move.+ - Con: a new paid path (query embedding) must be declared on the wire, which adds an additive response field and one more enum member.+ - Con: pre-filtered KNN over a rowid list is a new query shape in the vector store and needs a size guard.+- **Complexity**: large+- **Risk**: medium++### Variant 3: Price provenance persisted beside the vectors, with a guarded migration verb+- **Approach**: Make the price a stored fact of the index: `index_state` gains optional `price_usd_per_mtok`, `price_source` and `price_declared_at` for the active signature, every receipt row and the `maintenance_spend` metric copy them, and the operator declares the price through a `price_model` + `price_usd_per_mtok` pair in the `_brain.yaml` embeddings block with `o2b search price show | set` verbs. On top of that plan, build the deferred `o2b search migrate-embeddings --to <model>` verb (dry run, `--apply`, `--force-cost`) that prices a full re-embed at the target model's declared price and refuses when unknown. Card C is served by fixing `search(pathPrefix)` itself: the prefix becomes a KNN pre-filter instead of a post-filter, and the context pack's semantic mode routes through ordinary `search`; scoped backfill reuses the same plan.+- **Trade-offs**:+ - Pro: the receipt reads the price that was actually assumed at spend time, not a later reconstruction; old rows read as unrecorded.+ - Pro: fixes the prefix under-return defect for every `search` caller, not only beliefs; closes Card B(e).+ - Con: price is per account, but `_brain.yaml` is per vault; two vaults on one key must declare it twice and can drift.+ - Con: a migration verb is explicitly deferred by `restamp.ts`; adding it widens the release past the three cards and adds the most new surface.+ - Con: changing the KNN filter order touches the hot search path and its ranking tests; schema additions to `index_state` need a migration and census registration.+- **Complexity**: large+- **Risk**: high++### Recommended: Variant 2+**Rationale**: The release's argument is that one census produces one number everywhere, and only Variant 2 makes that a structural property rather than a four-site audit; Variant 1 leaves Card C's retrieval ask unmet and prices only one model, while Variant 3 pulls in a deferred migration verb and a hot-path KNN change that the cards do not require. Declaring prices on registry profiles follows the existing rule that only operator-declared names count, the plan object keeps spend behind the dry-run and apply idiom, and the injected scorer lets `packContext` stay synchronous for every caller it has today.++---++## Decision++Variant 2 is accepted. The release argues that one census produces one number everywhere: what a run announces, refuses, spends and receipts. Only a shared spend plan makes that structural. Variant 1 leaves four pricing sites to be audited by hand and does not serve the belief retrieval ask. Variant 3 pulls in the migration verb the cards do not require and a hot-path KNN change, and it stores price as per-vault policy although price is a fact about the account.++Three overrides, each recorded in `design.md` under "Chosen approach":++1. **Exact cosine instead of a pre-filtered KNN.** The belief set is bounded by the vault's preference notes, and `surprisal.ts` already reads stored vectors per note. Exact scoring adds no new `vec0` query shape and cannot under-return.+2. **No price field on registry profiles.** One model is active at a time, and the model-bound pair already prices it. A second price source would need a precedence rule and a consistency check for a case the pair covers. An unpriced switch is answered by the named refusal.+3. **Partial belief coverage is reported, not refused.** Refusing on any vectorless preference would disable the mode after every new preference until the next embedding pass. The mode refuses only when no kept candidate has a usable vector, and names every unembedded candidate otherwise.++## Premise outcomes that bound the variants++- t_0bed79f3: confirmed, with three corrections. There is no implicit default cap, visibility is partial already, and the scope is embeddings only.+- t_82c3b275: account-side model discovery and automatic model switching are refuted against the documented no-unrequested-network stance and the vector invalidation they would trigger. Env-key presence stays out of the capability tier. Keyless tiering and recall tooling already ship. What is built is the offline credential report and the provenance-gated vector carry-over.+- t_0431f64d: "beliefs have no embeddings" is refuted, because belief notes are indexed and vectorised like any document. What is built is a belief-scoped reader and a path-scoped backfill.diff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 26bbe6a0..1403daca 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -634,6 +634,8 @@ The lane's two vault-side knobs live in `Brain/_brain.yaml` under `maintenance:`Since v1.64.0 the lane's timeouts are honest and its embedding spend is named. A task killed at its safeguard deadline exits **6** (`probeIncomplete`), keyed on `timed_out` rows alone: the only proved fact is that the pass did not finish, which is not the same as a task failing; a proved failure (1) outranks a timeout, a timeout outranks a refusal (7), and the render says TIMED OUT with the safeguard detail. The reindex task stays keyword-only unless `maintenance_embeddings: true` (env `OPEN_SECOND_BRAIN_MAINTENANCE_EMBEDDINGS`, default `false`) opts it in; then it requests the embedding phase whenever the resolved semantic config can reach a provider, announces the pending-spend estimate before the pass, and journals the phase's own cost-gate result as the per-run receipt - model, tokens, estimated cost, and whether a bypass fired - on the task row, the journal line and the `maintenance_spend` metrics surface; the preview is an estimate by position, the receipt prices what the completed pass embedded, and a run killed mid-spend receipts nothing. `--force-cost` (MCP `force_cost`) bypasses a positive `embedding_cost_gate_usd` for this run, recorded on the receipt when it overrode a gate that would have refused. A safeguard timeout is journaled as `timed_out` and neither counts toward nor resets the failure streak that refuses a task.+Since v1.72.0 the spend is priced honestly. The banner prints `price unknown` instead of `$0.0000` when the embedding model has no known price, the receipt and the `maintenance_spend` metric carry `price_source` (`builtin`, `operator` or `unknown`) with a null estimate for an unknown price, and `o2b brain maintenance status` lists each receipt with its `price_source` (`unrecorded` for rows journaled before this release). Under a positive `embedding_cost_gate_usd` an unpriced model refuses the reindex task's embedding phase with `EMBEDDING_COST_UNPRICED` unless `--force-cost`; declare the price with `embedding_price_model` and `embedding_price_usd_per_mtok` (see "Embedding prices" below).+Since v1.65.0 the lane prints its own schedule and can carry an install's own upkeep. `o2b brain maintenance run --cron-template` prints a script (`~/.local/bin/osb-maintenance-<hash>.sh`, where `<hash>` is the first 8 hex characters of the SHA-256 of the resolved vault path, so each vault gets its own script, Hermes job and systemd units and re-rendering for the same vault keeps the name) and its scheduler lines - a crontab line and the Hermes form by default, a systemd user timer with `--format systemd` - with `--interval <N>m|h|d` defaulting to `1h`: the gates decide whether work happens and a gate skip exits 0, so an hourly schedule plus `--window` is the intended pattern. The script embeds the resolved vault, runs `o2b brain maintenance run --vault '<vault>' --json`, stays silent on exit 0 and prints the captured JSON and keeps the exit code otherwise. The rendered script calls `o2b` by name, so it expects `o2b` on the scheduler's PATH: cron jobs and systemd user services start with a minimal PATH, so add the install directory (usually `~/.local/bin`) to that PATH. The verb returns before the lease, the gates, the journal and the metrics, so printing a recipe leaves no trace. A bad interval, window or format, `--interval` or `--format` without `--cron-template`, a lane-run flag (`--force`, `--retry`, `--force-cost`, `--busy-minutes`, `--busy-threshold`, `--agent`, `--progress`, `--json`) beside `--cron-template`, and `status --cron-template` exit 2.Custom lane tasks (since v1.65.0) put an install's own upkeep under the lane's window, busy, pressure, lease and streak gates. They are declared in the machine config file, never in the vault: `maintenance_custom_<name>: <command>`, optionally `maintenance_custom_<name>_cwd: <absolute dir>` (default the running user's home directory; the vault is allowed when named) and `maintenance_custom_<name>_timeout_seconds: <N>` (default 120; a whole number from 1 to 1200). The lease is taken once per pass for 30 minutes, so the declared custom timeouts together stay within 1200 s, which leaves 600 s for the built-in tasks; in name order, a task whose timeout would take the sum past 1200 s is refused by name, and the full 8 tasks fit at the default. Declared tasks run only with the master switch `maintenance_custom_tasks: true` (env `OPEN_SECOND_BRAIN_MAINTENANCE_CUSTOM_TASKS`, default off; `0` turns it off on one host). Each runs as `custom:<name>`, where `<name>` matches `^[a-z][a-z0-9-]{0,31}$` (no underscores, so the suffixes stay unambiguous, and no task can be named `tasks`), after the four built-in tasks and stale-first with them; at most 8 are declared. The command runs through `sh -c` (`cmd.exe /d /s /c` on Windows) with stdin closed, stdout discarded and `O2B_VAULT` set. Its environment is this process's minus every variable whose name declares a credential (`*_API_KEY`, `*_TOKEN`, `*_SECRET`, `*PASSWORD*` and the other names the redactor treats as secrets); `PATH`, `HOME`, `LANG`, `LC_*`, `TZ`, `TMPDIR` and `O2B_VAULT` are always kept, and a command that needs a key reads it from its own configuration. The default working directory is not the vault because the vault syncs and is writable through the MCP write tools, and on Windows a bare command name resolves in the working directory first; use absolute executable paths. A `_cwd` that does not exist fails the task as `custom task <name>: cwd does not exist: <path>`. A non-zero exit is reported as `exit <N>: <stderr tail>`, redacted and capped at 4096 bytes. The task's outcome is its shell's exit status: it does not wait for processes the command left in the background, but those still belong to the task, and on POSIX the command's whole process group is killed (SIGTERM, then SIGKILL a second later) when the timeout elapses or the o2b process exits, whichever is first, also after the shell has exited (on Windows `taskkill /T /F` kills the tree while the shell runs). Past its timeout the run is journaled as timed out, and unlike a built-in task, a custom task's timeout counts toward its `failure_streak_limit` and exits 1 rather than 6, so a command that always hangs is refused like one that always fails. A shell that exits before its timeout keeps its exit status even when a background process still holds its stderr at the deadline. `--retry custom:<name>` (MCP `retry_tasks`) attempts a refused custom task alone; an undeclared name is refused with the registered list. A bad declaration (a bad name, an empty command, an out-of-range timeout, a relative `_cwd`, a suffix key without its command, a ninth task, a timeout over the budget) is named on stderr as `custom task refused: <reason>` (MCP `custom_task_errors`) and the valid ones still run; `status` says `custom tasks declared but maintenance_custom_tasks is off` while the switch is off, or `custom tasks declared but OPEN_SECOND_BRAIN_MAINTENANCE_CUSTOM_TASKS turns them off` when the env override did it. A custom task never calls a model on the lane's behalf and records no spend receipt, and the lane cannot police spend or side effects inside an operator's own command. No MCP parameter can add, edit or read a command. The config reader strips one pair of matching surrounding quotes from a value, so a command that starts and ends with the same quote character loses them; wrap such a command in single quotes (`maintenance_custom_tidy: '"/opt/my tool" --flag "x"'`).@@ -1806,14 +1808,43 @@ o2b search vector-backfill Run the vector phase ALONE for indexed chunks thatconfigured semantic capability tier, and contacts no provider.--apply is the only path that reaches a provider or writes a vector--force-cost bypasses the embedding cost gate for that run+ --path <prefix> scopes the run to chunks under a vault-relative+ prefix; repeatable, and validated like every other path prefix (an+ unsafe prefix is refused by name). The pending census, the estimate,+ the cost gate and the spend receipt all read the same scoped census,+ so `--path Brain/preferences/` prices and embeds only belief notes.+ The text report adds a `scope:` line and the next step it names keeps+ the scope. On Windows a prefix that would need quoting is not+ spliced into the advice: next_command is omitted and the text says+ `next: rerun this command with --apply and the same --path flags`.+ The prefix is a raw string prefix, not a directory: end a directory+ with `/`, or `Brain/pref` also matches `Brain/preferences-old/`. A+ leading `./` is dropped and `\` becomes `/`; an empty prefix is+ refused with INVALID_INPUT. A prefix that matches no indexed+ document is warned on stderr by name (`scope <prefix> matches no+ indexed document`) rather than reported like a fully embedded+ scope (since v1.72.0)--progress watches it; Ctrl-C stops it between embed batches--json emits dry_run, capability_tier, capability_code, chunks_total,- pending, embedded, retries, plus estimated_cost_usd only when the- model's price is known - a missing price is an absent key, never 0+ pending, embedded, retries, estimated_cost_usd, price_source, and+ path_prefixes on a scoped run, plus unmatched_path_prefixes when a+ prefix matches no document. Since v1.72.0 estimated_cost_usd is+ null (never 0, never omitted) when the model's price is unknown, and+ the text report prints `price unknown` for it. Since v1.72.0, when+ the configured gate would refuse the run unforced, --json adds+ gate_blocked: true and gate_reason (unpriced or over_cap), the+ dry-run text report adds `cost gate: would refuse (<reason>); add+ --force-cost or ...` naming the price pair or embedding_cost_gate_usd;+ the next step it names stays unforced. An --apply run that+ reached the provider adds spend {model, tokens, estimated_usd,+ price_source, forced}, its receipt. Both are absent otherwiseIdempotent; an --apply run that wrote vectors appends onevector-backfill Brain log evento2b search status Index status; since v0.36.0 also reports the active embeddingsignature (<provider>:<model>:<dimension>) and a refresh-cost estimate+ Since v1.72.0 the estimate is null and printed as `price unknown` when+ the model has no known price; --json adds refresh_price_source+ (builtin, operator or unknown)Since v1.64.0, once an index exists, status also printsevent_time: <with>/<documents> documents (earliest <ISO>, latest <ISO>,<n> in the last 30 days); --json carries it as the event_time object@@ -1872,6 +1903,20 @@ o2b search check Pre-flight diagnostics: vault, index directory, SQ`contradicted` - a record the data itself disproves, which is adifferent finding from ABI drift (the record disagrees with this build)and from an unrecorded token (no claim was ever made).++ Since v1.72.0, when no embedding key resolves, the report names where+ it looked, by name only: `key_sources_checked:` lists the sources in+ probe order (OPEN_SECOND_BRAIN_EMBEDDING_KEY, embedding_api_key, then+ the env-key names of the registered profile `embedding_provider`+ selects), and `key_present_under:` lists the other registered profiles+ whose env key is set, or says `none`. --json carries+ the same as `credential_sources` {consulted, present_elsewhere}. Only+ names you declared are consulted (config keys and your provider+ registry), no value is ever printed, and a configured setup's output+ is unchanged. The recommendations also name an embedding model with+ no known price (with the price pair to declare and what a positive+ gate does with it) and a price pair that names a model other than the+ active one.o2b search restamp Record this build's sqlite-vec version as the one the stored vectors areaccepted under - the repair for a drift confined toembedding_vec_version, which is an ABI marker rather than a property of@@ -2037,7 +2082,59 @@ built-in `openai-compat`, the offline `local` feature-hashing embedder(no cloud, no key, no model download; `embedding_dimension` default 256),`disabled`, or any name registered via `o2b search provider add`.`embedding_cost_gate_usd` (default 0 = off) refuses an embedding run whose-estimated spend exceeds it unless `--force-cost`.+estimated spend exceeds it unless `--force-cost`. Since v1.72.0 a blank+(whitespace-only) value of the gate or of its env twin+`OPEN_SECOND_BRAIN_EMBEDDING_COST_GATE` fails config resolution with+`INVALID_INPUT` instead of reading as a gate of 0, and a blank+`search_rerank_min_score` is refused the same way.++Embedding prices (since v1.72.0). Every estimate names where its price+came from: `builtin` (the frozen price table, and the local embedder,+which is free), `operator` (declared by you) or `unknown`. Declare a price+for a model the table does not list, or correct a table price, with the+operator price pair:++```yaml+embedding_price_model: nomic-embed-text:latest+embedding_price_usd_per_mtok: 0.02+```++The env twins are `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL` and+`OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK`, and they win over the+config keys as a pair: when either env twin is set, both halves come from+env and the config pair is ignored, so an env model never pairs with a+config rate. Set both keys or neither (a blank value counts as unset);+the rate is USD per million tokens, a plain non-negative decimal number+no larger than 1000000, and `0` declares the model free. A half pair, a+negative rate, a non-decimal rate (`0x10`, `1e3`, `Infinity`) or a rate+above 1000000 fails config resolution with `INVALID_INPUT` naming the key+or env variable that supplied it. The pair binds the price to one model+name (compared case-insensitively), so switching models never re-targets+it silently: `o2b search check` flags a declaration that names a model+other than the active one. A price is not part of the embedding identity,+so declaring or editing it never triggers a reindex. A loopback+`embedding_base_url` is not assumed to be free; declare `0` for a local+server that costs nothing.++An unknown price is reported as unknown, never as $0: the maintenance+banner, the backfill dry run and `search status` print `price unknown`,+the JSON estimates are `null`, and spend receipts carry `price_source`.+A fully embedded index has nothing pending to pay for, so `search status`+then prints no `refresh_cost_est` line and its JSON estimate is `0`.+Under a positive `embedding_cost_gate_usd`, an embedding run on a model+with no known price and pending chunks is refused with+`EMBEDDING_COST_UNPRICED`, because an unknown price cannot be checked+against a cap. The message names the model, both price keys and+`--force-cost`; the provider is never contacted. `--force-cost` passes the+refusal, and the receipt then records `forced: true`, `price_source:+unknown` and a null estimate. With the gate at 0 (the default) nothing is+refused.++Vector carry-over (since v1.72.0). When a note is edited, a chunk whose+content did not change keeps its stored vector, provided the vector was+written by the model and dimension the index records. Only changed chunks+are re-embedded and paid for. A paragraph moved within the note is+carried too; a paragraph moved into another note is re-embedded.`embedding_batch_tokens` (`OPEN_SECOND_BRAIN_EMBEDDING_BATCH_TOKENS`, sincev1.43.0) adds a per-request token budget beside `embedding_batch_size`: abatch closes on whichever cap fills first, so a run of long chunks cannotdiff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex 7385f97f..aec31bdc 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -1141,7 +1141,27 @@ Key behaviours, all driven from `Brain/_brain.yaml`-free `search_*` /(which resolves to `openai-compat` config after the built-ins). One`signature.ts` kernel canonicalises `<provider>:<model>:<dimension>`and prices it; `embedding_cost_gate_usd` refuses an over-budget run- unless `--force-cost`, and the local / unlisted-model price is 0.+ unless `--force-cost`.+- **Embedding spend (v1.72.0).** A price quote names its source:+ `builtin` (the frozen table; the local embedder is builtin and free),+ `operator` (the `embedding_price_model` / `embedding_price_usd_per_mtok`+ pair, which wins over the table for the model it names) or `unknown`.+ One spend plan owns the pending census (optionally scoped by path+ prefix), the model resolution, the token estimate, the quote and the+ gate verdict. The embedding phase, the maintenance preview, the+ backfill dry run and the `search status` refresh estimate all read it,+ so what is announced, refused, spent and receipted is one computation.+ An unknown price stays unknown (`null`, printed `price unknown`) rather+ than reading as $0, and under a positive gate it refuses with+ `EMBEDDING_COST_UNPRICED` unless forced. Price is never part of the+ embedding identity.+- **Vector carry-over (v1.72.0).** Replacing a document's chunks keeps+ the stored vector of every new chunk whose content hash matches an old+ chunk of the same document (duplicates match as a multiset), as long as+ the old row's model and dimension equal the identity `index_state`+ records; nothing is carried while that identity is unrecorded. Only the+ vectors not carried are purged, and the index run reports the count as+ `embeddingsReused`.- **Fusion modes (v0.36.0).** `search_fusion_mode` is `linear` (default,the weighted sum below) or `rrf` - Reciprocal Rank Fusion, which scoreseach candidate `Σ w_lane/(search_rrf_k + rank_in_lane)` across the@@ -1222,6 +1242,23 @@ semantic_weight·cosine + link_boost + recency_boost + entity_boost)`throws a typed `SearchError` so a misconfigured run cannot hide.The data-state case (zero embeddings) always warns and skips —running `o2b search index --embeddings` is the right answer there.+- **Semantic belief order (v1.72.0).** `brain_context_pack` with+ `query_mode: "semantic"` orders the curated belief notes by meaning+ rather than by shared words. The query is embedded once (with the+ query prefix, through the configured provider), and each belief note+ scores the maximum cosine over its stored chunk vectors; nothing new+ is embedded for the notes themselves. Only vectors written by the+ configured model at the query's dimension count, and the scores are+ read only for candidates that survive the reach filter. A note without+ a usable vector sorts after the scored notes of its tier and is named in the response's+ `semantic.unembedded`. The mode refuses by name when the semantic tier+ is blocked, when sqlite-vec is not loadable, and when no kept+ candidate has a usable vector (`BELIEF_VECTORS_MISSING`, naming+ `o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply`). The+ query embed is disclosed (model, price source, tokens, estimate); a+ remote caller's embed is refused with `EMBEDDING_COST_UNPRICED` when+ the model has no known price and `embedding_cost_gate_usd` is+ positive, while a local caller's embed is not gated, as for `search`.- **Atomic reindex.** `o2b search reindex` writes to`brain.sqlite.new`, renames to `brain.sqlite`, and keeps theprevious file as `brain.sqlite.bak`. If the main file is missing ondiff --git a/docs/mcp.md b/docs/mcp.mdindex e93ab0b6..d816b57c 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -2365,3 +2365,32 @@ format characters), when it contains NUL, or when it exceeds the cap.re-derives and archives only readable derived pages; and`brain_anticipatory_context` builds its bundle for the caller withoutreading or writing the shared cache, answering `cache_state: "miss"`.+- Since v1.72.0 `brain_context_pack` accepts `query_mode: "semantic"`,+ which orders the curated belief notes by the stored vectors of their+ chunks against one embedding of `query` (it still requires `query`).+ The response gains a `semantic` object: `model`, `price_source`,+ `query_tokens`, `estimated_usd` (null when the price is unknown),+ `scored` and `unembedded` (kept candidates with no usable vector, which+ sort after the scored ones within the same tier, then by recency;+ tier still decides first). The+ counts are taken after the reach filter, so a withheld page appears in+ neither. The mode refuses with a stable `error.data.code`: the semantic+ capability codes (`EMBEDDING_DISABLED`, `EMBEDDING_KEY_MISSING`) for a+ blocked tier, `VEC_EXTENSION_UNAVAILABLE`, and `BELIEF_VECTORS_MISSING`+ when no kept candidate has a usable vector, naming+ `o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply`.+ The query embed is one paid call per request, disclosed in the+ response, and refused with `EMBEDDING_COST_UNPRICED` for a remote+ caller when the model has no known price and `embedding_cost_gate_usd`+ is positive. `query` is capped at 2000 characters in every query mode,+ counted in Unicode code points as the schema's `maxLength` counts them,+ and `brain_search` counts its 2000-character cap the same way.+ `brain_recall_feedback`, which advertised the same cap without checking+ it, now refuses a longer `query` with `INVALID_PARAMS` before it records+ anything. The+ `substring` and `ranked` modes are otherwise unchanged. The embedding spend+ surfaces also change: `EMBEDDING_COST_UNPRICED` joins the stable error+ codes (an embedding run refused under a positive cost gate because the+ model has no known price), `brain_maintenance` spend receipts carry+ `price_source` with a null `estimated_usd` for an unknown price. No new+ tool.diff --git a/docs/updating.md b/docs/updating.mdindex dab54894..777fa15d 100644--- a/docs/updating.md+++ b/docs/updating.md@@ -47,6 +47,56 @@ instruction files such as `CLAUDE.md`/`AGENTS.md`, installed`.claude/skills/`) and warns with the exact replacement for any stalereference it finds (`removed-tool-reference`).+## Upgrading to 1.72.0++No step is required unless you set a positive `embedding_cost_gate_usd`,+left a numeric search setting blank, or an integration reads the JSON embedding cost estimates, which can now+be `null` whatever the gate (see below).++**A positive cost gate now refuses a model with no known price.**+Before this release, a model missing from the built-in price table was+priced at $0, so a gate never stopped it. Now its price is `unknown`,+and an embedding reindex, maintenance reindex or vector backfill with+pending chunks on that model is refused with `EMBEDDING_COST_UNPRICED`+while the gate is positive. This affects local servers (Ollama, LM+Studio, llama.cpp) and any model the table does not list. Run+`o2b search check`: it names the model and the keys to set. Declare the+price once to restore unattended runs:++```yaml+embedding_price_model: nomic-embed-text:latest+embedding_price_usd_per_mtok: 0+```++`0` declares the model free; use the provider's real rate otherwise.+`--force-cost` (MCP `force_cost`) passes a single run without a+declaration. With the gate at 0, the default, nothing is refused.++**Unknown prices read as unknown.** Where an estimate used to print+`$0.0000` for an unlisted model, the maintenance banner, the backfill+dry run and `search status` now print `price unknown`, and the JSON+estimates (`estimated_cost_usd`, `estimated_usd`, the status refresh+estimate) are `null`. A consumer that read `0` as free must treat `null`+as unknown. The backfill's `estimated_cost_usd` used to be omitted for an+unknown price and is now always present. Spend receipts, the+`maintenance_spend` metric, the backfill JSON and the status JSON gain+`price_source` (`refresh_price_source` on status); journal rows written+before this release list their price source as `unrecorded`.++**Edits re-embed less.** An edited note keeps the vectors of its+unchanged chunks, so the next embedding pass pays only for what changed.+Nothing needs reindexing to benefit.++`o2b search vector-backfill` gains a repeatable `--path`, and+`brain_context_pack` gains `query_mode: "semantic"`; both are additive.++**Blank numeric settings and long feedback queries are refused.** A+whitespace-only `embedding_cost_gate_usd` or `search_rerank_min_score`+(in config or through its env twin) used to read as 0; it now fails config+resolution with `INVALID_INPUT`, so remove the key or give it a number.+`brain_recall_feedback` now enforces its advertised 2000-character `query`+cap and refuses a longer query with `INVALID_PARAMS`.+## Upgrading to 1.71.0No step is required. Three changes are visible to an operator who runsdiff --git a/openclaw.plugin.json b/openclaw.plugin.jsonindex 611bb6ac..702fc487 100644--- a/openclaw.plugin.json+++ b/openclaw.plugin.json@@ -2,7 +2,7 @@"id": "open-second-brain","name": "Open Second Brain","description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",- "version": "1.71.0",+ "version": "1.72.0","activation": { "onStartup": true },"skills": ["./skills"],"contracts": {diff --git a/package.json b/package.jsonindex 1630cfd3..8fc042ae 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.71.0",+ "version": "1.72.0","private": false,"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.","keywords": [diff --git a/plugin.yaml b/plugin.yamlindex 16965859..bd1a43a9 100644--- a/plugin.yaml+++ b/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.71.0"+version: "1.72.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.jsonindex b51c373e..49263deb 100644--- a/plugins/codex/.codex-plugin/plugin.json+++ b/plugins/codex/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-second-brain",- "version": "1.71.0",+ "version": "1.72.0","description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.","author": {"name": "Open Second Brain contributors",diff --git a/plugins/codex/README.md b/plugins/codex/README.mdindex 29dd9aa7..0a57da76 100644--- a/plugins/codex/README.md+++ b/plugins/codex/README.md@@ -113,7 +113,7 @@ The full router with readiness criteria is [`install.md`](https://github.com/ite## What is new-1.71.0 stops prompt-time recall from repeating itself. With `recall_inject_enabled` on, the recall-inject hook now skips a note span it already showed in the session or that the SessionStart digest already delivered (`recall_inject_dedupe`, on by default), its note, character, time and confidence caps are config keys with env overrides, and an operator can lay the brief out as named slices declared in `Brain/_brain.yaml`. An opt-in chunked re-grounding (`reground_parts_enabled`, off by default) splits an oversized SessionStart payload into parts that the new `reground-deliver` hook hands out exactly once each on the following tool calls and prompts in Claude Code and Codex, so a post-compaction payload is no longer cut to a preview. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).+1.72.0 makes embedding spend honest. Every estimate names its price source (`builtin`, `operator` or `unknown`), `embedding_price_model` and `embedding_price_usd_per_mtok` declare the price of a model the built-in table does not list, and an unknown price is reported as `price unknown` instead of $0; under a positive `embedding_cost_gate_usd` such a model is refused with `EMBEDDING_COST_UNPRICED` until its price is declared or the run is forced. Edited notes keep the vectors of their unchanged chunks, `o2b search vector-backfill` accepts a repeatable `--path`, `o2b search check` names an unpriced model and where it looked for an embedding key, and `brain_context_pack` gains `query_mode: "semantic"`, which orders the curated belief notes by the stored vectors of their chunks. Every release is described in the [CHANGELOG](https://github.com/itechmeat/open-second-brain/blob/main/CHANGELOG.md).## Documentationdiff --git a/plugins/codex/skills/embeddings-setup/SKILL.md b/plugins/codex/skills/embeddings-setup/SKILL.mdindex b84f2d28..66772095 100644--- a/plugins/codex/skills/embeddings-setup/SKILL.md+++ b/plugins/codex/skills/embeddings-setup/SKILL.md@@ -25,7 +25,14 @@ is also present in the `--json` output for headless callers.Branch on what the report shows:-- `embedding_key: MISSING` → go to step 2.+- `embedding_key: MISSING` → go to step 2. The `key_sources_checked:`+ line names where the key was looked for, and `key_present_under:`+ names any other registered provider profile whose env key is set. If+ the key sits under another profile, the fix may be pointing+ `embedding_provider` at that profile rather than adding a key.+- A recommendation saying the embedding model has no known price, or+ that the price pair names another model → see "Declare the model's+ price" in step 2.- `vec_extension: unavailable` on macOS → go to step 3.- `vec_extension: unavailable` on Linux → go to step 4.- Everything OK but `semantic_enabled: false` (no embeddings yet) →@@ -73,6 +80,29 @@ instead. Every run that uses it prints one stderr warning per endpoint.user to paste their key in place of it. Recheck with `o2b searchcheck` after the user confirms.+### Declare the model's price++Spend estimates name their price source: `builtin` (the built-in table;+the `local` provider is free), `operator` (declared) or `unknown`. A+model the table does not list (most local servers, newer or niche+models) is `unknown`, and its estimates print `price unknown`. When the+user has set a positive `embedding_cost_gate_usd`, an unknown price+refuses embedding runs with `EMBEDDING_COST_UNPRICED` until the price is+declared. Ask the user for the provider's rate (USD per million tokens)+and write the pair beside the other embedding settings:++```bash+OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL=nomic-embed-text:latest+OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK=0+```++(or `embedding_price_model` and `embedding_price_usd_per_mtok` in the+o2b config). Set both or neither. `0` declares the model free; only use+it when the user confirms the endpoint costs nothing, because a+loopback URL can still be a paid proxy. The pair names one model, so+after a model switch `o2b search check` flags the stale declaration+until it is updated. Declaring a price never triggers a reindex.+## Step 3 — macOS: install Homebrew SQLiteApple ships `/usr/lib/libsqlite3.dylib` with@@ -121,6 +151,21 @@ This walks the vault, chunks every Markdown file, and computesembeddings. Cost scales linearly with vault size — a 150-file vaulttypically lands in a few seconds.+When the index already exists and only some chunks lack vectors,+price the work first with the dry run, then apply it:++```bash+o2b search vector-backfill+o2b search vector-backfill --apply+```++`--path <prefix>` (repeatable) limits the census, the estimate and the+spend to one part of the vault. For example,+`o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply`+embeds just the belief notes, which is what the `semantic` query mode of+`brain_context_pack` reads. Later edits re-embed only the chunks that+changed; unchanged chunks keep their vectors.+Verify semantic search works:```bashdiff --git a/plugins/hermes/_schemas.py b/plugins/hermes/_schemas.pyindex 5ba9980c..99602b1c 100644--- a/plugins/hermes/_schemas.py+++ b/plugins/hermes/_schemas.py@@ -531,18 +531,20 @@ STATIC_TOOL_SCHEMAS: tuple[dict[str, Any], ...] = ('description': 'Strict upper bound on the '"returned slice's token count."},'query': {'type': 'string',+ 'maxLength': 2000,'description': 'Optional query. Read as a ''case/Unicode-insensitive substring ''filter on topic + principle unless ''`query_mode` says otherwise.'},'query_mode': {'type': 'string',- 'enum': ['substring', 'ranked'],+ 'enum': ['substring', 'ranked', 'semantic'],'description': 'How `query` is read: '- '`substring` (default) filters, '- 'dropping misses as '- '`filter-miss`; `ranked` orders '- 'candidates by token overlap and '- 'excludes none.'},+ '`substring` (default) drops '+ 'misses; `ranked` orders by '+ 'token overlap; `semantic` '+ 'orders by stored belief '+ 'vectors, embedding the query '+ 'once.'},'focus_session': {'type': 'string','minLength': 1,'maxLength': 128,diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yamlindex 16965859..bd1a43a9 100644--- a/plugins/hermes/plugin.yaml+++ b/plugins/hermes/plugin.yaml@@ -1,5 +1,5 @@name: open-second-brain-version: "1.71.0"+version: "1.72.0"description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."author: "Open Second Brain contributors"memory_provider: truediff --git a/pyproject.toml b/pyproject.tomlindex f9332787..882d7c27 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"# CLI entry points (those moved to `package.json` `bin`).[project]name = "open-second-brain"-version = "1.71.0"+version = "1.72.0"description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."readme = "README.md"requires-python = ">=3.11"diff --git a/skills/embeddings-setup/SKILL.md b/skills/embeddings-setup/SKILL.mdindex b84f2d28..66772095 100644--- a/skills/embeddings-setup/SKILL.md+++ b/skills/embeddings-setup/SKILL.md@@ -25,7 +25,14 @@ is also present in the `--json` output for headless callers.Branch on what the report shows:-- `embedding_key: MISSING` → go to step 2.+- `embedding_key: MISSING` → go to step 2. The `key_sources_checked:`+ line names where the key was looked for, and `key_present_under:`+ names any other registered provider profile whose env key is set. If+ the key sits under another profile, the fix may be pointing+ `embedding_provider` at that profile rather than adding a key.+- A recommendation saying the embedding model has no known price, or+ that the price pair names another model → see "Declare the model's+ price" in step 2.- `vec_extension: unavailable` on macOS → go to step 3.- `vec_extension: unavailable` on Linux → go to step 4.- Everything OK but `semantic_enabled: false` (no embeddings yet) →@@ -73,6 +80,29 @@ instead. Every run that uses it prints one stderr warning per endpoint.user to paste their key in place of it. Recheck with `o2b searchcheck` after the user confirms.+### Declare the model's price++Spend estimates name their price source: `builtin` (the built-in table;+the `local` provider is free), `operator` (declared) or `unknown`. A+model the table does not list (most local servers, newer or niche+models) is `unknown`, and its estimates print `price unknown`. When the+user has set a positive `embedding_cost_gate_usd`, an unknown price+refuses embedding runs with `EMBEDDING_COST_UNPRICED` until the price is+declared. Ask the user for the provider's rate (USD per million tokens)+and write the pair beside the other embedding settings:++```bash+OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL=nomic-embed-text:latest+OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK=0+```++(or `embedding_price_model` and `embedding_price_usd_per_mtok` in the+o2b config). Set both or neither. `0` declares the model free; only use+it when the user confirms the endpoint costs nothing, because a+loopback URL can still be a paid proxy. The pair names one model, so+after a model switch `o2b search check` flags the stale declaration+until it is updated. Declaring a price never triggers a reindex.+## Step 3 — macOS: install Homebrew SQLiteApple ships `/usr/lib/libsqlite3.dylib` with@@ -121,6 +151,21 @@ This walks the vault, chunks every Markdown file, and computesembeddings. Cost scales linearly with vault size — a 150-file vaulttypically lands in a few seconds.+When the index already exists and only some chunks lack vectors,+price the work first with the dry run, then apply it:++```bash+o2b search vector-backfill+o2b search vector-backfill --apply+```++`--path <prefix>` (repeatable) limits the census, the estimate and the+spend to one part of the vault. For example,+`o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply`+embeds just the belief notes, which is what the `semantic` query mode of+`brain_context_pack` reads. Later edits re-embed only the chunks that+changed; unchanged chunks keep their vectors.+Verify semantic search works:```bashdiff --git a/src/cli/advisory-rail.ts b/src/cli/advisory-rail.tsindex 89a4668c..26fcb34e 100644--- a/src/cli/advisory-rail.ts+++ b/src/cli/advisory-rail.ts@@ -99,9 +99,23 @@ export function renderNextStepLine(step: NextStep): string {* its diagnostic yet, not a crash, and the doctor surfaces must be able to* report such an issue without inventing a command for it. The outcome* distinguishes that case from a suppressed one, so neither is silent.+ *+ * `nextCommand`, when given, replaces the registered command for a code+ * that IS registered - the registered one with the caller's own+ * arguments appended, such as a scope the run was limited to, so the+ * advice repeats the run instead of widening it. An unregistered code+ * stays unregistered: the override never invents an exit.*/-export function emitNextStep(code: string, stream: AdvisoryStream): AdvisoryEmission {- const nextStep = resolveNextStep(code);+export function emitNextStep(+ code: string,+ stream: AdvisoryStream,+ nextCommand?: string,+): AdvisoryEmission {+ const resolved = resolveNextStep(code);+ const nextStep =+ resolved !== null && nextCommand !== undefined+ ? Object.freeze({ ...resolved, nextCommand })+ : resolved;if (nextStep === null) {return Object.freeze({code,diff --git a/src/cli/brain/verbs/maintenance.ts b/src/cli/brain/verbs/maintenance.tsindex 23fe83d4..2f5978ee 100644--- a/src/cli/brain/verbs/maintenance.ts+++ b/src/cli/brain/verbs/maintenance.ts@@ -39,7 +39,12 @@ import {type LaneTaskId,type MaintenanceTaskResult,} from "../../../core/brain/maintenance/lane.ts";-import { listJournal, MAINTENANCE_JOURNAL_CAP } from "../../../core/brain/maintenance/journal.ts";+import {+ listJournal,+ MAINTENANCE_JOURNAL_CAP,+ recordedPriceSource,+ type MaintenanceSpendReceipt,+} from "../../../core/brain/maintenance/journal.ts";import {customTasksOffNotice,resolveCustomTasks,@@ -54,6 +59,11 @@ import {parseWindowBounds,renderMaintenanceCronTemplate,} from "../../maintenance-cron.ts";+import {+ formatEstimatedUsd,+ PRICE_UNKNOWN_LABEL,+ USD_DECIMALS,+} from "../../../core/search/embedding-spend.ts";import type { EmbeddingSpendPreview } from "../../../core/search/indexer.ts";import { resolveSearchConfig } from "../../../core/search/index.ts";import { onInterrupt } from "../../interrupt.ts";@@ -238,7 +248,9 @@ export async function cmdBrainMaintenance(argv: string[]): Promise<number> {// the task never ran; rendering it as FAILED would report an// attempt that did not happen.const outcome = e.ok === undefined ? "" : ` ${e.ok ? "ok" : "FAILED"}`;- ok(` ${e.ts} ${e.verdict}${e.task ? ` ${e.task}${outcome}` : ""}`);+ ok(+ ` ${e.ts} ${e.verdict}${e.task ? ` ${e.task}${outcome}` : ""}${journalReceiptSuffix(e.receipt)}`,+ );}}return MAINTENANCE_EXIT.ok;@@ -493,10 +505,29 @@ export function renderTaskLine(t: MaintenanceTaskResult): string {t.timed_out === true ? `TIMED OUT (${t.error})` : t.ok ? "ok" : `FAILED (${t.error})`;const line = `${t.name}: ${outcome} in ${t.duration_ms}ms`;if (t.receipt === undefined) return line;- return (- `${line} (tokens=${t.receipt.tokens}, ` +- `estimatedUsd=${t.receipt.estimatedUsd.toFixed(4)}, model=${t.receipt.model ?? "unknown"})`- );+ return `${line} (${receiptFields(t.receipt)})`;+}++/** A spend receipt's tokens, estimate and model, as the task line prints them. */+function receiptFields(receipt: MaintenanceSpendReceipt): string {+ // The journal read casts its rows unvalidated, so a row from another+ // build or a hand edit may carry no estimate or a non-number one; that+ // reads as an unknown price instead of breaking the whole listing.+ const estimate =+ typeof receipt.estimatedUsd === "number" && Number.isFinite(receipt.estimatedUsd)+ ? `estimatedUsd=${receipt.estimatedUsd.toFixed(USD_DECIMALS)}`+ : PRICE_UNKNOWN_LABEL;+ return `tokens=${receipt.tokens}, ${estimate}, model=${receipt.model ?? "unknown"}`;+}++/**+ * A journaled receipt for the status listing: the task line's fields plus+ * the recorded price source, so a row written before sources existed+ * reads as unrecorded instead of as a known price.+ */+function journalReceiptSuffix(receipt: MaintenanceSpendReceipt | undefined): string {+ if (receipt === undefined) return "";+ return ` (${receiptFields(receipt)}, price_source=${recordedPriceSource(receipt)})`;}/**@@ -508,10 +539,14 @@ export function renderTaskLine(t: MaintenanceTaskResult): string {* the refusal that can follow it print one spelling.*/export function formatSpendBanner(preview: EmbeddingSpendPreview, gateUsd: number): string {- const gate = gateUsd > 0 ? `$${gateUsd.toFixed(4)}` : "off";+ const gate = gateUsd > 0 ? formatEstimatedUsd(gateUsd) : "off";+ const estimate =+ preview.estimatedUsd === null+ ? PRICE_UNKNOWN_LABEL+ : `estimated ${formatEstimatedUsd(preview.estimatedUsd)}`;return (`embedding spend: model ${preview.model ?? "unknown"}, ${preview.pendingChunks} chunks pending, ` +- `estimated $${preview.estimatedUsd.toFixed(4)} (gate: ${gate})`+ `${estimate} (gate: ${gate})`);}diff --git a/src/cli/command-manifest.ts b/src/cli/command-manifest.tsindex 18c75ba4..7b8deedf 100644--- a/src/cli/command-manifest.ts+++ b/src/cli/command-manifest.ts@@ -893,6 +893,7 @@ export const CLI_COMMAND_MANIFEST: CliRootManifest = Object.freeze({flag("db", "string"),flag("apply", "boolean"),flag("force-cost", "boolean"),+ flag("path", "string-array"),flag("progress", "boolean"),],),diff --git a/src/cli/helpers.ts b/src/cli/helpers.tsindex 11111c2e..15bc5bdc 100644--- a/src/cli/helpers.ts+++ b/src/cli/helpers.ts@@ -3,6 +3,10 @@*/import { resolveVault } from "../core/config.ts";+import {+ EMBEDDING_KEY_CONFIG,+ EMBEDDING_KEY_ENV,+} from "../core/search/embeddings/credential-report.ts";export const NO_VAULT_ERROR ="error: no vault configured. Pass --vault <path> explicitly, " +@@ -102,8 +106,8 @@ export function resolveSemanticConfigState(// `discoverConfig().data` is typed `Record<string, string>`, but// this helper accepts the wider `unknown`-keyed map for forward// compat with future callers; coerce defensively without throwing.- const cfgKey = configData["embedding_api_key"];- const envKey = env["OPEN_SECOND_BRAIN_EMBEDDING_KEY"];+ const cfgKey = configData[EMBEDDING_KEY_CONFIG];+ const envKey = env[EMBEDDING_KEY_ENV];const keyPresent =(typeof cfgKey === "string" && cfgKey.trim().length > 0) ||(typeof envKey === "string" && envKey.trim().length > 0);diff --git a/src/cli/search/helpers.ts b/src/cli/search/helpers.tsindex 33ed43ab..7cf30a28 100644--- a/src/cli/search/helpers.ts+++ b/src/cli/search/helpers.ts@@ -36,6 +36,12 @@ export function flagString(flags: SearchVerbFlags, name: string): string | undefreturn typeof value === "string" ? value : undefined;}+/** Every value of a repeatable (`string-array`) flag, in argv order; empty when absent. */+export function flagStrings(flags: SearchVerbFlags, name: string): ReadonlyArray<string> {+ const value = flags[name];+ return Array.isArray(value) ? value : [];+}+/** True when the flag was passed; every boolean flag is opt-in. */export function flagBoolean(flags: SearchVerbFlags, name: string): boolean {return flags[name] === true;diff --git a/src/cli/search/verbs/check.ts b/src/cli/search/verbs/check.tsindex 80b7608d..eb0ccc70 100644--- a/src/cli/search/verbs/check.ts+++ b/src/cli/search/verbs/check.ts@@ -77,12 +77,14 @@ import { nextCommandField } from "../../../core/brain/next-step.ts";import { formatStampMismatch } from "../../../core/integrity/stamp.ts";import {indexCheck,+ resolveCredentialContext,serializeEmbedderRecordCensus,serializePendingVectorCensus,serializeStampMismatches,serializeVisibilityHonestyFinding,} from "../../../core/search/index.ts";import type {+ CredentialSourceReport,EmbedderRecordCensus,IndexCheckReport,PendingVectorCensus,@@ -105,6 +107,7 @@ import {flagBoolean,parseFlags,resolveConfig,+ resolveConfigPath,searchAdvisoryStream,VAULT_FLAGS,} from "../helpers.ts";@@ -389,6 +392,10 @@ function ok(passed: boolean): string {return passed ? "OK" : "MISSING";}+function serializeCredentialSources(report: CredentialSourceReport): Record<string, unknown> {+ return { consulted: report.consulted, present_elsewhere: report.presentElsewhere };+}+function jsonForCheck(r: IndexCheckReport): Record<string, unknown> {return {vault_readable: r.vaultReadable,@@ -422,6 +429,11 @@ function jsonForCheck(r: IndexCheckReport): Record<string, unknown> {...(r.embeddingAbi.length > 0? { embedding_abi: serializeStampMismatches(r.embeddingAbi) }: {}),+ // Only on a credential-missing tier, so a configured setup's JSON is+ // byte-identical. Names only: no env value is ever part of it.+ ...(r.credentialSources !== undefined+ ? { credential_sources: serializeCredentialSources(r.credentialSources) }+ : {}),warnings: r.warnings,fatal: r.fatal,recommendations: r.recommendations,@@ -530,6 +542,13 @@ function renderCheckHuman(r: IndexCheckReport): string {lines.push(`visibility_honesty: ${describeVisibilityHonesty(r.visibilityHonesty)}`);}for (const w of r.warnings) lines.push(`warning: ${w}`);+ // Under the keyless warning, and only on a credential-missing tier.+ if (r.credentialSources !== undefined) {+ const { consulted, presentElsewhere } = r.credentialSources;+ lines.push(`key_sources_checked: ${consulted.join(", ")}`);+ const holders = presentElsewhere.length > 0 ? presentElsewhere.join(", ") : "none";+ lines.push(`key_present_under: ${holders}`);+ }for (const f of r.fatal) lines.push(`fatal: ${f}`);if (r.recommendations.length > 0) {lines.push("");@@ -595,7 +614,15 @@ export async function cmdSearchCheck(argv: ReadonlyArray<string>): Promise<numbe// worse surprise than one that still does. `--no-probe` is for the// caller who cannot spend a network round-trip - an air-gapped machine,// a tight CI loop - and it reports `skipped` rather than a verdict.- const report = await indexCheck(cfg, { probeProvider: !flagBoolean(flags, "no-probe") });+ // The credential-source report reads names from this env and the+ // vault's registry, handed over explicitly; it never reads a value out.+ const report = await indexCheck(cfg, {+ probeProvider: !flagBoolean(flags, "no-probe"),+ credentialContext: resolveCredentialContext({+ vault: cfg.vault,+ configPath: resolveConfigPath(flags),+ }),+ });// Absent the flag nothing below runs, nothing is written, and the two// report shapes are byte-identical to what they were before it existed.const integrity = flagBoolean(flags, "integrity") ? await scanIntegrity(cfg) : null;diff --git a/src/cli/search/verbs/indexing.ts b/src/cli/search/verbs/indexing.tsindex 61eacc54..6598fbc6 100644--- a/src/cli/search/verbs/indexing.ts+++ b/src/cli/search/verbs/indexing.ts@@ -423,6 +423,9 @@ function jsonForStats(stats: IndexStats, cfg: ResolvedSearchConfig): unknown {chunks_total: stats.chunksTotal,embeddings_computed: stats.embeddingsComputed,embeddings_retries: stats.embeddingsRetries,+ // Conditional, like the field below: a run that carried no vector+ // over emits exactly the payload it emitted before this field existed.+ ...(stats.embeddingsReused > 0 ? { embeddings_reused: stats.embeddingsReused } : {}),// Conditional: a vault with nothing pending emits exactly the// payload it emitted before this field existed....(stats.eventAnchorsPending > 0@@ -463,9 +466,10 @@ function renderStatsHuman(stats: IndexStats, cfg: ResolvedSearchConfig): stringlines.push(` updated: ${stats.updated} files`);lines.push(` unchanged: ${stats.unchanged} files`);lines.push(` deleted: ${stats.deleted} files`);- if (stats.embeddingsComputed > 0 || stats.embeddingsRetries > 0) {+ if (stats.embeddingsComputed > 0 || stats.embeddingsRetries > 0 || stats.embeddingsReused > 0) {+ const reused = stats.embeddingsReused > 0 ? `, ${stats.embeddingsReused} reused` : "";lines.push(- ` embeddings: ${stats.embeddingsComputed} computed (${stats.embeddingsRetries} retries)`,+ ` embeddings: ${stats.embeddingsComputed} computed (${stats.embeddingsRetries} retries)${reused}`,);}// Above the errors block: these documents indexed fine, under adiff --git a/src/cli/search/verbs/status.ts b/src/cli/search/verbs/status.tsindex a44dddfc..b977662f 100644--- a/src/cli/search/verbs/status.ts+++ b/src/cli/search/verbs/status.ts@@ -3,6 +3,7 @@* indexer named as the exit when there is no index at all.*/+import { formatEstimatedUsd } from "../../../core/search/embedding-spend.ts";import {eventTimeStatus,indexStatus,@@ -68,8 +69,8 @@ function renderStatusHuman(s: IndexStatusSnapshot, eventTime: EventTimeStatus |lines.push(`embedding_model: ${s.embeddingModel ?? "(none)"}`);lines.push(`embedding_dimension: ${s.embeddingDimension ?? "(none)"}`);lines.push(`embedding_signature: ${s.embeddingSignature ?? "(disabled)"}`);- if (s.estimatedRefreshCostUsd > 0) {- lines.push(`refresh_cost_est: $${s.estimatedRefreshCostUsd.toFixed(4)}`);+ if (s.estimatedRefreshCostUsd === null || s.estimatedRefreshCostUsd > 0) {+ lines.push(`refresh_cost_est: ${formatEstimatedUsd(s.estimatedRefreshCostUsd)}`);}lines.push(`vec_extension: ${s.vecExtension}`);lines.push(`semantic_enabled: ${s.semanticEnabled}`);diff --git a/src/cli/search/verbs/vector-backfill.ts b/src/cli/search/verbs/vector-backfill.tsindex 883f66ec..b0957aaf 100644--- a/src/cli/search/verbs/vector-backfill.ts+++ b/src/cli/search/verbs/vector-backfill.ts@@ -41,8 +41,19 @@* the embedding phase is the part an operator would ever want to stop.*/+import {+ COST_GATE_KEY,+ EMBEDDING_GATE_REASON,+ FORCE_COST_FLAG,+ formatEstimatedUsd,+ type EmbeddingGateReason,+} from "../../../core/search/embedding-spend.ts";+import {+ EMBEDDING_PRICE_MODEL_KEY,+ EMBEDDING_PRICE_RATE_KEY,+} from "../../../core/search/embeddings/pricing.ts";import { appendLogEvent } from "../../../core/brain/log.ts";-import { nextCommandField } from "../../../core/brain/next-step.ts";+import { NEXT_COMMAND_KEY, resolveNextStep } from "../../../core/brain/next-step.ts";import {createSafeguard,OPERATION,@@ -61,13 +72,15 @@ import {planVectorBackfill,type VectorBackfillResult,} from "../../../core/search/vector-backfill.ts";-import { emitNextStep } from "../../advisory-rail.ts";+import { advisoryIsLegal, emitNextStep, type AdvisoryStream } from "../../advisory-rail.ts";+import { shellQuote } from "../../cron-recipe.ts";import { onInterrupt, reportInterrupted } from "../../interrupt.ts";import { info, ok } from "../../output.ts";import { attachProgress, reportProgressRefusal } from "../../progress-rail.ts";import {flagBoolean,flagString,+ flagStrings,parseFlags,resolveConfig,resolveConfigPath,@@ -82,6 +95,97 @@ import {*/const VECTORS_PENDING = SEMANTIC_VECTOR_CODE.pending;+/** The repeatable flag that limits the run to a vault-relative path prefix. */+const PATH_FLAG = "path";++/** A prefix that needs no shell quoting when echoed in a command. */+const PLAIN_SHELL_WORD = /^[A-Za-z0-9_./-]+$/u;++/**+ * `command` with one `--path <prefix>` per prefix, quoted only when+ * needed. Null on Windows when a prefix needs quoting: the quoting is+ * POSIX, which neither cmd.exe nor PowerShell reads back as the same+ * word, so the caller prints {@link RERUN_WITH_SCOPE_LINE}, which names+ * no command, instead of advising one that would not run as printed or+ * would run unscoped.+ */+export function scopedNextCommand(+ command: string,+ pathPrefixes: ReadonlyArray<string>,+ platform: NodeJS.Platform = process.platform,+): string | null {+ const plain = (prefix: string) => PLAIN_SHELL_WORD.test(prefix);+ if (platform === "win32" && !pathPrefixes.every(plain)) return null;+ return [+ command,+ ...pathPrefixes.map(+ (prefix) => `--${PATH_FLAG} ${plain(prefix) ? prefix : shellQuote(prefix)}`,+ ),+ ].join(" ");+}++/**+ * The line that replaces a scoped next step Windows cannot run as+ * printed. Deliberately not a command: a scope-free `--apply` copied as+ * printed would widen the spend to the whole vault.+ */+export const RERUN_WITH_SCOPE_LINE = `next: rerun this command with --apply and the same --${PATH_FLAG} flags`;++/** The next step a backfill run advises, resolved before anything prints. */+export interface BackfillNextStep {+ /** The registered exit the advice names. */+ readonly exitCode: string;+ /**+ * The runnable command, also the `next_command` JSON value. Absent+ * when the code has no command or when the scope cannot be quoted on+ * this platform.+ */+ readonly command: string | undefined;+ /** True when a scope was dropped because the platform cannot quote it. */+ readonly scopeDropped: boolean;+}++/**+ * Resolve the advice for `result`. A scoped run repeats its scope: the+ * registered `--apply` alone would widen the spend to the whole vault.+ * The command stays UNFORCED even when the gate would refuse it: the+ * gate is the operator's decision, so `--force-cost` is named only in+ * the human remedy line, never in a command a machine reader runs.+ */+export function backfillNextStep(+ result: Pick<VectorBackfillResult, "capability" | "pathPrefixes">,+ platform: NodeJS.Platform = process.platform,+): BackfillNextStep {+ // WHICH exit depends on what is in the operator's way. Naming+ // `--apply` while the credential is missing would advise a command that+ // cannot succeed, so a blocked tier names the tier's own exit instead.+ const exitCode = semanticCapabilityIsBlocked(result.capability)+ ? result.capability.code+ : VECTORS_PENDING;+ const registered = resolveNextStep(exitCode)?.nextCommand;+ if (registered === undefined || exitCode !== VECTORS_PENDING) {+ return { exitCode, command: registered, scopeDropped: false };+ }+ const scoped = scopedNextCommand(registered, result.pathPrefixes, platform);+ return scoped === null+ ? { exitCode, command: undefined, scopeDropped: true }+ : { exitCode, command: scoped, scopeDropped: false };+}++/**+ * Print `advice` on a human stream: the command, or, for a dropped+ * scope, the one rerun line that names no runnable command.+ */+export function emitBackfillNextStep(advice: BackfillNextStep, stream: AdvisoryStream): void {+ if (!advice.scopeDropped) {+ emitNextStep(advice.exitCode, stream, advice.command);+ return;+ }+ if (resolveNextStep(advice.exitCode) !== null && advisoryIsLegal(stream)) {+ info(RERUN_WITH_SCOPE_LINE);+ }+}+/** JSON payload shape. Snake case, matching every other search verb. */function jsonForResult(result: VectorBackfillResult): Record<string, unknown> {return {@@ -92,12 +196,39 @@ function jsonForResult(result: VectorBackfillResult): Record<string, unknown> {pending: result.pending,embedded: result.embedded,retries: result.retries,- // Absent rather than zero when the model carries no known price: a+ // Null rather than zero when the model carries no known price: a// missing price is not a free run.- ...(result.costKnown ? { estimated_cost_usd: result.estimatedCostUsd } : {}),+ estimated_cost_usd: result.estimatedCostUsd,+ price_source: result.priceSource,+ // Present only for a scoped run, so an unscoped payload is unchanged.+ ...(result.pathPrefixes.length > 0 ? { path_prefixes: result.pathPrefixes } : {}),+ ...(result.unmatchedPathPrefixes.length > 0+ ? { unmatched_path_prefixes: result.unmatchedPathPrefixes }+ : {}),+ // Present only when the unforced gate would refuse, and only for a+ // run that reached the provider, so an unblocked payload is unchanged.+ ...(result.blocked ? { gate_blocked: true, gate_reason: result.reason } : {}),+ ...(result.spend !== null+ ? {+ spend: {+ model: result.spend.model,+ tokens: result.spend.tokens,+ estimated_usd: result.spend.estimatedUsd,+ price_source: result.spend.priceSource,+ forced: result.spend.forced,+ },+ }+ : {}),};}+/** What lifts a gate refusal besides `--force-cost`, per reason. */+function gateRemedy(reason: EmbeddingGateReason | null): string {+ return reason === EMBEDDING_GATE_REASON.unpriced+ ? `set ${EMBEDDING_PRICE_MODEL_KEY} and ${EMBEDDING_PRICE_RATE_KEY}`+ : `raise ${COST_GATE_KEY}`;+}+async function renderHuman(result: VectorBackfillResult): Promise<void> {if (result.applied) {ok(`vector-backfill: wrote ${result.embedded} of ${result.pending} pending vector(s)`);@@ -106,8 +237,12 @@ async function renderHuman(result: VectorBackfillResult): Promise<void> {`vector-backfill dry-run: ${result.pending} of ${result.chunksTotal} chunk(s) have no vector`,);}- if (result.costKnown && result.pending > 0) {- info(` estimated cost: $${result.estimatedCostUsd.toFixed(4)}`);+ if (result.pathPrefixes.length > 0) info(` scope: ${result.pathPrefixes.join(", ")}`);+ if (result.pending > 0) info(` estimated cost: ${formatEstimatedUsd(result.estimatedCostUsd)}`);+ if (!result.applied && result.blocked) {+ info(+ ` cost gate: would refuse (${result.reason}); add ${FORCE_COST_FLAG} or ${gateRemedy(result.reason)}`,+ );}if (result.retries > 0) info(` provider retries: ${result.retries}`);// What the operator CONFIGURED, resolved from the registry - never a@@ -122,6 +257,8 @@ export async function cmdSearchVectorBackfill(argv: ReadonlyArray<string>): Prom...VAULT_FLAGS,apply: { type: "boolean" },"force-cost": { type: "boolean" },+ // Spelled out, not `[PATH_FLAG]`: the flag census reads literal keys.+ path: { type: "string-array" },progress: { type: "boolean" },json: { type: "boolean" },});@@ -145,6 +282,7 @@ export async function cmdSearchVectorBackfill(argv: ReadonlyArray<string>): Promresult = await planVectorBackfill(cfg, {apply,forceCost: flagBoolean(flags, "force-cost"),+ pathPrefixes: flagStrings(flags, PATH_FLAG),// The three seams this module declared and nothing produced until// now. The deadline is the `reindex` budget off the same ladder the// builders read, because this pass IS that run's embedding phase on@@ -182,27 +320,26 @@ export async function cmdSearchVectorBackfill(argv: ReadonlyArray<string>): Prom}}- // Work remains exactly when the run did not consume it.+ const advice = backfillNextStep(result);const pendingAfter = result.pending - result.embedded;- // WHICH exit, though, depends on what is in the operator's way. Naming- // `--apply` while the credential is missing would advise a command that- // cannot succeed, so a blocked tier names the tier's own exit instead.- const exitCode = semanticCapabilityIsBlocked(result.capability)- ? result.capability.code- : VECTORS_PENDING;+ // A typo'd scope has nothing pending and would otherwise read like a+ // fully embedded one; stderr, so the JSON on stdout stays parseable.+ for (const prefix of result.unmatchedPathPrefixes) {+ process.stderr.write(`warning: scope ${prefix} matches no indexed document\n`);+ }if (jsonRequested) {process.stdout.write(JSON.stringify({...jsonForResult(result),- ...(pendingAfter > 0 ? nextCommandField(exitCode) : {}),+ ...(pendingAfter > 0 && advice.command !== undefined+ ? { [NEXT_COMMAND_KEY]: advice.command }+ : {}),}) + "\n",);} else {await renderHuman(result);}- if (pendingAfter > 0) {- emitNextStep(exitCode, searchAdvisoryStream(argv, jsonRequested));- }+ if (pendingAfter > 0) emitBackfillNextStep(advice, searchAdvisoryStream(argv, jsonRequested));return 0;}diff --git a/src/core/brain/belief-semantic.ts b/src/core/brain/belief-semantic.tsnew file mode 100644index 00000000..a894eaa7--- /dev/null+++ b/src/core/brain/belief-semantic.ts@@ -0,0 +1,296 @@+/**+ * Belief vector scorer and loader (Honest Embedding Spend): the read side+ * of `brain_context_pack` `semantic` mode.+ *+ * Belief notes (`Brain/preferences/`, `Brain/retired/`) are ordinary+ * indexed documents, so their chunks already carry vectors after any+ * embedding pass. This module reads those stored vectors directly and+ * scores them by exact cosine against one query vector. It runs no KNN:+ * a prefix-filtered `semanticTopK` asks sqlite-vec for the nearest chunks+ * of the WHOLE vault and drops the rest afterwards, which under-returns+ * beliefs in a large vault, while the belief set is small enough to score+ * outright.+ *+ * Two halves:+ * - {@link scoreBeliefsByVector} is pure. A note's score is the maximum+ * cosine over its usable chunk vectors (the frontmatter chunk and the+ * body chunk both carry the principle, so a sum would double-count).+ * A row recorded under another model or dimension is not comparable+ * with the query and counts as unembedded, never as scored.+ * - {@link loadBeliefSemanticRelevance} opens the index, refuses a+ * blocked capability tier and a missing sqlite-vec by name BEFORE it+ * embeds anything, embeds the query exactly once with the `query`+ * prefix kind, and discloses that one spend: the model, the price+ * source, the query tokens and the cost (null when the price is+ * unknown). A remote caller's embed of an unpriced model is refused+ * under a positive cost gate, before the call.+ *+ * No belief row under the model anywhere is refused here, before the+ * embed, with `BELIEF_VECTORS_MISSING`: none in reach can have one. Any+ * other shortfall is the pack's to judge, because whether the caller's+ * KEPT candidates have vectors is only known after its reach filter.+ */++import { BRAIN_PREFERENCES_REL, BRAIN_RETIRED_REL } from "./path-constants.ts";+import {+ BLOCKED_TIER_ERROR_CODE,+ isBlockedCapability,+ resolveSemanticCapability,+ semanticCapabilityLabel,+} from "../search/capability-tier.ts";+import type { EmbeddingProvider } from "../search/embeddings/contract.ts";+import { activeSpendQuote, COST_GATE_KEY, formatEstimatedUsd } from "../search/embedding-spend.ts";+import { EMBEDDING_PRICE_SOURCE, type EmbeddingPriceSource } from "../search/embeddings/pricing.ts";+import { makeProvider } from "../search/embeddings/provider.ts";+import { estimateCostUsd, estimateTokens } from "../search/embeddings/signature.ts";+import { SearchError } from "../search/search-error.ts";+import { isRemotelyReadable } from "../graph/visibility.ts";+import { TRANSPORT_REACH, type TransportReach } from "../graph/transport-reach.ts";+import { contradictedAbiFields, formatEmbeddingAbiDrift, Store } from "../search/store.ts";+import type { ResolvedSearchConfig } from "../search/types.ts";+import { assertValidVector } from "../search/vector-guard.ts";++/** Vault-relative directory prefixes whose notes are scored as beliefs. */+export const BELIEF_SEMANTIC_PATH_PREFIXES: ReadonlyArray<string> = Object.freeze([+ `${BRAIN_PREFERENCES_REL}/`,+ `${BRAIN_RETIRED_REL}/`,+]);++/** The command that pays for the vectors the `semantic` mode reads. */+export const BELIEF_VECTORS_BACKFILL_COMMAND = `o2b search vector-backfill ${BELIEF_SEMANTIC_PATH_PREFIXES.map((prefix) => `--path ${prefix}`).join(" ")} --apply`;++/** One stored vector with the identity its `embeddings` row recorded. */+export interface StoredBeliefVector {+ readonly vector: Float32Array;+ readonly model: string;+ readonly dimension: number;+}++export interface BeliefVectorScoreInput {+ /** The model the query vector was produced by. */+ readonly model: string;+ readonly queryVector: ReadonlyArray<number> | Float32Array;+ /** Every belief note's stored vectors, keyed by vault-relative path. */+ readonly vectorsByPath: ReadonlyMap<string, ReadonlyArray<StoredBeliefVector>>;+}++export interface BeliefVectorScores {+ /** Maximum cosine per path, only for paths with a usable vector. */+ readonly relevanceByPath: ReadonlyMap<string, number>;+ /** Scored paths, highest relevance first, ties by path. */+ readonly order: ReadonlyArray<string>;+ /** Paths with no usable vector, sorted. */+ readonly unembedded: ReadonlyArray<string>;+ /** How many paths were scored. */+ readonly scored: number;+}++/** The one ungated spend of the semantic mode, disclosed rather than hidden. */+export interface BeliefSemanticQueryReport {+ readonly model: string;+ readonly priceSource: EmbeddingPriceSource;+ readonly queryTokens: number;+ /** Null when nobody stated the model's price. */+ readonly estimatedUsd: number | null;+}++export interface BeliefSemanticRelevance extends BeliefVectorScores {+ readonly report: BeliefSemanticQueryReport;+ /** The search lane's embedding-ABI drift warning when the index contradicts this build. */+ readonly warnings: ReadonlyArray<string>;+}++export interface BeliefSemanticDeps {+ /** The query embedder; the configured provider when omitted. */+ readonly provider?: EmbeddingProvider;+ /** Whether the store loads sqlite-vec; omitted loads it. */+ readonly loadVec?: boolean;+ /**+ * The caller's transport reach; local when omitted. Below local reach a+ * belief whose INDEXED visibility is not readable there loses its+ * vectors: they were captured from bytes reserved at index time, and a+ * live file rewritten since must not be ordered by them.+ */+ readonly reach?: TransportReach;+ /**+ * Whether the caller may be shown a belief, by vault-relative path; every+ * belief when omitted. A belief it rejects is left out of the read, so the+ * pre-embed refusal below answers for it exactly as for an absent page:+ * the pack drops it after its own reach and owner filters, and a refusal+ * decided over a wider set would tell the two apart.+ */+ readonly inView?: (path: string) => boolean;+}++/**+ * A refusal raised after the query embed still names that spend: the+ * provider answered, so the call was paid whatever is refused after it.+ * Shared by the loader (a query vector the guard rejects) and the pack (a+ * refusal decided over the beliefs it keeps), so both read the same.+ */+export function discloseSpentQuery(e: SearchError, spent: BeliefSemanticQueryReport): SearchError {+ return new SearchError(+ e.code,+ `${e.message} (the query embed was still spent: model ${spent.model}, ` ++ `price source ${spent.priceSource}, ${spent.queryTokens} query token(s), ` ++ `${formatEstimatedUsd(spent.estimatedUsd)})`,+ );+}++/** The context the query vector is validated under, named in a refusal. */+const QUERY_VECTOR_CONTEXT = "belief semantic query";++function cosine(a: ReadonlyArray<number> | Float32Array, b: Float32Array): number {+ let dot = 0;+ let normA = 0;+ let normB = 0;+ for (let i = 0; i < a.length; i++) {+ const x = a[i] as number;+ const y = b[i] as number;+ dot += x * y;+ normA += x * x;+ normB += y * y;+ }+ return dot / (Math.sqrt(normA) * Math.sqrt(normB));+}++function byPath(a: string, b: string): number {+ return a < b ? -1 : a > b ? 1 : 0;+}++/** Score belief notes by the best cosine of their usable stored vectors. */+export function scoreBeliefsByVector(input: BeliefVectorScoreInput): BeliefVectorScores {+ const dimension = input.queryVector.length;+ const relevanceByPath = new Map<string, number>();+ const unembedded: string[] = [];+ for (const [path, rows] of input.vectorsByPath) {+ let best: number | null = null;+ for (const row of rows) {+ if (row.model !== input.model) continue;+ if (row.dimension !== dimension || row.vector.length !== dimension) continue;+ const score = cosine(input.queryVector, row.vector);+ // A zero or non-finite row from an index written before the store+ // guard has no direction: unusable, never a NaN in the order.+ if (!Number.isFinite(score)) continue;+ if (best === null || score > best) best = score;+ }+ if (best === null) unembedded.push(path);+ else relevanceByPath.set(path, best);+ }+ const order = [...relevanceByPath.keys()].toSorted(+ (a, b) => relevanceByPath.get(b)! - relevanceByPath.get(a)! || byPath(a, b),+ );+ return {+ relevanceByPath,+ order: Object.freeze(order),+ unembedded: Object.freeze(unembedded.toSorted(byPath)),+ scored: relevanceByPath.size,+ };+}++function isBeliefPath(path: string): boolean {+ return BELIEF_SEMANTIC_PATH_PREFIXES.some((prefix) => path.startsWith(prefix));+}++/**+ * Embed `query` once and score every indexed belief note by its stored+ * vectors. Throws a named {@link SearchError} for a blocked tier+ * (`EMBEDDING_DISABLED`, `EMBEDDING_KEY_MISSING`) and for a store without+ * sqlite-vec (`VEC_EXTENSION_UNAVAILABLE`), each before any embed call.+ */+export async function loadBeliefSemanticRelevance(+ config: ResolvedSearchConfig,+ query: string,+ deps: BeliefSemanticDeps = {},+): Promise<BeliefSemanticRelevance> {+ const capability = resolveSemanticCapability(config.semantic);+ if (isBlockedCapability(capability)) {+ throw new SearchError(+ BLOCKED_TIER_ERROR_CODE[capability.tier],+ await semanticCapabilityLabel(capability.code),+ );+ }+ const store = await Store.open(config, {+ mode: "read",+ ...(deps.loadVec !== undefined ? { loadVec: deps.loadVec } : {}),+ });+ try {+ if (!store.vecLoaded()) {+ throw new SearchError(+ "VEC_EXTENSION_UNAVAILABLE",+ "semantic belief order unavailable: sqlite-vec extension not loaded",+ );+ }+ const reach = deps.reach ?? TRANSPORT_REACH.local;+ const inView = deps.inView ?? (() => true);+ const beliefDocs = [...store.listDocuments()].filter(+ ([path]) => isBeliefPath(path) && inView(path),+ );+ const indexed =+ reach === TRANSPORT_REACH.local+ ? new Map<string, ReadonlyArray<string>>()+ : store.indexedVisibilityByPaths(beliefDocs.map(([path]) => path));+ const vectorsByPath = new Map<string, ReadonlyArray<StoredBeliefVector>>();+ for (const [path, doc] of beliefDocs) {+ // Reads as unembedded, exactly as a page whose reserved bytes were never indexed.+ const readable = isRemotelyReadable(indexed.get(path) ?? [], reach);+ vectorsByPath.set(path, readable ? store.storedEmbeddingsForDocument(doc.id) : []);+ }+ const provider = deps.provider ?? makeProvider(config.semantic);+ const model = config.semantic.model ?? provider.model;+ // No belief row under the model anywhere means none in reach has one:+ // the pack's refusal is certain, so it is raised before the paid embed.+ // The dimension needs the query vector and is judged after it.+ const anyUsable = [...vectorsByPath.values()].some((rows) =>+ rows.some((r) => r.model === model),+ );+ if (!anyUsable) {+ throw new SearchError(+ "BELIEF_VECTORS_MISSING",+ `semantic belief order unavailable: no belief note has a stored vector ` ++ `for the configured model; run: ${BELIEF_VECTORS_BACKFILL_COMMAND}`,+ );+ }+ // Priced through the resolution every spend surface shares; `model`+ // above stays the row filter because it matches the indexer's stamp.+ const spend = activeSpendQuote(config);+ // A remote caller spends the operator's money on every request. Under+ // a positive gate an unpriced model is the spend the gate exists to+ // stop, so it is refused before the call; a local caller keeps the+ // disclosed, ungated embed, as for `search`.+ if (+ reach !== TRANSPORT_REACH.local &&+ config.semantic.costGateUsd > 0 &&+ spend.quote.source === EMBEDDING_PRICE_SOURCE.unknown+ ) {+ throw new SearchError(+ "EMBEDDING_COST_UNPRICED",+ "semantic belief order refused: the embedding model has no known price " ++ `and ${COST_GATE_KEY} is positive`,+ );+ }+ const [queryVector = []] = await provider.embed([query], "query");+ const queryTokens = estimateTokens([query]);+ const report: BeliefSemanticQueryReport = {+ model: spend.model ?? model,+ priceSource: spend.quote.source,+ queryTokens,+ estimatedUsd: estimateCostUsd(queryTokens, spend.quote),+ };+ try {+ assertValidVector(queryVector, QUERY_VECTOR_CONTEXT);+ } catch (e) {+ throw e instanceof SearchError ? discloseSpentQuery(e, report) : e;+ }+ const contradicted = contradictedAbiFields(store.embeddingAbiMismatches());+ return {+ ...scoreBeliefsByVector({ model, queryVector, vectorsByPath }),+ report,+ warnings: Object.freeze(+ contradicted.length > 0 ? [formatEmbeddingAbiDrift(contradicted)] : [],+ ),+ };+ } finally {+ await store.close();+ }+}diff --git a/src/core/brain/context-pack.ts b/src/core/brain/context-pack.tsindex 469b9555..6cc4b1b5 100644--- a/src/core/brain/context-pack.ts+++ b/src/core/brain/context-pack.ts@@ -24,6 +24,8 @@ import {type ContextSafetyReport,} from "./safety/context-guard.ts";import { brainDirs } from "./paths.ts";+import { BELIEF_VECTORS_BACKFILL_COMMAND } from "./belief-semantic.ts";+import { SearchError } from "../search/search-error.ts";import {collectPreferencePages,formatOwnerScopeWarning,@@ -99,8 +101,15 @@ const TIER_ORDER: ReadonlyArray<PageTier> = [* the budget - not a lexical accident - decides what is dropped, and the* curated pool, guard, tip preference, owner scope, and receipt the lane* exists for all still apply.+ *+ * `semantic` (Honest Embedding Spend) orders the same way, by meaning+ * instead of token overlap: the caller passes+ * {@link ContextPackOptions.semanticRelevance}, computed at the async+ * boundary from the stored vectors of the belief notes, and the pack reads+ * it in the slot `ranked` uses. The pack itself never embeds, so it stays+ * synchronous for every other caller.*/-export const CONTEXT_PACK_QUERY_MODES = ["substring", "ranked"] as const;+export const CONTEXT_PACK_QUERY_MODES = ["substring", "ranked", "semantic"] as const;export type ContextPackQueryMode = (typeof CONTEXT_PACK_QUERY_MODES)[number];@@ -110,6 +119,27 @@ export function isContextPackQueryMode(value: unknown): value is ContextPackQuer);}+export { BELIEF_VECTORS_BACKFILL_COMMAND };++/**+ * The relevance an unembedded candidate reads in `semantic` mode: below+ * any cosine, so it sorts after every scored candidate of its tier,+ * negative cosines included.+ */+const UNEMBEDDED_RELEVANCE = -2;++/**+ * A `semantic` pack was asked for without the relevance map that mode+ * reads. Packing by recency instead would answer a meaning query with an+ * order that ignores the query, so the call fails by name.+ */+export class ContextPackSemanticRelevanceMissingError extends Error {+ constructor() {+ super("context pack: query_mode semantic requires a semanticRelevance map");+ this.name = "ContextPackSemanticRelevanceMissingError";+ }+}+export interface ContextPackItem extends ContextTransformAnnotations {readonly id: string;readonly path: string;@@ -147,6 +177,17 @@ export interface ContextPackItem extends ContextTransformAnnotations {readonly safety?: ContextSafetyReport;}+/**+ * What the `semantic` mode could score, counted over the candidates that+ * survived the reach filter, so a withheld page is in neither member.+ */+export interface ContextPackSemanticReport {+ /** Kept candidates that had a relevance score. */+ readonly scored: number;+ /** Ids of kept candidates without one, sorted; they sort after the scored ones of their tier. */+ readonly unembedded: ReadonlyArray<string>;+}+export interface ContextPackSkipped {readonly id: string;readonly tokens: number;@@ -177,6 +218,8 @@ export interface ContextPackReport {* and `density` use.*/readonly stamp?: ContextPackStamp;+ /** Present only in `semantic` mode; see {@link ContextPackSemanticReport}. */+ readonly semantic?: ContextPackSemanticReport;}/**@@ -212,6 +255,14 @@ export interface ContextPackOptions {* Without a `query` the mode has nothing to act on and is inert.*/readonly queryMode?: ContextPackQueryMode;+ /**+ * Relevance per belief note for the `semantic` mode, keyed by the+ * note's vault-relative POSIX path. A candidate absent from the map has+ * no usable vector: it sorts after every scored candidate of its tier and is named in+ * {@link ContextPackReport.semantic}. Required in `semantic` mode and+ * read in no other.+ */+ readonly semanticRelevance?: ReadonlyMap<string, number>;/*** Per-memory character cap (v0.20.0): trim any single page's body to* this many code points before it consumes the token budget, so one@@ -454,6 +505,7 @@ export function packContext(export function packContext(vault: string, opts: ContextPackOptions): ContextPackReport;export function packContext(vault: string, opts: ContextPackOptions): ContextPackReport {const startedAtMs = Date.now();+ const semanticRelevance = semanticRelevanceFor(opts);if (!Number.isFinite(opts.maxTokens) || opts.maxTokens <= 0) {return finalizeContextPackReport(vault,@@ -473,7 +525,10 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac// predicate - is null in that mode and no candidate is ever excluded for// failing to contain the turn verbatim.const rankedQuery = opts.queryMode === "ranked" && opts.query ? opts.query : null;- const query = rankedQuery === null && opts.query ? normalizeForDedup(opts.query) : null;+ const query =+ rankedQuery === null && semanticRelevance === null && opts.query+ ? normalizeForDedup(opts.query)+ : null;// Opt-in language-agnostic prompt-injection containment (Unit 1).// Default off, so the surfaced bodies are byte-identical to the legacy// blocklist guard unless the vault enables the flag.@@ -524,6 +579,38 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac// The map is empty when the mode is off, so both sides read 0 and the// comparator falls straight through.const relevanceById = new Map<string, number>();+ // Semantic relevance reads the caller's map over the KEPT candidates+ // only, so the report below is post-reach by construction.+ const unembedded: string[] = [];+ let scored = 0;+ if (semanticRelevance !== null) {+ for (const c of candidates) {+ const score = semanticRelevance.get(canonicalNotePath(relative(vault, c.path)));+ if (score === undefined) unembedded.push(c.id);+ else scored += 1;+ relevanceById.set(c.id, score ?? UNEMBEDDED_RELEVANCE);+ }+ }+ // No kept candidate has a usable vector: a "semantic" order would be+ // pure recency. Refused here, before the receipt and telemetry are+ // emitted, so a refused pack leaves no record of a delivery. An empty+ // candidate set is an empty pack, not a refusal.+ if (semanticRelevance !== null && candidates.length > 0 && scored === 0) {+ throw new SearchError(+ "BELIEF_VECTORS_MISSING",+ `semantic belief order unavailable: none of the ${candidates.length} belief note(s) ` ++ `in reach has a stored vector for the configured model; run: ${BELIEF_VECTORS_BACKFILL_COMMAND}`,+ );+ }+ const semanticReport: { readonly semantic?: ContextPackSemanticReport } =+ semanticRelevance === null+ ? {}+ : {+ semantic: Object.freeze({+ scored,+ unembedded: Object.freeze(unembedded.toSorted()),+ }),+ };if (rankedQuery !== null) {const queryTokens = tokenise(rankedQuery);for (const c of candidates) {@@ -711,6 +798,7 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPacitems: Object.freeze(keptItems),skipped: Object.freeze(skipped),...withOptionalLanes(opts, keptItems),+ ...semanticReport,},startedAtMs,[],@@ -730,6 +818,7 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPacitems: Object.freeze(finalItems),skipped: Object.freeze(skipped),...withOptionalLanes(opts, finalItems),+ ...semanticReport,},startedAtMs,ownerScopeWarnings,@@ -737,6 +826,17 @@ export function packContext(vault: string, opts: ContextPackOptions): ContextPac);}+/**+ * The relevance map the `semantic` mode reads, or null in every other+ * mode. Throws {@link ContextPackSemanticRelevanceMissingError} when the+ * mode is `semantic` and the map is absent.+ */+function semanticRelevanceFor(opts: ContextPackOptions): ReadonlyMap<string, number> | null {+ if (opts.queryMode !== "semantic") return null;+ if (opts.semanticRelevance === undefined) throw new ContextPackSemanticRelevanceMissingError();+ return opts.semanticRelevance;+}+function finalizeContextPackReport(vault: string,opts: ContextPackOptions,diff --git a/src/core/brain/maintenance/journal.ts b/src/core/brain/maintenance/journal.tsindex 734f3bb1..d12db8d6 100644--- a/src/core/brain/maintenance/journal.ts+++ b/src/core/brain/maintenance/journal.ts@@ -7,6 +7,10 @@* sweep on append, matching the activation-store discipline.*/+import {+ isEmbeddingPriceSource,+ type EmbeddingPriceSource,+} from "../../search/embeddings/pricing.ts";import { appendFileSync, existsSync, readFileSync, writeFileSync } from "node:fs";import { mkdirSync } from "node:fs";import { dirname, join } from "node:path";@@ -112,9 +116,34 @@ export interface MaintenanceSpendReceipt {/** The model the pass named; null when the config leaves it unset. */readonly model: string | null;readonly tokens: number;- readonly estimatedUsd: number;+ /**+ * Null when nobody stated the model's price. A row written before price+ * sources existed keeps the number it was written with.+ */+ readonly estimatedUsd: number | null;/** True when `--force-cost` overrode a positive gate that would have blocked the run. */readonly forced: boolean;+ /**+ * Who stated the price the estimate used. Absent on rows written before+ * price sources existed: such a row's source is unrecorded, not known.+ */+ readonly priceSource?: EmbeddingPriceSource;+}++/** How a receipt written before price sources existed names its source. */+export const PRICE_SOURCE_UNRECORDED = "unrecorded";++/**+ * The price source a journaled receipt recorded, read back off disk:+ * {@link PRICE_SOURCE_UNRECORDED} for a row written before price sources+ * existed (or carrying a value this build does not know), never a guess.+ */+export function recordedPriceSource(+ receipt: MaintenanceSpendReceipt,+): EmbeddingPriceSource | typeof PRICE_SOURCE_UNRECORDED {+ return isEmbeddingPriceSource(receipt.priceSource)+ ? receipt.priceSource+ : PRICE_SOURCE_UNRECORDED;}/**diff --git a/src/core/brain/maintenance/reindex-task.ts b/src/core/brain/maintenance/reindex-task.tsindex a07b68bc..91e05a62 100644--- a/src/core/brain/maintenance/reindex-task.ts+++ b/src/core/brain/maintenance/reindex-task.ts@@ -18,6 +18,7 @@* banner for a pass that did not happen.*/+import type { EmbeddingPriceSource } from "../../search/embeddings/pricing.ts";import { resolveMaintenanceEmbeddings } from "../../config.ts";import {resolveSemanticCapability,@@ -57,7 +58,10 @@ export interface LaneSpendBlock {readonly banner?: {readonly model: string | null;readonly pendingChunks: number;- readonly estimatedUsd: number;+ /** Null when nobody stated the model's price. */+ readonly estimatedUsd: number | null;+ /** Who stated the price the estimate used. */+ readonly priceSource: EmbeddingPriceSource;readonly gateUsd: number;};readonly receipt?: MaintenanceSpendReceipt;@@ -121,6 +125,7 @@ export function createLaneReindex(opts: LaneReindexOptions): LaneReindex {model: receipt.model,tokens: receipt.tokens,estimated_usd: receipt.estimatedUsd,+ price_source: receipt.priceSource,forced: receipt.forced,lane: true,},@@ -150,6 +155,7 @@ export function createLaneReindex(opts: LaneReindexOptions): LaneReindex {model: banner.model,pendingChunks: banner.pendingChunks,estimatedUsd: banner.estimatedUsd,+ priceSource: banner.priceSource,gateUsd: opts.searchConfig.semantic.costGateUsd,},}diff --git a/src/core/brain/time.ts b/src/core/brain/time.tsindex 306b7140..fffd5600 100644--- a/src/core/brain/time.ts+++ b/src/core/brain/time.ts@@ -88,10 +88,19 @@ export function msToWholeDays(durationMs: number): number {* `nowMs` is required rather than defaulted so a caller that already* holds a pinned clock cannot accidentally measure against a second,* later one mid-scan.+ *+ * A negative age within the clock's own millisecond is age zero: `nowMs`+ * is a whole millisecond (`Date.now()`) while the filesystem stamps a+ * sub-millisecond mtime, so a file written in that millisecond would+ * otherwise read as stamped in the future - a precision artefact, not+ * clock skew. Every other age keeps its fraction, so an mtime a whole+ * millisecond or more ahead still reads negative.*/export function fileAgeMs(path: string, nowMs: number): number | null {try {- return nowMs - statSync(path).mtimeMs;+ const mtimeMs = statSync(path).mtimeMs;+ const ageMs = nowMs - mtimeMs;+ return ageMs < 0 && Math.floor(mtimeMs) === Math.floor(nowMs) ? 0 : ageMs;} catch {return null;}diff --git a/src/core/maintenance/ensure-current.ts b/src/core/maintenance/ensure-current.tsindex cd470a50..8d6c9348 100644--- a/src/core/maintenance/ensure-current.ts+++ b/src/core/maintenance/ensure-current.ts@@ -81,7 +81,8 @@ export interface EnsureCurrentResult {readonly brainUpgraded: ReadonlyArray<string>;/*** In `background` mode, what was done about the managed-file upgrade;- * `null` in the foreground, where {@link brainUpgradeOutcome} answers.+ * `null` in the foreground, where {@link brainUpgradeOutcome} answers,+ * and when the vault was removed before a worker could be started.*/readonly brainUpgradeSpawn: SelfHealUpgradeSpawnDecision | null;/** In the foreground, how the upgrade attempt ended; `null` in `background` mode. */@@ -186,7 +187,10 @@ function startSelfHealUpgrade(vault: string,configPath: string | undefined,now: Date,-): SelfHealUpgradeSpawnDecision {+): SelfHealUpgradeSpawnDecision | null {+ // The claim creates the lock's directory; a vault removed since the+ // caller's check is left alone and nothing is started.+ if (!existsSync(brainConfigPath(vault))) return null;if (selfHealUpgradeBackoffActive(readSelfHealUpgradeFailure(vault), now)) {return SELF_HEAL_UPGRADE_SPAWN.skippedBackoff;}diff --git a/src/core/maintenance/self-heal-upgrade.ts b/src/core/maintenance/self-heal-upgrade.tsindex 5acb9422..c9e8be6a 100644--- a/src/core/maintenance/self-heal-upgrade.ts+++ b/src/core/maintenance/self-heal-upgrade.ts@@ -112,6 +112,9 @@ export function runSelfHealUpgrade(// longer ours; the process that took it over does the work.token = ownsSelfHealUpgradeLock(vault, opts.lockToken) ? opts.lockToken : null;} else {+ // The claim creates the lock's directory, so a vault that is gone is+ // answered before it: claiming first would recreate it.+ if (vaultGone(vault)) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);try {token = claimSelfHealUpgradeLock(vault, now);} catch (e) {@@ -134,10 +137,19 @@ export function runSelfHealUpgrade(}}+/**+ * Whether `vault` is no longer an initialised vault. Asked again right+ * before every write, not only once at the start: a vault removed or moved+ * while the worker plans would otherwise come back as a failure marker, a+ * metrics row or rewritten managed files, recreating a directory nobody+ * wants.+ */+function vaultGone(vault: string): boolean {+ return !existsSync(brainConfigPath(vault));+}+function attempt(vault: string, now: Date): SelfHealUpgradeRun {- // A vault removed or moved since the spawn: writing a marker would- // recreate a directory nobody wants, so nothing is written at all.- if (!existsSync(brainConfigPath(vault))) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);+ if (vaultGone(vault)) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);if (selfHealUpgradeBackoffActive(readSelfHealUpgradeFailure(vault), now)) {return run(SELF_HEAL_UPGRADE_OUTCOME.backoff);}@@ -151,7 +163,11 @@ function attempt(vault: string, now: Date): SelfHealUpgradeRun {return run(SELF_HEAL_UPGRADE_OUTCOME.current);}pending = plan.files.filter((f) => f.status === "update").map((f) => f.path);+ if (vaultGone(vault)) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);const applied = applyUpgrade(vault, { now });+ // The metrics row creates its directory: a vault removed during the+ // rewrite would come back for it.+ if (vaultGone(vault)) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);clearSelfHealUpgradeFailure(vault);recordRow(vault, {outcome: SELF_HEAL_UPGRADE_OUTCOME.applied,@@ -160,6 +176,9 @@ function attempt(vault: string, now: Date): SelfHealUpgradeRun {});return run(SELF_HEAL_UPGRADE_OUTCOME.applied, applied.files_updated);} catch (e) {+ // A plan or an apply that failed because the vault went away under it+ // is not a failure to record: there is no vault left to record it in.+ if (vaultGone(vault)) return run(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);const error = message(e);try {recordSelfHealUpgradeFailure(vault, error, pending, now);diff --git a/src/core/search/capability-tier.ts b/src/core/search/capability-tier.tsindex 490d75f5..5408e7cc 100644--- a/src/core/search/capability-tier.ts+++ b/src/core/search/capability-tier.ts@@ -41,6 +41,7 @@* label lookup is async and every caller of it is already async.*/+import type { SearchErrorCode } from "./search-error.ts";import type { ResolvedEmbeddingConfig } from "./types.ts";/**@@ -140,6 +141,40 @@ export function semanticCapabilityIsBlocked(capability: SemanticCapability): booreturn capability.tier !== SEMANTIC_CAPABILITY_TIER.configured;}+/** The rungs of the capability ladder that BLOCK; `configured` is not one. */+export type BlockedCapabilityTier = Exclude<+ SemanticCapabilityTier,+ typeof SEMANTIC_CAPABILITY_TIER.configured+>;++/**+ * The typed failure each blocked rung refuses with, shared by every+ * surface that attempted semantic work and could not.+ *+ * One entry per rung, so adding a rung to the ladder is a type error here+ * rather than a request that silently degrades. `credential-missing`+ * raises `EMBEDDING_KEY_MISSING`; `disabled` raises `EMBEDDING_DISABLED`,+ * the code the null provider already raises for the same configuration+ * when the indexer meets it, so one condition has one code across the+ * tree.+ */+export const BLOCKED_TIER_ERROR_CODE: Readonly<Record<BlockedCapabilityTier, SearchErrorCode>> =+ Object.freeze({+ [SEMANTIC_CAPABILITY_TIER.disabled]: "EMBEDDING_DISABLED",+ [SEMANTIC_CAPABILITY_TIER.credentialMissing]: "EMBEDDING_KEY_MISSING",+ });++/**+ * {@link semanticCapabilityIsBlocked} as a TYPE predicate, which lets+ * {@link BLOCKED_TIER_ERROR_CODE} be indexed with no cast and no+ * unreachable default arm.+ */+export function isBlockedCapability(+ capability: SemanticCapability,+): capability is SemanticCapability & { readonly tier: BlockedCapabilityTier } {+ return semanticCapabilityIsBlocked(capability);+}+/*** The operator-facing label for a registered capability or vector code,* read from the diagnostics registry.diff --git a/src/core/search/embedding-spend.ts b/src/core/search/embedding-spend.tsnew file mode 100644index 00000000..b7a4a4b1--- /dev/null+++ b/src/core/search/embedding-spend.ts@@ -0,0 +1,200 @@+/**+ * The shared embedding spend plan (Honest Embedding Spend).+ *+ * Four places price embedding spend: the phase gate, the maintenance+ * preview, the vector-backfill dry run and the `search status` refresh+ * estimate. They all read {@link planEmbeddingSpend}, so what a run+ * announces, refuses, spends and receipts is one computation: one pending+ * census (optionally path-scoped), one model resolution, one token+ * estimate, one price quote and one gate verdict.+ *+ * The gate fails closed only under an explicit cap: a positive+ * `embedding_cost_gate_usd` refuses an unpriced model with pending work,+ * because for an operator who asked for a ceiling an unknown price is not+ * a price below it. A zero gate (the default) never blocks.+ */++import {+ EMBEDDING_PRICE_MODEL_KEY,+ EMBEDDING_PRICE_RATE_KEY,+ resolveEmbeddingPrice,+ type PriceQuote,+} from "./embeddings/pricing.ts";+import { estimateCostUsd, estimateTokens, LOCAL_EMBEDDING_MODEL } from "./embeddings/signature.ts";+import type { Store } from "./store.ts";+import type { PendingVectorScope } from "./store/chunks.ts";+import type { ResolvedSearchConfig } from "./types.ts";++/** Why a gate refused. The value leaves TypeScript in previews and JSON. */+export const EMBEDDING_GATE_REASON = Object.freeze({+ /** The known estimate strictly exceeds the positive gate. */+ overCap: "over_cap",+ /** The gate is positive and nobody stated the model's price. */+ unpriced: "unpriced",+} as const);++/** Closed union over {@link EMBEDDING_GATE_REASON}. */+export type EmbeddingGateReason =+ (typeof EMBEDDING_GATE_REASON)[keyof typeof EMBEDDING_GATE_REASON];++/** Membership list of {@link EMBEDDING_GATE_REASON}. */+export const EMBEDDING_GATE_REASONS: ReadonlyArray<EmbeddingGateReason> = Object.freeze([+ EMBEDDING_GATE_REASON.overCap,+ EMBEDDING_GATE_REASON.unpriced,+]);++/** Narrow a string read back off disk or across a tool boundary. */+export function isEmbeddingGateReason(value: unknown): value is EmbeddingGateReason {+ return (+ typeof value === "string" && (EMBEDDING_GATE_REASONS as ReadonlyArray<string>).includes(value)+ );+}++/** A gate verdict: `reason` is null exactly when the run is not blocked. */+export type EmbeddingGateVerdict =+ | { readonly blocked: false; readonly reason: null }+ | { readonly blocked: true; readonly reason: EmbeddingGateReason };++/** Outcome of a cost-gate evaluation for an embedding run. */+export type CostGateResult = EmbeddingGateVerdict & {+ readonly tokens: number;+ /** Null when the price is unknown; never 0 for an unknown price. */+ readonly estimatedUsd: number | null;+};++const PASS: EmbeddingGateVerdict = Object.freeze({ blocked: false, reason: null });++/**+ * Evaluate whether an embedding run should be blocked on estimated spend.+ * Never blocks when the gate is 0, the run is forced or nothing is+ * pending. Otherwise an unknown price blocks as `unpriced` and a known+ * estimate strictly over the gate blocks as `over_cap`.+ */+export function evaluateCostGate(opts: {+ texts: ReadonlyArray<string>;+ quote: PriceQuote;+ gateUsd: number;+ forced?: boolean;+}): CostGateResult {+ const tokens = estimateTokens(opts.texts);+ const estimatedUsd = estimateCostUsd(tokens, opts.quote);+ return { tokens, estimatedUsd, ...gateVerdict(opts, estimatedUsd) };+}++function gateVerdict(+ opts: { texts: ReadonlyArray<string>; gateUsd: number; forced?: boolean },+ estimatedUsd: number | null,+): EmbeddingGateVerdict {+ if (opts.gateUsd <= 0 || opts.forced === true || opts.texts.length === 0) return PASS;+ if (estimatedUsd === null) return { blocked: true, reason: EMBEDDING_GATE_REASON.unpriced };+ if (estimatedUsd > opts.gateUsd) return { blocked: true, reason: EMBEDDING_GATE_REASON.overCap };+ return PASS;+}++/**+ * The model an embedding request would name right now. The local+ * provider always embeds with its implicit model, whatever+ * `embedding_model` says; every other provider takes the configured+ * one, falling back to what the index recorded when the config leaves+ * it unset.+ */+export function activeEmbeddingModel(+ config: ResolvedSearchConfig,+ storedModel: string | null = null,+): string | null {+ if (config.semantic.provider === "local") return LOCAL_EMBEDDING_MODEL;+ return config.semantic.model ?? storedModel;+}++/**+ * The model a pass would name and its price quote, through the operator+ * pair. The one model resolution every spend surface shares.+ */+export function activeSpendQuote(config: ResolvedSearchConfig): {+ readonly model: string | null;+ readonly quote: PriceQuote;+} {+ const model = activeEmbeddingModel(config);+ return { model, quote: resolveEmbeddingPrice(model, config.semantic.priceOverride) };+}++/** One computation of what an embedding pass would spend. */+export interface EmbeddingSpendPlan {+ /** The scoped pending census: chunks with no vector yet. */+ readonly pending: ReadonlyArray<{ readonly chunkId: number; readonly content: string }>;+ readonly model: string | null;+ readonly tokens: number;+ readonly quote: PriceQuote;+ /** Null when the price is unknown. */+ readonly estimatedUsd: number | null;+ readonly gate: EmbeddingGateVerdict;+}++/** Options of {@link planEmbeddingSpend}. */+export interface EmbeddingSpendPlanOptions {+ /** Path scope of the pending census; vault-wide when absent. */+ readonly scope?: PendingVectorScope;+ /** A forced plan never blocks. */+ readonly forced?: boolean;+}++/**+ * Plan the spend of embedding every pending chunk in scope. Reads only:+ * the phase in `indexer.ts` remains the only spender and throws on a+ * blocked verdict.+ */+export function planEmbeddingSpend(+ store: Store,+ config: ResolvedSearchConfig,+ opts: EmbeddingSpendPlanOptions = {},+): EmbeddingSpendPlan {+ return priceSpendPlan(store.findChunksWithoutEmbeddings(opts.scope), config, opts.forced);+}++/**+ * Price `pending` under the current config: the model, the quote, the+ * estimate and the gate verdict, with no census read. The embedding+ * phase re-prices a plan a caller handed in this way, so the verdict it+ * gates on is never one the caller computed forced or under another+ * config.+ */+export function priceSpendPlan(+ pending: EmbeddingSpendPlan["pending"],+ config: ResolvedSearchConfig,+ forced?: boolean,+): EmbeddingSpendPlan {+ const { model, quote } = activeSpendQuote(config);+ const { tokens, estimatedUsd, ...gate } = evaluateCostGate({+ texts: pending.map((p) => p.content),+ quote,+ gateUsd: config.semantic.costGateUsd,+ ...(forced === undefined ? {} : { forced }),+ });+ return { pending, model, tokens, quote, estimatedUsd, gate };+}++/** What every spend surface prints in place of a dollar figure nobody can state. */+export const PRICE_UNKNOWN_LABEL = "price unknown";++/** Decimal places of every rendered USD amount, the gate's own spelling. */+export const USD_DECIMALS = 4;++/** A USD estimate as `$0.0123`, or {@link PRICE_UNKNOWN_LABEL} when null. */+export function formatEstimatedUsd(usd: number | null): string {+ return usd === null ? PRICE_UNKNOWN_LABEL : `$${usd.toFixed(USD_DECIMALS)}`;+}++/** The config key whose positive value turns an estimate into a refusal. */+export const COST_GATE_KEY = "embedding_cost_gate_usd";+/** The flag that passes a cost-gate refusal for one run. */+export const FORCE_COST_FLAG = "--force-cost";++/** The refusal text for an unpriced model under a positive gate. */+export function unpricedRefusalMessage(plan: EmbeddingSpendPlan, gateUsd: number): string {+ return (+ `embedding model ${plan.model ?? "(unset)"} has no known price, so ${plan.pending.length} ` ++ `chunk(s) cannot be checked against ${COST_GATE_KEY} ${formatEstimatedUsd(gateUsd)}. ` ++ `Declare its price with ${EMBEDDING_PRICE_MODEL_KEY} and ${EMBEDDING_PRICE_RATE_KEY}, ` ++ `or re-run with ${FORCE_COST_FLAG} to proceed.`+ );+}diff --git a/src/core/search/embeddings/credential-report.ts b/src/core/search/embeddings/credential-report.tsnew file mode 100644index 00000000..fc57e9e6--- /dev/null+++ b/src/core/search/embeddings/credential-report.ts@@ -0,0 +1,102 @@+/**+ * Where `search check` looked for an embedding credential (t_82c3b275).+ *+ * A `credential-missing` tier says that no key resolved. This report+ * says where the resolver looked, in its own probe order, and which+ * other registered profiles do have a key present, so an operator whose+ * key sits under another name can see it without reading the resolver.+ *+ * Three rules bound it:+ *+ * - Names only. An env value is tested for presence and never copied,+ * so nothing credential-shaped can leave this function.+ * - Operator-declared names only: the config key, its env override and+ * the env-key names of the profiles in the vault's own registry. No+ * vendor list of well-known variable names is consulted.+ * - It does not touch the tier. `resolveSemanticCapability` stays+ * derived from the resolved config alone; this report is computed+ * beside it, from an env the caller passes in explicitly.+ */++import { resolveSemanticCapability, SEMANTIC_CAPABILITY_TIER } from "../capability-tier.ts";+import {+ SearchError,+ type CredentialSourceReport,+ type ResolvedEmbeddingConfig,+} from "../types.ts";+import { envKeyList, type ProviderProfile } from "./registry.ts";++export type { CredentialSourceReport };++/** The env override of the explicit credential, which wins over the config key. */+export const EMBEDDING_KEY_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_KEY";+/** The flat config key of the explicit credential. */+export const EMBEDDING_KEY_CONFIG = "embedding_api_key";++/**+ * The explicit credential sources, in the order the resolver probes+ * them. They are consulted for every provider that needs a key, a+ * registered profile included: an explicit key wins over the profile's.+ */+export const EXPLICIT_CREDENTIAL_SOURCES: ReadonlyArray<string> = Object.freeze([+ EMBEDDING_KEY_ENV,+ EMBEDDING_KEY_CONFIG,+]);++/** Where the report reads names from, handed in by the caller rather than read from the process. */+export interface CredentialSourceContext {+ /** The registered profile the configuration names, or null for explicit config. */+ readonly activeProfile: string | null;+ readonly registry: ReadonlyArray<ProviderProfile>;+ readonly env: Readonly<Record<string, string | undefined>>;+}++export interface CredentialSourceInput extends CredentialSourceContext {+ readonly semantic: ResolvedEmbeddingConfig;+}++function hasPresentKey(+ profile: ProviderProfile,+ env: Readonly<Record<string, string | undefined>>,+): boolean {+ return envKeyList(profile.envKey).some((name) => {+ const value = env[name];+ return value !== undefined && value !== "";+ });+}++/**+ * The credential-source report, or null for every tier other than+ * `credential-missing` - a configured or disabled setup has nothing to+ * explain, so its check output stays byte-identical.+ *+ * Throws `SearchError("INVALID_INPUT")` naming the profile when+ * `activeProfile` is not in `registry`: config resolution only expands a+ * registered name, so a mismatch means the caller passed two different+ * registries, and a report built on either would name the wrong sources.+ */+export function describeCredentialSources(+ input: CredentialSourceInput,+): CredentialSourceReport | null {+ const { semantic, activeProfile, registry, env } = input;+ if (resolveSemanticCapability(semantic).tier !== SEMANTIC_CAPABILITY_TIER.credentialMissing) {+ return null;+ }+ let profileKeys: ReadonlyArray<string> = [];+ if (activeProfile !== null) {+ const profile = registry.find((p) => p.name === activeProfile);+ if (profile === undefined) {+ throw new SearchError(+ "INVALID_INPUT",+ `embedding provider profile "${activeProfile}" is not in the provider registry`,+ );+ }+ profileKeys = envKeyList(profile.envKey);+ }+ return Object.freeze({+ consulted: Object.freeze([...EXPLICIT_CREDENTIAL_SOURCES, ...profileKeys]),+ presentElsewhere: Object.freeze(+ registry.filter((p) => p.name !== activeProfile && hasPresentKey(p, env)).map((p) => p.name),+ ),+ });+}diff --git a/src/core/search/embeddings/presets.ts b/src/core/search/embeddings/presets.tsindex 61a5a03c..a977ac37 100644--- a/src/core/search/embeddings/presets.ts+++ b/src/core/search/embeddings/presets.ts@@ -149,14 +149,12 @@ export function findEmbeddingPreset(model: string): EmbeddingModelPreset | null* The input window declared for `model`, or `null` when this table does* not declare one.*- * `null` means UNKNOWN, and every caller must treat it as such. It is- * deliberately NOT the shape {@link pricePerMillionTokens} uses, which- * answers 0 for an unlisted model so the cost gate can never falsely- * block: an unknown price is safe to treat as free because the- * consequence of the fallback is that a gate declines to fire. An unknown- * window has the opposite polarity - treating it as "fits" would report a- * passing check for a condition nobody measured, which is the misleading- * silence this census exists to remove.+ * `null` means UNKNOWN, and every caller must treat it as such. This is+ * the same polarity the price resolver (`resolveEmbeddingPrice` in+ * `pricing.ts`) uses for an unlisted model: treating an unknown window as+ * "fits", like treating an unknown price as free, would report a passing+ * check for a condition nobody measured, which is the misleading silence+ * this census exists to remove.** Structural lookup by exact model string only. There is no family* heuristic here on purpose: `e5` prefixes are a property of thediff --git a/src/core/search/embeddings/pricing.ts b/src/core/search/embeddings/pricing.tsnew file mode 100644index 00000000..f3cb3c85--- /dev/null+++ b/src/core/search/embeddings/pricing.ts@@ -0,0 +1,100 @@+/**+ * Embedding price quotes (Honest Embedding Spend).+ *+ * One resolver answers "what does this model cost per million input+ * tokens, and who said so". The order is: the operator's declared price+ * pair when its model matches, then the frozen builtin table, then+ * unknown. An unknown price is reported as unknown (`usdPerMtok: null`),+ * never as 0, so a paid model nobody priced cannot read as free.+ */++import { canonicalToken, EMBEDDING_PRICING } from "./signature.ts";++/** Who stated an embedding price. The value leaves TypeScript in receipts and JSON. */+export const EMBEDDING_PRICE_SOURCE = Object.freeze({+ /** The frozen table shipped with this release stated the price. */+ builtin: "builtin",+ /** The operator declared the price through the config price pair. */+ operator: "operator",+ /** Nobody stated a price for the model. */+ unknown: "unknown",+} as const);++/** Closed union over {@link EMBEDDING_PRICE_SOURCE}. */+export type EmbeddingPriceSource =+ (typeof EMBEDDING_PRICE_SOURCE)[keyof typeof EMBEDDING_PRICE_SOURCE];++/** Membership list, in resolution order. */+export const EMBEDDING_PRICE_SOURCES: ReadonlyArray<EmbeddingPriceSource> = Object.freeze([+ EMBEDDING_PRICE_SOURCE.operator,+ EMBEDDING_PRICE_SOURCE.builtin,+ EMBEDDING_PRICE_SOURCE.unknown,+]);++/** Narrow a string read back off disk or across a tool boundary. */+export function isEmbeddingPriceSource(value: unknown): value is EmbeddingPriceSource {+ return (+ typeof value === "string" && (EMBEDDING_PRICE_SOURCES as ReadonlyArray<string>).includes(value)+ );+}++/** Config key naming the model the operator's declared price applies to. */+export const EMBEDDING_PRICE_MODEL_KEY = "embedding_price_model";+/** Config key carrying the operator's declared USD per million input tokens. */+export const EMBEDDING_PRICE_RATE_KEY = "embedding_price_usd_per_mtok";+/** Env twin of {@link EMBEDDING_PRICE_MODEL_KEY}. */+export const EMBEDDING_PRICE_MODEL_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL";+/** Env twin of {@link EMBEDDING_PRICE_RATE_KEY}. */+export const EMBEDDING_PRICE_RATE_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK";++/** The operator's declared price, bound to one model name. */+export interface EmbeddingPriceOverride {+ readonly model: string;+ readonly usdPerMtok: number;+}++/**+ * A resolved price. `usdPerMtok` is null exactly when `source` is+ * `unknown`; a known rate of 0 is a statement that the model is free.+ */+export type PriceQuote = KnownPriceQuote | UnknownPriceQuote;++/** A quote whose rate somebody stated. */+export interface KnownPriceQuote {+ readonly usdPerMtok: number;+ readonly source: typeof EMBEDDING_PRICE_SOURCE.builtin | typeof EMBEDDING_PRICE_SOURCE.operator;+}++/** A quote for a model nobody priced. */+export interface UnknownPriceQuote {+ readonly usdPerMtok: null;+ readonly source: typeof EMBEDDING_PRICE_SOURCE.unknown;+}++const UNKNOWN_QUOTE: PriceQuote = Object.freeze({+ usdPerMtok: null,+ source: EMBEDDING_PRICE_SOURCE.unknown,+});++/**+ * Resolve the price of `model`. The operator declaration wins when its+ * model equals `model` after the same canonicalisation the table uses,+ * so a vendor price change can be declared over a table rate. A+ * declaration for another model is ignored: the pair binds the price to+ * one name, so switching models never re-targets it.+ */+export function resolveEmbeddingPrice(+ model: string | null,+ override?: EmbeddingPriceOverride,+): PriceQuote {+ if (model === null) return UNKNOWN_QUOTE;+ const key = canonicalToken(model);+ if (override !== undefined && canonicalToken(override.model) === key) {+ return { usdPerMtok: override.usdPerMtok, source: EMBEDDING_PRICE_SOURCE.operator };+ }+ const rate = EMBEDDING_PRICING[key];+ if (typeof rate === "number" && Number.isFinite(rate) && rate >= 0) {+ return { usdPerMtok: rate, source: EMBEDDING_PRICE_SOURCE.builtin };+ }+ return UNKNOWN_QUOTE;+}diff --git a/src/core/search/embeddings/registry.ts b/src/core/search/embeddings/registry.tsindex e3e7811f..81d63aff 100644--- a/src/core/search/embeddings/registry.ts+++ b/src/core/search/embeddings/registry.ts@@ -34,7 +34,7 @@ export interface ProviderProfile {}/** Normalise `envKey` (string | list) to an ordered list of env-var names. */-function envKeyList(envKey: ProviderProfile["envKey"]): string[] {+export function envKeyList(envKey: ProviderProfile["envKey"]): string[] {const raw = typeof envKey === "string" ? [envKey] : [...envKey];return raw.map((k) => k.trim()).filter((k) => k !== "");}diff --git a/src/core/search/embeddings/signature.ts b/src/core/search/embeddings/signature.tsindex 37c13d7c..1fc1dbea 100644--- a/src/core/search/embeddings/signature.ts+++ b/src/core/search/embeddings/signature.ts@@ -32,7 +32,8 @@ export const LOCAL_EMBEDDING_MODEL = "hashing-ngram-v1";/** Sentinel for a null model/dimension so signatures stay parseable. */const NULL_FIELD = "?";-function canonicalToken(raw: string): string {+/** NFC, trim, lowercase: the one normalisation for identity and price keys. */+export function canonicalToken(raw: string): string {return raw.normalize("NFC").trim().toLowerCase();}@@ -77,11 +78,14 @@ export function signatureIdentityKnown(signature: string): boolean {}/**- * Best-effort embedding price in USD per million input tokens, keyed by- * model name. Rates drift as providers change pricing, so the table is- * deliberately small and any model NOT listed is treated as free- * (price 0) - the cost gate must never falsely block on a model whose- * rate we do not know. The local embedder is listed explicitly at 0.+ * Builtin embedding prices in USD per million input tokens, keyed by+ * canonical model name. Rates drift as providers change pricing, so the+ * table is deliberately small. A model NOT listed has an UNKNOWN price,+ * never a price of 0: `resolveEmbeddingPrice` (`pricing.ts`) reports it+ * as unknown, an explicit positive cost gate refuses to spend on it, and+ * the operator can state its rate with the `embedding_price_model` /+ * `embedding_price_usd_per_mtok` pair, which also wins over this table.+ * The local embedder is listed explicitly at 0, so it is known to be free.*/export const EMBEDDING_PRICING: Readonly<Record<string, number>> = Object.freeze({[LOCAL_EMBEDDING_MODEL]: 0,@@ -90,13 +94,6 @@ export const EMBEDDING_PRICING: Readonly<Record<string, number>> = Object.freeze"text-embedding-ada-002": 0.1,});-/** Per-million-token price for a model; 0 for the local/unknown/null case. */-export function pricePerMillionTokens(model: string | null): number {- if (model === null) return 0;- const rate = EMBEDDING_PRICING[canonicalToken(model)];- return typeof rate === "number" && Number.isFinite(rate) && rate > 0 ? rate : 0;-}-/*** Characters per estimated token. Lifted out of {@link estimateTokens}* because the ratio now has a second reader: the store-side oversize@@ -251,11 +248,28 @@ export function utf8ByteFloorUnderTokenBudget(tokens: number): number {);}-/** Estimated spend in USD for `tokens` against `model`'s rate. */-export function estimateCostUsd(tokens: number, model: string | null): number {- const rate = pricePerMillionTokens(model);- if (rate === 0) return 0;- return (tokens / 1_000_000) * rate;+/** Tokens per unit of a per-million-token rate. */+const TOKENS_PER_PRICED_UNIT = 1_000_000;++/**+ * The rate half of a price quote (`PriceQuote` in `pricing.ts`), stated+ * structurally so this kernel imports nothing from the resolver that+ * imports it: null means nobody stated the price.+ */+interface QuotedRate {+ readonly usdPerMtok: number | null;+}++/**+ * Estimated spend in USD for `tokens` at the quoted rate: null when the+ * price is unknown, so an unpriced model never reads as free.+ */+export function estimateCostUsd(tokens: number, quote: { readonly usdPerMtok: number }): number;+export function estimateCostUsd(tokens: number, quote: QuotedRate): number | null;+export function estimateCostUsd(tokens: number, quote: QuotedRate): number | null {+ if (quote.usdPerMtok === null) return null;+ if (quote.usdPerMtok === 0) return 0;+ return (tokens / TOKENS_PER_PRICED_UNIT) * quote.usdPerMtok;}/**@@ -272,28 +286,3 @@ export function isStaleSignature(active: string, stored: string): boolean {if (!signatureIdentityKnown(active) || !signatureIdentityKnown(stored)) return true;return active !== stored;}--/** Outcome of a cost-gate evaluation for an embedding run. */-export interface CostGateResult {- readonly tokens: number;- readonly estimatedUsd: number;- readonly blocked: boolean;-}--/**- * Evaluate whether an embedding run should be blocked on estimated spend.- * A run is blocked only when the gate is positive, the run is not forced,- * and the estimate strictly exceeds the gate. A zero gate (the default)- * and free models (local/unknown -> estimate 0) never block.- */-export function evaluateCostGate(opts: {- texts: ReadonlyArray<string>;- model: string | null;- gateUsd: number;- forced?: boolean;-}): CostGateResult {- const tokens = estimateTokens(opts.texts);- const estimatedUsd = estimateCostUsd(tokens, opts.model);- const blocked = opts.gateUsd > 0 && opts.forced !== true && estimatedUsd > opts.gateUsd;- return { tokens, estimatedUsd, blocked };-}diff --git a/src/core/search/embeddings/sunset.ts b/src/core/search/embeddings/sunset.tsindex 92233200..d8291d3e 100644--- a/src/core/search/embeddings/sunset.ts+++ b/src/core/search/embeddings/sunset.ts@@ -64,9 +64,8 @@* validation and this tool's own onboarding recommends a preview model* that no catalog here carries, so off-survey is the NORMAL case. This* is the same polarity `declaredInputWindowTokens` chose for an- * unlisted window and the opposite of `pricePerMillionTokens`, which- * may answer 0 for an unknown model only because its fallback makes a- * gate DECLINE to fire.+ * unlisted window and `resolveEmbeddingPrice` chose for an unlisted+ * price: unknown is reported as unknown, never as a passing value.** ## What an entry may honestly claim*diff --git a/src/core/search/index.ts b/src/core/search/index.tsindex 0f08b56b..b149c695 100644--- a/src/core/search/index.ts+++ b/src/core/search/index.ts@@ -20,10 +20,23 @@ import {loadProviderRegistry,expandRegisteredProvider,type ExpandedProvider,+ type ProviderProfile,} from "./embeddings/registry.ts";+import {+ EMBEDDING_KEY_CONFIG,+ EMBEDDING_KEY_ENV,+ type CredentialSourceContext,+} from "./embeddings/credential-report.ts";import { loadRerankRegistry, expandRegisteredRerankProvider } from "./rerank/registry.ts";import { decisionModelModeFor, resolveDecisionModelConfig } from "../decision-model/config.ts";import { resolveEmbeddingPrefixes } from "./embeddings/presets.ts";+import {+ EMBEDDING_PRICE_MODEL_ENV,+ EMBEDDING_PRICE_MODEL_KEY,+ EMBEDDING_PRICE_RATE_ENV,+ EMBEDDING_PRICE_RATE_KEY,+ type EmbeddingPriceOverride,+} from "./embeddings/pricing.ts";import { SearchError } from "./types.ts";import type {ResolvedEmbeddingConfig,@@ -51,6 +64,7 @@ export type {ExpandHitInput,ExpandHitResult,IndexCheckReport,+ CredentialSourceReport,EmbedderRecordCensus,IndexStats,IndexStatusSnapshot,@@ -310,6 +324,10 @@ function parsePositiveFloat(raw: string | null, fallback: number, fieldName: str/** Parse a non-negative finite float (e.g. a cost gate; 0 disables). */function parseNonNegativeFloat(raw: string | null, fallback: number, fieldName: string): number {if (raw === null) return fallback;+ // `Number(" ")` is 0: a blank gate would read as a gate switched off.+ if (raw.trim() === "") {+ throw new SearchError("INVALID_INPUT", `${fieldName} must be a number >= 0, got empty string`);+ }const n = Number(raw);if (!Number.isFinite(n) || n < 0) {throw new SearchError("INVALID_INPUT", `${fieldName} must be a number >= 0, got '${raw}'`);@@ -317,6 +335,73 @@ function parseNonNegativeFloat(raw: string | null, fallback: number, fieldName:return n;}+/** The highest declarable embedding rate, in USD per million tokens. */+const EMBEDDING_PRICE_RATE_CEILING = 1_000_000;++/**+ * Parse a declared embedding rate. Only a plain decimal spelling is a+ * price: `Number()` would read `0x10`, `1e3` or `Infinity` as numbers an+ * operator never wrote as a rate. A rate above the ceiling is a typo, not+ * a price, and would overflow every estimate built on it.+ */+function parseEmbeddingPriceRate(raw: string, fieldName: string): number {+ if (!/^\d+(\.\d+)?$/.test(raw)) {+ throw new SearchError(+ "INVALID_INPUT",+ `${fieldName} must be a plain decimal number >= 0 (for example 0.02), got '${raw}'`,+ );+ }+ const n = Number(raw);+ if (n > EMBEDDING_PRICE_RATE_CEILING) {+ throw new SearchError(+ "INVALID_INPUT",+ `${fieldName} must be at most ${EMBEDDING_PRICE_RATE_CEILING} USD per million tokens, got '${raw}'`,+ );+ }+ return n;+}++/**+ * The operator's declared embedding price: `embedding_price_model` and+ * `embedding_price_usd_per_mtok`, both or neither, overridable as a unit+ * by their `OPEN_SECOND_BRAIN_EMBEDDING_PRICE_*` env twins. A pair, not a+ * model-keyed map, because the flat parser splits a key on its first+ * colon and model ids carry colons (`nomic-embed-text:latest`); the value+ * side keeps them. Binding the rate to one model name means a model+ * switch never silently re-targets the price. The pair resolves from ONE+ * source: when either env twin is set both halves come from env, so an+ * env model never borrows a config rate into a price nobody declared.+ * Null when neither source sets either half.+ */+function resolveEmbeddingPriceOverride(+ env: NodeJS.ProcessEnv,+ config: Readonly<Record<string, string>>,+): EmbeddingPriceOverride | null {+ // A blank half is a missing half: `Number(" ")` is 0, so a whitespace+ // rate would otherwise declare an unpriced model free.+ const set = (layer: Readonly<Record<string, string | undefined>>, name: string) => {+ const value = layer[name]?.trim();+ return value === undefined || value === "" ? null : value;+ };+ const fromEnv =+ set(env, EMBEDDING_PRICE_MODEL_ENV) !== null || set(env, EMBEDDING_PRICE_RATE_ENV) !== null;+ const [layer, modelName, rateName] = fromEnv+ ? [env, EMBEDDING_PRICE_MODEL_ENV, EMBEDDING_PRICE_RATE_ENV]+ : [config, EMBEDDING_PRICE_MODEL_KEY, EMBEDDING_PRICE_RATE_KEY];+ const model = set(layer, modelName);+ const rateRaw = set(layer, rateName);+ if (model === null && rateRaw === null) return null;+ if (model === null || rateRaw === null) {+ const [present, missing] = model === null ? [rateName, modelName] : [modelName, rateName];+ throw new SearchError(+ "INVALID_INPUT",+ `${present} is set but ${missing} is not: declare both or neither`,+ );+ }+ const usdPerMtok = parseEmbeddingPriceRate(rateRaw, rateName);+ return Object.freeze({ model, usdPerMtok });+}+/*** Raw setting lookup that preserves an explicit empty string* (memory-write-path-integrity B2). Unlike `envOrConfig`, which folds `""`@@ -344,6 +429,12 @@ function rawSetting(*/function parseFiniteFloat(raw: string | null, fallback: number, fieldName: string): number {if (raw === null) return fallback;+ if (raw.trim() === "") {+ throw new SearchError(+ "INVALID_INPUT",+ `${fieldName} must be a finite number, got empty string`,+ );+ }const n = Number(raw);if (!Number.isFinite(n)) {throw new SearchError("INVALID_INPUT", `${fieldName} must be a finite number, got '${raw}'`);@@ -479,6 +570,40 @@ function resolveRegistryProvider(}}+/**+ * The names `search check` hands the credential-source report: the+ * registered profile `embedding_provider` selects (by the same env-over-+ * config rule and registry lookup {@link resolveSearchConfig} expands),+ * the vault's registry and the env. Fail-soft like the expansion itself:+ * `loadProviderRegistry` reads a missing or malformed registry as an empty+ * one, so the report names exactly the profiles the resolver can see, and a+ * name the registry does not hold is no profile.+ */+export function resolveCredentialContext(opts: {+ vault: string;+ configPath?: string;+ env?: NodeJS.ProcessEnv;+}): CredentialSourceContext {+ const env = opts.env ?? process.env;+ const config: Readonly<Record<string, string>> = opts.configPath+ ? discoverConfig(opts.configPath).data+ : {};+ const rawProvider = envOrConfig(+ env,+ config,+ "OPEN_SECOND_BRAIN_EMBEDDING_PROVIDER",+ "embedding_provider",+ );+ const registry: ReadonlyArray<ProviderProfile> = loadProviderRegistry(opts.vault);+ const activeProfile =+ rawProvider !== null &&+ !BUILTIN_PROVIDERS.has(rawProvider) &&+ registry.some((p) => p.name === rawProvider)+ ? rawProvider+ : null;+ return Object.freeze({ activeProfile, registry, env });+}+export function resolveSearchConfig(opts: {vault: string;configPath?: string;@@ -567,12 +692,7 @@ export function resolveSearchConfig(opts: {"OPEN_SECOND_BRAIN_EMBEDDING_MODEL","embedding_model",);- const explicitApiKey = envOrConfig(- env,- config,- "OPEN_SECOND_BRAIN_EMBEDDING_KEY",- "embedding_api_key",- );+ const explicitApiKey = envOrConfig(env, config, EMBEDDING_KEY_ENV, EMBEDDING_KEY_CONFIG);// Explicit config/env always wins over the registry profile's fields.const baseUrl = explicitBaseUrl ?? registryExpansion?.baseUrl ?? null;// The plain-http opt-out binds to the URL the operator wrote down: the@@ -658,6 +778,7 @@ export function resolveSearchConfig(opts: {DEFAULTS.costGateUsd,"embedding_cost_gate_usd",);+ const priceOverride = resolveEmbeddingPriceOverride(env, config);// Instruction prefixes (memory-write-path-integrity B2). Resolved with raw// presence, not `envOrConfig`, because an explicit empty string must DISABLE@@ -689,6 +810,7 @@ export function resolveSearchConfig(opts: {...(batchTokens === null ? {} : { batchTokens }),maxRetries,costGateUsd,+ ...(priceOverride === null ? {} : { priceOverride }),queryPrefix,passagePrefix,});diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.tsindex 52f35953..a5e3345e 100644--- a/src/core/search/indexer.ts+++ b/src/core/search/indexer.ts@@ -30,19 +30,33 @@ import { parseAuthoredAtSeconds } from "./authored-at.ts";import { CHUNKER_VERSION, chunkMarkdown } from "./chunker.ts";import { expandTextForCjkFts } from "./cjk-tokenizer.ts";import { declaredInputWindowTokens, passagePrefixSentByProvider } from "./embeddings/presets.ts";+import {+ activeEmbeddingModel,+ activeSpendQuote,+ COST_GATE_KEY,+ EMBEDDING_GATE_REASON,+ type EmbeddingGateReason,+ type EmbeddingSpendPlan,+ FORCE_COST_FLAG,+ formatEstimatedUsd,+ planEmbeddingSpend,+ priceSpendPlan,+ unpricedRefusalMessage,+} from "./embedding-spend.ts";+import { embeddingPriceRecommendations } from "./price-recommendations.ts";+import type { EmbeddingPriceSource } from "./embeddings/pricing.ts";import { makeProvider } from "./embeddings/provider.ts";import {embeddingSignature,- estimateCostUsd,- estimateTokens,- evaluateCostGate,isStaleSignature,- pricePerMillionTokens,- LOCAL_EMBEDDING_MODEL,MODEL_NATIVE_DIMENSION,type EmbeddingIdentity,} from "./embeddings/signature.ts";import { LOCAL_DEFAULT_DIMENSION } from "./embeddings/local-provider.ts";+import {+ describeCredentialSources,+ type CredentialSourceContext,+} from "./embeddings/credential-report.ts";import { resolveEventAnchor } from "./event-anchor.ts";import { extractLinks } from "./links.ts";import { resolveDocumentEventTimeWindow } from "./pipeline/event-time.ts";@@ -99,6 +113,7 @@ import {excludedCallableVisibilitySurfaces,} from "./visibility-surface-registry.ts";import type { ChunkInput, LinkInput } from "./store.ts";+import type { PendingVectorScope } from "./store/chunks.ts";import type {ChunkWindowCensus,EmbedderRecordCensus,@@ -165,6 +180,7 @@ interface MutableStats {deleted: number;chunksTotal: number;embeddingsComputed: number;+ embeddingsReused: number;embeddingsRetries: number;errors: Array<{ readonly path: string; readonly message: string }>;frontmatterNotices: DegradationNotice[];@@ -193,6 +209,7 @@ function newStats(): MutableStats {deleted: 0,chunksTotal: 0,embeddingsComputed: 0,+ embeddingsReused: 0,embeddingsRetries: 0,errors: [],frontmatterNotices: [],@@ -233,6 +250,7 @@ function freezeStats(s: MutableStats, durationMs: number): IndexStats {deleted: s.deleted,chunksTotal: s.chunksTotal,embeddingsComputed: s.embeddingsComputed,+ embeddingsReused: s.embeddingsReused,embeddingsRetries: s.embeddingsRetries,errors: Object.freeze([...s.errors]),frontmatterNotices: Object.freeze([...s.frontmatterNotices]),@@ -592,7 +610,11 @@ async function indexIntoRun(tokenCount: c.tokenCount,headingPath: c.headingPath,}));- const chunkIds = store.replaceChunks(docId, chunkInputs);+ // An unchanged chunk keeps its stored vector (vector carry-over),+ // so the embedding phase below never re-pays for it.+ const replaced = store.replaceDocumentChunks(docId, chunkInputs);+ const chunkIds = replaced.chunkIds;+ stats.embeddingsReused += replaced.embeddingsReused;const links: LinkInput[] = [];for (let i = 0; i < chunkResult.chunks.length; i++) {@@ -800,20 +822,6 @@ async function indexIntoRun(}}-/**- * The model an embedding request would name right now. The local- * provider has an implicit model string; every other provider takes the- * configured one, falling back to what the index recorded when the- * config leaves it unset.- */-function activeEmbeddingModel(- config: ResolvedSearchConfig,- storedModel: string | null = null,-): string | null {- if (config.semantic.provider === "local") return LOCAL_EMBEDDING_MODEL;- return config.semantic.model ?? storedModel;-}-/*** Canonical signature of the ACTIVE embedding configuration for status* reporting. Null when semantic search is disabled. Stored model and@@ -993,6 +1001,21 @@ export interface EmbeddingPhaseTally {export interface EmbeddingPhaseOptions {/** Bypass the configured spend ceiling for this run. */readonly forceCost?: boolean;+ /**+ * Path scope of the pending census. It reaches the census only through+ * the spend plan, so the gate, the refusal and the receipt read the+ * same chunks the phase embeds. Vault-wide when absent.+ */+ readonly scope?: PendingVectorScope;+ /**+ * A spend plan the caller already computed on this store and over the+ * same `scope`; the phase embeds its pending census instead of reading+ * it a second time, so a caller that reported the plan embeds exactly+ * what it priced. Its price and gate verdict are recomputed here from+ * the current config, so a forced or stale verdict is never trusted.+ * Recomputed in full when absent.+ */+ readonly plan?: EmbeddingSpendPlan;readonly safeguard?: import("../brain/safeguard.ts").Safeguard;readonly signal?: AbortSignal;/**@@ -1053,39 +1076,45 @@ export async function runEmbeddingPhase();}- const pending = store.findChunksWithoutEmbeddings();+ // A handed-in plan keeps its census but is priced and gated again here,+ // unforced: the verdict this phase acts on is always its own.+ const plan =+ opts.plan !== undefined+ ? priceSpendPlan(opts.plan.pending, config)+ : planEmbeddingSpend(store, config, opts.scope ? { scope: opts.scope } : {});+ const pending = plan.pending;if (pending.length === 0) return;const provider = makeProvider(config.semantic);const model = config.semantic.model ?? provider.model;- // Cost gate: estimate the spend for the whole pending set up front and- // refuse the run when it exceeds the configured ceiling, unless forced.- // The local provider (price 0) and unknown-price models never block.+ // Cost gate: the shared spend plan estimates the whole pending set up+ // front and the run is refused when it exceeds the configured ceiling,+ // unless forced.//- // The gate is evaluated UNFORCED so the refusal and the receipt read- // one verdict: `blocked && !forceCost` is exactly the blocked flag- // `evaluateCostGate` would have returned for `forced: forceCost`, and- // `blocked && forceCost` is the honest meaning of the receipt's- // `forced` - this run overrode a gate that would have refused it.- const gate = evaluateCostGate({- texts: pending.map((p) => p.content),- model,- gateUsd: config.semantic.costGateUsd,- });- if (gate.blocked && !forceCost) {+ // The plan is evaluated UNFORCED so the refusal and the receipt read+ // one verdict: `blocked && !forceCost` is exactly the blocked flag a+ // forced plan would have returned, and `blocked && forceCost` is the+ // honest meaning of the receipt's `forced` - this run overrode a gate+ // that would have refused it.+ if (plan.gate.blocked && !forceCost) {+ const gateUsd = config.semantic.costGateUsd;+ if (plan.gate.reason === EMBEDDING_GATE_REASON.unpriced) {+ throw new SearchError("EMBEDDING_COST_UNPRICED", unpricedRefusalMessage(plan, gateUsd));+ }throw new SearchError("EMBEDDING_COST_GATE",- `estimated embedding cost $${gate.estimatedUsd.toFixed(4)} for ${pending.length} chunk(s) ` +- `exceeds embedding_cost_gate_usd $${config.semantic.costGateUsd.toFixed(4)}. ` +- `Re-run with --force-cost to proceed or raise the gate.`,+ `estimated embedding cost ${formatEstimatedUsd(plan.estimatedUsd)} for ${pending.length} chunk(s) ` ++ `exceeds ${COST_GATE_KEY} ${formatEstimatedUsd(gateUsd)}. ` ++ `Re-run with ${FORCE_COST_FLAG} to proceed or raise the gate.`,);}stats.spend = {- model,- tokens: gate.tokens,- estimatedUsd: gate.estimatedUsd,- forced: gate.blocked && forceCost,+ model: plan.model,+ tokens: plan.tokens,+ estimatedUsd: plan.estimatedUsd,+ forced: plan.gate.blocked && forceCost,+ priceSource: plan.quote.source,};const batchSize = Math.max(1, config.semantic.batchSize);// Hand the provider a super-batch large enough to keep its internal@@ -1144,22 +1173,26 @@ export interface EmbeddingSpendPreview {/** Chunks the phase's own anti-join still finds vectorless. */readonly pendingChunks: number;readonly tokens: number;- readonly estimatedUsd: number;+ /** Null when nobody stated the model's price. */+ readonly estimatedUsd: number | null;+ /** Who stated the price the estimate used. */+ readonly priceSource: EmbeddingPriceSource;/*** True when the configured gate would refuse this spend unforced - the* fact a `--force-cost` receipt is honest about.*/readonly blocked: boolean;+ /** Why the gate would refuse; null when it would not. */+ readonly reason: EmbeddingGateReason | null;}/*** The spend preview the maintenance lane announces before its reindex* task and records on its journal row after it (t_9d155d0e).*- * Deliberately a READ over the same census and the same cost kernel- * {@link runEmbeddingPhase} gates on - the phase's guards, in its order,- * and its own `findChunksWithoutEmbeddings` anti-join and- * `evaluateCostGate` - so what a run ANNOUNCES and what it later RECORDS+ * Deliberately a READ over the same spend plan {@link runEmbeddingPhase}+ * gates on - the phase's guards, in its order, and `planEmbeddingSpend`+ * (`embedding-spend.ts`) - so what a run ANNOUNCES and what it later RECORDS* cannot drift from what would actually be refused or embedded. This* function never spends: the phase remains the only spender, and a* refusal it would throw stays its to throw.@@ -1184,21 +1217,16 @@ export async function estimatePendingEmbeddingSpend(}try {if (!store.vecLoaded()) return null;- const pending = store.findChunksWithoutEmbeddings();- if (pending.length === 0) return null;- const provider = makeProvider(config.semantic);- const model = config.semantic.model ?? provider.model;- const gate = evaluateCostGate({- texts: pending.map((p) => p.content),- model,- gateUsd: config.semantic.costGateUsd,- });+ const plan = planEmbeddingSpend(store, config);+ if (plan.pending.length === 0) return null;return {- model,- pendingChunks: pending.length,- tokens: gate.tokens,- estimatedUsd: gate.estimatedUsd,- blocked: gate.blocked,+ model: plan.model,+ pendingChunks: plan.pending.length,+ tokens: plan.tokens,+ estimatedUsd: plan.estimatedUsd,+ priceSource: plan.quote.source,+ blocked: plan.gate.blocked,+ reason: plan.gate.reason,};} finally {await store.close();@@ -1533,6 +1561,7 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStembeddingDimension: null,embeddingSignature: activeEmbeddingSignature(config),estimatedRefreshCostUsd: 0,+ refreshPriceSource: null,vecExtension: "unknown" as const,semanticEnabled: config.semantic.enabled,embeddingKeyPresent: !!config.semantic.apiKey,@@ -1635,16 +1664,24 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStwarnings.push(await formatChunkWindowMeasured(chunkWindow));}- // Best-effort spend estimate to bring stale/missing embeddings current.- // Only scan chunk content when the active model is actually priced.- const activeModel = activeEmbeddingModel(config, model);- let estimatedRefreshCostUsd = 0;- if (config.semantic.enabled && pricePerMillionTokens(activeModel) > 0) {- const pending = store.findChunksWithoutEmbeddings();- estimatedRefreshCostUsd = estimateCostUsd(- estimateTokens(pending.map((p) => p.content)),- activeModel,- );+ // Best-effort spend estimate to bring stale/missing embeddings current,+ // from the shared spend plan. Chunk content is scanned only when the+ // price is a positive known rate: a known-free model costs 0. An+ // unknown price is unknown only while chunks are pending - a fully+ // embedded index has nothing to pay for, so it reads 0, counted+ // without loading a chunk body.+ let estimatedRefreshCostUsd: number | null = 0;+ let refreshPriceSource: EmbeddingPriceSource | null = null;+ if (config.semantic.enabled) {+ const { quote } = activeSpendQuote(config);+ refreshPriceSource = quote.source;+ if (quote.usdPerMtok !== null && quote.usdPerMtok > 0) {+ estimatedRefreshCostUsd = planEmbeddingSpend(store, config).estimatedUsd;+ } else if (quote.usdPerMtok === null && store.countChunksWithoutEmbeddings() > 0) {+ estimatedRefreshCostUsd = null;+ } else {+ estimatedRefreshCostUsd = 0;+ }}return Object.freeze({@@ -1659,6 +1696,7 @@ export async function indexStatus(config: ResolvedSearchConfig): Promise<IndexStembeddingDimension: dim,embeddingSignature: activeEmbeddingSignature(config, model, dim),estimatedRefreshCostUsd,+ refreshPriceSource,vecExtension: store.vecLoaded() ? ("loaded" as const) : ("unavailable" as const),semanticEnabled: config.semantic.enabled,embeddingKeyPresent: !!config.semantic.apiKey,@@ -1794,6 +1832,13 @@ export interface IndexCheckOptions {* a call nobody made proves neither.*/readonly probeProvider?: boolean;+ /**+ * The env and provider registry the credential-source report reads+ * names from. Passed explicitly so the report never reads the process+ * on its own; absent, the report is not built and the result carries no+ * `credentialSources` field.+ */+ readonly credentialContext?: CredentialSourceContext;}export async function indexCheck(@@ -1936,6 +1981,13 @@ export async function indexCheck(// Rules match the design doc table; agents and operators read the// list to know what command to run next without learning the// internals of OSB.+ // Names only, and only for a credential-missing tier: every other tier+ // yields null and the result stays byte-identical.+ const credentialSources =+ opts?.credentialContext === undefined+ ? null+ : describeCredentialSources({ semantic: config.semantic, ...opts.credentialContext });+const recommendations = buildRecommendations({config,embeddingKeyResolved,@@ -1964,6 +2016,7 @@ export async function indexCheck(warnings: Object.freeze(warnings),fatal: Object.freeze(fatal),recommendations: Object.freeze(recommendations),+ ...(credentialSources === null ? {} : { credentialSources }),});}@@ -2086,6 +2139,13 @@ function buildRecommendations(input: BuildRecommendationsInput): string[] {);}+ // The price the cost gate reasons from (Honest Embedding Spend): an+ // unpriced active model and a declaration that prices another model.+ // Silent for priced and local models, so their output is unchanged.+ if (input.config.semantic.enabled) {+ recs.push(...embeddingPriceRecommendations(input.config));+ }+if (input.vecExtension === "unavailable") {if (process.platform === "darwin") {recs.push(diff --git a/src/core/search/pipeline/request.ts b/src/core/search/pipeline/request.tsindex d4e11774..4b675141 100644--- a/src/core/search/pipeline/request.ts+++ b/src/core/search/pipeline/request.ts@@ -88,10 +88,22 @@ export interface ResolvedSearchRequest {readonly hybridDeadlineMs: number | null;}-function assertSafePathPrefix(prefix: string | undefined): string | undefined {+/** The argument name a refused prefix is reported under unless the caller names its own. */+const PATH_PREFIX_ARGUMENT = "path_prefix";++/**+ * Refuse a vault-relative path prefix that could reach outside the vault:+ * a `..` segment, an absolute POSIX path or a Windows drive letter. The+ * refusal names the argument and the offending value, so a caller passing+ * several prefixes learns which one was refused.+ */+export function assertSafePathPrefix(+ prefix: string | undefined,+ argument: string = PATH_PREFIX_ARGUMENT,+): string | undefined {if (!prefix) return undefined;if (prefix.includes("..") || prefix.startsWith("/") || /^[A-Za-z]:/.test(prefix)) {- throw new SearchError("INVALID_INPUT", "path_prefix escapes vault");+ throw new SearchError("INVALID_INPUT", `${argument} escapes vault: ${JSON.stringify(prefix)}`);}return prefix;}diff --git a/src/core/search/price-recommendations.ts b/src/core/search/price-recommendations.tsnew file mode 100644index 00000000..4d0ff070--- /dev/null+++ b/src/core/search/price-recommendations.ts@@ -0,0 +1,75 @@+/**+ * `search check` hints about the embedding price (Honest Embedding Spend).+ *+ * Two findings, each one line, each silent when there is nothing to say:+ *+ * - The active model has no known price. The line names the price pair+ * the operator can declare and what the cost gate does meanwhile: a+ * gate of 0 never blocks, so the line says a positive gate would+ * refuse; a positive gate refuses unpriced embedding work now, so the+ * line says that it does.+ * - The declared price pair names a model other than the active one. A+ * stale declaration prices nothing, and nothing else would say so.+ *+ * A table-priced model, an operator-priced model and the local embedder+ * (priced at 0 by the table) produce no line, so their check output stays+ * byte-identical.+ */++import {+ activeSpendQuote,+ COST_GATE_KEY,+ FORCE_COST_FLAG,+ formatEstimatedUsd,+} from "./embedding-spend.ts";+import {+ EMBEDDING_PRICE_MODEL_KEY,+ EMBEDDING_PRICE_RATE_KEY,+ EMBEDDING_PRICE_SOURCE,+} from "./embeddings/pricing.ts";+import { canonicalToken } from "./embeddings/signature.ts";+import type { ResolvedSearchConfig } from "./types.ts";++/** What the cost gate does with an unpriced model under `gateUsd`. */+function gateConsequence(gateUsd: number): string {+ if (gateUsd > 0) {+ return (+ `${COST_GATE_KEY} is ${formatEstimatedUsd(gateUsd)}, so embedding reindexes and ` ++ `vector backfills refuse until the price is declared or the run passes ${FORCE_COST_FLAG}.`+ );+ }+ return (+ `${COST_GATE_KEY} is 0, so nothing is refused; a positive gate would refuse embedding ` ++ `reindexes and vector backfills on this model until its price is declared.`+ );+}++/**+ * The price hints for the model an embedding pass would name right now.+ * The model and its quote come from `activeSpendQuote`, the resolution+ * the spend plan uses, so the hint prices the model the gate prices. A+ * configuration that names no model has nothing to price.+ */+export function embeddingPriceRecommendations(config: ResolvedSearchConfig): string[] {+ const { model, quote } = activeSpendQuote(config);+ if (model === null) return [];+ const recs: string[] = [];+ if (quote.source === EMBEDDING_PRICE_SOURCE.unknown) {+ recs.push(+ `No price is known for embedding model "${model}", so its spend is reported as unknown. ` ++ `Declare it with ${EMBEDDING_PRICE_MODEL_KEY}: ${model} and ` ++ `${EMBEDDING_PRICE_RATE_KEY}: <USD per million tokens> (0 declares the model free). ` ++ `Meanwhile ${gateConsequence(config.semantic.costGateUsd)}`,+ );+ }+ const declared = config.semantic.priceOverride;+ if (declared !== undefined && canonicalToken(declared.model) !== canonicalToken(model)) {+ recs.push(+ `The declared embedding price names model "${declared.model}", but the active embedding ` ++ `model is "${model}", so the declaration prices nothing. Point ` ++ `${EMBEDDING_PRICE_MODEL_KEY} at the active model or remove both ` ++ `${EMBEDDING_PRICE_MODEL_KEY} and ${EMBEDDING_PRICE_RATE_KEY}.`,+ );+ }+ return recs;+}diff --git a/src/core/search/search-error.ts b/src/core/search/search-error.tsindex 3b7932ba..a2f1a5f8 100644--- a/src/core/search/search-error.ts+++ b/src/core/search/search-error.ts@@ -22,7 +22,9 @@ export const SEARCH_ERROR_CODES = ["EMBEDDING_DIMENSION_MISMATCH","EMBEDDING_INVALID_VECTOR","EMBEDDING_COST_GATE",+ "EMBEDDING_COST_UNPRICED","EMBEDDING_QUOTA_EXHAUSTED",+ "BELIEF_VECTORS_MISSING","RERANK_PROVIDER_HTTP","INDEX_LOCKED","INVALID_INPUT",diff --git a/src/core/search/semantic-phase.ts b/src/core/search/semantic-phase.tsindex bf970c8e..fdf956af 100644--- a/src/core/search/semantic-phase.ts+++ b/src/core/search/semantic-phase.ts@@ -6,9 +6,9 @@*/import {+ BLOCKED_TIER_ERROR_CODE,+ isBlockedCapability,resolveSemanticCapability,- SEMANTIC_CAPABILITY_TIER,- semanticCapabilityIsBlocked,semanticCapabilityLabel,} from "./capability-tier.ts";import { classifyEmbeddingError } from "./embeddings/openai-compat.ts";@@ -16,9 +16,8 @@ import { makeProvider } from "./embeddings/provider.ts";import { RETRIEVAL_DEGRADATION, noteDegradation } from "./retrieval-trail.ts";import { Store } from "./store.ts";import { EMBEDDING_QUOTA_MESSAGE, SearchError } from "./types.ts";-import type { SemanticCapability, SemanticCapabilityTier } from "./capability-tier.ts";import type { RetrievalDegradationSink } from "./retrieval-trail.ts";-import type { ResolvedSearchConfig, SearchErrorCode, SearchOptions } from "./types.ts";+import type { ResolvedSearchConfig, SearchOptions } from "./types.ts";export interface SemanticPolicy {/** caller asked for semantic on or off (true), or accepted the default (false). */@@ -53,41 +52,6 @@ export function semanticPoolSize(limit: number): number {return Math.max(limit * POOL_OVERFETCH, POOL_FLOOR);}-/** The rungs of the capability ladder that BLOCK; `configured` is not one. */-type BlockedCapabilityTier = Exclude<- SemanticCapabilityTier,- typeof SEMANTIC_CAPABILITY_TIER.configured->;--/**- * The typed failure an EXPLICIT semantic request reports per blocked rung.- *- * One entry per rung, so adding a rung to the ladder is a type error here- * rather than a request that silently degrades. `credential-missing` keeps- * `EMBEDDING_KEY_MISSING`, the code that arm has always raised; `disabled`- * raises `EMBEDDING_DISABLED`, the code the null provider already raises- * for the same configuration when the indexer meets it, so one condition- * has one code across the tree.- */-const BLOCKED_TIER_ERROR_CODE: Readonly<Record<BlockedCapabilityTier, SearchErrorCode>> =- Object.freeze({- [SEMANTIC_CAPABILITY_TIER.disabled]: "EMBEDDING_DISABLED",- [SEMANTIC_CAPABILITY_TIER.credentialMissing]: "EMBEDDING_KEY_MISSING",- });--/**- * {@link semanticCapabilityIsBlocked} as a TYPE predicate. The shared- * resolver exports it as a plain boolean and lives in a module this lane- * does not own, so the narrowing lives here - which lets the table above be- * total over exactly the blocked rungs, with no cast and no unreachable- * default arm.- */-function isBlockedCapability(- capability: SemanticCapability,-): capability is SemanticCapability & { readonly tier: BlockedCapabilityTier } {- return semanticCapabilityIsBlocked(capability);-}-interface SemanticPhaseOutcome {readonly attempted: boolean;readonly hits: ReturnType<Store["semanticTopK"]>;diff --git a/src/core/search/serialize.ts b/src/core/search/serialize.tsindex 58a71a91..b93b6267 100644--- a/src/core/search/serialize.ts+++ b/src/core/search/serialize.ts@@ -174,6 +174,7 @@ export function serializeIndexStatus(s: IndexStatusSnapshot): Record<string, unkembedding_dimension: s.embeddingDimension,embedding_signature: s.embeddingSignature,estimated_refresh_cost_usd: s.estimatedRefreshCostUsd,+ refresh_price_source: s.refreshPriceSource,vec_extension: s.vecExtension,semantic_enabled: s.semanticEnabled,embedding_key_present: s.embeddingKeyPresent,diff --git a/src/core/search/store.ts b/src/core/search/store.tsindex d2d277dd..11c302e3 100644--- a/src/core/search/store.ts+++ b/src/core/search/store.ts@@ -412,9 +412,21 @@ export class Store {}/**- * Atomically replace every chunk for a document. Old vec rows are- * removed first; FTS5 stays in sync via the chunks_ai/ad/au triggers.- * Returns the new chunk ids in `chunkIndex` order.+ * Atomically replace every chunk for a document, carrying the stored+ * vector of every unchanged chunk (same content hash, recorded model+ * and dimension) and purging only the rest. FTS5 stays in sync via the+ * chunks_ai/ad/au triggers. Returns the new ids and the carried count.+ */+ replaceDocumentChunks(+ documentId: number,+ input: ReadonlyArray<chunks.ChunkInput>,+ ): chunks.ChunkReplacement {+ return chunks.replaceDocumentChunks(this.db, this.vecExtensionLoaded, documentId, input);+ }++ /**+ * {@link Store.replaceDocumentChunks}, returning only the new chunk+ * ids in `chunkIndex` order.*/replaceChunks(documentId: number, input: ReadonlyArray<chunks.ChunkInput>): number[] {return chunks.replaceChunks(this.db, this.vecExtensionLoaded, documentId, input);@@ -453,10 +465,23 @@ export class Store {/*** Chunks that have no row in `embeddings`. Used by the indexer to- * populate vectors after a fresh index or after the model-change drop.+ * populate vectors after a fresh index or after the model-change drop,+ * optionally scoped to path prefixes.*/- findChunksWithoutEmbeddings(): Array<{ chunkId: number; content: string }> {- return chunks.findChunksWithoutEmbeddings(this.db);+ findChunksWithoutEmbeddings(+ scope?: chunks.PendingVectorScope,+ ): Array<{ chunkId: number; content: string }> {+ return chunks.findChunksWithoutEmbeddings(this.db, scope);+ }++ /** How many chunks the scoped census above would return, without their bodies. */+ countChunksWithoutEmbeddings(scope?: chunks.PendingVectorScope): number {+ return chunks.countChunksWithoutEmbeddings(this.db, scope);+ }++ /** How many documents sit under one path prefix of a pending-vector scope. */+ countDocumentsUnderPrefix(prefix: string): number {+ return documents.countDocumentsUnderPrefix(this.db, prefix);}// ── embeddings ─────────────────────────────────────────────────────────────@@ -497,6 +522,11 @@ export class Store {return vectors.embeddingForChunk(this.db, this.vecExtensionLoaded, chunkId);}+ /** Every stored vector of a document's chunks with its recorded model and dimension. */+ storedEmbeddingsForDocument(documentId: number): vectors.StoredChunkEmbedding[] {+ return vectors.storedEmbeddingsForDocument(this.db, this.vecExtensionLoaded, documentId);+ }+getEmbeddingHash(chunkId: number): string | null {return vectors.getEmbeddingHash(this.db, chunkId);}diff --git a/src/core/search/store/chunks.ts b/src/core/search/store/chunks.tsindex f583e882..c207b9a3 100644--- a/src/core/search/store/chunks.ts+++ b/src/core/search/store/chunks.ts@@ -9,7 +9,12 @@ import { Database } from "bun:sqlite";import { SearchError } from "../types.ts";import { nowIso, sqlPlaceholders } from "./sql.ts";-import { purgeVecRowsByChunkIds, purgeVecRowsForDocument } from "./vectors.ts";+import {+ matchCarriedVectors,+ readCarryCandidates,+ restoreCarriedVectors,+} from "./vector-carry-over.ts";+import { purgeVecRowsByChunkIds } from "./vectors.ts";export interface ChunkInput {readonly chunkIndex: number;@@ -119,21 +124,47 @@ export function getChunksByDocument(db: Database, documentId: number): ChunkRow[}));}+/** What one document's chunk replacement wrote. */+export interface ChunkReplacement {+ /** The new chunk ids, in `chunkIndex` order. */+ readonly chunkIds: number[];+ /**+ * New chunks that kept a stored vector of an old chunk with the same+ * `content_hash` (vector carry-over), so the embedding phase will not+ * pay for them again.+ */+ readonly embeddingsReused: number;+}+/**- * Atomically replace every chunk for a document. Old vec rows are- * removed first; FTS5 stays in sync via the chunks_ai/ad/au triggers.- * Returns the new chunk ids in `chunkIndex` order.+ * Atomically replace every chunk for a document, carrying the stored+ * vector of every new chunk whose content an old chunk of this document+ * already embedded under the recorded model and dimension (the rule and+ * its guard live in `vector-carry-over.ts`). Only the vec rows that are+ * not carried are purged; the old `chunk_vec_map` and `embeddings` rows+ * go with their chunks through the FK cascade, and the carried ones are+ * re-attached to the new ids. FTS5 stays in sync via the+ * chunks_ai/ad/au triggers.*/-export function replaceChunks(+export function replaceDocumentChunks(db: Database,vecLoaded: boolean,documentId: number,chunks: ReadonlyArray<ChunkInput>,-): number[] {+): ChunkReplacement {const ids: number[] = [];+ let embeddingsReused = 0;db.exec("BEGIN");try {- purgeVecRowsForDocument(db, vecLoaded, documentId);+ const carried = matchCarriedVectors(+ readCarryCandidates(db, vecLoaded, documentId),+ chunks.map((c) => c.contentHash),+ );+ const keptChunkIds = new Set(carried.map((m) => m.candidate.chunkId));+ const purged = chunksForDocument(db, documentId)+ .map((c) => c.id)+ .filter((id) => !keptChunkIds.has(id));+ purgeVecRowsByChunkIds(db, vecLoaded, purged);db.run("DELETE FROM chunks WHERE document_id = ?", [documentId]);const insert = db.prepare<{ id: number },@@ -160,12 +191,27 @@ export function replaceChunks(if (!row) throw new SearchError("INDEX_UNREADABLE", "chunk insert returned no id");ids.push(row.id);}+ restoreCarriedVectors(db, ids, carried);+ embeddingsReused = carried.length;db.exec("COMMIT");} catch (e) {db.exec("ROLLBACK");throw e;}- return ids;+ return { chunkIds: ids, embeddingsReused };+}++/**+ * {@link replaceDocumentChunks} for callers that need only the new ids+ * (in `chunkIndex` order). The carry-over applies all the same.+ */+export function replaceChunks(+ db: Database,+ vecLoaded: boolean,+ documentId: number,+ chunks: ReadonlyArray<ChunkInput>,+): number[] {+ return replaceDocumentChunks(db, vecLoaded, documentId, chunks).chunkIds;}/**@@ -207,20 +253,55 @@ export function countChunks(db: Database): number {return db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM chunks").get()?.n ?? 0;}+/**+ * Optional path scope of the pending-vector census: only chunks whose+ * document path starts with one of the prefixes. An absent or empty list+ * means vault-wide, and then the census SQL is exactly the unscoped form.+ */+export interface PendingVectorScope {+ readonly pathPrefixes?: ReadonlyArray<string>;+}++const PENDING_ANTI_JOIN =+ "FROM chunks c LEFT JOIN embeddings e ON e.chunk_id = c.id WHERE e.chunk_id IS NULL";++/**+ * The anti-join's FROM/WHERE text and bindings for a scope. A scoped+ * census joins `documents` and matches each prefix with the+ * `substr(path, 1, length(?)) = ?` form the vector prefix filter uses,+ * which needs no LIKE wildcard escaping.+ */+function pendingCensusClause(scope: PendingVectorScope | undefined): {+ sql: string;+ bindings: string[];+} {+ const prefixes = scope?.pathPrefixes ?? [];+ if (prefixes.length === 0) return { sql: PENDING_ANTI_JOIN, bindings: [] };+ const matches = prefixes.map(() => "substr(d.path, 1, length(?)) = ?").join(" OR ");+ return {+ sql:+ "FROM chunks c JOIN documents d ON d.id = c.document_id " ++ "LEFT JOIN embeddings e ON e.chunk_id = c.id " ++ `WHERE e.chunk_id IS NULL AND (${matches})`,+ bindings: prefixes.flatMap((prefix) => [prefix, prefix]),+ };+}+/*** Chunks that have no row in `embeddings`. Used by the indexer to- * populate vectors after a fresh index or after the model-change drop.+ * populate vectors after a fresh index or after the model-change drop,+ * optionally scoped to path prefixes.*/export function findChunksWithoutEmbeddings(db: Database,+ scope?: PendingVectorScope,): Array<{ chunkId: number; content: string }> {+ const { sql, bindings } = pendingCensusClause(scope);const rows = db- .query<{ id: number; content: string }, []>(- "SELECT c.id AS id, c.content AS content FROM chunks c " +- "LEFT JOIN embeddings e ON e.chunk_id = c.id " +- "WHERE e.chunk_id IS NULL ORDER BY c.id",+ .query<{ id: number; content: string }, string[]>(+ `SELECT c.id AS id, c.content AS content ${sql} ORDER BY c.id`,)- .all();+ .all(...bindings);return rows.map((r) => ({ chunkId: r.id, content: r.content }));}@@ -228,23 +309,16 @@ export function findChunksWithoutEmbeddings(* How MANY chunks have no row in `embeddings`, without materialising a* single one of their bodies.*- * The same anti-join {@link findChunksWithoutEmbeddings} walks, as a- * `COUNT(*)`. The row-returning form is the indexer's work queue and- * loads every pending chunk's full `content` into JS memory to hand it- * to a provider; a diagnostic that only wants the number must not pay- * that, which is the whole reason this second query exists rather than- * a `.length` on the first.+ * The same anti-join {@link findChunksWithoutEmbeddings} walks, under the+ * same scope, as a `COUNT(*)`. The row-returning form is the indexer's+ * work queue and loads every pending chunk's full `content` into JS+ * memory to hand it to a provider; a diagnostic that only wants the+ * number must not pay that, which is the whole reason this second query+ * exists rather than a `.length` on the first.*/-export function countChunksWithoutEmbeddings(db: Database): number {- return (- db- .query<{ n: number }, []>(- "SELECT COUNT(*) AS n FROM chunks c " +- "LEFT JOIN embeddings e ON e.chunk_id = c.id " +- "WHERE e.chunk_id IS NULL",- )- .get()?.n ?? 0- );+export function countChunksWithoutEmbeddings(db: Database, scope?: PendingVectorScope): number {+ const { sql, bindings } = pendingCensusClause(scope);+ return db.query<{ n: number }, string[]>(`SELECT COUNT(*) AS n ${sql}`).get(...bindings)?.n ?? 0;}export function hydrateChunks(diff --git a/src/core/search/store/documents.ts b/src/core/search/store/documents.tsindex 69383f68..0cb6277e 100644--- a/src/core/search/store/documents.ts+++ b/src/core/search/store/documents.ts@@ -455,3 +455,17 @@ export function documentTitles(export function countDocuments(db: Database): number {return db.query<{ c: number }, []>("SELECT count(*) AS c FROM documents").get()?.c ?? 0;}++/**+ * How many documents sit under one raw path prefix, matched with the+ * same `substr` form the pending-vector census uses so the two agree.+ */+export function countDocumentsUnderPrefix(db: Database, prefix: string): number {+ return (+ db+ .query<{ c: number }, [string, string]>(+ "SELECT count(*) AS c FROM documents WHERE substr(path, 1, length(?)) = ?",+ )+ .get(prefix, prefix)?.c ?? 0+ );+}diff --git a/src/core/search/store/vector-carry-over.ts b/src/core/search/store/vector-carry-over.tsnew file mode 100644index 00000000..60cfef5f--- /dev/null+++ b/src/core/search/store/vector-carry-over.ts@@ -0,0 +1,167 @@+/**+ * Vector carry-over on edit (t_82c3b275): the rule that lets a chunk+ * replacement keep the stored vector of a chunk whose text did not+ * change, so an edit to one paragraph re-embeds, and re-pays for, that+ * paragraph alone.+ *+ * The key is `content_hash` within one document. The embedded text is+ * exactly `chunks.content` and `content_hash = sha256(content)`, so two+ * chunks with one hash embed to one vector. The query and passage+ * prefixes are store-level, and any change to them already clears every+ * vector through `ensureEmbeddingModel`, so they need no place in the key.+ *+ * The guard is provenance: an old row is carried only when its+ * `embeddings.model` and `dimension` equal the identity `index_state`+ * records. When that identity is unrecorded, nothing is carried - there+ * is no recorded claim to vouch for the old rows, and a carried vector+ * that does not belong to the active model would be silent garbage.+ */++import { Database } from "bun:sqlite";++import { SearchError } from "../types.ts";+import { EMBEDDING_DIMENSION_STATE_KEY, EMBEDDING_MODEL_STATE_KEY, getState } from "./state.ts";+import { nowIso } from "./sql.ts";++/** One stored vector of the document being replaced, eligible to be carried. */+export interface CarryCandidate {+ /** The old chunk id the vector is attached to before the replacement. */+ readonly chunkId: number;+ readonly contentHash: string;+ readonly vecRowid: number;+ readonly model: string;+ readonly dimension: number;+ readonly embeddingHash: string;+ readonly createdAt: string;+}++/** A new chunk position paired with the old vector it keeps. */+export interface CarriedVector<C> {+ /** Index into the replacement chunk list, not `chunk_index`. */+ readonly position: number;+ readonly candidate: C;+}++/**+ * Pair each new chunk hash with an unused old candidate of the same+ * hash. Duplicated hashes match as a multiset: the k-th occurrence of a+ * hash in the new list takes the k-th old candidate of that hash, both+ * in list order (the caller hands candidates in `chunk_index` order).+ * Pure, so the pairing rule is testable without a store.+ */+export function matchCarriedVectors<C extends { readonly contentHash: string }>(+ candidates: ReadonlyArray<C>,+ newHashes: ReadonlyArray<string>,+): Array<CarriedVector<C>> {+ const queues = new Map<string, C[]>();+ for (const candidate of candidates) {+ const queue = queues.get(candidate.contentHash);+ if (queue === undefined) queues.set(candidate.contentHash, [candidate]);+ else queue.push(candidate);+ }+ const matches: Array<CarriedVector<C>> = [];+ newHashes.forEach((hash, position) => {+ const candidate = queues.get(hash)?.shift();+ if (candidate !== undefined) matches.push({ position, candidate });+ });+ return matches;+}++/** The embedding identity `index_state` records, or null when either half is absent. */+function recordedIdentity(db: Database): { model: string; dimension: number } | null {+ const model = getState(db, EMBEDDING_MODEL_STATE_KEY);+ const rawDimension = getState(db, EMBEDDING_DIMENSION_STATE_KEY);+ if (model === null || rawDimension === null) return null;+ const dimension = Number(rawDimension);+ if (!Number.isInteger(dimension) || dimension <= 0) return null;+ return { model, dimension };+}++/**+ * The stored vectors of one document that a replacement may carry, in+ * `chunk_index` order. Empty when sqlite-vec is not loaded (there is no+ * vector table to keep rows in, and the replacement behaves as it always+ * did), when the identity is unrecorded, and for every row whose model+ * or dimension differs from the recorded identity.+ */+export function readCarryCandidates(+ db: Database,+ vecLoaded: boolean,+ documentId: number,+): CarryCandidate[] {+ if (!vecLoaded) return [];+ const identity = recordedIdentity(db);+ if (identity === null) return [];+ return db+ .query<+ {+ chunk_id: number;+ content_hash: string;+ vec_rowid: number;+ model: string;+ dimension: number;+ embedding_hash: string;+ created_at: string;+ },+ [number, string, number]+ >(+ "SELECT c.id AS chunk_id, c.content_hash AS content_hash, m.vec_rowid AS vec_rowid, e.model AS model, " ++ " e.dimension AS dimension, e.embedding_hash AS embedding_hash, e.created_at AS created_at " ++ "FROM chunks c " ++ "JOIN chunk_vec_map m ON m.chunk_id = c.id " ++ "JOIN embeddings e ON e.chunk_id = c.id " ++ "WHERE c.document_id = ? AND e.model = ? AND e.dimension = ? " ++ "ORDER BY c.chunk_index",+ )+ .all(documentId, identity.model, identity.dimension)+ .map((r) => ({+ chunkId: r.chunk_id,+ contentHash: r.content_hash,+ vecRowid: r.vec_rowid,+ model: r.model,+ dimension: r.dimension,+ embeddingHash: r.embedding_hash,+ createdAt: r.created_at,+ }));+}++/**+ * Re-attach carried vectors to the new chunk ids: one `chunk_vec_map`+ * row pointing at the kept `chunk_vec` row, and one `embeddings` row+ * copying the old provenance. `created_at` keeps the instant the vector+ * was computed; `updated_at` records the carry. Runs inside the+ * caller's transaction.+ */+export function restoreCarriedVectors(+ db: Database,+ newChunkIds: ReadonlyArray<number>,+ matches: ReadonlyArray<CarriedVector<CarryCandidate>>,+): void {+ if (matches.length === 0) return;+ const insertMap = db.prepare<unknown, [number, number]>(+ "INSERT INTO chunk_vec_map(chunk_id, vec_rowid) VALUES (?, ?)",+ );+ const insertEmbedding = db.prepare<unknown, [number, string, number, string, string, string]>(+ "INSERT INTO embeddings(chunk_id, model, dimension, embedding_hash, created_at, updated_at) " ++ "VALUES (?, ?, ?, ?, ?, ?)",+ );+ const now = nowIso();+ for (const { position, candidate } of matches) {+ const chunkId = newChunkIds[position];+ if (chunkId === undefined) {+ throw new SearchError(+ "INDEX_UNREADABLE",+ `carried vector position ${position} has no new chunk id`,+ );+ }+ insertMap.run(chunkId, candidate.vecRowid);+ insertEmbedding.run(+ chunkId,+ candidate.model,+ candidate.dimension,+ candidate.embeddingHash,+ candidate.createdAt,+ now,+ );+ }+}diff --git a/src/core/search/store/vectors.ts b/src/core/search/store/vectors.tsindex 8c303822..f1952701 100644--- a/src/core/search/store/vectors.ts+++ b/src/core/search/store/vectors.ts@@ -11,9 +11,9 @@import { Database } from "bun:sqlite";import {+ BLOCKED_TIER_ERROR_CODE,+ isBlockedCapability,resolveSemanticCapability,- SEMANTIC_CAPABILITY_TIER,- type SemanticCapabilityTier,} from "../capability-tier.ts";import { LOCAL_EMBEDDING_MODEL } from "../embeddings/signature.ts";import { dropVecTable, ensureVecTable } from "../schema.ts";@@ -232,7 +232,11 @@ export function embeddingForChunk().get(chunkId);if (!row) return null;- const bytes = row.embedding;+ return vectorFromBlob(row.embedding);+}++/** Copy a stored vec0 blob into a `Float32Array`. */+function vectorFromBlob(bytes: Uint8Array): Float32Array {// Copy instead of viewing: a pooled buffer with a non-4-byte-aligned// byteOffset would make the Float32Array constructor throw.return new Float32Array(@@ -240,6 +244,50 @@ export function embeddingForChunk();}+/** One stored chunk vector with the identity its `embeddings` row recorded. */+export interface StoredChunkEmbedding {+ readonly chunkId: number;+ readonly vector: Float32Array;+ readonly model: string;+ readonly dimension: number;+}++/**+ * Every stored vector of one document's chunks, in `chunk_index` order,+ * each with the model and dimension recorded when it was written. A+ * reader that scores against a fresh query vector needs the identity per+ * row: a vector left by an older model is not comparable and must be+ * told apart from a current one, which {@link embeddingForChunk} cannot.+ */+export function storedEmbeddingsForDocument(+ db: Database,+ vecLoaded: boolean,+ documentId: number,+): StoredChunkEmbedding[] {+ if (!vecLoaded) {+ throw new SearchError(+ "VEC_EXTENSION_UNAVAILABLE",+ "sqlite-vec extension not loaded; cannot read stored embeddings",+ );+ }+ return db+ .query<{ chunk_id: number; embedding: Uint8Array; model: string; dimension: number }, [number]>(+ "SELECT c.id AS chunk_id, v.embedding AS embedding, e.model AS model, " ++ "e.dimension AS dimension FROM chunks c " ++ "JOIN chunk_vec_map m ON m.chunk_id = c.id " ++ "JOIN chunk_vec v ON v.rowid = m.vec_rowid " ++ "JOIN embeddings e ON e.chunk_id = c.id " ++ "WHERE c.document_id = ? ORDER BY c.chunk_index",+ )+ .all(documentId)+ .map((r) => ({+ chunkId: r.chunk_id,+ vector: vectorFromBlob(r.embedding),+ model: r.model,+ dimension: r.dimension,+ }));+}+export function getEmbeddingHash(db: Database, chunkId: number): string | null {const row = db.query<{ embedding_hash: string }, [number]>(@@ -318,19 +366,6 @@ export interface EmbeddingRebuildGate {readonly semantic: ResolvedEmbeddingConfig;}-/**- * The typed error each blocked capability tier refuses with. Both codes- * pre-exist in the closed `SEARCH_ERROR_CODES` union and carry exactly- * these meanings: a `disabled` configuration computes no embeddings at- * all, and a `credential-missing` one cannot reach its provider.- */-const BLOCKED_TIER_ERROR_CODE: Readonly<- Record<Exclude<SemanticCapabilityTier, "configured">, SearchErrorCode>-> = Object.freeze({- [SEMANTIC_CAPABILITY_TIER.disabled]: "EMBEDDING_DISABLED",- [SEMANTIC_CAPABILITY_TIER.credentialMissing]: "EMBEDDING_KEY_MISSING",-});-/*** Whether any `chunks` rows exist - the source-material half of the* verify-before-replace gate.@@ -381,7 +416,7 @@ function rebuildRefusalBeforeClear(const stored = countEmbeddings(db);if (stored === 0) return null;const capability = resolveSemanticCapability(gate.semantic);- if (capability.tier === SEMANTIC_CAPABILITY_TIER.configured) return null;+ if (!isBlockedCapability(capability)) return null;const material = hasChunkRows(db) ? "" : "; no chunk material remains to rebuild from";return {code: BLOCKED_TIER_ERROR_CODE[capability.tier],diff --git a/src/core/search/types.ts b/src/core/search/types.tsindex 2dabb246..c1033e06 100644--- a/src/core/search/types.ts+++ b/src/core/search/types.ts@@ -21,6 +21,7 @@ import type { StampMismatch } from "../integrity/stamp.ts";import type { ReconciliationOutcome, ReconciliationReport } from "../reconciliation-report.ts";import type { VaultPathRule, VaultScopeRules } from "../vault-scope/defaults.ts";import type { MaintenanceSpendReceipt } from "../brain/maintenance/journal.ts";+import type { EmbeddingPriceOverride, EmbeddingPriceSource } from "./embeddings/pricing.ts";import type { DegreePredicate } from "./property-filter.ts";import type { TemporalIntent } from "./temporal-intent.ts";import type { FtsMatchMode } from "./fts-match-mode.ts";@@ -237,6 +238,11 @@ export interface IndexStats {readonly deleted: number;readonly chunksTotal: number;readonly embeddingsComputed: number;+ /**+ * Stored vectors this run kept for chunks whose content did not change+ * across an edit (vector carry-over), and therefore did not re-embed.+ */+ readonly embeddingsReused: number;readonly embeddingsRetries: number;readonly errors: ReadonlyArray<{readonly path: string;@@ -376,10 +382,13 @@ export interface IndexStatusSnapshot {readonly embeddingSignature: string | null;/*** Best-effort USD estimate to (re-)embed the chunks that currently- * lack a current embedding, at the active model's rate. 0 for the- * local/unknown-price case.+ * lack a current embedding, at the active model's quoted rate, from the+ * shared spend plan. 0 for a known-free model, semantic search off or+ * no index; null when nobody stated the model's price.*/- readonly estimatedRefreshCostUsd: number;+ readonly estimatedRefreshCostUsd: number | null;+ /** Who stated the price of the refresh estimate; null when semantic search is off or there is no index. */+ readonly refreshPriceSource: EmbeddingPriceSource | null;readonly vecExtension: VecExtensionState;readonly semanticEnabled: boolean;readonly embeddingKeyPresent: boolean;@@ -598,6 +607,21 @@ export interface IndexCheckReport {* key so headless callers (Hermes cron, CI) can act on them.*/readonly recommendations: ReadonlyArray<string>;+ /**+ * Where the resolver looked for an embedding credential, by name only+ * (Honest Embedding Spend). Present only on a `credential-missing` tier+ * and only when the caller handed the check a credential context, so a+ * configured or disabled setup reports byte-identically.+ */+ readonly credentialSources?: CredentialSourceReport;+}++/** The names a `credential-missing` check consulted, and where a key does exist. */+export interface CredentialSourceReport {+ /** Credential sources consulted, in probe order. */+ readonly consulted: ReadonlyArray<string>;+ /** Other registered profiles whose env key is present, in registry order. */+ readonly presentElsewhere: ReadonlyArray<string>;}/**@@ -1238,6 +1262,13 @@ export interface ResolvedEmbeddingConfig {* run whose estimated cost exceeds this is refused unless forced.*/readonly costGateUsd: number;+ /**+ * The operator's declared price for one named model (Honest Embedding+ * Spend), from `embedding_price_model` + `embedding_price_usd_per_mtok`.+ * Absent when neither key is set. Price is never part of the embedding+ * identity, so declaring or editing it never triggers a reindex.+ */+ readonly priceOverride?: EmbeddingPriceOverride;/*** Active instruction prefix for a search query* (memory-write-path-integrity B2). Resolved from the preset default anddiff --git a/src/core/search/vector-backfill.ts b/src/core/search/vector-backfill.tsindex 4a4934c8..4231c992 100644--- a/src/core/search/vector-backfill.ts+++ b/src/core/search/vector-backfill.ts@@ -36,10 +36,15 @@*/import { resolveSemanticCapability, type SemanticCapability } from "./capability-tier.ts";-import { estimateCostUsd, estimateTokens, pricePerMillionTokens } from "./embeddings/signature.ts";-import { runEmbeddingPhase } from "./indexer.ts";+import { planEmbeddingSpend, type EmbeddingGateReason } from "./embedding-spend.ts";+import type { EmbeddingPriceSource } from "./embeddings/pricing.ts";+import { runEmbeddingPhase, type EmbeddingPhaseTally } from "./indexer.ts";+import { assertSafePathPrefix } from "./pipeline/request.ts";+import { SearchError } from "./search-error.ts";import { Store } from "./store.ts";+import type { PendingVectorScope } from "./store/chunks.ts";import type { ResolvedSearchConfig } from "./types.ts";+import type { MaintenanceSpendReceipt } from "../brain/maintenance/journal.ts";import {OPERATION,progressCounter,@@ -52,6 +57,13 @@ export interface VectorBackfillOptions {readonly apply?: boolean;/** Bypass the configured spend ceiling for this run. */readonly forceCost?: boolean;+ /**+ * Vault-relative path prefixes the run is limited to. Each one is+ * validated by {@link assertSafePathPrefix}; the census, the estimate,+ * the gate and the receipt all read this scope. Empty or absent means+ * the whole vault.+ */+ readonly pathPrefixes?: ReadonlyArray<string>;readonly safeguard?: import("../brain/safeguard.ts").Safeguard;readonly signal?: AbortSignal;/**@@ -77,14 +89,70 @@ export interface VectorBackfillResult {/** Provider retries this run consumed (0 on a dry run). */readonly retries: number;/**- * Estimated spend for the pending set, from the estimator that already- * governs the indexer's cost gate. `0` when the model carries no known- * price - which is an absent price, not a free run, and the human- * report says so rather than printing `$0.0000`.+ * Estimated spend for the pending set, from the shared spend plan the+ * indexer's cost gate reads. Null when nobody stated the model's price+ * - an absent price, not a free run, and the report says so rather+ * than printing `$0.0000`.*/- readonly estimatedCostUsd: number;- /** True when {@link estimatedCostUsd} could be derived at all. */- readonly costKnown: boolean;+ readonly estimatedCostUsd: number | null;+ /** Who stated the price the estimate used. */+ readonly priceSource: EmbeddingPriceSource;+ /** True when the configured gate would refuse this spend unforced. */+ readonly blocked: boolean;+ /** Why the gate would refuse; null when it would not. */+ readonly reason: EmbeddingGateReason | null;+ /**+ * The prefixes the run was limited to, normalised (no leading `./`,+ * `/` separators); empty for the whole vault.+ */+ readonly pathPrefixes: ReadonlyArray<string>;+ /**+ * The prefixes that match no indexed document. A scope that matches+ * nothing has nothing pending, and without this list it would read+ * exactly like a fully embedded scope.+ */+ readonly unmatchedPathPrefixes: ReadonlyArray<string>;+ /**+ * The spend receipt of an applied run that reached the provider, from+ * the same scoped plan the dry run reports; null otherwise.+ */+ readonly spend: MaintenanceSpendReceipt | null;+}++/** The argument name an unsafe prefix is refused under. */+const PATH_PREFIX_ARGUMENT = "path prefix";++/**+ * The spelling a prefix is matched under: Windows separators become `/`+ * and a leading `./` is dropped, so `./Brain/` and `Brain\preferences\` match the+ * stored `Brain/...` paths. Matching itself stays a raw string prefix.+ */+function normalisePathPrefix(prefix: string): string {+ let normal = prefix.replaceAll("\\", "/");+ while (normal.startsWith("./")) normal = normal.slice(2);+ return normal;+}++/**+ * Normalise and validate every prefix by name; an empty list scopes+ * nothing. An empty or blank prefix is refused rather than skipped: it+ * would match every document and widen a scoped run to the whole vault+ * while the report still echoed a scope.+ */+function normalisePathPrefixes(pathPrefixes: ReadonlyArray<string>): ReadonlyArray<string> {+ return Object.freeze(+ pathPrefixes.map((raw) => {+ const prefix = normalisePathPrefix(raw);+ if (prefix.trim() === "") {+ throw new SearchError(+ "INVALID_INPUT",+ `${PATH_PREFIX_ARGUMENT} is empty: ${JSON.stringify(raw)}`,+ );+ }+ assertSafePathPrefix(prefix, PATH_PREFIX_ARGUMENT);+ return prefix;+ }),+ );}/**@@ -113,19 +181,24 @@ async function planVectorBackfillRun(progress: ProgressCounter,): Promise<VectorBackfillResult> {const apply = opts.apply === true;+ const pathPrefixes = normalisePathPrefixes(opts.pathPrefixes ?? []);+ const scope: PendingVectorScope | undefined =+ pathPrefixes.length > 0 ? { pathPrefixes } : undefined;const capability = resolveSemanticCapability(config.semantic);const store = await Store.open(config, { mode: apply ? "write" : "read" });try {- const pendingChunks = store.findChunksWithoutEmbeddings();+ const plan = planEmbeddingSpend(store, config, scope ? { scope } : {});const chunksTotal = store.counts().chunks;- const model = config.semantic.model;- const tokens = estimateTokens(pendingChunks.map((c) => c.content));- const estimatedCostUsd = estimateCostUsd(tokens, model);- const tally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ const unmatchedPathPrefixes = Object.freeze(+ pathPrefixes.filter((prefix) => store.countDocumentsUnderPrefix(prefix) === 0),+ );+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };- if (apply && pendingChunks.length > 0) {+ if (apply && plan.pending.length > 0) {await runEmbeddingPhase(store, config, tally, {forceCost: opts.forceCost === true,+ ...(scope ? { scope } : {}),+ plan,...(opts.safeguard !== undefined ? { safeguard: opts.safeguard } : {}),...(opts.signal !== undefined ? { signal: opts.signal } : {}),progress,@@ -136,14 +209,16 @@ async function planVectorBackfillRun(applied: apply,capability,chunksTotal,- pending: pendingChunks.length,+ pending: plan.pending.length,embedded: tally.embeddingsComputed,retries: tally.embeddingsRetries,- estimatedCostUsd,- // Whether the MODEL has a price, not whether this run costs- // anything: an empty pending set costs nothing and that is not the- // same statement as "the price of this model is unknown".- costKnown: pricePerMillionTokens(model) > 0,+ estimatedCostUsd: plan.estimatedUsd,+ priceSource: plan.quote.source,+ blocked: plan.gate.blocked,+ reason: plan.gate.reason,+ pathPrefixes,+ unmatchedPathPrefixes,+ spend: tally.spend ?? null,});} finally {await store.close();diff --git a/src/mcp/brain/pack-tools.ts b/src/mcp/brain/pack-tools.tsindex 8f52bf22..89b0702e 100644--- a/src/mcp/brain/pack-tools.ts+++ b/src/mcp/brain/pack-tools.ts@@ -17,6 +17,12 @@ import {resolveSearchFocusContextPack,} from "../../core/config.ts";import { resolveSearchConfig } from "../../core/search/index.ts";+import { SearchError } from "../../core/search/search-error.ts";+import {+ discloseSpentQuery,+ loadBeliefSemanticRelevance,+ type BeliefSemanticRelevance,+} from "../../core/brain/belief-semantic.ts";import { readActiveSessionFocus } from "../../core/search/session-focus.ts";import {CONTEXT_PACK_QUERY_MODES,@@ -59,7 +65,8 @@ import {} from "../../core/brain/context-presets.ts";import { extractPreCompactRecords } from "../../core/brain/pre-compact-extract.ts";import { EventTraceSelectorError, resolveLogEventTraces } from "../../core/brain/event-trace.ts";-import { gatedOwnerScopeView } from "../../core/brain/owner-scope-view.ts";+import { gatedOwnerScopeView, ownerScopeView } from "../../core/brain/owner-scope-view.ts";+import { resolveOwnerScopeDelivery } from "../../core/brain/preferences-collect.ts";import { BRAIN_LOG_EVENT_KIND_SET, type BrainLogEventKind } from "../../core/brain/types.ts";import { INTERNAL_ERROR, INVALID_PARAMS, MCPError } from "../protocol.ts";import type { ServerContext, ToolDefinition } from "../tool-contract.ts";@@ -68,6 +75,12 @@ import { readableAtContextReachOrUndefined } from "./reach-readable.ts";import { reachView } from "../../core/brain/reach-view.ts";import { TRANSPORT_REACH } from "../../core/graph/transport-reach.ts";import { vaultPathField } from "../vault-path-field.ts";+import {+ exceedsMcpQueryCap,+ MCP_QUERY_CAP_MESSAGE,+ MCP_QUERY_MAX_CHARS,+ searchErrorToMcp,+} from "../search-tools.ts";import { MCP_PREVIEW_BUDGET } from "../preview-budget.ts";import {AGENT_SCOPE_SCHEMA,@@ -111,6 +124,53 @@ function vaultRelative(vault: string, absOrRel: string): string {return isAbsolute(absOrRel) ? relative(vault, absOrRel) : absOrRel;}+/** Run `fn`, answering a SearchError with its stable code on the wire. */+async function withSearchErrorsOnWire<T>(+ fn: () => Promise<T>,+ refine: (e: SearchError) => SearchError = (e) => e,+): Promise<T> {+ try {+ return await fn();+ } catch (e) {+ if (e instanceof SearchError) throw searchErrorToMcp(refine(e));+ throw e;+ }+}++/**+ * A pack refusal raised after the query embed still names that spend. The+ * loader's pre-embed check reads every belief in view, while the pack keeps+ * fewer (no tombstones, chain tips only, and a dimension the query vector+ * decides), so the embed can be paid and the pack refused all the same.+ */+function discloseSpentPackQuery(+ semantic: BeliefSemanticRelevance | null,+): (e: SearchError) => SearchError {+ return (e) => {+ if (semantic === null || e.code !== "BELIEF_VECTORS_MISSING") return e;+ return discloseSpentQuery(e, semantic.report);+ };+}++/**+ * The `semantic` member of a semantic-mode response: the one ungated+ * spend (the query embed) disclosed beside what the pack could score+ * after the reach filter.+ */+function semanticReportField(+ loaded: BeliefSemanticRelevance,+ packed: NonNullable<ReturnType<typeof packContext>["semantic"]>,+): Record<string, unknown> {+ return {+ model: loaded.report.model,+ price_source: loaded.report.priceSource,+ query_tokens: loaded.report.queryTokens,+ estimated_usd: loaded.report.estimatedUsd,+ scored: packed.scored,+ unembedded: packed.unembedded,+ };+}+async function toolBrainContextPack(ctx: ServerContext,args: Record<string, unknown>,@@ -137,6 +197,16 @@ async function toolBrainContextPack("brain_context_pack: query_mode requires query; a mode with nothing to read is inert",);}+ // The semantic query is embedded once, ungated: a blank one buys nothing.+ if (queryMode === "semantic" && query.trim() === "") {+ throw new MCPError(+ INVALID_PARAMS,+ "brain_context_pack: query_mode requires a non-empty query",+ );+ }+ }+ if (query !== undefined && exceedsMcpQueryCap(query)) {+ throw new MCPError(INVALID_PARAMS, `brain_context_pack: ${MCP_QUERY_CAP_MESSAGE}`);}const includeLanes = coerceBool(args, "lanes");const cacheStable = coerceBool(args, "cache_stable");@@ -239,37 +309,72 @@ async function toolBrainContextPack(// core holds absolute paths; the view answers over vault-relative ones.const view = reachView(ctx.vault, contextReach(ctx));const remote = view.reach !== TRANSPORT_REACH.local;- const report = packContext(ctx.vault, {- ...(remote ? { visible: (abs: string) => view.visible(vaultRelative(ctx.vault, abs)) } : {}),- maxTokens,- ...(agentScope !== undefined ? { agentScope } : {}),- ...(densityRanking ? { densityRanking: true } : {}),- ...(sessionFocus !== null ? { sessionFocus } : {}),- ...(query ? { query } : {}),- ...(queryMode !== null ? { queryMode } : {}),- ...(includeLanes ? { includeLanes: true } : {}),- ...(receipt !== undefined ? { receipt } : {}),- ...(adequacy !== undefined ? { recallAdequacy: adequacy } : {}),- ...(answerableVerdict !== undefined ? { decisionAnswerable: answerableVerdict } : {}),- ...(cacheStable || dedupRepeated- ? {- transforms: {- ...(cacheStable ? { cacheStableOrdering: true } : {}),- ...(dedupRepeated ? { deduplicateRepeatedContext: true } : {}),- },- }- : {}),- ...(maxCharsPerMemory !== undefined ? { maxCharsPerMemory } : {}),- ...(maxTotalChars !== undefined ? { maxTotalChars } : {}),- ...(configuredDegradation(ctx.vault) !== undefined- ? { degradation: configuredDegradation(ctx.vault)! }- : {}),- ...(telemetry !== undefined ? { telemetry } : {}),- // The synthesized attention-flow block has no page of its own to ask- // the reach view about, so a remote caller does not get it (fail- // closed, as the per-item filter this replaced did).- ...(attentionFlowIds.length > 0 && !remote ? { attentionFlowIds } : {}),- });+ // Semantic mode (Honest Embedding Spend): the one async step - the query+ // embed and the stored-vector read - happens here, so `packContext`+ // stays synchronous for every other caller. Every refusal is a named+ // SearchError: a blocked tier and a missing sqlite-vec from the loader,+ // before any embed, and `BELIEF_VECTORS_MISSING` from the pack, after+ // the reach filter and before a receipt is written. The loader reads+ // only the beliefs this caller's pack can keep - the same reach and+ // owner views the pack applies - so its own pre-embed refusal never+ // answers for a withheld belief differently than for an absent one.+ const semantic =+ queryMode === "semantic" && query !== undefined+ ? await withSearchErrorsOnWire(() => {+ const owners = ownerScopeView(+ ctx.vault,+ resolveOwnerScopeDelivery(ctx.vault, agentScope).enforcedScope,+ );+ return loadBeliefSemanticRelevance(+ resolveSearchConfig({ vault: ctx.vault, configPath: ctx.configPath ?? undefined }),+ query,+ { reach: view.reach, inView: (rel) => view.visible(rel) && owners.visible(rel) },+ );+ })+ : null;+ const report = await withSearchErrorsOnWire(+ async () =>+ packContext(ctx.vault, {+ ...(remote+ ? { visible: (abs: string) => view.visible(vaultRelative(ctx.vault, abs)) }+ : {}),+ maxTokens,+ ...(agentScope !== undefined ? { agentScope } : {}),+ ...(densityRanking ? { densityRanking: true } : {}),+ ...(sessionFocus !== null ? { sessionFocus } : {}),+ ...(query ? { query } : {}),+ ...(queryMode !== null ? { queryMode } : {}),+ ...(includeLanes ? { includeLanes: true } : {}),+ ...(receipt !== undefined ? { receipt } : {}),+ ...(adequacy !== undefined ? { recallAdequacy: adequacy } : {}),+ ...(answerableVerdict !== undefined ? { decisionAnswerable: answerableVerdict } : {}),+ ...(cacheStable || dedupRepeated+ ? {+ transforms: {+ ...(cacheStable ? { cacheStableOrdering: true } : {}),+ ...(dedupRepeated ? { deduplicateRepeatedContext: true } : {}),+ },+ }+ : {}),+ ...(maxCharsPerMemory !== undefined ? { maxCharsPerMemory } : {}),+ ...(maxTotalChars !== undefined ? { maxTotalChars } : {}),+ ...(configuredDegradation(ctx.vault) !== undefined+ ? { degradation: configuredDegradation(ctx.vault)! }+ : {}),+ ...(telemetry !== undefined ? { telemetry } : {}),+ // The synthesized attention-flow block has no page of its own to ask+ // the reach view about, so a remote caller does not get it (fail+ // closed, as the per-item filter this replaced did).+ ...(attentionFlowIds.length > 0 && !remote ? { attentionFlowIds } : {}),+ ...(semantic !== null ? { semanticRelevance: semantic.relevanceByPath } : {}),+ }),+ discloseSpentPackQuery(semantic),+ );+ const warnings = [+ ...(report.warnings ?? []),+ ...(semantic?.warnings ?? []),+ ...(answerable?.kind === "ignored" ? [answerable.warning] : []),+ ];return {vault_path: vaultPathField(ctx),max_tokens: report.maxTokens,@@ -304,13 +409,9 @@ async function toolBrainContextPack(// one surface that could not say a memory it injected is contested.// `brain_pre_compress_pack` has forwarded them all along; absent when// empty, so a warning-free pack stays byte-identical.- ...(report.warnings || answerable?.kind === "ignored"- ? {- warnings: [- ...(report.warnings ?? []),- ...(answerable?.kind === "ignored" ? [answerable.warning] : []),- ],- }+ ...(warnings.length > 0 ? { warnings } : {}),+ ...(semantic !== null && report.semantic !== undefined+ ? { semantic: semanticReportField(semantic, report.semantic) }: {}),...(adequacy !== undefined? {@@ -937,6 +1038,7 @@ export const PACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([},query: {type: "string",+ maxLength: MCP_QUERY_MAX_CHARS,description:"Optional query. Read as a case/Unicode-insensitive substring filter on topic + principle unless `query_mode` says otherwise.",},@@ -944,7 +1046,7 @@ export const PACK_TOOLS: ReadonlyArray<ToolDefinition> = Object.freeze([type: "string",enum: [...CONTEXT_PACK_QUERY_MODES],description:- "How `query` is read: `substring` (default) filters, dropping misses as `filter-miss`; `ranked` orders candidates by token overlap and excludes none.",+ "How `query` is read: `substring` (default) drops misses; `ranked` orders by token overlap; `semantic` orders by stored belief vectors, embedding the query once.",},focus_session: {type: "string",diff --git a/src/mcp/search-tools.ts b/src/mcp/search-tools.tsindex 81aaf52b..82ae4ee4 100644--- a/src/mcp/search-tools.ts+++ b/src/mcp/search-tools.ts@@ -105,6 +105,27 @@ import { emitGatedTelemetry } from "../core/brain/continuity/emit.ts";import { recordQueryDemand, recordRecallAdequacyDemand } from "../core/brain/query-demand.ts";const MCP_LIMIT_MAX = 50;++/** The longest `query` a recall tool accepts, advertised and enforced from here. */+export const MCP_QUERY_MAX_CHARS = 2000;++/** The refusal text of a query over {@link MCP_QUERY_MAX_CHARS}, shared by every capped tool. */+export const MCP_QUERY_CAP_MESSAGE = `argument 'query' exceeds ${MCP_QUERY_MAX_CHARS} characters`;++/**+ * Whether `query` is longer than {@link MCP_QUERY_MAX_CHARS}, counted in+ * code points as the advertised JSON Schema `maxLength` counts them: a+ * character outside the Basic Multilingual Plane is one character, not the+ * two UTF-16 units `String.length` would see.+ */+export function exceedsMcpQueryCap(query: string): boolean {+ if (query.length <= MCP_QUERY_MAX_CHARS) return false;+ let count = 0;+ for (const _ of query) {+ if (++count > MCP_QUERY_MAX_CHARS) return true;+ }+ return false;+}const MCP_CONTENT_MAX = 600;const SEARCH_TIMEOUT_MS = 10_000;/** Surfaced rows named as source refs on one recall-telemetry record. */@@ -116,7 +137,7 @@ const SEARCH_INPUT_SCHEMA: Record<string, unknown> = {query: {type: "string",minLength: 1,- maxLength: 2000,+ maxLength: MCP_QUERY_MAX_CHARS,description:"What to recall from the vault. Matched against the index by keyword, semantics, or both.",},@@ -923,8 +944,8 @@ async function toolBrainSearch(if (typeof query !== "string" || query.trim() === "") {throw new MCPError(INVALID_PARAMS, "missing required argument: query");}- if (query.length > 2000) {- throw new MCPError(INVALID_PARAMS, "argument 'query' exceeds 2000 characters");+ if (exceedsMcpQueryCap(query)) {+ throw new MCPError(INVALID_PARAMS, MCP_QUERY_CAP_MESSAGE);}let limit = 10;@@ -1422,7 +1443,7 @@ const RECALL_FEEDBACK_INPUT_SCHEMA: Record<string, unknown> = {query: {type: "string",minLength: 1,- maxLength: 2000,+ maxLength: MCP_QUERY_MAX_CHARS,description: "The query that produced the judged result; re-run to recover its layer scores.",},result_path: {@@ -1462,6 +1483,9 @@ async function toolBrainRecallFeedback(args: Record<string, unknown>,): Promise<unknown> {const query = coerceStr(args, "query")!;+ if (exceedsMcpQueryCap(query)) {+ throw new MCPError(INVALID_PARAMS, MCP_QUERY_CAP_MESSAGE);+ }const resultPath = coerceStr(args, "result_path")!;const verdict = coerceStr(args, "verdict")!;if (verdict !== "up" && verdict !== "down") {@@ -2001,6 +2025,7 @@ export async function buildSearchStatusBlock(ctx: ServerContext): Promise<Recordconst {embedding_signature: _embeddingSignature,estimated_refresh_cost_usd: _estimatedRefreshCostUsd,+ refresh_price_source: _refreshPriceSource,warnings,...rest} = serializeIndexStatus(snap);diff --git a/tests/cli/brain-maintenance.test.ts b/tests/cli/brain-maintenance.test.tsindex 19046379..dcc5180b 100644--- a/tests/cli/brain-maintenance.test.ts+++ b/tests/cli/brain-maintenance.test.ts@@ -22,9 +22,20 @@ import {type MaintenanceSpendReceipt,type MaintenanceTaskResult,} from "../../src/core/brain/maintenance/lane.ts";-import { listJournal, MAINTENANCE_JOURNAL_CAP } from "../../src/core/brain/maintenance/journal.ts";+import {+ appendJournal,+ listJournal,+ MAINTENANCE_JOURNAL_CAP,+ MAINTENANCE_SPEND_METRIC,+ MAINTENANCE_VERDICT,+} from "../../src/core/brain/maintenance/journal.ts";+import { listMetrics } from "../../src/core/brain/metrics.ts";import { currentLease, MAINTENANCE_LEASE_NAME } from "../../src/core/brain/maintenance/lease.ts";import { MAINTENANCE_EMBEDDINGS_ENV } from "../../src/core/config.ts";+import {+ isEmbeddingPriceSource,+ type EmbeddingPriceSource,+} from "../../src/core/search/embeddings/pricing.ts";import { MAINTENANCE_FAILURE_STREAK_LIMIT_DEFAULT } from "../../src/core/brain/policy/blocks/maintenance.ts";import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";import { startFakeHttp, type FakeHttp } from "../helpers/fake-http.ts";@@ -510,6 +521,125 @@ describe("renderTaskLine", () => {expect(line).toContain("reindex: ok in 1ms");expect(line).toContain("(tokens=38110, estimatedUsd=0.0076, model=text-embedding-3-small)");});++ test("a receipt for an unknown price says so instead of a dollar figure", () => {+ const receipt: MaintenanceSpendReceipt = {+ model: "zembed-1",+ tokens: 120,+ estimatedUsd: null,+ forced: true,+ priceSource: "unknown",+ };+ const line = renderTaskLine({ ...okRow(LANE_TASK.reindex), receipt });+ expect(line).toContain("(tokens=120, price unknown, model=zembed-1)");+ expect(line).not.toContain("$0");+ });+});++describe("the journal listing", () => {+ test("a receipt written before price sources renders as unrecorded, its estimate as written", async () => {+ const init = await runCli(["brain", "init", "--vault", vault], { env: baseEnv() });+ expect(init.returncode).toBe(0);+ appendJournal(vault, {+ ts: "2026-09-01T00:00:00Z",+ holder: "test",+ verdict: MAINTENANCE_VERDICT.run,+ task: LANE_TASK.reindex,+ ok: true,+ receipt: {+ model: "text-embedding-3-small",+ tokens: 500,+ estimatedUsd: 0.0123,+ forced: false,+ },+ });+ appendJournal(vault, {+ ts: "2026-09-02T00:00:00Z",+ holder: "test",+ verdict: MAINTENANCE_VERDICT.run,+ task: LANE_TASK.reindex,+ ok: true,+ receipt: {+ model: "zembed-1",+ tokens: 40,+ estimatedUsd: null,+ forced: true,+ priceSource: "unknown",+ },+ });+ const status = await runCli(["brain", "maintenance", "status", "--vault", vault], {+ env: baseEnv(),+ });+ expect(status.returncode).toBe(0);+ expect(status.stdout).toContain(+ "(tokens=500, estimatedUsd=0.0123, model=text-embedding-3-small, price_source=unrecorded)",+ );+ expect(status.stdout).toContain(+ "(tokens=40, price unknown, model=zembed-1, price_source=unknown)",+ );+ });+});++describe("a journaled receipt without a numeric estimate", () => {+ test("reads price unknown and never breaks the listing", async () => {+ const init = await runCli(["brain", "init", "--vault", vault], { env: baseEnv() });+ expect(init.returncode).toBe(0);+ // Rows another build, another device's shard or a hand edit could+ // write: the journal read casts them, so the listing must not trust+ // the estimate's type.+ const receipts = [+ { model: "m-absent", tokens: 3, forced: false },+ { model: "m-string", tokens: 4, forced: false, estimatedUsd: "x" },+ ] as unknown as ReadonlyArray<MaintenanceSpendReceipt>;+ for (const [i, receipt] of receipts.entries()) {+ appendJournal(vault, {+ ts: `2026-09-0${i + 4}T00:00:00Z`,+ holder: "test",+ verdict: MAINTENANCE_VERDICT.run,+ task: LANE_TASK.reindex,+ ok: true,+ receipt,+ });+ }+ const status = await runCli(["brain", "maintenance", "status", "--vault", vault], {+ env: baseEnv(),+ });+ expect(status.returncode).toBe(0);+ expect(status.stdout).toContain(+ "(tokens=3, price unknown, model=m-absent, price_source=unrecorded)",+ );+ expect(status.stdout).toContain(+ "(tokens=4, price unknown, model=m-string, price_source=unrecorded)",+ );+ });+});++describe("a receipt whose price source this build does not know", () => {+ test("renders as unrecorded, never as the stray value", async () => {+ expect(isEmbeddingPriceSource("vendor")).toBe(false);+ const init = await runCli(["brain", "init", "--vault", vault], { env: baseEnv() });+ expect(init.returncode).toBe(0);+ appendJournal(vault, {+ ts: "2026-09-03T00:00:00Z",+ holder: "test",+ verdict: MAINTENANCE_VERDICT.run,+ task: LANE_TASK.reindex,+ ok: true,+ receipt: {+ model: "voyage-3",+ tokens: 70,+ estimatedUsd: 0.0001,+ forced: false,+ priceSource: "vendor" as EmbeddingPriceSource,+ },+ });+ const status = await runCli(["brain", "maintenance", "status", "--vault", vault], {+ env: baseEnv(),+ });+ expect(status.returncode).toBe(0);+ expect(status.stdout).toContain("model=voyage-3, price_source=unrecorded)");+ expect(status.stdout).not.toContain("vendor");+ });});describe("formatSpendBanner", () => {@@ -521,7 +651,9 @@ describe("formatSpendBanner", () => {pendingChunks: 214,tokens: 1,estimatedUsd: 0.011,+ priceSource: "builtin",blocked: false,+ reason: null,},0,),@@ -531,11 +663,36 @@ describe("formatSpendBanner", () => {);expect(formatSpendBanner(- { model: null, pendingChunks: 1, tokens: 0, estimatedUsd: 0, blocked: false },+ {+ model: null,+ pendingChunks: 1,+ tokens: 0,+ estimatedUsd: 0,+ priceSource: "builtin",+ blocked: false,+ reason: null,+ },0.5,),).toBe("embedding spend: model unknown, 1 chunks pending, estimated $0.0000 (gate: $0.5000)");});++ test("an unknown price prints price unknown, never a dollar figure", () => {+ expect(+ formatSpendBanner(+ {+ model: "zembed-1",+ pendingChunks: 3,+ tokens: 9,+ estimatedUsd: null,+ priceSource: "unknown",+ blocked: true,+ reason: "unpriced",+ },+ 0.5,+ ),+ ).toBe("embedding spend: model zembed-1, 3 chunks pending, price unknown (gate: $0.5000)");+ });});/** Initialize the vault, drop one note in, and index it keyword-only. */@@ -580,7 +737,13 @@ describe("the spend surface end to end", () => {expect(first.returncode).toBe(0);const payload = JSON.parse(first.stdout) as {spend?: {- banner: { model: string; pendingChunks: number; estimatedUsd: number; gateUsd: number };+ banner: {+ model: string;+ pendingChunks: number;+ estimatedUsd: number;+ priceSource: string;+ gateUsd: number;+ };receipt: MaintenanceSpendReceipt;};tasks: Array<{ name: string; ok: boolean; receipt?: MaintenanceSpendReceipt }>;@@ -593,6 +756,11 @@ describe("the spend surface end to end", () => {expect(payload.spend?.receipt).toEqual(reindex?.receipt);expect(payload.spend?.receipt.tokens).toBeGreaterThan(0);expect(payload.spend?.receipt.forced).toBe(false);+ // Who stated the price rides the banner, the receipt and the metric.+ expect(payload.spend?.banner.priceSource).toBe("builtin");+ expect(payload.spend?.receipt.priceSource).toBe("builtin");+ const metric = listMetrics(vault, { surface: MAINTENANCE_SPEND_METRIC }).at(-1);+ expect(metric?.payload["price_source"]).toBe("builtin");// The receipt the lane journaled agrees with the row it reported.const status = await runCli(["brain", "maintenance", "status", "--vault", vault, "--json"], {diff --git a/tests/cli/search-check-provider.test.ts b/tests/cli/search-check-provider.test.tsindex 32eb6459..9cf24813 100644--- a/tests/cli/search-check-provider.test.ts+++ b/tests/cli/search-check-provider.test.ts@@ -43,12 +43,13 @@ import { join } from "node:path";import { INSTALL_EXIT } from "../../src/cli/install/install.ts";import { exitCodeForCheck, SEARCH_CHECK_EXIT } from "../../src/cli/search/verbs/check.ts";+import { providerRegistryPath } from "../../src/core/search/embeddings/registry.ts";import { PROVIDER_PROBE } from "../../src/core/search/provider-probe.ts";import type { IndexCheckReport } from "../../src/core/search/types.ts";import { startFakeHttp, type FakeHttp } from "../helpers/fake-http.ts";import { createTempVault, writeMd } from "../helpers/search-fixtures.ts";import { runCli } from "../helpers/run-cli.ts";-import { FAKE_PROVIDER_KEY } from "../helpers/fake-credentials.ts";+import { FAKE_PROVIDER_KEY, fakeCredential } from "../helpers/fake-credentials.ts";/** Short enough for a test, long enough that a healthy loopback answer beats it. */const SHORT_REQUEST_TIMEOUT_MS = 120;@@ -279,3 +280,190 @@ test("every probe state maps to exactly one exit, and a machine fault outranks a// the number there must not have to learn a second one here.expect(SEARCH_CHECK_EXIT.providerUnreachable).toBe(INSTALL_EXIT.mcpUnreachable);});++// ─────────────────────────────────────────────────────────────────────────────+// Price recommendations (Honest Embedding Spend, task 7)+//+// `fake-model` is in no price table, so the configured stub is an+// unpriced model. The recommendation arms have their own suite+// (tests/core/search/check-price-recommendations.test.ts); these pin that+// `search check` carries the lines in both report shapes and stays silent+// once the model is priced.+// ─────────────────────────────────────────────────────────────────────────────++const PRICE_KEYS = ["embedding_price_model", "embedding_price_usd_per_mtok"] as const;++function priceRecommendations(run: CheckRun): string[] {+ const recs = (run.payload["recommendations"] ?? []) as string[];+ return recs.filter((r) => PRICE_KEYS.some((key) => r.includes(key)));+}++test("an unpriced model gets one price recommendation in both report shapes", async () => {+ await writeConfiguredProvider();++ const json = await runCheck("--json", "--no-probe");+ const lines = priceRecommendations(json);+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain('"fake-model"');+ expect(lines[0]).toContain("a positive gate would refuse");++ const human = await runCheck("--no-probe");+ expect(human.stdout).toContain(` - ${lines[0]}`);+});++test("an unpriced model under a positive gate is told that backfills refuse", async () => {+ await writeConfiguredProvider(["embedding_cost_gate_usd: 0.5"]);++ const lines = priceRecommendations(await runCheck("--json", "--no-probe"));+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain("vector backfills refuse");+});++test("a declared pair for another model yields one stale-declaration recommendation", async () => {+ await writeConfiguredProvider([+ 'embedding_model: "text-embedding-3-small"',+ "embedding_price_model: acme-embed-retired",+ "embedding_price_usd_per_mtok: 0.07",+ ]);++ const lines = priceRecommendations(await runCheck("--json", "--no-probe"));+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain('names model "acme-embed-retired"');+});++test("an operator-priced model adds no price recommendation", async () => {+ await writeConfiguredProvider([+ "embedding_price_model: fake-model",+ "embedding_price_usd_per_mtok: 0.07",+ ]);++ expect(priceRecommendations(await runCheck("--json", "--no-probe"))).toEqual([]);+});++// ─────────────────────────────────────────────────────────────────────────────+// Credential-source report (Honest Embedding Spend, task 8)+//+// A credential-missing tier names where the resolver looked and which+// other registered profiles hold a key, by name only. The pure report has+// its own suite (tests/core/search/credential-report.test.ts); these pin+// the wiring: the env and registry `search check` hands it, both report+// shapes, and that no env value reaches either.+// ─────────────────────────────────────────────────────────────────────────────++const ACTIVE_PROFILE = "acme-embed";+const ACTIVE_ENV_KEY = "ACME_EMBED_TEST_KEY";+const OTHER_PROFILE = "beta-embed";+const OTHER_ENV_KEY = "BETA_EMBED_TEST_KEY";+/** A credential-shaped placeholder that must never be printed. */+const PRIVATE_BODY = fakeCredential("beta", "-private-", "0d9b");++/** The keyless env every credential run starts from, so a shell key cannot leak in. */+const KEYLESS_ENV: Readonly<Record<string, string>> = Object.freeze({+ OPEN_SECOND_BRAIN_EMBEDDING_KEY: "",+ [ACTIVE_ENV_KEY]: "",+});++async function writeProviderRegistry(): Promise<void> {+ await Bun.write(+ providerRegistryPath(vault),+ JSON.stringify([+ {+ name: ACTIVE_PROFILE,+ baseUrl: "https://acme.invalid/v1",+ defaultModel: "acme-1",+ envKey: ACTIVE_ENV_KEY,+ },+ {+ name: OTHER_PROFILE,+ baseUrl: "https://beta.invalid/v1",+ defaultModel: "beta-1",+ envKey: OTHER_ENV_KEY,+ },+ ]),+ );+}++async function writeKeylessProfile(): Promise<void> {+ await Bun.write(+ configPath,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ `embedding_provider: ${ACTIVE_PROFILE}`,+ "",+ ].join("\n"),+ );+}++async function runCheckWithEnv(+ env: Readonly<Record<string, string>>,+ ...extra: ReadonlyArray<string>+): Promise<CheckRun> {+ const r = await runCli(+ ["search", "check", "--vault", vault, "--db", dbPath, "--config", configPath, ...extra],+ { env: { OPEN_SECOND_BRAIN_CONFIG: configPath, ...KEYLESS_ENV, ...env } },+ );+ const json = extra.includes("--json")+ ? (JSON.parse(r.stdout) as Record<string, unknown>)+ : ({} as Record<string, unknown>);+ return { payload: json, returncode: r.returncode, stdout: r.stdout };+}++test("a keyless registry profile names the sources consulted and the profile that holds a key", async () => {+ await writeProviderRegistry();+ await writeKeylessProfile();++ const run = await runCheckWithEnv({ [OTHER_ENV_KEY]: PRIVATE_BODY }, "--json", "--no-probe");++ expect(run.payload["embedding_key_resolved"]).toBe(false);+ expect(run.payload["credential_sources"]).toEqual({+ consulted: ["OPEN_SECOND_BRAIN_EMBEDDING_KEY", "embedding_api_key", ACTIVE_ENV_KEY],+ present_elsewhere: [OTHER_PROFILE],+ });+ expect(run.stdout).not.toContain(PRIVATE_BODY);+});++test("the human report renders the credential sources and never an env value", async () => {+ await writeProviderRegistry();+ await writeKeylessProfile();++ const run = await runCheckWithEnv({ [OTHER_ENV_KEY]: PRIVATE_BODY }, "--no-probe");++ expect(run.stdout).toContain(+ `key_sources_checked: OPEN_SECOND_BRAIN_EMBEDDING_KEY, embedding_api_key, ${ACTIVE_ENV_KEY}`,+ );+ expect(run.stdout).toContain(`key_present_under: ${OTHER_PROFILE}`);+ expect(run.stdout).not.toContain(PRIVATE_BODY);+});++test("explicit keyless config consults the explicit pair and says when no profile holds a key", async () => {+ await Bun.write(+ configPath,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ "",+ ].join("\n"),+ );++ const json = await runCheckWithEnv({}, "--json", "--no-probe");+ expect(json.payload["credential_sources"]).toEqual({+ consulted: ["OPEN_SECOND_BRAIN_EMBEDDING_KEY", "embedding_api_key"],+ present_elsewhere: [],+ });+ const human = await runCheckWithEnv({}, "--no-probe");+ expect(human.stdout).toContain("key_present_under: none\n");+});++test("a configured tier carries no credential report in either shape", async () => {+ await writeProviderRegistry();+ await writeConfiguredProvider();++ const json = await runCheckWithEnv({ [OTHER_ENV_KEY]: PRIVATE_BODY }, "--json", "--no-probe");+ expect("credential_sources" in json.payload).toBe(false);+ const human = await runCheckWithEnv({ [OTHER_ENV_KEY]: PRIVATE_BODY }, "--no-probe");+ expect(human.stdout).not.toContain("key_sources_checked:");+ expect(human.stdout).not.toContain("key_present_under:");+});diff --git a/tests/cli/search-index-reuse.test.ts b/tests/cli/search-index-reuse.test.tsnew file mode 100644index 00000000..7e08dacf--- /dev/null+++ b/tests/cli/search-index-reuse.test.ts@@ -0,0 +1,87 @@+/**+ * `o2b search index` reports the vectors an edit carried over: the count+ * the carry-over exists to save is shown in the JSON stats and on the+ * human `embeddings:` line, and a run that carried nothing keeps the+ * payload it emitted before the field existed.+ */++import { afterEach, beforeEach, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import { runCli } from "../helpers/run-cli.ts";+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";++const VEC_LOADABLE = sqliteVecLoadable();++let tmp: string;+let vault: string;+let configPath: string;++/** Three sections small enough that each is its own chunk. */+const SECTIONS = [+ "# Alpha\n\nThe first section talks about compost and soil.",+ "# Beta\n\nThe second section talks about tomatoes in spring.",+ "# Gamma\n\nThe third section talks about watering at dawn.",+] as const;+const EDITED_BETA = "# Beta\n\nThe second section now talks about peppers.";++beforeEach(() => {+ tmp = mkdtempSync(join(tmpdir(), "o2b-index-reuse-"));+ vault = join(tmp, "vault");+ mkdirSync(join(vault, "Notes"), { recursive: true });+ configPath = join(tmp, "config.yaml");+ writeFileSync(+ configPath,+ [+ `vault: "${vault}"`,+ 'search_semantic_enabled: "true"',+ 'embedding_provider: "local"',+ 'search_chunk_size: "60"',+ 'search_chunk_overlap: "0"',+ 'search_chunk_min_size: "1"',+ ].join("\n") + "\n",+ );+});++afterEach(() => {+ rmSync(tmp, { recursive: true, force: true });+});++function writeNote(sections: ReadonlyArray<string>): void {+ writeFileSync(join(vault, "Notes", "garden.md"), sections.join("\n\n"));+}++async function index(json: boolean): Promise<string> {+ const argv = ["search", "index", "--embeddings", ...(json ? ["--json"] : [])];+ const r = await runCli(argv, { env: { OPEN_SECOND_BRAIN_CONFIG: configPath } });+ expect(r.returncode).toBe(0);+ return r.stdout;+}++function statsOf(stdout: string): Record<string, unknown> {+ return (JSON.parse(stdout) as { stats: Record<string, unknown> }).stats;+}++test.skipIf(!VEC_LOADABLE)(+ "an edit reports the carried vectors as embeddings_reused in JSON",+ async () => {+ writeNote(SECTIONS);+ const first = statsOf(await index(true));+ expect(first).not.toHaveProperty("embeddings_reused");++ writeNote([SECTIONS[0], EDITED_BETA, SECTIONS[2]]);+ const second = statsOf(await index(true));+ expect(second["embeddings_computed"]).toBe(1);+ expect(second["embeddings_reused"]).toBe((first["embeddings_computed"] as number) - 1);+ },+);++test.skipIf(!VEC_LOADABLE)("the human embeddings line names the reused count", async () => {+ writeNote(SECTIONS);+ expect(await index(false)).not.toContain("reused");++ writeNote([SECTIONS[0], EDITED_BETA, SECTIONS[2]]);+ expect(await index(false)).toMatch(/embeddings: 1 computed \(0 retries\), \d+ reused/);+});diff --git a/tests/cli/search-vector-backfill-path.test.ts b/tests/cli/search-vector-backfill-path.test.tsnew file mode 100644index 00000000..b5b14876--- /dev/null+++ b/tests/cli/search-vector-backfill-path.test.ts@@ -0,0 +1,433 @@+/**+ * Path-scoped vector backfill (Honest Embedding Spend, task 10).+ *+ * Paying for vectors was all-or-nothing across the vault: an operator who+ * wanted only the belief notes embedded (the vectors the `semantic`+ * context-pack mode reads) had to price and pay for every chunk. The+ * backfill now takes repeatable path prefixes, validated by the search+ * layer's `assertSafePathPrefix`, and threads them through the one spend+ * plan, so the dry run, the cost gate's refusal, the embedding phase and+ * the spend receipt all read the same scoped census. The CLI takes the+ * scope as a repeatable `--path`, echoes it, and names a scoped `--apply`+ * as the next step, never the vault-wide one.+ *+ * The only embedding endpoint is the loopback fake the search fixtures+ * ship. Deliberately not covered here: the unscoped census and apply+ * mechanics (search-vector-backfill.test.ts) and the price wording+ * (search-vector-backfill-price.test.ts).+ */++import { afterEach, beforeEach, expect, spyOn, test } from "bun:test";+import { join } from "node:path";++import { indexVault } from "../../src/core/search/indexer.ts";+import { SearchError } from "../../src/core/search/search-error.ts";+import { Store } from "../../src/core/search/store.ts";+import { planVectorBackfill } from "../../src/core/search/vector-backfill.ts";+import {+ backfillNextStep,+ emitBackfillNextStep,+ RERUN_WITH_SCOPE_LINE,+ scopedNextCommand,+} from "../../src/cli/search/verbs/vector-backfill.ts";+import { FAKE_PROVIDER_KEY } from "../helpers/fake-credentials.ts";+import { startFakeHttp, type FakeHttp } from "../helpers/fake-http.ts";+import { runCli } from "../helpers/run-cli.ts";+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";++const VEC_LOADABLE = sqliteVecLoadable();+const BELIEFS = "Brain/preferences/";+const MODEL = "c10-model";+/** Priced so that one note's chunks fit under the gate and four notes' do not. */+const USD_PER_MTOK = 1_000;++let vault: string;+let dbPath: string;+let cleanup: () => void;+let server: FakeHttp;++beforeEach(async () => {+ const v = createTempVault("c10-vector-backfill-path");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+ server = await startFakeHttp();+});++afterEach(async () => {+ await server.close();+ cleanup();+});++function semanticConfig(costGateUsd = 0): ReturnType<typeof makeConfig> {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: server.url,+ model: MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ dimension: 4,+ timeoutMs: 5_000,+ costGateUsd,+ priceOverride: { model: MODEL, usdPerMtok: USD_PER_MTOK },+ },+ });+}++/** One belief note and three other notes, indexed with no vectors. */+async function seed(): Promise<void> {+ writeMd(vault, `${BELIEFS}pref-brevity.md`, "# Brevity\n\nKeep answers short.");+ writeMd(vault, "Notes/a.md", "# A\n\nFirst note about something that matters.");+ writeMd(vault, "Notes/b.md", "# B\n\nSecond note discussing other things at length.");+ writeMd(vault, "Notes/c.md", "# C\n\nThird note with yet more words in it.");+ await indexVault(semanticConfig());+}++test.skipIf(!VEC_LOADABLE)(+ "the scoped dry run prices only the chunks under the prefix",+ async () => {+ await seed();+ const all = await planVectorBackfill(semanticConfig());+ const scoped = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });++ expect(scoped.pathPrefixes).toEqual([BELIEFS]);+ expect(all.pathPrefixes).toEqual([]);+ expect(scoped.pending).toBeGreaterThan(0);+ expect(scoped.pending).toBeLessThan(all.pending);+ expect(scoped.estimatedCostUsd!).toBeLessThan(all.estimatedCostUsd!);+ expect(scoped.chunksTotal).toBe(all.chunksTotal);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "--apply embeds only the scoped chunks and its receipt agrees with the dry run",+ async () => {+ await seed();+ const dry = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });+ // The applied run embeds from the plan it priced: one census read,+ // not a second one the receipt could disagree with.+ const census = spyOn(Store.prototype, "findChunksWithoutEmbeddings");+ let applied: Awaited<ReturnType<typeof planVectorBackfill>>;+ try {+ applied = await planVectorBackfill(semanticConfig(), {+ pathPrefixes: [BELIEFS],+ apply: true,+ });+ expect(census).toHaveBeenCalledTimes(1);+ } finally {+ census.mockRestore();+ }++ expect(applied.embedded).toBe(dry.pending);+ expect(applied.spend).not.toBeNull();+ expect(applied.spend!.estimatedUsd).toBe(dry.estimatedCostUsd);+ expect(applied.spend!.priceSource).toBe(dry.priceSource);++ const rest = await planVectorBackfill(semanticConfig());+ const afterScoped = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });+ expect(afterScoped.pending).toBe(0);+ expect(rest.pending).toBeGreaterThan(0);+ },+);++test.skipIf(!VEC_LOADABLE)("the cost gate reads the scoped census", async () => {+ await seed();+ const scopedDry = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });+ const allDry = await planVectorBackfill(semanticConfig());+ // A gate between the two estimates: the scoped spend fits, the vault-wide one does not.+ const gate = (scopedDry.estimatedCostUsd! + allDry.estimatedCostUsd!) / 2;++ const refusal = await planVectorBackfill(semanticConfig(gate), { apply: true }).catch(+ (e: unknown) => e,+ );+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_COST_GATE");++ const scopedPlan = await planVectorBackfill(semanticConfig(gate), { pathPrefixes: [BELIEFS] });+ expect(scopedPlan.blocked).toBe(false);+ const scopedApply = await planVectorBackfill(semanticConfig(gate), {+ pathPrefixes: [BELIEFS],+ apply: true,+ });+ expect(scopedApply.embedded).toBe(scopedDry.pending);+});++test.skipIf(!VEC_LOADABLE)("an unpriced refusal counts the scoped census", async () => {+ await seed();+ const unpriced = makeConfig({+ vault,+ dbPath,+ semantic: { ...semanticConfig(1).semantic, priceOverride: undefined },+ });+ const scopedDry = await planVectorBackfill(unpriced, { pathPrefixes: [BELIEFS] });+ expect(scopedDry.blocked).toBe(true);++ const refusal = await planVectorBackfill(unpriced, {+ pathPrefixes: [BELIEFS],+ apply: true,+ }).catch((e: unknown) => e);+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_COST_UNPRICED");+ expect((refusal as SearchError).message).toContain(`${scopedDry.pending} chunk(s)`);+});++test.each(["../outside/", "Brain/../Notes/", "/etc/", "//server/share/", "C:/Users/"])(+ "an unsafe prefix is refused by name (%p)",+ async (prefix) => {+ const refusal = await planVectorBackfill(semanticConfig(), { pathPrefixes: [prefix] }).catch(+ (e: unknown) => e,+ );+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("INVALID_INPUT");+ expect((refusal as SearchError).message).toContain(prefix);+ },+);++/** A CLI config file for the fake provider. */+async function cliConfig(): Promise<string> {+ const config = join(vault, "cli-config.yaml");+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ `embedding_model: ${MODEL}`,+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "",+ ].join("\n"),+ );+ return config;+}++async function backfillCli(config: string, extra: ReadonlyArray<string>) {+ return await runCli(+ ["search", "vector-backfill", "--vault", vault, "--db", dbPath, "--config", config, ...extra],+ { env: { OPEN_SECOND_BRAIN_CONFIG: config } },+ );+}++test.skipIf(!VEC_LOADABLE)(+ "--path is repeatable, echoed, and the next step keeps the scope",+ async () => {+ await seed();+ writeMd(vault, "Brain/retired/pref-old.md", "# Old\n\nAnswer at length.");+ await indexVault(semanticConfig());+ const config = await cliConfig();+ const scoped = await planVectorBackfill(semanticConfig(), {+ pathPrefixes: [BELIEFS, "Brain/retired/"],+ });++ const json = await backfillCli(config, [+ "--path",+ BELIEFS,+ "--path",+ "Brain/retired/",+ "--json",+ ]);+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect(payload["path_prefixes"]).toEqual([BELIEFS, "Brain/retired/"]);+ expect(payload["pending"]).toBe(scoped.pending);+ expect(payload["next_command"]).toBe(+ "o2b search vector-backfill --apply --path Brain/preferences/ --path Brain/retired/",+ );++ const human = await backfillCli(config, ["--path", BELIEFS]);+ expect(human.returncode).toBe(0);+ expect(human.stdout).toContain(`scope: ${BELIEFS}`);+ expect(human.stdout).toContain(+ "next: o2b search vector-backfill --apply --path Brain/preferences/",+ );+ },+);++test.skipIf(!VEC_LOADABLE)("an unscoped run keeps its report and next step", async () => {+ await seed();+ const config = await cliConfig();+ const json = await backfillCli(config, ["--json"]);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expect("path_prefixes" in payload).toBe(false);+ expect(payload["next_command"]).toBe("o2b search vector-backfill --apply");+});++test.skipIf(!VEC_LOADABLE)("--apply --path embeds only the scoped chunks", async () => {+ await seed();+ const config = await cliConfig();+ const dry = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });+ const applied = await backfillCli(config, ["--path", BELIEFS, "--apply", "--json"]);+ expect(applied.returncode).toBe(0);+ const payload = JSON.parse(applied.stdout) as Record<string, unknown>;+ expect(payload["embedded"]).toBe(dry.pending);+ const rest = await planVectorBackfill(semanticConfig());+ expect(rest.pending).toBeGreaterThan(0);+});++test.skipIf(!VEC_LOADABLE).each([+ ["", []],+ ["", ["--apply"]],+ [" ", ["--apply"]],+ ["./", ["--apply"]],+] as const)(+ "an empty --path %p exits INVALID_INPUT and embeds nothing (%p)",+ async (prefix, extra) => {+ await seed();+ const config = await cliConfig();+ const before = await planVectorBackfill(semanticConfig());+ const run = await backfillCli(config, ["--path", prefix, ...extra]);+ expect(run.returncode).toBe(2);+ expect(run.stderr).toContain("[INVALID_INPUT]");+ expect(run.stderr).toContain("path prefix is empty");+ expect(run.stderr).toContain(JSON.stringify(prefix));+ const after = await planVectorBackfill(semanticConfig());+ expect(after.pending).toBe(before.pending);+ },+);++test.skipIf(!VEC_LOADABLE).each(["./Brain/preferences/", "Brain\\preferences\\"])(+ "a %p prefix matches the same notes as its plain form",+ async (prefix) => {+ await seed();+ const plain = await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS] });+ const spelled = await planVectorBackfill(semanticConfig(), { pathPrefixes: [prefix] });+ expect(spelled.pending).toBe(plain.pending);+ expect(spelled.pathPrefixes).toEqual([BELIEFS]);+ },+);++test.skipIf(!VEC_LOADABLE)(+ "a scope that matches no document is named, not reported as done",+ async () => {+ await seed();+ const config = await cliConfig();+ const run = await backfillCli(config, ["--path", "Brain/prefs/", "--path", BELIEFS, "--json"]);+ expect(run.returncode).toBe(0);+ expect(run.stderr).toContain("scope Brain/prefs/ matches no indexed document");+ expect(run.stderr).not.toContain(`scope ${BELIEFS} matches`);+ const payload = JSON.parse(run.stdout) as Record<string, unknown>;+ expect(payload["unmatched_path_prefixes"]).toEqual(["Brain/prefs/"]);++ const human = await backfillCli(config, ["--path", "Brain/prefs/"]);+ expect(human.stderr).toContain("scope Brain/prefs/ matches no indexed document");+ },+);++test.skipIf(!VEC_LOADABLE)(+ "a fully embedded scope that matches documents warns nothing",+ async () => {+ await seed();+ await planVectorBackfill(semanticConfig(), { pathPrefixes: [BELIEFS], apply: true });+ const config = await cliConfig();+ const run = await backfillCli(config, ["--path", BELIEFS, "--json"]);+ expect(run.returncode).toBe(0);+ expect(run.stderr).not.toContain("matches no indexed document");+ const payload = JSON.parse(run.stdout) as Record<string, unknown>;+ expect(payload["pending"]).toBe(0);+ expect("unmatched_path_prefixes" in payload).toBe(false);+ },+);++/**+ * A prefix that needs quoting is quoted POSIX-style; on Windows, where+ * that quoting would not run as printed, the key is omitted instead.+ */+function expectQuotedNextStep(payload: Record<string, unknown>, posix: string): void {+ // Pending work is what makes an absent key mean "dropped", not "nothing to advise".+ expect(payload["pending"]).toBeGreaterThan(0);+ if (process.platform === "win32") expect("next_command" in payload).toBe(false);+ else expect(payload["next_command"]).toBe(posix);+}++test.skipIf(!VEC_LOADABLE)("a scoped next step quotes a prefix with a quote", async () => {+ await seed();+ writeMd(vault, "Notes/it's/d.md", "# D\n\nA note in a folder whose name has a quote.");+ await indexVault(semanticConfig());+ const config = await cliConfig();+ const json = await backfillCli(config, ["--path", "Notes/it's/", "--json"]);+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expectQuotedNextStep(payload, "o2b search vector-backfill --apply --path 'Notes/it'\\''s/'");+});++test.skipIf(!VEC_LOADABLE)("a scoped next step quotes a prefix with a space", async () => {+ await seed();+ writeMd(vault, "Notes/with space/d.md", "# D\n\nA note in a folder whose name has a space.");+ await indexVault(semanticConfig());+ const config = await cliConfig();+ const json = await backfillCli(config, ["--path", "Notes/with space/", "--json"]);+ expect(json.returncode).toBe(0);+ const payload = JSON.parse(json.stdout) as Record<string, unknown>;+ expectQuotedNextStep(payload, "o2b search vector-backfill --apply --path 'Notes/with space/'");++ // The human advice: the quoted scope off Windows, the rerun line that+ // names no runnable command on it.+ const human = await backfillCli(config, ["--path", "Notes/with space/"]);+ const advice = human.stdout + human.stderr;+ if (process.platform === "win32") {+ expect(advice).toContain(RERUN_WITH_SCOPE_LINE);+ expect(advice).not.toContain("'Notes/with space/'");+ } else {+ expect(advice).toContain("--path 'Notes/with space/'");+ expect(advice).not.toContain(RERUN_WITH_SCOPE_LINE);+ }+});++const APPLY_COMMAND = "o2b search vector-backfill --apply";++test("a scope Windows cannot quote drops the scoped next step", () => {+ // POSIX quoting is not a word cmd.exe or PowerShell reads back, so the+ // advice names no command and asks for a rerun with the same flags.+ expect(scopedNextCommand(APPLY_COMMAND, ["Notes/with space/"], "win32")).toBeNull();+ expect(scopedNextCommand(APPLY_COMMAND, [BELIEFS, "Notes/it's/"], "win32")).toBeNull();+});++test("a plain scope stays in the next step on every platform", () => {+ for (const platform of ["win32", "linux", "darwin"] as const) {+ expect(scopedNextCommand(APPLY_COMMAND, [BELIEFS], platform)).toBe(+ `${APPLY_COMMAND} --path ${BELIEFS}`,+ );+ }+ expect(scopedNextCommand(APPLY_COMMAND, ["Notes/with space/"], "linux")).toBe(+ `${APPLY_COMMAND} --path 'Notes/with space/'`,+ );+});++test.skipIf(!VEC_LOADABLE)(+ "a gate-blocked scope Windows cannot quote advises no runnable command",+ async () => {+ await seed();+ writeMd(vault, "a b/d.md", "# D\n\nA note in a folder whose name has a space.");+ await indexVault(semanticConfig());+ // Priced far above a one-cent gate, so the unforced run is blocked.+ const result = await planVectorBackfill(semanticConfig(0.000_001), { pathPrefixes: ["a b/"] });+ expect(result.blocked).toBe(true);+ expect(result.pending).toBeGreaterThan(0);++ const advice = backfillNextStep(result, "win32");+ expect(advice.command).toBeUndefined();+ const written: string[] = [];+ const spy = spyOn(process.stdout, "write").mockImplementation((chunk) => {+ written.push(String(chunk));+ return true;+ });+ try {+ emitBackfillNextStep(advice, {+ command: "search",+ argv: ["vector-backfill"],+ jsonRequested: false,+ });+ } finally {+ spy.mockRestore();+ }+ const lines = written.join("").split("\n");+ expect(lines).toContain(RERUN_WITH_SCOPE_LINE);+ expect(lines.some((line) => /^next: .*--apply( --force-cost)?$/u.test(line))).toBe(false);+ expect(lines.some((line) => line.includes("--force-cost"))).toBe(false);+ },+);diff --git a/tests/cli/search-vector-backfill-price.test.ts b/tests/cli/search-vector-backfill-price.test.tsnew file mode 100644index 00000000..980614f9--- /dev/null+++ b/tests/cli/search-vector-backfill-price.test.ts@@ -0,0 +1,230 @@+/**+ * Price honesty on `o2b search vector-backfill` and `o2b search status`+ * (Honest Embedding Spend).+ *+ * Pins the defect that an unpriced model was reported with its estimate+ * omitted (backfill JSON) or dropped (status), so a reader could not tell+ * "free" from "price unknown". Both verbs now carry the price source, and+ * an unknown price reads `price unknown` in text and null in JSON, never+ * `$0.0000`. A table model keeps its text output unchanged.+ *+ * Deliberately not covered here: the dry run's census and apply+ * mechanics (search-vector-backfill.test.ts) and the plan arithmetic+ * (embedding-spend-preview.test.ts).+ */+import { afterEach, beforeEach, expect, test } from "bun:test";+import { join } from "node:path";++import { indexVault } from "../../src/core/search/indexer.ts";+import { planVectorBackfill } from "../../src/core/search/vector-backfill.ts";+import { FAKE_PROVIDER_KEY } from "../helpers/fake-credentials.ts";+import { startFakeHttp } from "../helpers/fake-http.ts";+import { createTempVault, makeConfig, writeMd } from "../helpers/search-fixtures.ts";+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";+import { runCli } from "../helpers/run-cli.ts";++const VEC_LOADABLE = sqliteVecLoadable();+const UNPRICED_MODEL = "zembed-1";+const TABLE_MODEL = "text-embedding-3-small";++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("vector-backfill-price");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => cleanup());++/** Index two notes keyword-only and write a CLI config naming `model`. */+async function seed(+ model: string,+ baseUrl = "http://127.0.0.1:9",+ extra: ReadonlyArray<string> = [],+): Promise<string> {+ writeMd(vault, "a.md", "# A\n\nFirst note about something.");+ writeMd(vault, "b.md", "# B\n\nSecond note discussing other things.");+ await indexVault(makeConfig({ vault, dbPath }));+ const config = join(vault, "cli-config.yaml");+ await Bun.write(+ config,+ [+ `vault: "${vault}"`,+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${baseUrl}"`,+ `embedding_model: ${model}`,+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ ...extra,+ "",+ ].join("\n"),+ );+ return config;+}++async function cli(config: string, argv: ReadonlyArray<string>) {+ const run = await runCli([...argv, "--vault", vault, "--db", dbPath, "--config", config], {+ env: { OPEN_SECOND_BRAIN_CONFIG: config },+ });+ expect(run.returncode).toBe(0);+ return run.stdout;+}++test.skipIf(!VEC_LOADABLE)(+ "an unpriced backfill says price unknown and null, never $0",+ async () => {+ const config = await seed(UNPRICED_MODEL);+ const payload = JSON.parse(+ await cli(config, ["search", "vector-backfill", "--json"]),+ ) as Record<string, unknown>;+ expect(payload["estimated_cost_usd"]).toBeNull();+ expect(payload["price_source"]).toBe("unknown");+ const human = await cli(config, ["search", "vector-backfill"]);+ expect(human).toContain("estimated cost: price unknown");+ expect(human).not.toContain("$0.0000");+ },+);++test.skipIf(!VEC_LOADABLE)("a table-priced backfill names its builtin source", async () => {+ const config = await seed(TABLE_MODEL);+ const payload = JSON.parse(await cli(config, ["search", "vector-backfill", "--json"])) as Record<+ string,+ unknown+ >;+ expect(payload["estimated_cost_usd"]).toBeGreaterThan(0);+ expect(payload["price_source"]).toBe("builtin");+ expect(await cli(config, ["search", "vector-backfill"])).toMatch(+ /estimated cost: \$\d+\.\d{4}\n/,+ );+});++const POSITIVE_GATE = "embedding_cost_gate_usd: 5";++test.skipIf(!VEC_LOADABLE)(+ "a dry run under a positive gate names the refusal its --apply would meet",+ async () => {+ const config = await seed(UNPRICED_MODEL, undefined, [POSITIVE_GATE]);+ const payload = JSON.parse(+ await cli(config, ["search", "vector-backfill", "--json"]),+ ) as Record<string, unknown>;+ expect(payload["gate_blocked"]).toBe(true);+ expect(payload["gate_reason"]).toBe("unpriced");+ // The advised step stays unforced; only the remedy line names the bypass.+ expect(payload["next_command"]).toBe("o2b search vector-backfill --apply");+ const human = await cli(config, ["search", "vector-backfill"]);+ expect(human).toContain(+ "cost gate: would refuse (unpriced); add --force-cost or set " ++ "embedding_price_model and embedding_price_usd_per_mtok",+ );+ expect(human).toContain("next: o2b search vector-backfill --apply\n");+ },+);++test.skipIf(!VEC_LOADABLE)(+ "a dry run over the gate names the over_cap refusal and the gate key as its remedy",+ async () => {+ // A declared rate at the ceiling prices the two notes far above the gate.+ const config = await seed(UNPRICED_MODEL, undefined, [+ POSITIVE_GATE,+ `embedding_price_model: ${UNPRICED_MODEL}`,+ 'embedding_price_usd_per_mtok: "1000000"',+ ]);+ const payload = JSON.parse(+ await cli(config, ["search", "vector-backfill", "--json"]),+ ) as Record<string, unknown>;+ expect(payload["gate_blocked"]).toBe(true);+ expect(payload["gate_reason"]).toBe("over_cap");+ expect(payload["next_command"]).toBe("o2b search vector-backfill --apply");+ const human = await cli(config, ["search", "vector-backfill"]);+ expect(human).toContain(+ "cost gate: would refuse (over_cap); add --force-cost or raise embedding_cost_gate_usd",+ );+ expect(human).toContain("next: o2b search vector-backfill --apply\n");+ },+);++test.skipIf(!VEC_LOADABLE)("without a gate the dry run carries no gate keys", async () => {+ const config = await seed(UNPRICED_MODEL);+ const payload = JSON.parse(await cli(config, ["search", "vector-backfill", "--json"])) as Record<+ string,+ unknown+ >;+ expect("gate_blocked" in payload).toBe(false);+ expect("gate_reason" in payload).toBe(false);+ expect("spend" in payload).toBe(false);+ expect(payload["next_command"]).toBe("o2b search vector-backfill --apply");+ expect(await cli(config, ["search", "vector-backfill"])).not.toContain("cost gate:");+});++test.skipIf(!VEC_LOADABLE)("a forced apply names its spend receipt", async () => {+ const server = await startFakeHttp();+ try {+ const config = await seed(UNPRICED_MODEL, server.url, [POSITIVE_GATE]);+ const payload = JSON.parse(+ await cli(config, ["search", "vector-backfill", "--apply", "--force-cost", "--json"]),+ ) as Record<string, unknown>;+ expect(payload["embedded"]).toBe(2);+ expect(payload["spend"]).toEqual({+ model: UNPRICED_MODEL,+ tokens: expect.any(Number),+ estimated_usd: null,+ price_source: "unknown",+ forced: true,+ });+ } finally {+ await server.close();+ }+});++test("status of an unpriced model says price unknown and reports its source", async () => {+ const config = await seed(UNPRICED_MODEL);+ const payload = JSON.parse(await cli(config, ["search", "status", "--json"])) as Record<+ string,+ unknown+ >;+ expect(payload["estimated_refresh_cost_usd"]).toBeNull();+ expect(payload["refresh_price_source"]).toBe("unknown");+ const human = await cli(config, ["search", "status"]);+ expect(human).toContain("refresh_cost_est: price unknown");+ expect(human).not.toContain("$0.0000");+});++test.skipIf(!VEC_LOADABLE)(+ "status of a fully embedded index on an unpriced model prints no unknown spend",+ async () => {+ const server = await startFakeHttp();+ try {+ const config = await seed(UNPRICED_MODEL, server.url);+ await planVectorBackfill(+ makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: server.url,+ model: UNPRICED_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ dimension: 4,+ timeoutMs: 5_000,+ },+ }),+ { apply: true },+ );+ const payload = JSON.parse(await cli(config, ["search", "status", "--json"])) as Record<+ string,+ unknown+ >;+ expect(payload["estimated_refresh_cost_usd"]).toBe(0);+ expect(payload["refresh_price_source"]).toBe("unknown");+ expect(await cli(config, ["search", "status"])).not.toContain("refresh_cost_est");+ } finally {+ await server.close();+ }+ },+);diff --git a/tests/core/architecture/verdict-vocabulary-census.test.ts b/tests/core/architecture/verdict-vocabulary-census.test.tsindex 104d21f4..0be82e55 100644--- a/tests/core/architecture/verdict-vocabulary-census.test.ts+++ b/tests/core/architecture/verdict-vocabulary-census.test.ts@@ -200,6 +200,16 @@ import {READINESS_STATUS,READINESS_STATUSES,} from "../../../src/core/doctor-readiness.ts";+import {+ EMBEDDING_GATE_REASON,+ EMBEDDING_GATE_REASONS,+ isEmbeddingGateReason,+} from "../../../src/core/search/embedding-spend.ts";+import {+ EMBEDDING_PRICE_SOURCE,+ EMBEDDING_PRICE_SOURCES,+ isEmbeddingPriceSource,+} from "../../../src/core/search/embeddings/pricing.ts";import {isRetrievalDegradationCode,RETRIEVAL_DEGRADATION,@@ -556,6 +566,24 @@ const CENSUS: ReadonlyArray<VocabularyUnderCensus> = Object.freeze([guard: isReadinessStatus,},{ name: "GATE_MODE", values: GATE_MODE, members: GATE_MODES, guard: isGateMode },+ {+ // Who stated an embedding price: the value leaves TypeScript in spend+ // receipts, the maintenance_spend metric and the JSON surfaces, so a+ // reader can tell a builtin or operator price from an unknown one.+ name: "EMBEDDING_PRICE_SOURCE",+ values: EMBEDDING_PRICE_SOURCE,+ members: EMBEDDING_PRICE_SOURCES,+ guard: isEmbeddingPriceSource,+ },+ {+ // Why the embedding cost gate refused: over the cap on a known price,+ // or unpriced under an explicit positive gate. Rendered in previews and+ // JSON so a refusal says which of the two it is.+ name: "EMBEDDING_GATE_REASON",+ values: EMBEDDING_GATE_REASON,+ members: EMBEDDING_GATE_REASONS,+ guard: isEmbeddingGateReason,+ },{name: "SCHEMA_PACK_INTEGRITY",values: SCHEMA_PACK_INTEGRITY,@@ -1747,7 +1775,7 @@ const SCANNED = scanVocabularies(SOURCE_TREE);* How many four-piece vocabularies `src/` currently holds. Measured, and* kept as an equality rather than a floor - see the population test.*/-const VOCABULARY_POPULATION = 90;+const VOCABULARY_POPULATION = 92;const REGISTERED = new Map(CENSUS.map((entry) => [entry.name, entry] as const));describe("verdict vocabulary census", () => {diff --git a/tests/core/brain/belief-semantic.test.ts b/tests/core/brain/belief-semantic.test.tsnew file mode 100644index 00000000..301dad9d--- /dev/null+++ b/tests/core/brain/belief-semantic.test.ts@@ -0,0 +1,513 @@+/**+ * Belief vector scorer and loader (Honest Embedding Spend, task 11).+ *+ * Pins the read side of `brain_context_pack` `semantic` mode: the pure+ * scorer takes the maximum cosine per belief note over the vectors it is+ * handed, counts a row written by another model or at another dimension+ * as unembedded rather than scored, and orders ties by path; the+ * store-backed loader reads the stored vectors of `Brain/preferences/`+ * and `Brain/retired/` notes, embeds the query exactly once with the+ * `query` prefix kind, refuses a blocked tier and a missing sqlite-vec+ * by name BEFORE any query embed, and discloses the model, the price+ * source, the query tokens and the nullable query cost.+ *+ * Deliberately not covered here: the reach filter and the+ * `BELIEF_VECTORS_MISSING` refusal, which are decided after the pack+ * knows its kept set (context-pack-semantic.test.ts and the MCP tool+ * test).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";++import {+ BELIEF_VECTORS_BACKFILL_COMMAND,+ loadBeliefSemanticRelevance,+ scoreBeliefsByVector,+ type StoredBeliefVector,+} from "../../../src/core/brain/belief-semantic.ts";+import { TRANSPORT_REACH } from "../../../src/core/graph/transport-reach.ts";+import { EMBEDDING_PRICE_SOURCE } from "../../../src/core/search/embeddings/pricing.ts";+import { LOCAL_EMBEDDING_MODEL } from "../../../src/core/search/embeddings/signature.ts";+import type { EmbedKind, EmbeddingProvider } from "../../../src/core/search/embeddings/contract.ts";+import { indexVault } from "../../../src/core/search/indexer.ts";+import { EMBEDDING_DIMENSION_STATE_KEY, Store } from "../../../src/core/search/store.ts";+import { SearchError } from "../../../src/core/search/search-error.ts";+import type {+ ResolvedEmbeddingConfig,+ ResolvedSearchConfig,+} from "../../../src/core/search/types.ts";+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++const MODEL = "c11-model";+const OTHER_MODEL = "c11-older-model";+const DIMENSION = 4;+const QUERY = "how should replies be phrased";+const VEC_LOADABLE = sqliteVecLoadable();++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ ({ vault, dbPath, cleanup } = createTempVault("c11-belief-semantic"));+});++afterEach(() => cleanup());++function unit(values: number[]): number[] {+ const norm = Math.hypot(...values);+ return values.map((v) => v / norm);+}++function row(values: number[], model = MODEL, dimension = values.length): StoredBeliefVector {+ return { vector: Float32Array.from(unit(values)), model, dimension };+}++function vecConfig(semantic: Partial<ResolvedEmbeddingConfig> = {}): ResolvedSearchConfig {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ baseUrl: "https://x/v1",+ model: MODEL,+ apiKey: "k",+ dimension: DIMENSION,+ ...semantic,+ },+ });+}++/** A provider that records every embed call and answers with one fixed vector. */+function countingProvider(vector: number[]): {+ provider: EmbeddingProvider;+ calls: Array<{ texts: ReadonlyArray<string>; kind: EmbedKind | undefined }>;+} {+ const calls: Array<{ texts: ReadonlyArray<string>; kind: EmbedKind | undefined }> = [];+ const provider: EmbeddingProvider = {+ name: "c11-counting",+ model: MODEL,+ dimension: vector.length,+ async embed(texts, kind) {+ calls.push({ texts, kind });+ return texts.map(() => [...vector]);+ },+ async ping() {+ return { ok: true, dimension: vector.length };+ },+ };+ return { provider, calls };+}++/** A provider that fails the test the moment anything asks it to embed. */+const throwingProvider: EmbeddingProvider = {+ name: "c11-throwing",+ model: MODEL,+ dimension: DIMENSION,+ async embed() {+ throw new Error("the query must not be embedded before the refusal");+ },+ async ping() {+ return { ok: false, reason: "never pinged" };+ },+};++function writeBelief(relPath: string, id: string, principle: string): void {+ writeMd(+ vault,+ relPath,+ `---\nid: ${id}\ntopic: ${id}\nprinciple: ${principle}\n---\n\n## Principle\n\n${principle}\n`,+ );+}++async function plant(+ config: ResolvedSearchConfig,+ byPath: Record<string, { vector: number[]; model?: string }>,+): Promise<void> {+ const store = await Store.open(config, { mode: "write" });+ try {+ for (const [path, planted] of Object.entries(byPath)) {+ const docId = store.getDocumentIdByPath(path);+ expect(docId).not.toBeNull();+ for (const chunk of store.chunksForDocument(docId!)) {+ store.vecUpsert(+ chunk.id,+ unit(planted.vector),+ planted.model ?? MODEL,+ DIMENSION,+ `eh-${path}-${chunk.id}`,+ );+ }+ }+ } finally {+ await store.close();+ }+}++describe("scoreBeliefsByVector", () => {+ test("scores each path by its best chunk and orders by score, then path", () => {+ const result = scoreBeliefsByVector({+ model: MODEL,+ queryVector: unit([1, 0, 0, 0]),+ vectorsByPath: new Map([+ ["Brain/preferences/b.md", [row([0, 1, 0, 0]), row([1, 0.1, 0, 0])]],+ ["Brain/preferences/a.md", [row([1, 0.1, 0, 0])]],+ ["Brain/preferences/c.md", [row([0, 0, 1, 0])]],+ ]),+ });+ expect(result.order).toEqual([+ "Brain/preferences/a.md",+ "Brain/preferences/b.md",+ "Brain/preferences/c.md",+ ]);+ expect(result.relevanceByPath.get("Brain/preferences/a.md")).toBeCloseTo(+ result.relevanceByPath.get("Brain/preferences/b.md")!,+ 12,+ );+ expect(result.relevanceByPath.get("Brain/preferences/c.md")).toBeCloseTo(0, 12);+ expect(result.scored).toBe(3);+ expect(result.unembedded).toEqual([]);+ });++ test("a note's relevance is its best chunk, not the sum of its chunks", () => {+ const result = scoreBeliefsByVector({+ model: MODEL,+ queryVector: [1, 0, 0, 0],+ vectorsByPath: new Map([+ ["Brain/preferences/a.md", [row([0.6, 0.8, 0, 0]), row([0.6, 0, 0.8, 0])]],+ ["Brain/preferences/b.md", [row([0.9, Math.sqrt(0.19), 0, 0])]],+ ]),+ });+ expect(result.order).toEqual(["Brain/preferences/b.md", "Brain/preferences/a.md"]);+ expect(result.relevanceByPath.get("Brain/preferences/a.md")).toBeCloseTo(0.6, 6);+ });++ test("rows from another model or dimension count as unembedded, never as scored", () => {+ const result = scoreBeliefsByVector({+ model: MODEL,+ queryVector: unit([1, 0, 0, 0]),+ vectorsByPath: new Map([+ ["Brain/preferences/stale-model.md", [row([1, 0, 0, 0], OTHER_MODEL)]],+ ["Brain/preferences/stale-dim.md", [row([1, 0, 0], MODEL)]],+ ["Brain/preferences/none.md", []],+ ["Brain/preferences/fresh.md", [row([0, 1, 0, 0]), row([1, 0, 0, 0], OTHER_MODEL)]],+ ]),+ });+ expect(result.scored).toBe(1);+ expect([...result.relevanceByPath.keys()]).toEqual(["Brain/preferences/fresh.md"]);+ expect(result.relevanceByPath.get("Brain/preferences/fresh.md")).toBeCloseTo(0, 12);+ expect(result.unembedded).toEqual([+ "Brain/preferences/none.md",+ "Brain/preferences/stale-dim.md",+ "Brain/preferences/stale-model.md",+ ]);+ });++ test("a stored vector longer than its recorded dimension is unusable, never a truncated score", () => {+ // Recorded at the query's dimension, but the blob holds more values:+ // a cosine over the first DIMENSION of them is a score of a vector+ // nobody stored.+ const query = unit([1, 0, 0, 0]);+ const scores = scoreBeliefsByVector({+ model: MODEL,+ queryVector: query,+ vectorsByPath: new Map([+ ["Brain/preferences/long.md", [{ ...row([1, 0, 0, 0, 1]), dimension: DIMENSION }]],+ ["Brain/preferences/ok.md", [row([0, 1, 0, 0])]],+ ]),+ });+ expect(scores.unembedded).toEqual(["Brain/preferences/long.md"]);+ expect(scores.order).toEqual(["Brain/preferences/ok.md"]);+ });++ // The store guard rejects such rows only since v1.32.0; an older index+ // can still hold one, and its NaN cosine must not reach the order.+ test("a stored zero or non-finite vector is unusable, never a NaN relevance", () => {+ const stored = (values: number[]): StoredBeliefVector => ({+ vector: Float32Array.from(values),+ model: MODEL,+ dimension: values.length,+ });+ const result = scoreBeliefsByVector({+ model: MODEL,+ queryVector: unit([1, 0, 0, 0]),+ vectorsByPath: new Map([+ ["Brain/preferences/zero.md", [stored([0, 0, 0, 0])]],+ ["Brain/preferences/nan.md", [stored([Number.NaN, 0, 0, 0])]],+ ["Brain/preferences/mixed.md", [stored([0, 0, 0, 0]), row([1, 0, 0, 0])]],+ ]),+ });+ expect([...result.relevanceByPath.keys()]).toEqual(["Brain/preferences/mixed.md"]);+ expect(result.relevanceByPath.get("Brain/preferences/mixed.md")).toBeCloseTo(1, 6);+ expect(result.unembedded).toEqual(["Brain/preferences/nan.md", "Brain/preferences/zero.md"]);+ });+});++describe("loadBeliefSemanticRelevance", () => {+ test.skipIf(!VEC_LOADABLE)(+ "reads stored belief vectors, embeds the query once and discloses the spend",+ async () => {+ writeBelief("Brain/preferences/pref-near.md", "pref-near", "Keep answers short");+ writeBelief("Brain/preferences/pref-far.md", "pref-far", "Use tabs in makefiles");+ writeBelief("Brain/retired/pref-old.md", "pref-old", "Answer in long form");+ writeBelief("Brain/preferences/pref-stale.md", "pref-stale", "Reply politely");+ writeBelief("Brain/preferences/pref-new.md", "pref-new", "Never guess");+ writeMd(vault, "Notes/elsewhere.md", "# Elsewhere\n\nA note that is not a belief.\n");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, {+ "Brain/preferences/pref-near.md": { vector: [1, 0.05, 0, 0] },+ "Brain/preferences/pref-far.md": { vector: [0, 0, 0, 1] },+ "Brain/retired/pref-old.md": { vector: [0.7, 0.7, 0, 0] },+ "Brain/preferences/pref-stale.md": { vector: [1, 0, 0, 0], model: OTHER_MODEL },+ "Notes/elsewhere.md": { vector: [1, 0, 0, 0] },+ });+ const { provider, calls } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect(calls).toEqual([{ texts: [QUERY], kind: "query" }]);+ expect(loaded.order).toEqual([+ "Brain/preferences/pref-near.md",+ "Brain/retired/pref-old.md",+ "Brain/preferences/pref-far.md",+ ]);+ expect(loaded.relevanceByPath.has("Notes/elsewhere.md")).toBe(false);+ expect(loaded.unembedded).toEqual([+ "Brain/preferences/pref-new.md",+ "Brain/preferences/pref-stale.md",+ ]);+ expect(loaded.scored).toBe(3);+ expect(loaded.report.model).toBe(MODEL);+ expect(loaded.report.priceSource).toBe(EMBEDDING_PRICE_SOURCE.unknown);+ expect(loaded.report.estimatedUsd).toBeNull();+ expect(loaded.report.queryTokens).toBeGreaterThan(0);+ expect(loaded.warnings).toEqual([]);+ },+ );++ test.skipIf(!VEC_LOADABLE)("an operator-priced model reports its query cost", async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig({ priceOverride: { model: MODEL, usdPerMtok: 2 } });+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ const { provider } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect(loaded.report.priceSource).toBe(EMBEDDING_PRICE_SOURCE.operator);+ expect(loaded.report.estimatedUsd).toBeCloseTo((loaded.report.queryTokens / 1_000_000) * 2, 12);+ });++ test.skipIf(!VEC_LOADABLE)(+ "the local provider prices its query as the local model, not a leftover embedding_model",+ async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig({ provider: "local", apiKey: null });+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ const { provider } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect(loaded.scored).toBe(1);+ expect(loaded.report.model).toBe(LOCAL_EMBEDDING_MODEL);+ expect(loaded.report.priceSource).toBe(EMBEDDING_PRICE_SOURCE.builtin);+ expect(loaded.report.estimatedUsd).toBe(0);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "no belief row under the model refuses with BELIEF_VECTORS_MISSING before any query embed",+ async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ writeBelief("Brain/preferences/pref-b.md", "pref-b", "Reply politely");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, {+ "Brain/preferences/pref-b.md": { vector: [1, 0, 0, 0], model: OTHER_MODEL },+ });++ const refusal = await loadBeliefSemanticRelevance(config, QUERY, {+ provider: throwingProvider,+ }).catch((e: unknown) => e);++ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("BELIEF_VECTORS_MISSING");+ expect((refusal as SearchError).message).toContain(BELIEF_VECTORS_BACKFILL_COMMAND);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a row under the model at another dimension still embeds and reads as unembedded",+ async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ const { provider, calls } = countingProvider(unit([1, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect(calls).toHaveLength(1);+ expect(loaded.scored).toBe(0);+ expect(loaded.unembedded).toEqual(["Brain/preferences/pref-a.md"]);+ },+ );++ test("a disabled tier refuses with EMBEDDING_DISABLED before any query embed", async () => {+ const config = vecConfig({ enabled: false });+ const refusal = await loadBeliefSemanticRelevance(config, QUERY, {+ provider: throwingProvider,+ }).catch((e: unknown) => e);+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_DISABLED");+ });++ test("a missing credential refuses with EMBEDDING_KEY_MISSING before any query embed", async () => {+ const config = vecConfig({ apiKey: null });+ const refusal = await loadBeliefSemanticRelevance(config, QUERY, {+ provider: throwingProvider,+ }).catch((e: unknown) => e);+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_KEY_MISSING");+ });++ // No belief note on purpose: with one, the per-document vector read+ // throws the same code, so only an empty belief set proves the guard+ // runs before the query embed.+ test("a store without sqlite-vec refuses with VEC_EXTENSION_UNAVAILABLE before any query embed", async () => {+ writeMd(vault, "notes/n.md", "# A note\n\nNot a belief.\n");+ const config = vecConfig();+ await indexVault(config);+ const refusal = await loadBeliefSemanticRelevance(config, QUERY, {+ provider: throwingProvider,+ loadVec: false,+ }).catch((e: unknown) => e);+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("VEC_EXTENSION_UNAVAILABLE");+ });++ test.skipIf(!VEC_LOADABLE)(+ "a sibling directory sharing the prefix string is not a belief",+ async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ writeBelief("Brain/preferences-archive/x.md", "pref-x", "Keep answers short");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, {+ "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] },+ "Brain/preferences-archive/x.md": { vector: [1, 0, 0, 0] },+ });+ const { provider } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect([...loaded.relevanceByPath.keys()]).toEqual(["Brain/preferences/pref-a.md"]);+ expect(loaded.unembedded).toEqual([]);+ },+ );++ for (const [label, vector] of [+ ["an empty", []],+ ["a non-finite", [Number.NaN, 0, 0, 0]],+ ] as const) {+ test.skipIf(!VEC_LOADABLE)(+ `${label} query vector is refused with EMBEDDING_INVALID_VECTOR`,+ async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ const { provider } = countingProvider([...vector]);++ const refusal = await loadBeliefSemanticRelevance(config, QUERY, { provider }).catch(+ (e: unknown) => e,+ );++ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_INVALID_VECTOR");+ expect((refusal as SearchError).message).toContain("belief semantic query");+ // The provider answered, so the call was paid: the refusal names it.+ expect((refusal as SearchError).message).toContain("the query embed was still spent");+ expect((refusal as SearchError).message).toContain(`model ${MODEL}`);+ expect((refusal as SearchError).message).toContain("price source unknown");+ },+ );+ }++ describe("an unpriced query embed under a positive cost gate", () => {+ async function seed(semantic: Partial<ResolvedEmbeddingConfig>): Promise<ResolvedSearchConfig> {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig(semantic);+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ return config;+ }++ test.skipIf(!VEC_LOADABLE)(+ "is refused for a remote caller with EMBEDDING_COST_UNPRICED before anything is embedded",+ async () => {+ const config = await seed({ costGateUsd: 1 });++ const refusal = await loadBeliefSemanticRelevance(config, QUERY, {+ provider: throwingProvider,+ reach: TRANSPORT_REACH.remote,+ }).catch((e: unknown) => e);++ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("EMBEDDING_COST_UNPRICED");+ },+ );++ test.skipIf(!VEC_LOADABLE)("is embedded once and disclosed for a local caller", async () => {+ const config = await seed({ costGateUsd: 1 });+ const { provider, calls } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, {+ provider,+ reach: TRANSPORT_REACH.local,+ });++ expect(calls).toHaveLength(1);+ expect(loaded.report.priceSource).toBe(EMBEDDING_PRICE_SOURCE.unknown);+ });++ test.skipIf(!VEC_LOADABLE)(+ "is embedded for a remote caller once the model is priced or the gate is off",+ async () => {+ await seed({});+ for (const semantic of [+ { costGateUsd: 1, priceOverride: { model: MODEL, usdPerMtok: 2 } },+ { costGateUsd: 0 },+ ]) {+ const config = vecConfig(semantic);+ const { provider, calls } = countingProvider(unit([1, 0, 0, 0]));+ // eslint-disable-next-line no-await-in-loop+ await loadBeliefSemanticRelevance(config, QUERY, {+ provider,+ reach: TRANSPORT_REACH.remote,+ });+ expect(calls).toHaveLength(1);+ }+ },+ );+ });++ test.skipIf(!VEC_LOADABLE)("a contradicted embedding identity reaches the warnings", async () => {+ writeBelief("Brain/preferences/pref-a.md", "pref-a", "Keep answers short");+ const config = vecConfig();+ await indexVault(config);+ await plant(config, { "Brain/preferences/pref-a.md": { vector: [1, 0, 0, 0] } });+ const store = await Store.open(config, { mode: "write" });+ store.setState(EMBEDDING_DIMENSION_STATE_KEY, "8");+ await store.close();+ const { provider } = countingProvider(unit([1, 0, 0, 0]));++ const loaded = await loadBeliefSemanticRelevance(config, QUERY, { provider });++ expect(loaded.warnings).toHaveLength(1);+ expect(loaded.warnings[0]).toContain(EMBEDDING_DIMENSION_STATE_KEY);+ });+});diff --git a/tests/core/brain/context-pack-semantic.test.ts b/tests/core/brain/context-pack-semantic.test.tsnew file mode 100644index 00000000..53e8016b--- /dev/null+++ b/tests/core/brain/context-pack-semantic.test.ts@@ -0,0 +1,256 @@+/**+ * Semantic query mode on the context-pack lane (Honest Embedding Spend,+ * task 12).+ *+ * `ranked` orders by token overlap, so a paraphrase that shares no word+ * with the belief it means reads relevance 0 and falls to recency. The+ * `semantic` mode reads a relevance map the async boundary computed from+ * stored vectors, keyed by vault-relative path, in the same comparator+ * slot `ranked` uses. Pinned here:+ * 1. A zero-overlap paraphrase orders the closest belief first within+ * its tier, and tier stays the coarse gate.+ * 2. `semantic.scored` and `semantic.unembedded` are computed after the+ * reach filter, so a withheld page appears in neither, and its score+ * cannot change the order.+ * 3. A `semantic` call without a map throws a named error instead of+ * silently packing by recency.+ * 4. When no kept candidate is scored the pack refuses with+ * `BELIEF_VECTORS_MISSING`, naming the scoped backfill, before any+ * receipt is written; a withheld page with a vector cannot lift the+ * refusal, and an empty candidate set is an empty pack.+ * 5. `substring` and `ranked` carry no `semantic` report.+ *+ * Not covered: how the map is built (belief-semantic.test.ts) and the+ * MCP refusal when nothing kept is scored (context-pack-tool.test.ts).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++import {+ BELIEF_VECTORS_BACKFILL_COMMAND,+ ContextPackSemanticRelevanceMissingError,+ packContext,+} from "../../../src/core/brain/context-pack.ts";+import { listContextReceipts } from "../../../src/core/brain/context-receipts.ts";+import { SearchError } from "../../../src/core/search/search-error.ts";++let vault: string;++/** Shares no token with any principle below. */+const PARAPHRASE = "how terse should my replies be";++beforeEach(() => {+ vault = mkdtempSync(join(tmpdir(), "o2b-c12-pack-semantic-"));+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+ mkdirSync(join(vault, "Brain", "retired"), { recursive: true });+});++afterEach(() => {+ rmSync(vault, { recursive: true, force: true });+});++function writePref(+ slug: string,+ fields: { principle: string; tier?: string; created_at: string },+): void {+ const lines = ["---", `id: pref-${slug}`, `topic: ${slug}`, `principle: ${fields.principle}`];+ if (fields.tier) lines.push(`tier: ${fields.tier}`);+ lines.push(`created_at: ${fields.created_at}`, "---", "", fields.principle);+ writeFileSync(join(vault, "Brain", "preferences", `pref-${slug}.md`), lines.join("\n"));+}++function rel(slug: string): string {+ return `Brain/preferences/pref-${slug}.md`;+}++function writeBeliefs(): void {+ writePref("brevity", {+ principle: "keep answers short",+ tier: "core",+ created_at: "2026-01-01T00:00:00Z",+ });+ writePref("deploy", {+ principle: "never ship on friday",+ tier: "core",+ created_at: "2026-01-03T00:00:00Z",+ });+ writePref("imports", {+ principle: "use explicit imports only",+ tier: "core",+ created_at: "2026-01-02T00:00:00Z",+ });+ writePref("tabs", {+ principle: "indent makefiles with tabs",+ tier: "peripheral",+ created_at: "2026-01-04T00:00:00Z",+ });+}++const RELEVANCE: ReadonlyMap<string, number> = new Map([+ [rel("brevity"), 0.92],+ [rel("deploy"), 0.11],+ [rel("tabs"), 0.99],+]);++describe("packContext semantic mode", () => {+ test("a zero-overlap paraphrase orders the closest belief first within its tier", () => {+ writeBeliefs();+ const ranked = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "ranked",+ });+ expect(ranked.items[0]!.id).toBe("pref-deploy");++ const report = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: RELEVANCE,+ });+ expect(report.items.map((i) => i.id)).toEqual([+ "pref-brevity",+ "pref-deploy",+ "pref-imports",+ "pref-tabs",+ ]);+ expect(report.skipped).toEqual([]);+ expect(report.semantic).toEqual({ scored: 3, unembedded: ["pref-imports"] });+ });++ test("a scored belief with a negative cosine still ranks above an unembedded one", () => {+ writePref("brevity", {+ principle: "keep answers short",+ tier: "core",+ created_at: "2026-01-01T00:00:00Z",+ });+ writePref("imports", {+ principle: "use explicit imports only",+ tier: "core",+ created_at: "2026-01-02T00:00:00Z",+ });+ const report = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: new Map([[rel("brevity"), -0.4]]),+ });+ expect(report.items.map((i) => i.id)).toEqual(["pref-brevity", "pref-imports"]);+ expect(report.semantic).toEqual({ scored: 1, unembedded: ["pref-imports"] });+ });++ test("unembedded is sorted by id even where recency orders the items the other way", () => {+ writePref("brevity", {+ principle: "keep answers short",+ tier: "core",+ created_at: "2026-01-01T00:00:00Z",+ });+ writePref("alpha", {+ principle: "the oldest",+ tier: "core",+ created_at: "2026-01-02T00:00:00Z",+ });+ writePref("zeta", {+ principle: "the newest",+ tier: "core",+ created_at: "2026-01-09T00:00:00Z",+ });+ const report = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: new Map([[rel("brevity"), 0.5]]),+ });+ expect(report.items.map((i) => i.id)).toEqual(["pref-brevity", "pref-zeta", "pref-alpha"]);+ expect(report.semantic).toEqual({ scored: 1, unembedded: ["pref-alpha", "pref-zeta"] });+ });++ test("scored and unembedded are computed after the reach filter", () => {+ writeBeliefs();+ const withheld = join(vault, rel("brevity"));+ const report = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: RELEVANCE,+ visible: (abs) => abs !== withheld,+ });+ expect(report.items.map((i) => i.id)).toEqual(["pref-deploy", "pref-imports", "pref-tabs"]);+ expect(report.semantic).toEqual({ scored: 2, unembedded: ["pref-imports"] });++ const hiddenUnembedded = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: RELEVANCE,+ visible: (abs) => abs !== join(vault, rel("imports")),+ });+ expect(hiddenUnembedded.semantic).toEqual({ scored: 3, unembedded: [] });+ });++ test("a semantic call without a relevance map throws a named error", () => {+ writeBeliefs();+ expect(() =>+ packContext(vault, { maxTokens: 10_000, query: PARAPHRASE, queryMode: "semantic" }),+ ).toThrow(ContextPackSemanticRelevanceMissingError);+ });++ test("no scored kept candidate refuses by name before any receipt is written", () => {+ writeBeliefs();+ const onlyBrevityVisible = join(vault, rel("brevity"));+ const refusal = (() => {+ try {+ packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: new Map([[rel("deploy"), 0.5]]),+ visible: (abs) => abs === onlyBrevityVisible,+ receipt: { trigger: "context_pack", host: "test" },+ });+ } catch (e) {+ return e;+ }+ return null;+ })();+ expect(refusal).toBeInstanceOf(SearchError);+ expect((refusal as SearchError).code).toBe("BELIEF_VECTORS_MISSING");+ expect((refusal as SearchError).message).toContain(BELIEF_VECTORS_BACKFILL_COMMAND);+ expect(BELIEF_VECTORS_BACKFILL_COMMAND).toBe(+ "o2b search vector-backfill --path Brain/preferences/ --path Brain/retired/ --apply",+ );+ expect(listContextReceipts(vault, {})).toEqual([]);++ // Control: the same call with the scored page in reach does write one.+ packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: new Map([[rel("deploy"), 0.5]]),+ receipt: { trigger: "context_pack", host: "test" },+ });+ expect(listContextReceipts(vault, {}).length).toBe(1);+ });++ test("an empty candidate set is an empty pack, not a refusal", () => {+ const report = packContext(vault, {+ maxTokens: 10_000,+ query: PARAPHRASE,+ queryMode: "semantic",+ semanticRelevance: new Map(),+ });+ expect(report.items).toEqual([]);+ expect(report.semantic).toEqual({ scored: 0, unembedded: [] });+ });++ test("substring and ranked carry no semantic report", () => {+ writeBeliefs();+ for (const queryMode of ["substring", "ranked"] as const) {+ const report = packContext(vault, { maxTokens: 10_000, query: "short", queryMode });+ expect("semantic" in report).toBe(false);+ }+ });+});diff --git a/tests/core/brain/time.test.ts b/tests/core/brain/time.test.tsindex 7517b3c4..be2d38a5 100644--- a/tests/core/brain/time.test.ts+++ b/tests/core/brain/time.test.ts@@ -76,6 +76,43 @@ describe("fileAgeMs", () => {}});+ // `nowMs` is a whole millisecond (`Date.now()`), while the filesystem+ // stamps a sub-millisecond mtime: a file written within the clock's own+ // millisecond would otherwise read a fraction of a millisecond in the+ // future, and `msToWholeDays` would floor that to a whole day ago.+ test("a file written within the clock's millisecond is age zero, not in the future", () => {+ const dir = makeTempDir();+ try {+ const path = join(dir, "fresh.md");+ writeFileSync(path, "x");+ const stampedMs = NOW_MS + 0.5;+ utimesSync(path, stampedMs / 1000, stampedMs / 1000);+ // The fixture has to carry the fraction before the product is asked.+ expect(statSync(path).mtimeMs).toBeGreaterThan(NOW_MS);+ expect(fileAgeMs(path, NOW_MS)).toBe(0);+ expect(msToWholeDays(fileAgeMs(path, NOW_MS) ?? Number.NaN)).toBe(0);+ } finally {+ rmSync(dir, { recursive: true, force: true });+ }+ });++ // Only the same-millisecond artefact is absorbed: an older file keeps its+ // fractional age, so one just under a day old is not rounded up into it.+ test("a file just under a day old keeps its fraction and reads zero days", () => {+ const dir = makeTempDir();+ try {+ const path = join(dir, "almost-a-day.md");+ writeFileSync(path, "x");+ const stampedMs = NOW_MS - MS_PER_DAY + 0.5;+ utimesSync(path, stampedMs / 1000, stampedMs / 1000);+ expect(statSync(path).mtimeMs).toBeGreaterThan(NOW_MS - MS_PER_DAY);+ expect(fileAgeMs(path, NOW_MS)).toBeLessThan(MS_PER_DAY);+ expect(msToWholeDays(fileAgeMs(path, NOW_MS) ?? Number.NaN)).toBe(0);+ } finally {+ rmSync(dir, { recursive: true, force: true });+ }+ });+test("returns null for a path that does not exist", () => {const dir = makeTempDir();try {diff --git a/tests/core/maintenance/self-heal-reindex.test.ts b/tests/core/maintenance/self-heal-reindex.test.tsindex c0c26891..b39f6e5f 100644--- a/tests/core/maintenance/self-heal-reindex.test.ts+++ b/tests/core/maintenance/self-heal-reindex.test.ts@@ -32,6 +32,7 @@ import {import { cmdSearchReindex } from "../../../src/cli/search/verbs/indexing.ts";import { resolveSearchConfig } from "../../../src/core/search/index.ts";import { acquireWriterLock } from "../../../src/core/search/store/writer-lock.ts";+import { waitForSelfHealUpgradeWorker } from "../../helpers/self-heal-children.ts";let vault: string;let configHome: string;@@ -61,12 +62,22 @@ beforeEach(() => {bootstrapBrain(vault, { configPath });});-afterEach(() => {- if (prevConfigEnv === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];- else process.env["OPEN_SECOND_BRAIN_CONFIG"] = prevConfigEnv;- rmSync(vault, { recursive: true, force: true });- rmSync(configHome, { recursive: true, force: true });-});+afterEach(async () => {+ // Every background `ensureVaultCurrent` also starts the detached+ // managed-file upgrade worker, which no test here waits for. Removing the+ // vault under it lets its metrics row recreate the directory after the+ // test is gone, so the removal waits for that worker first. The reindex+ // children are awaited by the tests that spawn them, and two tests write+ // spawn rows no child will ever pair.+ try {+ await waitForSelfHealUpgradeWorker(vault, CHILD_BUDGET_MS);+ } finally {+ if (prevConfigEnv === undefined) delete process.env["OPEN_SECOND_BRAIN_CONFIG"];+ else process.env["OPEN_SECOND_BRAIN_CONFIG"] = prevConfigEnv;+ rmSync(vault, { recursive: true, force: true });+ rmSync(configHome, { recursive: true, force: true });+ }+}, CHILD_BUDGET_MS + 10_000);function dbPath(): string {return resolveSearchConfig({ vault, configPath }).dbPath;diff --git a/tests/core/maintenance/self-heal-upgrade.test.ts b/tests/core/maintenance/self-heal-upgrade.test.tsindex cbc71f3b..d439a28e 100644--- a/tests/core/maintenance/self-heal-upgrade.test.ts+++ b/tests/core/maintenance/self-heal-upgrade.test.ts@@ -8,7 +8,7 @@* (and with it the whole upgrade) cannot proceed.*/-import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test";import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { dirname, join } from "node:path";@@ -16,6 +16,7 @@ import { dirname, join } from "node:path";import { bootstrapBrain } from "../../../src/core/brain/init.ts";import { listMetrics } from "../../../src/core/brain/metrics.ts";import { brainDirs, brainManualPath } from "../../../src/core/brain/paths.ts";+import * as upgradeModule from "../../../src/core/brain/upgrade.ts";import { planUpgrade } from "../../../src/core/brain/upgrade.ts";import { atomicWriteFileSync } from "../../../src/core/fs-atomic.ts";import {@@ -313,3 +314,86 @@ describe("operator surfaces after a failure", () => {expect(planUpgrade(vault).pending).toBe(0);});});++describe("a vault removed while the worker runs", () => {+ // The worker checks the vault once before planning. A vault removed (or+ // moved) after that check must not come back as a failure marker, a+ // metrics row or rewritten managed files.+ const realPlan = upgradeModule.planUpgrade;++ test("between the plan and the rewrite: nothing is written, no directory recreated", () => {+ makeManualStale();+ const spy = spyOn(upgradeModule, "planUpgrade").mockImplementationOnce((v) => {+ const plan = realPlan(v);+ rmSync(vault, { recursive: true, force: true });+ return plan;+ });+ try {+ const run = runSelfHealUpgrade(vault, { now: T0 });++ expect(run.outcome).toBe(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);+ expect(run.filesUpdated).toEqual([]);+ expect(existsSync(vault)).toBe(false);+ } finally {+ spy.mockRestore();+ }+ });++ test("during the plan: the failure is not recorded into a recreated vault", () => {+ makeManualStale();+ const spy = spyOn(upgradeModule, "planUpgrade").mockImplementationOnce(() => {+ rmSync(vault, { recursive: true, force: true });+ throw new Error("ENOENT: the vault went away mid-plan");+ });+ try {+ const run = runSelfHealUpgrade(vault, { now: T0 });++ expect(run.outcome).toBe(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);+ expect(existsSync(vault)).toBe(false);+ } finally {+ spy.mockRestore();+ }+ });++ test("before the lock: a missing vault is not created for the claim", () => {+ const missing = join(root, "gone");+ const run = runSelfHealUpgrade(missing, { now: T0 });++ expect(run.outcome).toBe(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);+ expect(existsSync(missing)).toBe(false);+ });++ test("right after the rewrite: no metrics row recreates the vault", () => {+ makeManualStale();+ const realApply = upgradeModule.applyUpgrade;+ const spy = spyOn(upgradeModule, "applyUpgrade").mockImplementationOnce((v, opts) => {+ const applied = realApply(v, opts);+ rmSync(vault, { recursive: true, force: true });+ return applied;+ });+ try {+ const run = runSelfHealUpgrade(vault, { now: T0 });++ expect(run.outcome).toBe(SELF_HEAL_UPGRADE_OUTCOME.notInitialized);+ expect(existsSync(vault)).toBe(false);+ } finally {+ spy.mockRestore();+ }+ });++ test("a failure in a vault that is still there is recorded as before", () => {+ makeManualStale();+ const spy = spyOn(upgradeModule, "planUpgrade").mockImplementationOnce(() => {+ throw new Error("plan failed for another reason");+ });+ try {+ const run = runSelfHealUpgrade(vault, { now: T0 });++ expect(run.outcome).toBe(SELF_HEAL_UPGRADE_OUTCOME.failed);+ expect(readSelfHealUpgradeFailure(vault)!.error).toBe("plan failed for another reason");+ expect(upgradeRows().map((r) => r["outcome"])).toEqual([SELF_HEAL_UPGRADE_OUTCOME.failed]);+ } finally {+ spy.mockRestore();+ }+ });+});diff --git a/tests/core/search/check-price-recommendations.test.ts b/tests/core/search/check-price-recommendations.test.tsnew file mode 100644index 00000000..9078b2fa--- /dev/null+++ b/tests/core/search/check-price-recommendations.test.ts@@ -0,0 +1,139 @@+/**+ * `search check` recommendations about the embedding price (Honest+ * Embedding Spend, task 7).+ *+ * The defect pinned here: a model outside the frozen price table was+ * priced at 0 everywhere, and `search check` said nothing about it, so+ * an operator learnt that a model was unpriced only when a gated run was+ * refused. A declared price pair for a model that is no longer active+ * was equally silent while it priced nothing.+ *+ * Covered: the unpriced line under a gate of 0 and under a positive+ * gate, the stale-declaration line, and the silence of table-priced,+ * operator-priced and local models.+ *+ * Deliberately not covered: the refusal itself (`EMBEDDING_COST_UNPRICED`+ * belongs to the spend plan's suites) and the other recommendation arms+ * (`check-recommendations.test.ts`).+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";++import {+ EMBEDDING_PRICE_MODEL_KEY,+ EMBEDDING_PRICE_RATE_KEY,+} from "../../../src/core/search/embeddings/pricing.ts";+import { indexCheck } from "../../../src/core/search/indexer.ts";+import { embeddingPriceRecommendations } from "../../../src/core/search/price-recommendations.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";+import { createTempVault, makeConfig } from "../../helpers/search-fixtures.ts";++const UNPRICED_MODEL = "acme-embed-unlisted-1";+const TABLE_MODEL = "text-embedding-3-small";+const POSITIVE_GATE_USD = 0.5;+const DECLARED_RATE = 0.07;++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("check-price-recs");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++function configured(semantic: Partial<ResolvedEmbeddingConfig>) {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: "https://embeddings.invalid/v1",+ apiKey: FAKE_PROVIDER_KEY,+ ...semantic,+ },+ });+}++async function priceLines(semantic: Partial<ResolvedEmbeddingConfig>): Promise<string[]> {+ const report = await indexCheck(configured(semantic), { probeProvider: false });+ return report.recommendations.filter(+ (r) => r.includes(EMBEDDING_PRICE_MODEL_KEY) || r.includes(EMBEDDING_PRICE_RATE_KEY),+ );+}++describe("indexCheck price recommendations", () => {+ test("an unpriced model under gate 0 gets one line naming both keys and what a gate would do", async () => {+ const lines = await priceLines({ model: UNPRICED_MODEL, costGateUsd: 0 });+ expect(lines).toHaveLength(1);+ const [line] = lines;+ expect(line).toContain(UNPRICED_MODEL);+ expect(line).toContain(EMBEDDING_PRICE_MODEL_KEY);+ expect(line).toContain(EMBEDDING_PRICE_RATE_KEY);+ expect(line).toContain("a positive gate would refuse");+ });++ test("an unpriced model under a positive gate is told that backfills refuse", async () => {+ const lines = await priceLines({ model: UNPRICED_MODEL, costGateUsd: POSITIVE_GATE_USD });+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain("vector backfills refuse");+ expect(lines[0]).toContain("--force-cost");+ });++ test("a declared pair for another model yields one stale-declaration line", async () => {+ const lines = await priceLines({+ model: TABLE_MODEL,+ priceOverride: { model: UNPRICED_MODEL, usdPerMtok: DECLARED_RATE },+ });+ expect(lines).toHaveLength(1);+ expect(lines[0]).toContain(`names model "${UNPRICED_MODEL}"`);+ expect(lines[0]).toContain(`"${TABLE_MODEL}"`);+ });++ test("a table-priced model adds no line", async () => {+ expect(await priceLines({ model: TABLE_MODEL })).toEqual([]);+ });++ test("an operator-priced model adds no line, its declared name compared canonically", async () => {+ expect(+ await priceLines({+ model: UNPRICED_MODEL,+ priceOverride: { model: ` ${UNPRICED_MODEL.toUpperCase()} `, usdPerMtok: DECLARED_RATE },+ }),+ ).toEqual([]);+ });++ test("the local embedder adds no line, whatever embedding_model says", async () => {+ expect(await priceLines({ provider: "local", model: UNPRICED_MODEL })).toEqual([]);+ });++ test("a disabled semantic configuration adds no line", async () => {+ expect(await priceLines({ enabled: false, model: UNPRICED_MODEL })).toEqual([]);+ });+});++describe("embeddingPriceRecommendations", () => {+ test("a configuration that names no model has nothing to price", () => {+ expect(embeddingPriceRecommendations(configured({ model: null }))).toEqual([]);+ });++ test("a stale declaration is reported beside the unpriced line, unpriced first", () => {+ const recs = embeddingPriceRecommendations(+ configured({+ model: UNPRICED_MODEL,+ priceOverride: { model: TABLE_MODEL, usdPerMtok: DECLARED_RATE },+ }),+ );+ expect(recs).toHaveLength(2);+ expect(recs[0]).toContain(`No price is known for embedding model "${UNPRICED_MODEL}"`);+ expect(recs[1]).toContain(`names model "${TABLE_MODEL}"`);+ });+});diff --git a/tests/core/search/credential-report.test.ts b/tests/core/search/credential-report.test.tsnew file mode 100644index 00000000..bf3f44ce--- /dev/null+++ b/tests/core/search/credential-report.test.ts@@ -0,0 +1,200 @@+/**+ * Credential-source report for `search check` (t_82c3b275, task 8).+ *+ * The defect pinned here: a `credential-missing` tier told the operator+ * that no key resolved, and nothing more. It did not say WHERE it had+ * looked, so an operator whose key sat under a registered profile's env+ * name - or under the wrong one - had to read the resolver to find out.+ * The report names the sources consulted, in probe order, and the other+ * registered profiles whose env key is present. Names only: a value is+ * tested for presence and never copied anywhere.+ *+ * Covered: the explicit-config arm, the registry-profile arm (single+ * key and probe list), the present-elsewhere list, the silence of every+ * non-blocked or disabled tier, and a sweep proving no env value reaches+ * the report.+ *+ * Deliberately not covered: the tier itself (`resolveSemanticCapability`+ * is unchanged and has its own suite) and the rendering in `search+ * check`, which `tests/cli/search-check-provider.test.ts` pins.+ */++import { describe, expect, test } from "bun:test";++import {+ describeCredentialSources,+ EXPLICIT_CREDENTIAL_SOURCES,+} from "../../../src/core/search/embeddings/credential-report.ts";+import type { ProviderProfile } from "../../../src/core/search/embeddings/registry.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import { fakeCredential } from "../../helpers/fake-credentials.ts";++const ACTIVE_PROFILE = "acme-embed";+const ACTIVE_ENV_KEYS = ["ACME_EMBED_KEY_PRIMARY", "ACME_EMBED_KEY_FALLBACK"] as const;+const OTHER_PRESENT_PROFILE = "beta-embed";+const OTHER_PRESENT_ENV_KEY = "BETA_EMBED_KEY";+const OTHER_ABSENT_PROFILE = "gamma-embed";+const OTHER_ABSENT_ENV_KEY = "GAMMA_EMBED_KEY";+const OTHER_LIST_PROFILE = "delta-embed";+const OTHER_LIST_ENV_KEYS = ["DELTA_EMBED_KEY_ONE", "DELTA_EMBED_KEY_TWO"] as const;++/** Credential-shaped placeholders: every one must stay out of the report. */+const PRIVATE_BODY_BETA = fakeCredential("beta", "-private-", "4c1d");+const PRIVATE_BODY_DELTA = fakeCredential("delta", "-private-", "77ae");++const REGISTRY: ReadonlyArray<ProviderProfile> = Object.freeze([+ {+ name: ACTIVE_PROFILE,+ baseUrl: "https://acme.invalid/v1",+ defaultModel: "acme-1",+ envKey: ACTIVE_ENV_KEYS,+ },+ {+ name: OTHER_PRESENT_PROFILE,+ baseUrl: "https://beta.invalid/v1",+ defaultModel: "beta-1",+ envKey: OTHER_PRESENT_ENV_KEY,+ },+ {+ name: OTHER_ABSENT_PROFILE,+ baseUrl: "https://gamma.invalid/v1",+ defaultModel: "gamma-1",+ envKey: OTHER_ABSENT_ENV_KEY,+ },+ {+ name: OTHER_LIST_PROFILE,+ baseUrl: "https://delta.invalid/v1",+ defaultModel: "delta-1",+ envKey: OTHER_LIST_ENV_KEYS,+ },+]);++const ENV_WITH_OTHER_KEYS: Readonly<Record<string, string | undefined>> = Object.freeze({+ [OTHER_PRESENT_ENV_KEY]: PRIVATE_BODY_BETA,+ [OTHER_ABSENT_ENV_KEY]: "",+ [OTHER_LIST_ENV_KEYS[1]]: PRIVATE_BODY_DELTA,+});++function semantic(overrides: Partial<ResolvedEmbeddingConfig> = {}): ResolvedEmbeddingConfig {+ return Object.freeze({+ enabled: true,+ provider: "openai-compat",+ baseUrl: "https://acme.invalid/v1",+ model: "acme-1",+ apiKey: null,+ dimension: null,+ timeoutMs: 10_000,+ concurrency: 4,+ batchSize: 32,+ costGateUsd: 0,+ maxRetries: 3,+ ...overrides,+ });+}++describe("describeCredentialSources", () => {+ test("explicit config consults the env override, then the config key", () => {+ const report = describeCredentialSources({+ semantic: semantic(),+ activeProfile: null,+ registry: REGISTRY,+ env: {},+ });+ expect(report).toEqual({+ consulted: ["OPEN_SECOND_BRAIN_EMBEDDING_KEY", "embedding_api_key"],+ presentElsewhere: [],+ });+ expect(report?.consulted).toEqual([...EXPLICIT_CREDENTIAL_SOURCES]);+ });++ test("a registry profile adds its env-key names in probe order", () => {+ const report = describeCredentialSources({+ semantic: semantic(),+ activeProfile: ACTIVE_PROFILE,+ registry: REGISTRY,+ env: {},+ });+ expect(report?.consulted).toEqual([...EXPLICIT_CREDENTIAL_SOURCES, ...ACTIVE_ENV_KEYS]);+ });++ test("other registered profiles whose env key is present are listed by name, in registry order", () => {+ const report = describeCredentialSources({+ semantic: semantic(),+ activeProfile: ACTIVE_PROFILE,+ registry: REGISTRY,+ env: ENV_WITH_OTHER_KEYS,+ });+ expect(report?.presentElsewhere).toEqual([OTHER_PRESENT_PROFILE, OTHER_LIST_PROFILE]);+ });++ test("explicit config lists every registered profile with a present key", () => {+ const report = describeCredentialSources({+ semantic: semantic(),+ activeProfile: null,+ registry: REGISTRY,+ env: { ...ENV_WITH_OTHER_KEYS, [ACTIVE_ENV_KEYS[0]]: PRIVATE_BODY_BETA },+ });+ expect(report?.presentElsewhere).toEqual([+ ACTIVE_PROFILE,+ OTHER_PRESENT_PROFILE,+ OTHER_LIST_PROFILE,+ ]);+ });++ test("no env value ever reaches the report", () => {+ const report = describeCredentialSources({+ semantic: semantic(),+ activeProfile: ACTIVE_PROFILE,+ registry: REGISTRY,+ env: ENV_WITH_OTHER_KEYS,+ });+ const serialized = JSON.stringify(report);+ for (const privateBody of [PRIVATE_BODY_BETA, PRIVATE_BODY_DELTA]) {+ expect(serialized).not.toContain(privateBody);+ }+ });++ test("a configured tier reports nothing", () => {+ expect(+ describeCredentialSources({+ semantic: semantic({ apiKey: PRIVATE_BODY_BETA }),+ activeProfile: null,+ registry: REGISTRY,+ env: ENV_WITH_OTHER_KEYS,+ }),+ ).toBeNull();+ });++ test("the keyless local provider reports nothing", () => {+ expect(+ describeCredentialSources({+ semantic: semantic({ provider: "local" }),+ activeProfile: null,+ registry: REGISTRY,+ env: {},+ }),+ ).toBeNull();+ });++ test("a disabled tier reports nothing", () => {+ expect(+ describeCredentialSources({+ semantic: semantic({ enabled: false }),+ activeProfile: null,+ registry: REGISTRY,+ env: {},+ }),+ ).toBeNull();+ });++ test("a named profile the registry does not hold is refused by name", () => {+ expect(() =>+ describeCredentialSources({+ semantic: semantic(),+ activeProfile: "missing-profile",+ registry: REGISTRY,+ env: {},+ }),+ ).toThrow(/missing-profile/);+ });+});diff --git a/tests/core/search/embedding-spend-preview.test.ts b/tests/core/search/embedding-spend-preview.test.tsindex bd9590a1..68a993a9 100644--- a/tests/core/search/embedding-spend-preview.test.ts+++ b/tests/core/search/embedding-spend-preview.test.ts@@ -13,9 +13,16 @@* the pass's walk and a walk that adds chunks (a dream or an agent* wrote since the last index) makes the preview an estimate by* position. The receipt is what the pass actually refused or priced.+ *+ * - The unpriced refusal (Honest Embedding Spend): under an explicit+ * positive gate a model nobody priced is refused with+ * EMBEDDING_COST_UNPRICED before any provider contact, instead of being+ * estimated at $0 and let through. A forced run records the unknown+ * price as a null estimate with `priceSource: unknown`.*/import { afterEach, beforeEach, expect, test } from "bun:test";+import { Database } from "bun:sqlite";import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";@@ -23,11 +30,18 @@ import { join } from "node:path";import {embeddingSpendOf,estimatePendingEmbeddingSpend,+ indexStatus,runEmbeddingPhase,type EmbeddingPhaseTally,} from "../../../src/core/search/indexer.ts";+import { planEmbeddingSpend } from "../../../src/core/search/embedding-spend.ts";+import { planVectorBackfill } from "../../../src/core/search/vector-backfill.ts";import { indexVault, resolveSearchConfig } from "../../../src/core/search/index.ts";import { Store } from "../../../src/core/search/store.ts";+import {+ resolveEmbeddingPrice,+ type KnownPriceQuote,+} from "../../../src/core/search/embeddings/pricing.ts";import {estimateCostUsd,estimateTokens,@@ -38,6 +52,13 @@ import type { ResolvedSearchConfig } from "../../../src/core/search/types.ts";import { startFakeHttp, type FakeHttp } from "../../helpers/fake-http.ts";import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";+/** The builtin quote of the table model the gate fixtures price against. */+const TABLE_QUOTE: KnownPriceQuote = (() => {+ const quote = resolveEmbeddingPrice("text-embedding-3-small");+ if (quote.usdPerMtok === null) throw new Error("text-embedding-3-small lost its table price");+ return quote;+})();+let tmp: string;let vault: string;@@ -108,7 +129,7 @@ test("a vault with no index cannot spend: null", async () => {test("pending chunks preview the kernel's own numbers at the resolved model", async () => {const contents = ["alpha beta gamma delta", "second chunk of prose"];const tokens = estimateTokens(contents);- const usd = estimateCostUsd(tokens, "text-embedding-3-small");+ const usd = estimateCostUsd(tokens, TABLE_QUOTE);// A positive gate set BELOW the kernel's estimate: the run would be// refused without a forced bypass, and the preview says so.const config = configWith({@@ -219,12 +240,43 @@ test("the phase records its own gate result on the tally, at the resolved model"tokens: estimateTokens(contents),estimatedUsd: 0,forced: false,+ priceSource: "builtin",});} finally {await store.close();}});+/**+ * Pinned split, deliberately: the receipt names the model the price was+ * resolved for - the local provider always embeds with its implicit+ * model - while the vectors keep the stamp `embedding_model` gives them,+ * because changing the stamp would re-key (and drop) every existing+ * local index. The local model is free, so the money statement holds.+ */+test("a local receipt names the priced model while vectors keep the configured stamp", async () => {+ const config = configWith({+ search_semantic_enabled: "true",+ embedding_provider: "local",+ embedding_model: "custom-local",+ });+ const store = await openSeeded(config, ["alpha beta gamma"]);+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ try {+ await runEmbeddingPhase(store, config, tally, {});+ } finally {+ await store.close();+ }+ expect(tally.spend?.model).toBe(LOCAL_EMBEDDING_MODEL);+ const db = new Database(config.dbPath, { readonly: true });+ try {+ const stamps = db.query<{ model: string }, []>("SELECT DISTINCT model FROM embeddings").all();+ expect(stamps.map((row) => row.model)).toEqual(["custom-local"]);+ } finally {+ db.close();+ }+});+test("nothing pending: the phase runs, embeds nothing, and records no receipt", async () => {const config = configWith({search_semantic_enabled: "true",@@ -273,7 +325,7 @@ test("a positive gate refuses unforced and records nothing; forced records the bexpect(forced.spend?.model).toBe("text-embedding-3-small");expect(forced.spend?.tokens).toBe(estimateTokens(contents));expect(forced.spend?.estimatedUsd).toBe(- estimateCostUsd(forced.spend?.tokens ?? 0, "text-embedding-3-small"),+ estimateCostUsd(forced.spend?.tokens ?? 0, TABLE_QUOTE),);expect(forced.spend?.estimatedUsd).toBeGreaterThan(0);expect(server.callCount()).toBeGreaterThan(0);@@ -285,6 +337,29 @@ test("a positive gate refuses unforced and records nothing; forced records the b}});+test("--force-cost under the cap is not recorded as forced", async () => {+ let server: FakeHttp | null = null;+ try {+ server = await startFakeHttp();+ const config = configWith({+ ...PRICED_MODEL,+ embedding_base_url: server.url,+ embedding_cost_gate_usd: "100",+ });+ const store = await openSeeded(config, ["a short chunk well under the cap"]);+ try {+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ await runEmbeddingPhase(store, config, tally, { forceCost: true });+ expect(tally.embeddingsComputed).toBe(1);+ expect(tally.spend?.forced).toBe(false);+ } finally {+ await store.close();+ }+ } finally {+ await server?.close();+ }+});+test("embeddingSpendOf reads the receipt off a completed run's stats", async () => {const config = configWith({search_semantic_enabled: "true",@@ -320,3 +395,208 @@ test("embeddingSpendOf reads the receipt off a completed run's stats", async ()expect(offlineStats.backend).toBe("offline");expect(embeddingSpendOf(offlineStats)).toBeUndefined();});++// ── unpriced models under an explicit gate ───────────────────────────────────++const UNPRICED_MODEL = "zembed-1";++test("a positive gate refuses an unpriced model by name, before any provider contact", async () => {+ let server: FakeHttp | null = null;+ try {+ server = await startFakeHttp();+ const config = configWith({+ search_semantic_enabled: "true",+ embedding_provider: "openai-compat",+ embedding_base_url: server.url,+ embedding_model: UNPRICED_MODEL,+ embedding_api_key: FAKE_PROVIDER_KEY,+ embedding_cost_gate_usd: "100",+ });+ const store = await openSeeded(config, ["a short chunk the price of which nobody stated"]);+ try {+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ const refusal = await runEmbeddingPhase(store, config, tally, {}).then(+ () => null,+ (error: unknown) => error,+ );+ expect(refusal).toBeInstanceOf(SearchError);+ const error = refusal as SearchError;+ expect(error.code).toBe("EMBEDDING_COST_UNPRICED");+ for (const named of [+ UNPRICED_MODEL,+ "embedding_price_model",+ "embedding_price_usd_per_mtok",+ "--force-cost",+ ]) {+ expect(error.message).toContain(named);+ }+ expect(tally.spend).toBeUndefined();+ expect(server.callCount()).toBe(0);++ const forced: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ await runEmbeddingPhase(store, config, forced, { forceCost: true });+ expect(forced.embeddingsComputed).toBe(1);+ expect(forced.spend).toMatchObject({+ model: UNPRICED_MODEL,+ forced: true,+ priceSource: "unknown",+ estimatedUsd: null,+ });+ } finally {+ await store.close();+ }+ } finally {+ await server?.close();+ }+});++test("a handed-in plan is re-gated, so a forced or stale verdict cannot pass the gate", async () => {+ let server: FakeHttp | null = null;+ try {+ server = await startFakeHttp();+ const config = configWith({+ search_semantic_enabled: "true",+ embedding_provider: "openai-compat",+ embedding_base_url: server.url,+ embedding_model: UNPRICED_MODEL,+ embedding_api_key: FAKE_PROVIDER_KEY,+ embedding_cost_gate_usd: "100",+ });+ const store = await openSeeded(config, ["a chunk whose plan claims the gate passed"]);+ try {+ const forcedPlan = planEmbeddingSpend(store, config, { forced: true });+ expect(forcedPlan.gate.blocked).toBe(false);+ const unforced = planEmbeddingSpend(store, config);+ const stalePlan = { ...unforced, gate: { blocked: false as const, reason: null } };+ for (const plan of [forcedPlan, stalePlan]) {+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ // eslint-disable-next-line no-await-in-loop -- two plans, one after the other+ const refusal = await runEmbeddingPhase(store, config, tally, { plan }).then(+ () => null,+ (error: unknown) => error,+ );+ expect((refusal as SearchError).code).toBe("EMBEDDING_COST_UNPRICED");+ expect(tally.spend).toBeUndefined();+ }+ expect(server.callCount()).toBe(0);+ } finally {+ await store.close();+ }+ } finally {+ await server?.close();+ }+});++test("the over-cap refusal keeps its code and message", async () => {+ const config = configWith({ ...PRICED_MODEL, embedding_cost_gate_usd: "0.000001" });+ const store = await openSeeded(config, ["pricing text repeated to exceed the gate. ".repeat(8)]);+ try {+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ const refusal = await runEmbeddingPhase(store, config, tally, {}).then(+ () => null,+ (error: unknown) => error,+ );+ expect((refusal as SearchError).code).toBe("EMBEDDING_COST_GATE");+ expect((refusal as SearchError).message).toMatch(+ /^estimated embedding cost \$\d+\.\d{4} for 1 chunk\(s\) exceeds embedding_cost_gate_usd \$0\.0000\. Re-run with --force-cost to proceed or raise the gate\.$/,+ );+ } finally {+ await store.close();+ }+});++// ── every surface reads the one plan ────────────────────────────────────────++const OPERATOR_PRICED = {+ embedding_price_model: UNPRICED_MODEL,+ embedding_price_usd_per_mtok: "0.05",+};++test("preview, backfill dry run, status and the phase report one estimate and source", async () => {+ let server: FakeHttp | null = null;+ try {+ server = await startFakeHttp();+ const config = configWith({+ search_semantic_enabled: "true",+ embedding_provider: "openai-compat",+ embedding_base_url: server.url,+ embedding_model: UNPRICED_MODEL,+ embedding_api_key: FAKE_PROVIDER_KEY,+ ...OPERATOR_PRICED,+ });+ const contents = ["first chunk the operator priced", "second chunk the operator priced"];+ await storeWithChunks(config, contents);++ const preview = await estimatePendingEmbeddingSpend(config);+ const dryRun = await planVectorBackfill(config);+ const status = await indexStatus(config);+ expect(preview?.estimatedUsd).toBeGreaterThan(0);+ expect(preview?.priceSource).toBe("operator");+ expect(dryRun.estimatedCostUsd).toBe(preview?.estimatedUsd ?? -1);+ expect(dryRun.priceSource).toBe("operator");+ expect(status.estimatedRefreshCostUsd).toBe(preview?.estimatedUsd ?? -1);+ expect(status.refreshPriceSource).toBe("operator");++ const store = await Store.open(config, { mode: "write" });+ try {+ const tally: EmbeddingPhaseTally = { embeddingsComputed: 0, embeddingsRetries: 0 };+ await runEmbeddingPhase(store, config, tally, {});+ expect(tally.spend).toMatchObject({+ tokens: preview?.tokens ?? -1,+ estimatedUsd: preview?.estimatedUsd ?? -1,+ priceSource: "operator",+ });+ } finally {+ await store.close();+ }+ } finally {+ await server?.close();+ }+});++test("an unknown price reads null on every surface and blocks as unpriced under a gate", async () => {+ const config = configWith({+ ...PRICED_MODEL,+ embedding_model: UNPRICED_MODEL,+ embedding_cost_gate_usd: "100",+ });+ await storeWithChunks(config, ["a chunk nobody priced"]);++ const preview = await estimatePendingEmbeddingSpend(config);+ expect(preview).toMatchObject({+ estimatedUsd: null,+ priceSource: "unknown",+ blocked: true,+ reason: "unpriced",+ });+ const dryRun = await planVectorBackfill(config);+ expect(dryRun).toMatchObject({+ estimatedCostUsd: null,+ priceSource: "unknown",+ blocked: true,+ reason: "unpriced",+ });+ const status = await indexStatus(config);+ expect(status.estimatedRefreshCostUsd).toBeNull();+ expect(status.refreshPriceSource).toBe("unknown");++ // The gate off: nothing blocks, the estimate stays unknown.+ const ungated = configWith({ ...PRICED_MODEL, embedding_model: UNPRICED_MODEL });+ expect(await estimatePendingEmbeddingSpend(ungated)).toMatchObject({+ estimatedUsd: null,+ blocked: false,+ reason: null,+ });+});++test("a local index prices as a known zero on the backfill and on status", async () => {+ const local = configWith({ search_semantic_enabled: "true", embedding_provider: "local" });+ await storeWithChunks(local, ["local chunk"]);+ expect(await planVectorBackfill(local)).toMatchObject({+ estimatedCostUsd: 0,+ priceSource: "builtin",+ });+ const status = await indexStatus(local);+ expect(status.estimatedRefreshCostUsd).toBe(0);+ expect(status.refreshPriceSource).toBe("builtin");+});diff --git a/tests/core/search/embedding-spend.test.ts b/tests/core/search/embedding-spend.test.tsnew file mode 100644index 00000000..204f484c--- /dev/null+++ b/tests/core/search/embedding-spend.test.ts@@ -0,0 +1,230 @@+/**+ * The shared embedding spend plan (Honest Embedding Spend).+ *+ * Pins the defect that a positive cost gate never refused an unpriced+ * model, because an unlisted price was estimated at 0. The gate verdict+ * now carries a reason: `over_cap` for a known price over the cap and+ * `unpriced` for an unknown price under an explicit positive gate. It+ * also pins that `planEmbeddingSpend` is one computation of the pending+ * census, the model, the tokens, the quote and the verdict, optionally+ * scoped to path prefixes.+ *+ * Deliberately not covered here: the phase that throws on a blocked+ * verdict (indexer.embeddings.test.ts) and the surfaces that render the+ * plan (embedding-spend-preview.test.ts, search-vector-backfill.test.ts).+ */+import { afterEach, beforeEach, expect, test } from "bun:test";++import {+ EMBEDDING_GATE_REASON,+ evaluateCostGate,+ planEmbeddingSpend,+} from "../../../src/core/search/embedding-spend.ts";+import {+ EMBEDDING_PRICE_SOURCE,+ resolveEmbeddingPrice,+} from "../../../src/core/search/embeddings/pricing.ts";+import { LOCAL_EMBEDDING_MODEL } from "../../../src/core/search/embeddings/signature.ts";+import { indexVault } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import type { ResolvedEmbeddingConfig } from "../../../src/core/search/types.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";++const TABLE_MODEL = "text-embedding-3-small";+const UNLISTED_MODEL = "zembed-1";+/** ~1M tokens at chars/4: $0.02 on the table model. */+const MILLION_TOKEN_TEXTS = ["x".repeat(4_000_000)];+const LOW_GATE_USD = 0.001;+const HIGH_GATE_USD = 100;+const MICRO_USD = 0.000001;++// ── pure gate verdicts ───────────────────────────────────────────────────────++test("a zero gate never blocks, priced or not", () => {+ for (const model of [TABLE_MODEL, UNLISTED_MODEL]) {+ const verdict = evaluateCostGate({+ texts: MILLION_TOKEN_TEXTS,+ quote: resolveEmbeddingPrice(model),+ gateUsd: 0,+ });+ expect(verdict.blocked).toBe(false);+ expect(verdict.reason).toBeNull();+ }+});++test("a positive gate with a known price blocks only over the cap, as over_cap", () => {+ const quote = resolveEmbeddingPrice(TABLE_MODEL);+ const over = evaluateCostGate({ texts: MILLION_TOKEN_TEXTS, quote, gateUsd: LOW_GATE_USD });+ expect(over).toMatchObject({ blocked: true, reason: EMBEDDING_GATE_REASON.overCap });+ const under = evaluateCostGate({ texts: MILLION_TOKEN_TEXTS, quote, gateUsd: HIGH_GATE_USD });+ expect(under).toMatchObject({ blocked: false, reason: null, tokens: 1_000_000 });+ expect(under.estimatedUsd).toBeGreaterThan(0);+});++test("an estimate exactly at the gate passes; one micro-dollar over blocks", () => {+ const quote = resolveEmbeddingPrice(TABLE_MODEL);+ const estimate = evaluateCostGate({+ texts: MILLION_TOKEN_TEXTS,+ quote,+ gateUsd: 0,+ }).estimatedUsd!;+ expect(estimate).toBeGreaterThan(MICRO_USD);+ const at = evaluateCostGate({ texts: MILLION_TOKEN_TEXTS, quote, gateUsd: estimate });+ expect(at.blocked).toBe(false);+ const below = evaluateCostGate({+ texts: MILLION_TOKEN_TEXTS,+ quote,+ gateUsd: estimate - MICRO_USD,+ });+ expect(below.blocked).toBe(true);+});++test("a positive gate with an unknown price and pending work blocks as unpriced", () => {+ const verdict = evaluateCostGate({+ texts: ["one pending chunk"],+ quote: resolveEmbeddingPrice(UNLISTED_MODEL),+ gateUsd: HIGH_GATE_USD,+ });+ expect(verdict).toMatchObject({+ blocked: true,+ reason: EMBEDDING_GATE_REASON.unpriced,+ estimatedUsd: null,+ });+});++test("a forced run never blocks", () => {+ for (const model of [TABLE_MODEL, UNLISTED_MODEL]) {+ const verdict = evaluateCostGate({+ texts: MILLION_TOKEN_TEXTS,+ quote: resolveEmbeddingPrice(model),+ gateUsd: LOW_GATE_USD,+ forced: true,+ });+ expect(verdict).toMatchObject({ blocked: false, reason: null });+ }+});++test("an empty pending set never blocks, even unpriced", () => {+ const verdict = evaluateCostGate({+ texts: [],+ quote: resolveEmbeddingPrice(UNLISTED_MODEL),+ gateUsd: LOW_GATE_USD,+ });+ expect(verdict).toMatchObject({ blocked: false, reason: null, tokens: 0 });+});++test("the local model never reads unpriced", () => {+ const verdict = evaluateCostGate({+ texts: MILLION_TOKEN_TEXTS,+ quote: resolveEmbeddingPrice(LOCAL_EMBEDDING_MODEL),+ gateUsd: LOW_GATE_USD,+ });+ expect(verdict).toMatchObject({ blocked: false, reason: null, estimatedUsd: 0 });+});++// ── the store-backed plan ────────────────────────────────────────────────────++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("embedding-spend");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => cleanup());++function semanticConfig(semantic: Partial<ResolvedEmbeddingConfig>) {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: "http://127.0.0.1:9",+ model: UNLISTED_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ dimension: 4,+ timeoutMs: 5_000,+ concurrency: 1,+ batchSize: 8,+ costGateUsd: 0,+ maxRetries: 1,+ ...semantic,+ },+ });+}++async function indexedStore(semantic: Partial<ResolvedEmbeddingConfig>) {+ writeMd(vault, "Brain/preferences/pref-a.md", "# Pref A\n\nA belief about tea.");+ writeMd(vault, "notes/b.md", "# B\n\nA plain note about coffee.");+ await indexVault(makeConfig({ vault, dbPath }));+ const config = semanticConfig(semantic);+ return { config, store: await Store.open(config, { mode: "read" }) };+}++test("the plan reports the pending set, model, tokens, quote and verdict in one place", async () => {+ const { config, store } = await indexedStore({ costGateUsd: HIGH_GATE_USD });+ try {+ const plan = planEmbeddingSpend(store, config);+ expect(plan.pending.length).toBeGreaterThan(0);+ expect(plan.model).toBe(UNLISTED_MODEL);+ expect(plan.tokens).toBeGreaterThan(0);+ expect(plan.quote.source).toBe(EMBEDDING_PRICE_SOURCE.unknown);+ expect(plan.estimatedUsd).toBeNull();+ expect(plan.gate).toEqual({ blocked: true, reason: EMBEDDING_GATE_REASON.unpriced });+ } finally {+ await store.close();+ }+});++test("the plan prices through the operator pair", async () => {+ const { config, store } = await indexedStore({+ costGateUsd: HIGH_GATE_USD,+ priceOverride: { model: UNLISTED_MODEL, usdPerMtok: 0.05 },+ });+ try {+ const plan = planEmbeddingSpend(store, config);+ expect(plan.quote).toEqual({ usdPerMtok: 0.05, source: EMBEDDING_PRICE_SOURCE.operator });+ expect(plan.estimatedUsd).toBeGreaterThan(0);+ expect(plan.gate).toEqual({ blocked: false, reason: null });+ } finally {+ await store.close();+ }+});++test("the local provider plans the local model whatever embedding_model says", async () => {+ const { config, store } = await indexedStore({+ provider: "local",+ model: UNLISTED_MODEL,+ costGateUsd: LOW_GATE_USD,+ });+ try {+ const plan = planEmbeddingSpend(store, config);+ expect(plan.model).toBe(LOCAL_EMBEDDING_MODEL);+ expect(plan.estimatedUsd).toBe(0);+ expect(plan.gate).toEqual({ blocked: false, reason: null });+ } finally {+ await store.close();+ }+});++test("a scoped plan reads only the chunks under its prefixes", async () => {+ const { config, store } = await indexedStore({});+ try {+ const all = planEmbeddingSpend(store, config);+ const scoped = planEmbeddingSpend(store, config, {+ scope: { pathPrefixes: ["Brain/preferences/"] },+ });+ expect(scoped.pending.length).toBeGreaterThan(0);+ expect(scoped.pending.length).toBeLessThan(all.pending.length);+ expect(scoped.pending.some((p) => p.content.includes("coffee"))).toBe(false);+ expect(scoped.tokens).toBeLessThan(all.tokens);+ } finally {+ await store.close();+ }+});diff --git a/tests/core/search/embeddings.cost-gate.test.ts b/tests/core/search/embeddings.cost-gate.test.tsindex 610946d6..7fece17a 100644--- a/tests/core/search/embeddings.cost-gate.test.ts+++ b/tests/core/search/embeddings.cost-gate.test.ts@@ -3,51 +3,11 @@ import { mkdtempSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";-import {- evaluateCostGate,- LOCAL_EMBEDDING_MODEL,-} from "../../../src/core/search/embeddings/signature.ts";+import { activeEmbeddingModel } from "../../../src/core/search/embedding-spend.ts";+import { embeddingSignature } from "../../../src/core/search/embeddings/signature.ts";import { resolveSearchConfig } from "../../../src/core/search/index.ts";--// ── pure cost-gate kernel ────────────────────────────────────────────────────--test("evaluateCostGate estimates spend for a priced model", () => {- // 4,000,000 chars -> ~1,000,000 tokens at chars/4.- const texts = ["x".repeat(4_000_000)];- const r = evaluateCostGate({ texts, model: "text-embedding-3-small", gateUsd: 0 });- expect(r.tokens).toBe(1_000_000);- expect(r.estimatedUsd).toBeGreaterThan(0);-});--test("evaluateCostGate blocks when the estimate exceeds a positive gate", () => {- const texts = ["x".repeat(4_000_000)]; // ~1M tokens- const r = evaluateCostGate({ texts, model: "text-embedding-3-small", gateUsd: 0.001 });- expect(r.blocked).toBe(true);-});--test("evaluateCostGate does not block when forced", () => {- const texts = ["x".repeat(4_000_000)];- const r = evaluateCostGate({- texts,- model: "text-embedding-3-small",- gateUsd: 0.001,- forced: true,- });- expect(r.blocked).toBe(false);-});--test("a zero gate (default) never blocks", () => {- const texts = ["x".repeat(40_000_000)];- const r = evaluateCostGate({ texts, model: "text-embedding-3-small", gateUsd: 0 });- expect(r.blocked).toBe(false);-});--test("the local model never blocks regardless of gate or volume", () => {- const texts = ["x".repeat(40_000_000)];- const r = evaluateCostGate({ texts, model: LOCAL_EMBEDDING_MODEL, gateUsd: 0.0001 });- expect(r.estimatedUsd).toBe(0);- expect(r.blocked).toBe(false);-});+import { SearchError } from "../../../src/core/search/search-error.ts";+import type { ResolvedSearchConfig } from "../../../src/core/search/types.ts";// ── config parsing ───────────────────────────────────────────────────────────@@ -74,3 +34,215 @@ test("embedding_cost_gate_usd is parsed from config", () => {const cfg = resolveSearchConfig({ vault: tmp, configPath: config });expect(cfg.semantic.costGateUsd).toBe(2.5);});++const COST_GATE_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_COST_GATE";++/** Resolve `lines` with `value` (or nothing) as the gate env twin, restoring it after. */+function resolveGate(lines: ReadonlyArray<string>, value?: string): ResolvedSearchConfig {+ writeFileSync(config, [`vault: "${tmp}"`, ...lines, ""].join("\n"));+ const saved = process.env[COST_GATE_ENV];+ if (value === undefined) delete process.env[COST_GATE_ENV];+ else process.env[COST_GATE_ENV] = value;+ try {+ return resolveSearchConfig({ vault: tmp, configPath: config });+ } finally {+ if (saved === undefined) delete process.env[COST_GATE_ENV];+ else process.env[COST_GATE_ENV] = saved;+ }+}++test("a blank cost gate is refused, never read as a gate of 0", () => {+ // `Number(" ")` is 0, which would silently switch the gate off.+ for (const run of [+ () => resolveGate(["embedding_cost_gate_usd: 1"], " "),+ () => resolveGate(['embedding_cost_gate_usd: " "']),+ ]) {+ const refusal = refusalOf(run);+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain("embedding_cost_gate_usd must be a number >= 0");+ expect(refusal.message).toContain("got empty string");+ }+});++test("an empty cost gate env twin still counts as unset", () => {+ expect(resolveGate(["embedding_cost_gate_usd: 1"], "").semantic.costGateUsd).toBe(1);+});++test("a blank rerank floor is refused, never read as 0", () => {+ const refusal = refusalOf(() => resolveGate(['search_rerank_min_score: " "']));+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain("must be a finite number, got empty string");+});++// ── operator price pair ──────────────────────────────────────────────────────++const PRICE_MODEL_KEY = "embedding_price_model";+const PRICE_RATE_KEY = "embedding_price_usd_per_mtok";+const PRICE_MODEL_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_PRICE_MODEL";+const PRICE_RATE_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_PRICE_USD_PER_MTOK";++/** Resolve with exactly `env` as the price-pair env layer, restoring it after. */+function resolveWith(lines: ReadonlyArray<string>, env: Record<string, string> = {}) {+ writeFileSync(config, [`vault: "${tmp}"`, ...lines, ""].join("\n"));+ const saved = {+ [PRICE_MODEL_ENV]: process.env[PRICE_MODEL_ENV],+ [PRICE_RATE_ENV]: process.env[PRICE_RATE_ENV],+ };+ delete process.env[PRICE_MODEL_ENV];+ delete process.env[PRICE_RATE_ENV];+ Object.assign(process.env, env);+ try {+ return resolveSearchConfig({ vault: tmp, configPath: config });+ } finally {+ for (const [key, value] of Object.entries(saved)) {+ if (value === undefined) delete process.env[key];+ else process.env[key] = value;+ }+ }+}++function refusalOf(run: () => unknown): SearchError {+ try {+ run();+ } catch (error) {+ if (error instanceof SearchError) return error;+ throw error;+ }+ throw new Error("expected config resolution to refuse");+}++test("the price pair resolves from config into priceOverride", () => {+ const cfg = resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "0.05"`]);+ expect(cfg.semantic.priceOverride).toEqual({ model: "zembed-1", usdPerMtok: 0.05 });+});++test("the env pair wins over the config pair", () => {+ const cfg = resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "0.05"`], {+ [PRICE_MODEL_ENV]: "voyage-3",+ [PRICE_RATE_ENV]: "0.06",+ });+ expect(cfg.semantic.priceOverride).toEqual({ model: "voyage-3", usdPerMtok: 0.06 });+});++test("a colon-bearing model id survives the flat parser", () => {+ const cfg = resolveWith([`${PRICE_MODEL_KEY}: nomic-embed-text:latest`, `${PRICE_RATE_KEY}: 0`]);+ expect(cfg.semantic.priceOverride).toEqual({ model: "nomic-embed-text:latest", usdPerMtok: 0 });+});++test("a half pair is refused, naming the missing key", () => {+ const noRate = refusalOf(() => resolveWith([`${PRICE_MODEL_KEY}: zembed-1`]));+ expect(noRate.code).toBe("INVALID_INPUT");+ expect(noRate.message).toContain(PRICE_RATE_KEY);+ const noModel = refusalOf(() => resolveWith([`${PRICE_RATE_KEY}: "0.05"`]));+ expect(noModel.code).toBe("INVALID_INPUT");+ expect(noModel.message).toContain(PRICE_MODEL_KEY);+});++test("an env model never pairs with a config rate", () => {+ const refusal = refusalOf(() =>+ resolveWith([`${PRICE_RATE_KEY}: "0.05"`], { [PRICE_MODEL_ENV]: "voyage-3" }),+ );+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain(`${PRICE_MODEL_ENV} is set but ${PRICE_RATE_ENV} is not`);+});++test("a half-set env pair is refused even over a full config pair", () => {+ const refusal = refusalOf(() =>+ resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "0.05"`], {+ [PRICE_RATE_ENV]: "0.06",+ }),+ );+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain(`${PRICE_RATE_ENV} is set but ${PRICE_MODEL_ENV} is not`);+});++test("a bad env rate is refused under the env name", () => {+ const refusal = refusalOf(() =>+ resolveWith([], { [PRICE_MODEL_ENV]: "voyage-3", [PRICE_RATE_ENV]: "cheap" }),+ );+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain(PRICE_RATE_ENV);+});++test("a negative or non-numeric rate is refused by name", () => {+ for (const rate of ["-1", "cheap", "NaN", "Infinity", "-Infinity", "0x10", "1e3", "1e308"]) {+ const refusal = refusalOf(() =>+ resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "${rate}"`]),+ );+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain(PRICE_RATE_KEY);+ }+});++test("a rate that is not a plain decimal is refused with an example", () => {+ for (const rate of [".5", "5."]) {+ const refusal = refusalOf(() =>+ resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "${rate}"`]),+ );+ expect(refusal.message).toBe(+ `${PRICE_RATE_KEY} must be a plain decimal number >= 0 (for example 0.02), got '${rate}'`,+ );+ }+});++test("a blank half is a missing half, never a declared free price", () => {+ // `Number(" ")` is 0: a whitespace rate used to resolve to an operator+ // price of $0 - the unknown-reads-as-free defect this pair exists to end.+ const blankRate = refusalOf(() =>+ resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: " "`]),+ );+ expect(blankRate.code).toBe("INVALID_INPUT");+ expect(blankRate.message).toContain(`${PRICE_MODEL_KEY} is set but ${PRICE_RATE_KEY} is not`);+ const blankEnvRate = refusalOf(() =>+ resolveWith([], { [PRICE_MODEL_ENV]: "voyage-3", [PRICE_RATE_ENV]: " " }),+ );+ expect(blankEnvRate.message).toContain(`${PRICE_MODEL_ENV} is set but ${PRICE_RATE_ENV} is not`);+ const blankModel = refusalOf(() =>+ resolveWith([], { [PRICE_MODEL_ENV]: " ", [PRICE_RATE_ENV]: "0.05" }),+ );+ expect(blankModel.message).toContain(`${PRICE_RATE_ENV} is set but ${PRICE_MODEL_ENV} is not`);+});++test("a rate above the ceiling is refused by name; the ceiling itself resolves", () => {+ const refusal = refusalOf(() =>+ resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "1000001"`]),+ );+ expect(refusal.code).toBe("INVALID_INPUT");+ expect(refusal.message).toContain(PRICE_RATE_KEY);+ expect(refusal.message).toContain("1000000");+ const ceiling = resolveWith([`${PRICE_MODEL_KEY}: zembed-1`, `${PRICE_RATE_KEY}: "1000000"`]);+ expect(ceiling.semantic.priceOverride).toEqual({ model: "zembed-1", usdPerMtok: 1_000_000 });+});++test("without the pair priceOverride is absent", () => {+ const cfg = resolveWith([]);+ expect("priceOverride" in cfg.semantic).toBe(false);+});++function signatureOf(cfg: ResolvedSearchConfig): string {+ return embeddingSignature({+ provider: cfg.semantic.provider,+ model: cfg.semantic.model,+ dimension: cfg.semantic.dimension,+ });+}++test("declaring a price never changes the embedding signature", () => {+ const model = "text-embedding-3-small";+ const plain = resolveWith([`embedding_model: ${model}`]);+ const priced = resolveWith([+ `embedding_model: ${model}`,+ `${PRICE_MODEL_KEY}: ${model}`,+ `${PRICE_RATE_KEY}: "0.5"`,+ ]);+ expect(priced.semantic.priceOverride).toBeDefined();+ expect(signatureOf(priced)).toBe(signatureOf(plain));+});++test("the configured embedding model wins over the stored one; the stored one fills a gap", () => {+ const configured = resolveWith(["embedding_model: voyage-3"]);+ expect(activeEmbeddingModel(configured, "zembed-1")).toBe("voyage-3");+ const unset = resolveWith([]);+ expect(unset.semantic.model).toBeNull();+ expect(activeEmbeddingModel(unset, "zembed-1")).toBe("zembed-1");+});diff --git a/tests/core/search/embeddings.pricing.test.ts b/tests/core/search/embeddings.pricing.test.tsnew file mode 100644index 00000000..7cd11f9d--- /dev/null+++ b/tests/core/search/embeddings.pricing.test.ts@@ -0,0 +1,89 @@+/**+ * Price quotes for embedding models (Honest Embedding Spend).+ *+ * Pins the defect that a model outside the frozen price table was priced+ * at 0, so an unpriced paid model read as free everywhere a cost was+ * estimated. The resolver now answers with a quote that names who stated+ * the price (`builtin` table, `operator` declaration) or says the price is+ * `unknown`, and an unknown quote yields a null estimate, never 0.+ *+ * Deliberately not covered here: config parsing of the operator pair+ * (embeddings.cost-gate.test.ts) and the gate verdicts built on a quote+ * (embedding-spend.test.ts).+ */+import { test, expect } from "bun:test";++import {+ EMBEDDING_PRICE_SOURCE,+ resolveEmbeddingPrice,+} from "../../../src/core/search/embeddings/pricing.ts";+import {+ estimateCostUsd,+ LOCAL_EMBEDDING_MODEL,+} from "../../../src/core/search/embeddings/signature.ts";++const TABLE_MODEL = "text-embedding-3-small";+const TABLE_RATE = 0.02;+const UNLISTED_MODEL = "zembed-1";+const OPERATOR_RATE = 0.05;++test("a table model resolves to its builtin rate", () => {+ expect(resolveEmbeddingPrice(TABLE_MODEL)).toEqual({+ usdPerMtok: TABLE_RATE,+ source: EMBEDDING_PRICE_SOURCE.builtin,+ });+});++test("the local model resolves to a builtin rate of 0, known to be free", () => {+ expect(resolveEmbeddingPrice(LOCAL_EMBEDDING_MODEL)).toEqual({+ usdPerMtok: 0,+ source: EMBEDDING_PRICE_SOURCE.builtin,+ });+});++test("an unlisted model and a null model resolve to an unknown price", () => {+ const unknown = { usdPerMtok: null, source: EMBEDDING_PRICE_SOURCE.unknown };+ expect(resolveEmbeddingPrice(UNLISTED_MODEL)).toEqual(unknown);+ expect(resolveEmbeddingPrice(null)).toEqual(unknown);+});++test("an operator declaration prices an unlisted model after canonicalisation", () => {+ const override = { model: " ZEmbed-1 ", usdPerMtok: OPERATOR_RATE };+ expect(resolveEmbeddingPrice(UNLISTED_MODEL, override)).toEqual({+ usdPerMtok: OPERATOR_RATE,+ source: EMBEDDING_PRICE_SOURCE.operator,+ });+});++test("an operator declaration re-prices a table model and may declare it free", () => {+ expect(resolveEmbeddingPrice(TABLE_MODEL, { model: TABLE_MODEL, usdPerMtok: 0.03 })).toEqual({+ usdPerMtok: 0.03,+ source: EMBEDDING_PRICE_SOURCE.operator,+ });+ expect(resolveEmbeddingPrice(UNLISTED_MODEL, { model: UNLISTED_MODEL, usdPerMtok: 0 })).toEqual({+ usdPerMtok: 0,+ source: EMBEDDING_PRICE_SOURCE.operator,+ });+});++test("a declaration for another model leaves the table answer unchanged", () => {+ const override = { model: "other-model", usdPerMtok: OPERATOR_RATE };+ const unknown = { usdPerMtok: null, source: EMBEDDING_PRICE_SOURCE.unknown };+ expect(resolveEmbeddingPrice(TABLE_MODEL, override)).toEqual({+ usdPerMtok: TABLE_RATE,+ source: EMBEDDING_PRICE_SOURCE.builtin,+ });+ expect(resolveEmbeddingPrice(UNLISTED_MODEL, override)).toEqual(unknown);+ expect(resolveEmbeddingPrice(null, override)).toEqual(unknown);+});++test("estimateCostUsd scales tokens by the quoted rate", () => {+ const quote = resolveEmbeddingPrice(TABLE_MODEL);+ expect(estimateCostUsd(1_000_000, quote)).toBeCloseTo(TABLE_RATE, 9);+ expect(estimateCostUsd(500_000, quote)).toBeCloseTo(TABLE_RATE / 2, 9);+});++test("estimateCostUsd is 0 for a known-free quote and null for an unknown one", () => {+ expect(estimateCostUsd(10_000_000, resolveEmbeddingPrice(LOCAL_EMBEDDING_MODEL))).toBe(0);+ expect(estimateCostUsd(10_000_000, resolveEmbeddingPrice(UNLISTED_MODEL))).toBeNull();+});diff --git a/tests/core/search/embeddings.signature.test.ts b/tests/core/search/embeddings.signature.test.tsindex 94a7d089..3695ef34 100644--- a/tests/core/search/embeddings.signature.test.ts+++ b/tests/core/search/embeddings.signature.test.ts@@ -2,9 +2,7 @@ import { test, expect } from "bun:test";import {embeddingSignature,- pricePerMillionTokens,estimateTokens,- estimateCostUsd,isStaleSignature,signatureIdentityKnown,EMBEDDING_PRICING,@@ -44,16 +42,6 @@ test("two identities with the same fields share a signature; different dim diffeexpect(a).not.toBe(c);});-test("pricePerMillionTokens returns a positive rate for a known remote model", () => {- expect(pricePerMillionTokens("text-embedding-3-small")).toBeGreaterThan(0);-});--test("pricePerMillionTokens is 0 for the local model and for unknown models", () => {- expect(pricePerMillionTokens(LOCAL_EMBEDDING_MODEL)).toBe(0);- expect(pricePerMillionTokens("some-model-nobody-priced")).toBe(0);- expect(pricePerMillionTokens(null)).toBe(0);-});-test("local model is present in the pricing table at 0", () => {expect(EMBEDDING_PRICING[LOCAL_EMBEDDING_MODEL]).toBe(0);});@@ -67,18 +55,6 @@ test("estimateTokens sums a chars/4 heuristic and rounds up", () => {expect(estimateTokens(["", ""])).toBe(0);});-test("estimateCostUsd scales tokens by the per-million rate", () => {- // 1,000,000 tokens * rate == rate USD.- const rate = pricePerMillionTokens("text-embedding-3-small");- expect(estimateCostUsd(1_000_000, "text-embedding-3-small")).toBeCloseTo(rate, 9);- expect(estimateCostUsd(500_000, "text-embedding-3-small")).toBeCloseTo(rate / 2, 9);-});--test("estimateCostUsd is 0 for local/unknown models regardless of token count", () => {- expect(estimateCostUsd(10_000_000, LOCAL_EMBEDDING_MODEL)).toBe(0);- expect(estimateCostUsd(10_000_000, "unknown")).toBe(0);-});-test("isStaleSignature is true only when active and stored differ", () => {expect(isStaleSignature("local:m:256", "local:m:256")).toBe(false);expect(isStaleSignature("local:m:256", "local:m:512")).toBe(true);diff --git a/tests/core/search/pending-vectors.test.ts b/tests/core/search/pending-vectors.test.tsindex b069d744..32878a7b 100644--- a/tests/core/search/pending-vectors.test.ts+++ b/tests/core/search/pending-vectors.test.ts@@ -44,6 +44,8 @@ import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtuimport { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";+const VEC_LOADABLE = sqliteVecLoadable();+let vault: string;let dbPath: string;let cleanup: () => void;@@ -98,6 +100,35 @@ test("the count agrees with the row walk it replaces", async () => {}});+test("a path-scoped census returns only chunks under the prefixes, row walk and count agreeing", async () => {+ writeMd(vault, "Brain/preferences/pref-a.md", "# Pref A\n\nA belief.");+ writeMd(vault, "Brain/retired/pref-b.md", "# Pref B\n\nA retired belief.");+ writeMd(vault, "notes/c.md", "# C\n\nA plain note.");+ await indexVault(makeConfig({ vault, dbPath }));++ const db = new Database(dbPath, { readonly: true });+ try {+ const pathOf = db.query<{ path: string }, [number]>(+ "SELECT d.path AS path FROM chunks c JOIN documents d ON d.id = c.document_id WHERE c.id = ?",+ );+ const all = findChunksWithoutEmbeddings(db);+ for (const pathPrefixes of [["Brain/preferences/"], ["Brain/preferences/", "Brain/retired/"]]) {+ const scoped = findChunksWithoutEmbeddings(db, { pathPrefixes });+ expect(scoped.length).toBeGreaterThan(0);+ expect(scoped.length).toBeLessThan(all.length);+ for (const row of scoped) {+ const path = pathOf.get(row.chunkId)?.path ?? "";+ expect(pathPrefixes.some((prefix) => path.startsWith(prefix))).toBe(true);+ }+ expect(countChunksWithoutEmbeddings(db, { pathPrefixes })).toBe(scoped.length);+ }+ expect(findChunksWithoutEmbeddings(db, { pathPrefixes: [] })).toEqual(all);+ expect(countChunksWithoutEmbeddings(db, { pathPrefixes: [] })).toBe(all.length);+ } finally {+ db.close();+ }+});+test("an absent index is unrecorded and names the path, never a pending count of zero", () => {const peek = peekPendingVectorsSync(dbPath);expect(peek.kind).toBe("absent");@@ -110,12 +141,14 @@ test("a file that is not a readable index is unrecorded with the reason the openexpect(peek.kind).toBe("unreadable");});-test("a fully embedded vault emits no reindex recommendation", async () => {- if (!sqliteVecLoadable()) return;+test.skipIf(!VEC_LOADABLE)("a fully embedded vault emits no reindex recommendation", async () => {writeMd(vault, "a.md", "# A\n\nA note with vectors.");- await indexVault(cfg(), { embeddings: true });+ // `fake-model` is in no price table; declaring its price keeps the+ // unpriced-model hint out of a report that must say nothing at all.+ const priced = cfg({ priceOverride: { model: "fake-model", usdPerMtok: 0 } });+ await indexVault(priced, { embeddings: true });- const report = await indexCheck(cfg());+ const report = await indexCheck(priced);expect(report.pendingVectors.verdict).toBe("measured");if (report.pendingVectors.verdict === "measured") {expect(report.pendingVectors.pending).toBe(0);diff --git a/tests/core/search/vector-carry-over.test.ts b/tests/core/search/vector-carry-over.test.tsnew file mode 100644index 00000000..1fd1074f--- /dev/null+++ b/tests/core/search/vector-carry-over.test.ts@@ -0,0 +1,413 @@+/**+ * Vector carry-over on edit (t_82c3b275, task 9).+ *+ * The defect pinned here: `replaceChunks` purged every stored vector of+ * a document before it rewrote the chunks, so an edit to one paragraph+ * re-embedded, and re-paid for, every paragraph that did not change.+ * The rule now is that a new chunk whose `content_hash` equals an old+ * chunk's hash in the same document keeps that chunk's vector, provided+ * the old row's model and dimension equal the identity `index_state`+ * records. Only the vectors that are not carried are purged.+ *+ * Covered: the pure multiset matcher (edit, move, duplicates), the+ * store seam (carry, purge of the uncarried vec rows, the model and+ * dimension guards, the unrecorded-identity guard, vec not loaded), the+ * `embeddingsReused` tally of an index run, the embedder record audit on+ * carried rows, a KNN query that finds the carried chunk under its+ * new id, and the one-document scope of the carry census (two notes+ * sharing a paragraph never trade vectors).+ *+ * Deliberately not covered: carry-over ACROSS documents (a paragraph+ * moved into another note is re-embedded by design - the key is scoped+ * to one document) and any change to the model-change lifecycle, which+ * still clears every vector through `ensureEmbeddingModel`.+ */++import { afterEach, beforeEach, describe, expect, test } from "bun:test";+import { Database } from "bun:sqlite";++import { indexVault } from "../../../src/core/search/indexer.ts";+import { Store } from "../../../src/core/search/store.ts";+import type { ChunkInput } from "../../../src/core/search/store/chunks.ts";+import { readEmbedderRecordCensusSync } from "../../../src/core/search/store/embedder-audit.ts";+import {+ EMBEDDING_DIMENSION_STATE_KEY,+ EMBEDDING_MODEL_STATE_KEY,+} from "../../../src/core/search/store/state.ts";+import { matchCarriedVectors } from "../../../src/core/search/store/vector-carry-over.ts";+import { loadVecExtension } from "../../../src/core/search/store/vectors.ts";+import type { ResolvedSearchConfig } from "../../../src/core/search/types.ts";+import { FAKE_PROVIDER_KEY } from "../../helpers/fake-credentials.ts";+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";+import { sqliteVecLoadable } from "../../helpers/sqlite-vec.ts";++const VEC_LOADABLE = sqliteVecLoadable();++const STORE_MODEL = "carry-model";+const STORE_DIMENSION = 4;+const NOTE_PATH = "Notes/carry.md";++let vault: string;+let dbPath: string;+let cleanup: () => void;++beforeEach(() => {+ const v = createTempVault("vector-carry-over");+ vault = v.vault;+ dbPath = v.dbPath;+ cleanup = v.cleanup;+});++afterEach(() => {+ cleanup();+});++function storeConfig(): ResolvedSearchConfig {+ return makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "openai-compat",+ baseUrl: "https://embeddings.invalid/v1",+ model: STORE_MODEL,+ apiKey: FAKE_PROVIDER_KEY,+ dimension: STORE_DIMENSION,+ },+ });+}++function chunkInput(chunkIndex: number, contentHash: string): ChunkInput {+ return {+ chunkIndex,+ content: `content ${contentHash}`,+ contentHash,+ startLine: chunkIndex + 1,+ endLine: chunkIndex + 1,+ tokenCount: 2,+ };+}++function unitVector(seed: number): number[] {+ const raw = [seed + 1, seed * 2 + 1, 3 - seed, 1];+ const norm = Math.hypot(...raw);+ return raw.map((v) => v / norm);+}++const SEED_HASHES = ["h0", "h1", "h2"] as const;++/** A write store holding one document of three embedded chunks. */+async function seededStore(): Promise<{ store: Store; docId: number; oldIds: number[] }> {+ const store = await Store.open(storeConfig(), { mode: "write" });+ const docId = store.upsertDocument({+ path: NOTE_PATH,+ title: null,+ contentHash: "doc-v1",+ mtime: 0,+ size: 1,+ });+ const oldIds = store.replaceChunks(+ docId,+ SEED_HASHES.map((h, i) => chunkInput(i, h)),+ );+ oldIds.forEach((id, i) => {+ store.vecUpsert(id, unitVector(i), STORE_MODEL, STORE_DIMENSION, `emb-${i}`);+ });+ return { store, docId, oldIds };+}++function vecRowCount(path: string): number {+ const db = new Database(path, { readonly: true });+ try {+ if (loadVecExtension(db) === null) throw new Error("sqlite-vec failed to load in the probe");+ return db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM chunk_vec").get()?.n ?? 0;+ } finally {+ db.close();+ }+}++/** The two timestamps of one chunk's `embeddings` row, read off disk. */+function embeddingTimes(path: string, chunkId: number): { createdAt: string; updatedAt: string } {+ const db = new Database(path, { readonly: true });+ try {+ const row = db+ .query<{ created_at: string; updated_at: string }, [number]>(+ "SELECT created_at, updated_at FROM embeddings WHERE chunk_id = ?",+ )+ .get(chunkId);+ if (row === null) throw new Error(`no embeddings row for chunk ${chunkId}`);+ return { createdAt: row.created_at, updatedAt: row.updated_at };+ } finally {+ db.close();+ }+}++function old(hashes: ReadonlyArray<string>): Array<{ contentHash: string; ref: string }> {+ return hashes.map((contentHash, i) => ({ contentHash, ref: `old-${i}` }));+}++describe("matchCarriedVectors", () => {+ test("an edited chunk is the only unmatched position", () => {+ const matches = matchCarriedVectors(old(["a", "b", "c"]), ["a", "b2", "c"]);+ expect(matches.map((m) => [m.position, m.candidate.ref])).toEqual([+ [0, "old-0"],+ [2, "old-2"],+ ]);+ });++ test("a moved chunk is matched at its new position", () => {+ const matches = matchCarriedVectors(old(["a", "b", "c"]), ["c", "a", "b"]);+ expect(matches.map((m) => [m.position, m.candidate.ref])).toEqual([+ [0, "old-2"],+ [1, "old-0"],+ [2, "old-1"],+ ]);+ });++ test("duplicated hashes match as a multiset in order", () => {+ const matches = matchCarriedVectors(old(["a", "a", "b"]), ["a", "b", "a", "a"]);+ expect(matches.map((m) => [m.position, m.candidate.ref])).toEqual([+ [0, "old-0"],+ [1, "old-2"],+ [2, "old-1"],+ ]);+ });++ test("no shared hash carries nothing", () => {+ expect(matchCarriedVectors(old(["a"]), ["b"])).toEqual([]);+ });+});++describe("replaceDocumentChunks carry-over", () => {+ test.skipIf(!VEC_LOADABLE)(+ "an edit keeps the vectors of the unchanged chunks and purges only the replaced one",+ async () => {+ const { store, docId } = await seededStore();+ const replaced = store.replaceDocumentChunks(docId, [+ chunkInput(0, "h0"),+ chunkInput(1, "h1-edited"),+ chunkInput(2, "h2"),+ ]);+ const [first, edited, last] = replaced.chunkIds;+ expect(replaced.embeddingsReused).toBe(2);+ expect(store.getEmbeddingHash(first!)).toBe("emb-0");+ expect(store.getEmbeddingHash(last!)).toBe("emb-2");+ expect(store.getEmbeddingHash(edited!)).toBeNull();+ expect(store.findChunksWithoutEmbeddings().map((p) => p.chunkId)).toEqual([edited!]);+ expect(Array.from(store.embeddingForChunk(last!) ?? [])).toEqual(+ Array.from(Float32Array.from(unitVector(2))),+ );+ const hits = store.semanticTopK(unitVector(2), { limit: 1 });+ expect(hits.map((h) => h.chunkId)).toEqual([last!]);+ await store.close();+ expect(vecRowCount(dbPath)).toBe(2);+ const census = readEmbedderRecordCensusSync(dbPath);+ expect(census.verdict).toBe("audited");+ if (census.verdict === "audited") expect(census.outcome).toBe("complete");+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a carried vector keeps its created_at and moves its updated_at",+ async () => {+ const { store, docId, oldIds } = await seededStore();+ const before = embeddingTimes(dbPath, oldIds[0]!);+ await Bun.sleep(5);+ const replaced = store.replaceDocumentChunks(docId, [+ chunkInput(0, "h0"),+ chunkInput(1, "h1-edited"),+ chunkInput(2, "h2"),+ ]);+ await store.close();+ const after = embeddingTimes(dbPath, replaced.chunkIds[0]!);+ expect(after.createdAt).toBe(before.createdAt);+ expect(after.updatedAt).not.toBe(before.updatedAt);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "the plain replaceChunks form carries too and still returns the ids",+ async () => {+ const { store, docId } = await seededStore();+ const ids = store.replaceChunks(docId, [chunkInput(0, "h2")]);+ expect(store.getEmbeddingHash(ids[0]!)).toBe("emb-2");+ await store.close();+ expect(vecRowCount(dbPath)).toBe(1);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a row whose model differs from the recorded identity is not carried",+ async () => {+ const { store, docId } = await seededStore();+ store.setState(EMBEDDING_MODEL_STATE_KEY, "another-model");+ const replaced = store.replaceDocumentChunks(+ docId,+ SEED_HASHES.map((h, i) => chunkInput(i, h)),+ );+ expect(replaced.embeddingsReused).toBe(0);+ expect(store.countEmbeddings()).toBe(0);+ await store.close();+ expect(vecRowCount(dbPath)).toBe(0);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a row whose dimension differs from the recorded identity is not carried",+ async () => {+ const { store, docId } = await seededStore();+ store.setState(EMBEDDING_DIMENSION_STATE_KEY, String(STORE_DIMENSION * 2));+ const replaced = store.replaceDocumentChunks(+ docId,+ SEED_HASHES.map((h, i) => chunkInput(i, h)),+ );+ expect(replaced.embeddingsReused).toBe(0);+ expect(store.countEmbeddings()).toBe(0);+ await store.close();+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "nothing is carried when the embedding identity is unrecorded",+ async () => {+ const { store, docId } = await seededStore();+ store.deleteState(EMBEDDING_MODEL_STATE_KEY);+ const replaced = store.replaceDocumentChunks(+ docId,+ SEED_HASHES.map((h, i) => chunkInput(i, h)),+ );+ expect(replaced.embeddingsReused).toBe(0);+ expect(store.countEmbeddings()).toBe(0);+ await store.close();+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "with vec not loaded the replacement behaves as before: nothing carried",+ async () => {+ const seeded = await seededStore();+ await seeded.store.close();+ const store = await Store.open(storeConfig(), { mode: "write", loadVec: false });+ expect(store.vecLoaded()).toBe(false);+ const replaced = store.replaceDocumentChunks(+ seeded.docId,+ SEED_HASHES.map((h, i) => chunkInput(i, h)),+ );+ expect(replaced.embeddingsReused).toBe(0);+ expect(replaced.chunkIds).toHaveLength(3);+ expect(store.countEmbeddings()).toBe(0);+ await store.close();+ },+ );+});++/** The `chunk_vec_map` rows of one document's chunks, as `chunk_id:vec_rowid`. */+function vecMapRows(path: string, docId: number): string[] {+ const db = new Database(path, { readonly: true });+ try {+ return db+ .query<{ chunk_id: number; vec_rowid: number }, [number]>(+ "SELECT m.chunk_id AS chunk_id, m.vec_rowid AS vec_rowid FROM chunk_vec_map m " ++ "JOIN chunks c ON c.id = m.chunk_id WHERE c.document_id = ? ORDER BY m.chunk_id",+ )+ .all(docId)+ .map((r) => `${r.chunk_id}:${r.vec_rowid}`);+ } finally {+ db.close();+ }+}++describe("carry-over stays inside one document", () => {+ test.skipIf(!VEC_LOADABLE)(+ "a paragraph two notes share carries only the edited note's own vector",+ async () => {+ const store = await Store.open(storeConfig(), { mode: "write" });+ const docOf = (path: string) =>+ store.upsertDocument({ path, title: null, contentHash: path, mtime: 0, size: 1 });+ const docA = docOf("Notes/a.md");+ const docB = docOf("Notes/b.md");+ // A's shared paragraph sits AFTER B's in chunk_index order, so a+ // census that leaked across documents would hand A B's vector first.+ const idsA = store.replaceChunks(docA, [chunkInput(0, "only-a"), chunkInput(1, "shared")]);+ const idsB = store.replaceChunks(docB, [chunkInput(0, "shared"), chunkInput(1, "only-b")]);+ idsA.forEach((id, i) =>+ store.vecUpsert(id, unitVector(i), STORE_MODEL, STORE_DIMENSION, `a-${i}`),+ );+ idsB.forEach((id, i) =>+ store.vecUpsert(id, unitVector(i + 2), STORE_MODEL, STORE_DIMENSION, `b-${i}`),+ );+ const oldRowidsA = vecMapRows(dbPath, docA).map((row) => row.split(":")[1]);+ const rowsB = vecMapRows(dbPath, docB);++ const replaced = store.replaceDocumentChunks(docA, [+ chunkInput(0, "only-a-edited"),+ chunkInput(1, "shared"),+ ]);+ expect(replaced.embeddingsReused).toBe(1);+ const carried = vecMapRows(dbPath, docA).map((row) => row.split(":")[1]);+ expect(carried).toHaveLength(1);+ for (const rowid of carried) expect(oldRowidsA).toContain(rowid);+ expect(vecMapRows(dbPath, docB)).toEqual(rowsB);+ const sharedA = replaced.chunkIds[1]!;+ expect(store.getEmbeddingHash(sharedA)).toBe("a-1");++ store.deleteDocument("Notes/b.md");+ expect(Array.from(store.embeddingForChunk(sharedA) ?? [])).toEqual(+ Array.from(Float32Array.from(unitVector(1))),+ );+ expect(store.semanticTopK(unitVector(1), { limit: 1 }).map((h) => h.chunkId)).toEqual([+ sharedA,+ ]);+ await store.close();+ },+ );+});++describe("an index run tallies carried vectors", () => {+ /** The offline local embedder, with chunks small enough that each section is its own. */+ function localRunConfig(): ResolvedSearchConfig {+ const base = makeConfig({+ vault,+ dbPath,+ semantic: {+ enabled: true,+ provider: "local",+ baseUrl: null,+ model: null,+ apiKey: null,+ dimension: 64,+ costGateUsd: 0,+ },+ });+ return { ...base, chunkSize: 60, chunkOverlap: 0, chunkMinSize: 1 };+ }++ const SECTIONS = [+ "# Alpha\n\nThe first section talks about compost and soil.",+ "# Beta\n\nThe second section talks about tomatoes in spring.",+ "# Gamma\n\nThe third section talks about watering at dawn.",+ ] as const;++ test.skipIf(!VEC_LOADABLE)(+ "an edit re-embeds only the changed chunk and reports the rest as reused",+ async () => {+ writeMd(vault, NOTE_PATH, SECTIONS.join("\n\n"));+ const cfg = localRunConfig();+ const first = await indexVault(cfg, { embeddings: true });+ expect(first.embeddingsReused).toBe(0);+ const chunks = first.embeddingsComputed;+ expect(chunks).toBeGreaterThanOrEqual(SECTIONS.length);++ writeMd(+ vault,+ NOTE_PATH,+ [SECTIONS[0], "# Beta\n\nThe second section now talks about peppers.", SECTIONS[2]].join(+ "\n\n",+ ),+ );+ const second = await indexVault(cfg, { embeddings: true });+ expect(second.embeddingsComputed).toBe(1);+ expect(second.embeddingsReused).toBe(chunks - 1);+ },+ );+});diff --git a/tests/helpers/self-heal-children.ts b/tests/helpers/self-heal-children.tsindex 99177332..e7429b36 100644--- a/tests/helpers/self-heal-children.ts+++ b/tests/helpers/self-heal-children.ts@@ -52,9 +52,23 @@ export async function waitForSelfHealChildren(if (pending.length > 0) {throw new Error(`self-heal reindex children never finished: ${pending.join(", ")}`);}- // The detached managed-file upgrade worker (#216) holds a lock its parent- // claimed before the spawn and releases it when it ends, so the lock's- // absence is that worker's completion event.+ await waitForSelfHealUpgradeWorker(vault, deadline - Date.now());+}++/**+ * Wait for the detached managed-file upgrade worker (#216) alone. Every+ * background `ensureVaultCurrent` starts one, so a test that drives the+ * reindex children itself, or writes synthetic spawn rows no child will+ * ever pair, still has this worker to wait for before removing its vault.+ *+ * The worker holds a lock its parent claimed before the spawn and releases+ * it when it ends, so the lock's absence is that worker's completion event.+ */+export async function waitForSelfHealUpgradeWorker(+ vault: string,+ budgetMs = CHILD_BUDGET_MS,+): Promise<void> {+ const deadline = Date.now() + budgetMs;const lock = selfHealUpgradeLockPath(vault);while (existsSync(lock) && Date.now() < deadline) {// eslint-disable-next-line no-await-in-loopdiff --git a/tests/mcp/context-pack-tool.test.ts b/tests/mcp/context-pack-tool.test.tsindex c7a433e6..bc6cc91c 100644--- a/tests/mcp/context-pack-tool.test.ts+++ b/tests/mcp/context-pack-tool.test.ts@@ -13,13 +13,24 @@ import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../src/mcp/indeimport { buildToolTable } from "../../src/mcp/tools.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { CONTEXT_GUARD_PLACEHOLDER } from "../../src/core/brain/safety/context-guard.ts";+import { brainConfigPath } from "../../src/core/brain/paths.ts";import { persistTension } from "../../src/core/brain/tensions.ts";+import { BELIEF_VECTORS_BACKFILL_COMMAND } from "../../src/core/brain/context-pack.ts";+import { LOCAL_EMBEDDING_MODEL } from "../../src/core/search/embeddings/signature.ts";+import { EMBEDDING_PRICE_SOURCE } from "../../src/core/search/embeddings/pricing.ts";+import { indexVault, resolveSearchConfig } from "../../src/core/search/index.ts";+import { EMBEDDING_DIMENSION_STATE_KEY, Store } from "../../src/core/search/store.ts";+import { startFakeHttp } from "../helpers/fake-http.ts";+import { sqliteVecLoadable } from "../helpers/sqlite-vec.ts";+import { readRpcErrorCode } from "../helpers/tool-error-envelope.ts";let tmp: string;let vault: string;let configHome: string;let configPath: string;const savedEnv: Record<string, string | undefined> = {};+/** Counted once: a runner without sqlite-vec reports these tests as skipped, never as passed. */+const VEC_LOADABLE = sqliteVecLoadable();beforeEach(() => {tmp = mkdtempSync(join(tmpdir(), "o2b-mcp-context-pack-"));@@ -359,17 +370,88 @@ describe("brain_context_pack tool — ranked query mode", () => {expect(r.error!.message).toContain("query");});+ test("a blank semantic query is refused before anything is embedded", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, {+ max_tokens: 10_000,+ query: " ",+ query_mode: "semantic",+ });+ expect(r.error?.code).toBe(-32602);+ expect(r.error!.message).toContain("query_mode requires a non-empty query");+ });++ test("a query over 2000 characters is refused, as the schema declares", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, {+ max_tokens: 10_000,+ query: "x".repeat(2001),+ query_mode: "semantic",+ });+ expect(r.error?.code).toBe(-32602);+ expect(r.error!.message).toContain("exceeds 2000 characters");+ const tool = buildToolTable("full").find((t) => t.name === "brain_context_pack")!;+ const schema = tool.inputSchema as { properties: Record<string, { maxLength?: number }> };+ expect(schema.properties["query"]!.maxLength).toBe(2000);+ });++ test("a query of exactly 2000 characters is not refused for its length", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, { max_tokens: 10_000, query: "x".repeat(2000) });+ expect(r.error?.message ?? "").not.toContain("exceeds 2000 characters");+ });++ // JSON Schema `maxLength` counts code points, so the enforced cap does+ // too: a character outside the Basic Multilingual Plane is one+ // character, not the two UTF-16 units of its surrogate pair.+ const ASTRAL = "\u{1D11E}";++ test("the cap counts code points: 1001 astral characters are accepted", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, {+ max_tokens: 10_000,+ query: ASTRAL.repeat(1001),+ query_mode: "ranked",+ });+ expect(r.error?.message ?? "").not.toContain("exceeds 2000 characters");+ });++ test("the cap counts code points: 2001 astral characters are refused", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, {+ max_tokens: 10_000,+ query: ASTRAL.repeat(2001),+ query_mode: "ranked",+ });+ expect(r.error?.code).toBe(-32602);+ expect(r.error!.message).toContain("exceeds 2000 characters");+ });++ test("the cap applies in every query mode, the default substring reading included", async () => {+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRaw(server, { max_tokens: 10_000, query: "x".repeat(2001) });+ expect(r.error?.code).toBe(-32602);+ expect(r.error!.message).toContain("exceeds 2000 characters");+ });+test("an unknown query_mode names the accepted set", async () => {const server = new MCPServer({ vault, configPath });await initialize(server);const r = await callPackRaw(server, {max_tokens: 10_000,query: "x",- query_mode: "semantic",+ query_mode: "fuzzy",});expect(r.error).toBeDefined();expect(r.error!.message).toContain("substring");expect(r.error!.message).toContain("ranked");+ expect(r.error!.message).toContain("semantic");});test("the declaration is closed, described, and states the pairing", () => {@@ -380,8 +462,9 @@ describe("brain_context_pack tool — ranked query mode", () => {additionalProperties?: boolean;};const declared = schema.properties["query_mode"]!;- expect(declared.enum).toEqual(["substring", "ranked"]);- expect(declared.description!.length).toBeGreaterThan(0);+ expect(declared.enum).toEqual(["substring", "ranked", "semantic"]);+ expect(declared.description).toContain("semantic");+ expect(declared.description!.endsWith(".")).toBe(true);expect(schema.additionalProperties).toBe(false);expect(schema.dependentRequired!["query_mode"]).toEqual(["query"]);});@@ -547,3 +630,403 @@ describe("brain_context_pack tool — remote reach", () => {expect(out["tokens_used"]).toBe(items.reduce((n, i) => n + i.tokens, 0));});});++/** A core belief note; `extra` adds raw frontmatter lines. */+function writeBelief(id: string, principle: string, extra = ""): void {+ writeFileSync(+ join(vault, "Brain", "preferences", `${id}.md`),+ `---\nid: ${id}\ntopic: ${id}\nprinciple: ${principle}\ntier: core\n${extra}created_at: 2026-05-01T00:00:00Z\n---\n\n${principle}\n`,+ );+}++/** Index the vault under the test config, with or without an embedding pass. */+async function indexBeliefs(embeddings: boolean): Promise<void> {+ await indexVault(resolveSearchConfig({ vault, configPath }), { embeddings });+}++/** Plant one unit vector under `model` on every chunk of each indexed path. */+async function plantVectors(paths: ReadonlyArray<string>, model: string): Promise<void> {+ const store = await Store.open(resolveSearchConfig({ vault, configPath }), { mode: "write" });+ try {+ for (const path of paths) {+ for (const chunk of store.chunksForDocument(store.getDocumentIdByPath(path)!)) {+ store.vecUpsert(chunk.id, [1, 0, 0, 0], model, 4, `planted-${chunk.id}`);+ }+ }+ } finally {+ await store.close();+ }+}++/** One `brain_context_pack` call returning the raw JSON-RPC response. */+async function callPackRpc(+ server: MCPServer,+ args: Record<string, unknown>,+): Promise<{ error?: { code: number; message: string; data?: unknown } }> {+ return (await server.handleRequest({+ jsonrpc: JSONRPC_VERSION,+ id: 12,+ method: "tools/call",+ params: { name: "brain_context_pack", arguments: args },+ })) as { error?: { code: number; message: string; data?: unknown } };+}++describe("brain_context_pack tool — semantic query mode", () => {+ /** Shares tokens with the brevity belief's body only through the local n-gram embedder. */+ const TURN = "keep my answers brief and short";+ const LOCAL_SEMANTIC_CONFIG = [+ "search_semantic_enabled: true",+ "embedding_provider: local",+ "embedding_dimension: 256",+ ];++ function writeLocalSemanticConfig(): void {+ atomicWriteFileSync(+ configPath,+ [`vault: ${vault}`, "agent_name: claude", ...LOCAL_SEMANTIC_CONFIG, ""].join("\n"),+ );+ }++ const SEMANTIC_ARGS = { max_tokens: 10_000, query: TURN, query_mode: "semantic" };+ /** The env override for the embedding credential, unset for the key-missing case. */+ const EMBEDDING_KEY_ENV = "OPEN_SECOND_BRAIN_EMBEDDING_KEY";++ test.skipIf(!VEC_LOADABLE)(+ "orders by stored vectors and carries the semantic report",+ async () => {+ writeLocalSemanticConfig();+ writeBelief("pref-deploy", "never ship on friday");+ writeBelief("pref-brevity", "keep answers short and brief");+ await indexBeliefs(true);+ writeBelief("pref-new", "a belief written after the last embedding pass");+ await indexBeliefs(false);+ const server = new MCPServer({ vault, configPath });+ await initialize(server);++ const out = await callPack(server, SEMANTIC_ARGS);++ const items = out["items"] as Array<{ id: string }>;+ expect(items[0]!.id).toBe("pref-brevity");+ expect(items.map((i) => i.id).toSorted()).toEqual([+ "pref-brevity",+ "pref-deploy",+ "pref-new",+ ]);+ const semantic = out["semantic"] as Record<string, unknown>;+ expect(semantic["model"]).toBe(LOCAL_EMBEDDING_MODEL);+ expect(semantic["price_source"]).toBe(EMBEDDING_PRICE_SOURCE.builtin);+ expect(semantic["estimated_usd"]).toBe(0);+ expect(semantic["query_tokens"]).toBeGreaterThan(0);+ expect(semantic["scored"]).toBe(2);+ expect(semantic["unembedded"]).toEqual(["pref-new"]);+ },+ );++ test("a disabled tier surfaces EMBEDDING_DISABLED on the wire", async () => {+ writeBelief("pref-brevity", "keep answers short and brief");+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRpc(server, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("EMBEDDING_DISABLED");+ });++ test("a missing credential surfaces EMBEDDING_KEY_MISSING on the wire", async () => {+ atomicWriteFileSync(+ configPath,+ [+ `vault: ${vault}`,+ "agent_name: claude",+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ "embedding_base_url: https://embeddings.invalid/v1",+ "embedding_model: c13-model",+ "",+ ].join("\n"),+ );+ const savedKey = process.env[EMBEDDING_KEY_ENV];+ delete process.env[EMBEDDING_KEY_ENV];+ try {+ writeBelief("pref-brevity", "keep answers short and brief");+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRpc(server, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("EMBEDDING_KEY_MISSING");+ } finally {+ if (savedKey !== undefined) process.env[EMBEDDING_KEY_ENV] = savedKey;+ }+ });++ test.skipIf(!VEC_LOADABLE)(+ "no kept belief with a vector refuses with BELIEF_VECTORS_MISSING naming the backfill",+ async () => {+ writeLocalSemanticConfig();+ writeBelief("pref-brevity", "keep answers short and brief");+ await indexBeliefs(false);+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRpc(server, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("BELIEF_VECTORS_MISSING");+ expect(r.error!.message).toContain(BELIEF_VECTORS_BACKFILL_COMMAND);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a refusal after the query embed still discloses that spend",+ async () => {+ writeLocalSemanticConfig();+ // The loader sees the tombstoned belief's vector and embeds; the pack+ // drops the tombstone and refuses over what it keeps.+ writeBelief("pref-gone", "keep answers short and brief", "_status: tombstoned\n");+ await indexBeliefs(true);+ writeBelief("pref-live", "a live belief with no vector yet");+ await indexBeliefs(false);+ const server = new MCPServer({ vault, configPath });+ await initialize(server);+ const r = await callPackRpc(server, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("BELIEF_VECTORS_MISSING");+ expect(r.error!.message).toContain("the query embed was still spent");+ expect(r.error!.message).toContain(`model ${LOCAL_EMBEDDING_MODEL}`);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a reach-withheld page with a vector never lifts the refusal",+ async () => {+ writeLocalSemanticConfig();+ writeBelief("pref-reserved", "keep answers short and brief", "visibility: [private]\n");+ await indexBeliefs(true);+ writeBelief("pref-public", "a public belief with no vector yet");+ await indexBeliefs(false);++ const local = new MCPServer({ vault, configPath }, { reach: "local" });+ await initialize(local);+ const seen = await callPack(local, SEMANTIC_ARGS);+ expect((seen["semantic"] as Record<string, unknown>)["scored"]).toBe(1);++ const remote = new MCPServer({ vault, configPath }, { reach: "remote" });+ await initialize(remote);+ const r = await callPackRpc(remote, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("BELIEF_VECTORS_MISSING");+ expect(JSON.stringify(r)).not.toContain("pref-reserved");+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "scored on the wire counts kept candidates only, not every belief with a vector",+ async () => {+ writeLocalSemanticConfig();+ // Indexed public with a vector, reserved since: the loader still reads+ // its vector vault-wide, the pack's reach filter drops the live page.+ writeBelief("pref-reserved", "keep answers short and brief");+ writeBelief("pref-public", "never ship on friday");+ await indexBeliefs(true);+ writeBelief("pref-reserved", "keep answers short and brief", "visibility: [private]\n");+ const remote = new MCPServer({ vault, configPath }, { reach: "remote" });+ await initialize(remote);++ const out = await callPack(remote, SEMANTIC_ARGS);++ const semantic = out["semantic"] as Record<string, unknown>;+ expect(semantic["scored"]).toBe(1);+ expect(semantic["unembedded"]).toEqual([]);+ expect(JSON.stringify(out)).not.toContain("pref-reserved");+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "an unpriced openai-compat model reports a null cost and an unknown price on the wire",+ async () => {+ const server = await startFakeHttp();+ try {+ atomicWriteFileSync(+ configPath,+ [+ `vault: ${vault}`,+ "agent_name: claude",+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${server.url}"`,+ "embedding_model: c13-unpriced-model",+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "",+ ].join("\n"),+ );+ writeBelief("pref-brevity", "keep answers short and brief");+ await indexBeliefs(false);+ await plantVectors(["Brain/preferences/pref-brevity.md"], "c13-unpriced-model");+ const mcp = new MCPServer({ vault, configPath });+ await initialize(mcp);++ const out = await callPack(mcp, SEMANTIC_ARGS);++ const semantic = out["semantic"] as Record<string, unknown>;+ expect(semantic["model"]).toBe("c13-unpriced-model");+ expect(semantic["price_source"]).toBe(EMBEDDING_PRICE_SOURCE.unknown);+ expect(semantic["estimated_usd"]).toBeNull();+ expect(semantic["scored"]).toBe(1);+ } finally {+ await server.close();+ }+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a remote caller's unpriced query embed under a positive gate is refused before it is sent",+ async () => {+ const fake = await startFakeHttp();+ try {+ atomicWriteFileSync(+ configPath,+ [+ `vault: ${vault}`,+ "agent_name: claude",+ "search_semantic_enabled: true",+ "embedding_provider: openai-compat",+ `embedding_base_url: "${fake.url}"`,+ "embedding_model: c13-unpriced-model",+ "embedding_api_key: test-key",+ "embedding_dimension: 4",+ "embedding_cost_gate_usd: 1",+ "",+ ].join("\n"),+ );+ writeBelief("pref-brevity", "keep answers short and brief");+ await indexBeliefs(false);+ await plantVectors(["Brain/preferences/pref-brevity.md"], "c13-unpriced-model");++ const remote = new MCPServer({ vault, configPath }, { reach: "remote" });+ await initialize(remote);+ const r = await callPackRpc(remote, SEMANTIC_ARGS);+ expect(readRpcErrorCode(r)).toBe("EMBEDDING_COST_UNPRICED");+ expect(fake.callCount()).toBe(0);++ // The operator's own transport keeps the disclosed, ungated embed.+ const local = new MCPServer({ vault, configPath }, { reach: "local" });+ await initialize(local);+ const out = await callPack(local, SEMANTIC_ARGS);+ expect(fake.callCount()).toBe(1);+ expect((out["semantic"] as Record<string, unknown>)["price_source"]).toBe(+ EMBEDDING_PRICE_SOURCE.unknown,+ );+ } finally {+ await fake.close();+ }+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a contradicted embedding identity reaches the wire warnings",+ async () => {+ writeLocalSemanticConfig();+ writeBelief("pref-brevity", "keep answers short and brief");+ await indexBeliefs(true);+ const store = await Store.open(resolveSearchConfig({ vault, configPath }), { mode: "write" });+ store.setState(EMBEDDING_DIMENSION_STATE_KEY, "999");+ await store.close();+ const server = new MCPServer({ vault, configPath });+ await initialize(server);++ const out = await callPack(server, SEMANTIC_ARGS);++ const warnings = out["warnings"] as string[];+ expect(warnings.some((w) => w.includes(EMBEDDING_DIMENSION_STATE_KEY))).toBe(true);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "vectors captured while a page was reserved never order it for a remote caller",+ async () => {+ const PRIVATE_BODY = "secret salary negotiation number is high";+ const PUBLIC = "use tabs for indentation in makefiles";+ const probe = {+ max_tokens: 10_000,+ query: "secret salary negotiation number",+ query_mode: "semantic",+ };+ /** One remote pack over a fresh vault, `seed` deciding what the last index run saw. */+ async function remotePack(seed: () => Promise<void>): Promise<Record<string, unknown>> {+ rmSync(vault, { recursive: true, force: true });+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+ writeLocalSemanticConfig();+ await seed();+ const remote = new MCPServer({ vault, configPath }, { reach: "remote" });+ await initialize(remote);+ const out = await callPack(remote, probe);+ return { items: out["items"], semantic: out["semantic"] };+ }++ const reservedThenRewritten = await remotePack(async () => {+ writeBelief("pref-a", "keep answers short and brief");+ writeBelief("pref-b", PRIVATE_BODY, "visibility: [private]\n");+ await indexBeliefs(true);+ writeBelief("pref-b", PUBLIC);+ });+ const neverReserved = await remotePack(async () => {+ writeBelief("pref-a", "keep answers short and brief");+ await indexBeliefs(true);+ writeBelief("pref-b", PUBLIC);+ });++ expect(JSON.stringify(reservedThenRewritten)).toBe(JSON.stringify(neverReserved));+ },+ );++ /** One raw pack over a fresh vault, `seed` deciding what the index and the live files hold. */+ async function freshPackRpc(+ reach: "local" | "remote",+ seed: () => Promise<void>,+ ): Promise<string> {+ rmSync(vault, { recursive: true, force: true });+ mkdirSync(join(vault, "Brain", "preferences"), { recursive: true });+ writeLocalSemanticConfig();+ await seed();+ const server = new MCPServer({ vault, configPath }, { reach });+ await initialize(server);+ return JSON.stringify(await callPackRpc(server, SEMANTIC_ARGS));+ }++ test.skipIf(!VEC_LOADABLE)(+ "a belief reserved since its vector was stored answers as if it were absent",+ async () => {+ const withheld = await freshPackRpc("remote", async () => {+ writeBelief("pref-withheld", "keep answers short and brief");+ await indexBeliefs(true);+ writeBelief("pref-public", "a public belief with no vector yet");+ await indexBeliefs(false);+ writeBelief("pref-withheld", "keep answers short and brief", "visibility: [private]\n");+ });+ const absent = await freshPackRpc("remote", async () => {+ writeBelief("pref-public", "a public belief with no vector yet");+ await indexBeliefs(false);+ });++ expect(withheld).toBe(absent);+ },+ );++ test.skipIf(!VEC_LOADABLE)(+ "a belief another owner holds answers as if it were absent under the owner gate",+ async () => {+ const gate = () =>+ writeFileSync(+ brainConfigPath(vault),+ "schema_version: 1\nintegrity:\n owner_scope_delivery: fail\n",+ );+ const withheld = await freshPackRpc("local", async () => {+ gate();+ writeBelief("pref-foreign", "keep answers short and brief", "owner: another-agent\n");+ await indexBeliefs(true);+ writeBelief("pref-own", "a belief of this agent with no vector yet");+ await indexBeliefs(false);+ });+ const absent = await freshPackRpc("local", async () => {+ gate();+ writeBelief("pref-own", "a belief of this agent with no vector yet");+ await indexBeliefs(false);+ });++ expect(withheld).toBe(absent);+ },+ );+});diff --git a/tests/mcp/search.test.ts b/tests/mcp/search.test.tsindex b0aac26e..15ca10e2 100644--- a/tests/mcp/search.test.ts+++ b/tests/mcp/search.test.ts@@ -8,7 +8,7 @@*/import { afterEach, beforeEach, expect, test } from "bun:test";-import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";@@ -19,6 +19,7 @@ import { LATEST_SCHEMA_VERSION } from "../../src/core/search/schema.ts";import { atomicWriteFileSync } from "../../src/core/fs-atomic.ts";import { listRecallTelemetry } from "../../src/core/brain/recall-telemetry.ts";import { readQueryDemand } from "../../src/core/brain/query-demand.ts";+import { feedbackDir } from "../../src/core/search/feedback.ts";let tmp: string;let vault: string;@@ -360,3 +361,33 @@ test("brain_search surfaces authoredAt when present and omits it otherwise", asyexpect(dated!["authoredAt"]).toBe(Math.floor(Date.parse("2026-05-20T10:00:00Z") / 1000));expect("authoredAt" in plain!).toBe(false);});++test("brain_search counts its query cap in code points, as the advertised maxLength does", async () => {+ const astral = "\u{1F600}";+ const server = makeServer();+ await initialize(server);+ const within = (await call(server, "brain_search", { query: astral.repeat(1001) })) as any;+ expect(within?.error?.message ?? "").not.toContain("exceeds 2000 characters");+ const over = (await call(server, "brain_search", { query: astral.repeat(2001) })) as any;+ expect(over?.error?.code).toBe(-32602);+ expect(over.error.message).toContain("exceeds 2000 characters");+});++test("brain_recall_feedback enforces its advertised query maxLength before recording", async () => {+ const astral = "\u{1F600}";+ writeMd("note.md", "# Note\n\nthe quarterly ledger reconciliation runbook\n");+ await indexVault(resolveSearchConfig({ vault, configPath }));+ const server = makeServer();+ await initialize(server);+ const feedback = (query: string) =>+ call(server, "brain_recall_feedback", { query, result_path: "note.md", verdict: "up" });++ const over = (await feedback(astral.repeat(2001))) as any;+ expect(over?.error?.code).toBe(-32602);+ expect(over.error.message).toContain("exceeds 2000 characters");+ expect(existsSync(feedbackDir(vault))).toBe(false);++ const within = extractToolResult(await feedback(astral.repeat(2000)));+ expect(within["recorded"]).toBe(true);+ expect(readdirSync(feedbackDir(vault))).toHaveLength(1);+});diff --git a/tests/python/test_static_schemas.py b/tests/python/test_static_schemas.pyindex aa8e5aaf..e6bdc458 100644--- a/tests/python/test_static_schemas.py+++ b/tests/python/test_static_schemas.py@@ -65,6 +65,23 @@ class StaticSchemaIntegrityTests(unittest.TestCase):self.assertNotIn("__mutated__", second[0]["parameters"]["properties"])+def _kill_tree(proc: subprocess.Popen) -> None:+ """Kill the server and everything it started.++ On Windows ``o2b`` is ``o2b.cmd``: ``proc`` is cmd.exe and Bun is its+ child, so ``proc.kill()`` alone would leave Bun holding the stdout pipe+ and the stderr log open.+ """+ if os.name == "nt":+ subprocess.run( # noqa: S603, S607 - fixed argv, test-only+ ["taskkill", "/F", "/T", "/PID", str(proc.pid)],+ check=False,+ capture_output=True,+ )+ else:+ proc.kill()++def _live_memory_tool_projection(vault: str, diagnostics) -> list[dict]:"""Fetch tools/list from a live ``o2b mcp`` and project the curated subset.@@ -97,7 +114,7 @@ def _live_memory_tool_projection(vault: str, diagnostics) -> list[dict]:)# Deadline for the whole handshake: if the server stays alive but never# answers, the timer kills it, readline() sees EOF, and the caller skips.- watchdog = threading.Timer(_HANDSHAKE_TIMEOUT, proc.kill)+ watchdog = threading.Timer(_HANDSHAKE_TIMEOUT, _kill_tree, args=(proc,))watchdog.start()try:def request(rid: int, method: str, params: dict) -> dict:@@ -136,11 +153,14 @@ def _live_memory_tool_projection(vault: str, diagnostics) -> list[dict]:for stream in (proc.stdin, proc.stdout):if stream is not None:stream.close()- proc.terminate()+ # Closed stdin is the server's shutdown signal. Wait for the whole+ # launcher chain to exit before the caller removes the directory that+ # holds the stderr log: on Windows terminate() stops cmd.exe but not+ # its Bun child, which keeps the log open and fails the cleanup.try:proc.wait(timeout=_HANDSHAKE_TIMEOUT)except subprocess.TimeoutExpired:- proc.kill()+ _kill_tree(proc)proc.wait(timeout=_HANDSHAKE_TIMEOUT)return [{field: tool.get(field) for field in _EMBEDDED_FIELDS}diff --git a/tests/setup-env-scrub.test.ts b/tests/setup-env-scrub.test.tsnew file mode 100644index 00000000..6ea86fbc--- /dev/null+++ b/tests/setup-env-scrub.test.ts@@ -0,0 +1,51 @@+/**+ * The suite preload removes `OPEN_SECOND_BRAIN_EMBEDDING_*` twins a+ * developer exports for daily use, unless the run opts out. Proved in a+ * child `bun test` run so the variable is really present at preload time.+ */+import { afterEach, expect, test } from "bun:test";+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";++const LEAKED = "OPEN_SECOND_BRAIN_EMBEDDING_MODEL";+const KEEP_OPT_OUT = "O2B_TEST_KEEP_EMBEDDING_ENV";+const REPO_ROOT = join(import.meta.dir, "..");+let probeDir: string | undefined;++afterEach(() => {+ if (probeDir !== undefined) rmSync(probeDir, { recursive: true, force: true });+ probeDir = undefined;+});++function runProbe(extraEnv: Record<string, string>): { code: number; output: string } {+ probeDir = mkdtempSync(join(tmpdir(), "o2b-env-scrub-"));+ const probe = join(probeDir, "env-probe.test.ts");+ writeFileSync(+ probe,+ `import { test } from "bun:test";\n` ++ `test("probe", () => { console.log("seen=" + String(process.env["${LEAKED}"])); });\n`,+ );+ // A suite run with the opt-out set would hand it to every probe; each+ // probe states it explicitly instead.+ const { [KEEP_OPT_OUT]: _inherited, ...baseEnv } = process.env;+ const child = Bun.spawnSync([process.execPath, "test", probe], {+ cwd: REPO_ROOT,+ env: { ...baseEnv, ...extraEnv },+ stdout: "pipe",+ stderr: "pipe",+ });+ return { code: child.exitCode ?? -1, output: child.stdout.toString() + child.stderr.toString() };+}++test("an exported embedding twin is gone before the first test file loads", () => {+ const run = runProbe({ [LEAKED]: "leaked-model" });+ expect(run.code).toBe(0);+ expect(run.output).toContain("seen=undefined");+});++test(`${KEEP_OPT_OUT}=1 keeps it`, () => {+ const run = runProbe({ [LEAKED]: "leaked-model", [KEEP_OPT_OUT]: "1" });+ expect(run.code).toBe(0);+ expect(run.output).toContain("seen=leaked-model");+});diff --git a/tests/setup.ts b/tests/setup.tsindex d33a5e5e..737e83da 100644--- a/tests/setup.ts+++ b/tests/setup.ts@@ -103,6 +103,18 @@ if (process.env["OPEN_SECOND_BRAIN_DEDUP_CACHE_DIR"] === undefined) {process.env["OPEN_SECOND_BRAIN_DEDUP_CACHE_DIR"] = cacheRoot;}+// The embedding settings resolve from `OPEN_SECOND_BRAIN_EMBEDDING_*`+// twins of the config keys (model, provider, key, price pair, cost gate).+// A developer who exports one for daily use would otherwise change what+// every config-resolving test means, so the suite starts without them. A+// run that wants them kept sets `O2B_TEST_KEEP_EMBEDDING_ENV=1`; a test+// that needs one sets it itself.+if (process.env["O2B_TEST_KEEP_EMBEDDING_ENV"] !== "1") {+ for (const name of Object.keys(process.env)) {+ if (name.startsWith("OPEN_SECOND_BRAIN_EMBEDDING_")) delete process.env[name];+ }+}+// Every temp entry a test mints lands under ONE per-run root, and the run// fails if any of them is still there after the last file (issue #194).//diff --git a/uv.lock b/uv.lockindex 9e289048..585171ab 100644--- a/uv.lock+++ b/uv.lock@@ -4,5 +4,5 @@ requires-python = ">=3.11"[[package]]name = "open-second-brain"-version = "1.71.0"+version = "1.72.0"source = { editable = "." }