Skip to content

Local-first ๐Ÿง  memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly ๐Ÿ˜ด dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

fix(brain): harden reindex swap, self-heal, hot paths, and continuity contracts (1.24.0) (#125)

Sol Aitken1cde572ff4e1
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index a8a85ed0..0dc97149 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.23.1",
+ "version": "1.24.0",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index 7f32d806..c54848fc 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.23.1",
+ "version": "1.24.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"skills": "./skills",
"hooks": "./hooks/hooks.json",
diff --git a/.gitignore b/.gitignore
index c1cf3069..1a296f78 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@ sandbox-vault/
.codegraph/
.orphaned_at
.claude/scheduled_tasks.lock
+.claude/settings.local.json
.open-second-brain/bench-runs/
# Local Kanban/git worktrees (never committed; exclude from lint/fmt)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index de2bca27..a3044e73 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,27 +5,90 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.24.0] - 2026-07-06
+
+A hardening and correctness pass over the PR #121 refactor. Thirteen fixes
+across four classes land as one minor because the `basename` column addition
+forces a `LATEST_SCHEMA_VERSION` bump and reindex on upgrade, and because the
+`created_at` validation tightening turns previously-accepted garbage into
+`INVALID_PARAMS`. The kernel still calls no LLM, and every new field is off or
+absent by default with byte-identical output when unconfigured.
+
+### Added
+
+- **Self-heal on an unreadable search index.** `openReadOrSelfHeal` now rebuilds
+ on `INDEX_UNREADABLE` alongside the existing `INDEX_MISSING` and
+ `SCHEMA_MISMATCH` cases, so a corrupted but present index self-heals instead of
+ surfacing a raw read error.
+
+### Changed
+
+- **`basename` column and index in the search store.** Resolved-wikilink
+ discovery is now an equality join over a populated `documents.basename` column
+ instead of a `documents` SUBSTR scan per link, removing a scan that grew with
+ vault age. The column requires a `LATEST_SCHEMA_VERSION` bump, so every
+ existing install reindexes once on next open. The reindex is absorbed together
+ with the Han-bigram tokenizer change below, so users pay one reindex, not two.
+- **Hot-path memoization.** The entity index, backlink index, and heal regex are
+ now memoized with mtime and generation keys so they are not rebuilt on every
+ query. The memos are opt-in by construction: they introduce a bounded staleness
+ window (a write inside mtime granularity can serve one stale read), which is
+ documented at the call site and pinned by write-then-read invalidation tests.
+- **`links normalize --mode short` rewritten over a basename-to-paths Map.**
+ Output is byte-identical to before, captured by a characterization fixture.
+- **Continuity latest-record queries exit early in reverse chronological order.**
+ The working-memory recompute no longer scans the full history when only the
+ most recent record is needed.
+
+### Fixed
+
+- **Search index rebuild race and crash-restore clobber.** A separate reindex
+ lock (`acquireReindexLock` / `isReindexInProgress`, distinct from the per-Store
+ writer lock) now guards the staging build so a concurrent reindex cannot swap
+ an empty staging database over the live index, and the crash-restore preamble
+ no longer clobbers a freshly built index with a stale `.bak`. Three regression
+ tests pin the path.
+- **`created_at` validation tightened.** Continuity and query-demand records now
+ require a full canonical UTC timestamp; a `YYYY-MM`-prefix timestamp that was
+ previously accepted now fails with a structured `INVALID_PARAMS`. This is the
+ contract-tightening gate that places these fixes behind a minor version.
+- **Surrogate-safe card snippet truncation.** Snippet truncation now cuts on code
+ points instead of UTF-16 code units, so an emoji or other astral character at
+ the boundary no longer splits into a lone U+FFFD on the `cards` disclosure
+ surface.
+- **Second-precision `since` / `until` filtering.** Filter bounds run through
+ timestamp normalization before the lexical compare, so millisecond and second
+ precision are handled consistently and up to a second of records is no longer
+ dropped at each boundary.
+- **Secret-exec audit redaction.** The `command` field logged by the secret-exec
+ path is now redacted before it reaches the audit trail, so a foreign bare
+ credential is no longer persisted in cleartext.
+- **Han-bigram tokenizer restricted to contiguous Han spans.** The CJK bigram
+ pass no longer emits cross-script bigrams (`nๅฎž`) for mixed tokens, matching
+ the docstring's Han-only scope. Pure-Han and pure-ASCII tokenization are
+ unchanged.
+
## [1.23.1] - 2026-07-06
### Fixed
-- **`o2b init` fails on native Windows** โ€” `resolve()` produces backslash
+- **`o2b init` fails on native Windows** -- `resolve()` produces backslash
paths that `setConfigValue()` rejects via `CONFIG_VALUE_REJECTED_CHARS`.
Vault path is now normalized to forward slashes before persisting; forward
slashes work identically in Node/Bun fs APIs on Windows. (#119)
-- **Hermes memory provider bridge fails on Windows** โ€” `scripts/o2b` is a
+- **Hermes memory provider bridge fails on Windows** -- `scripts/o2b` is a
bash script that `subprocess.Popen` cannot execute directly (Windows
delegates to `cmd.exe`, not a POSIX shell). Added `_resolve_command()`
which detects the platform and falls back to `bun run <entry> mcp` when
`o2b` is not available as a native executable.
-- **`detectTooling()` fails to find tools on Windows** โ€” PATH was split on
+- **`detectTooling()` fails to find tools on Windows** -- PATH was split on
`:` only (POSIX); now uses `;` on Windows. Executable probing now honours
`PATHEXT` (`.exe`, `.cmd`, `.bat`, `.com`) instead of hardcoding `.exe`.
-- **MCP `inputSchema` not recognized by Hermes adapters (all platforms)** โ€”
+- **MCP `inputSchema` not recognized by Hermes adapters (all platforms)** --
Hermes adapters (Anthropic, OpenAI, Bedrock) expect tool schemas under
`parameters`, but MCP serves them as `inputSchema`. Without this remap
brain tools had no visible arguments on any OS. Both live and static
- fallback schemas now remap `inputSchema` โ†’ `parameters` at the provider
+ fallback schemas now remap `inputSchema` -> `parameters` at the provider
boundary.
## [1.23.0] - 2026-07-05
@@ -6228,6 +6291,8 @@ plugin config (vault field)`, and exits with a clear
- Sandbox vault and plugin manifest fixtures for tests.
- GitHub release workflow for tag-based and manually dispatched releases.
+[1.24.0]: https://github.com/itechmeat/open-second-brain/compare/v1.23.1...v1.24.0
+[1.23.1]: https://github.com/itechmeat/open-second-brain/compare/v1.23.0...v1.23.1
[1.23.0]: https://github.com/itechmeat/open-second-brain/compare/v1.22.0...v1.23.0
[1.22.0]: https://github.com/itechmeat/open-second-brain/compare/v1.21.0...v1.22.0
[1.21.0]: https://github.com/itechmeat/open-second-brain/compare/v1.20.0...v1.21.0
diff --git a/README.md b/README.md
index 0f94adaa..00caa69d 100644
--- a/README.md
+++ b/README.md
@@ -8,7 +8,7 @@ Open Second Brain plugs into [Hermes Agent](https://github.com/NousResearch/herm
## What is new
-Open Second Brain 1.23.0 adds two optional precision layers that complete the retrieval-precision loop started in 1.22.0, each off by default with byte-identical output when unconfigured. A semantic entity dedup pass surfaces lexical entity-name variants as proposal-only alias-merge candidates when entity_semantic_dedup_enabled is set, via a deterministic jaccard layer with an embedding-cosine reader that reuses the configured embedding provider, and never rewrites the deterministic identity key. An optional cross-encoder rerank stage re-scores the top-K fused search candidates jointly against the query through an OpenAI-compatible endpoint (search_rerank_enabled, off by default), most valuable on the thorough profile, with zero HTTP cost and unchanged ordering when disabled and a graceful degrade to heuristic ordering on endpoint error. The kernel still calls no LLM.
+Open Second Brain 1.24.0 is a hardening and correctness pass. The search index rebuild is now guarded by a separate reindex lock so a concurrent or interrupted reindex cannot swap an empty or stale database over the live index, and an unreadable index self-heals instead of surfacing a raw read error. Wikilink resolution runs over a new `documents.basename` column and index instead of a per-link SUBSTR scan, and the entity, backlink, and heal hot paths are memoized so they are not rebuilt on every query. Continuity records now require a full canonical timestamp, card snippet truncation no longer splits an astral character into a lone replacement character, and `since` / `until` filtering handles second precision consistently. The secret-exec audit trail redacts the command field before it is persisted, and the CJK bigram tokenizer stays inside contiguous Han spans. The `basename` column requires a schema bump, so every existing install reindexes once on next open. The kernel still calls no LLM.
## Why
diff --git a/docs/brainstorm/pr121-hardening-correctness/cli-output/claude.md b/docs/brainstorm/pr121-hardening-correctness/cli-output/claude.md
new file mode 100644
index 00000000..c06f6c46
--- /dev/null
+++ b/docs/brainstorm/pr121-hardening-correctness/cli-output/claude.md
@@ -0,0 +1,42 @@
+## Variant 1 โ€” Card-sequential single minor (1.24.0), risk-ordered
+
+**Approach:** Ship all 13 fixes as one minor on the shared branch, committed one card at a time in risk order: land t_f27d80fe first (rebase `ec3bad27` in as-is, then extend it with the `INDEX_UNREADABLE`โ†’self-heal change as a follow-on commit), then the low-risk correctness cluster (B.1โ€“B.4 + the Han-bigram tokenizer), then the contract-neutral perf items (C.1, C.3โ€“C.6), and finally C.2 as the release-defining commit carrying the `LATEST_SCHEMA_VERSION` bump, the `basename` column, and the version/CHANGELOG bump. One reindex at release absorbs both C.2 and the tokenizer's re-tokenization.
+
+**Trade-offs:**
+- Each card stays a contiguous, independently-revertable commit cluster โ€” matches the "one card at a time / each unit revertable" convention exactly.
+- Data-loss fix reaches `main` first and stays green through every later commit; perf work rebases on a stabilized base, not vice-versa.
+- Self-heal expansion rides with t_f27d80fe (its natural home) and stays decoupled from the reindex-forcing schema bump โ€” two independent safety/contract surfaces reviewed separately.
+- The whole minor is gated behind one reindex because of C.2, so even pure-correctness fixes only take effect post-reindex for users โ€” acceptable for a minor, but not a fast path for the P4 data-loss fix.
+- Longest single branch lifetime; later cards carry the most rebase surface.
+
+**Complexity:** medium
+**Risk:** low
+
+## Variant 2 โ€” Two-release split: correctness patch (1.23.1) then perf+schema minor (1.24.0)
+
+**Approach:** Cut two releases from the shared branch. First a patch: t_f27d80fe (data-loss) + the contract-neutral parts of t_2ba5f0c9 (B.2 surrogate, B.3 precision, B.4 audit redaction) โ€” no schema change, no reindex, ships immediately. Then a minor stacked on the patch: all of t_1224c740, C.2's schema bump, the Han-bigram tokenizer, and B.1's `created_at` tightening, all gated behind a single reindex.
+
+**Trade-offs:**
+- Gets the highest-priority silent-data-loss fix to users fastest, with zero reindex burden.
+- Two version bumps + two CHANGELOG headings + two PRs on a protected `main` โ€” doubles the release ceremony that CLAUDE.md mandates per PR.
+- Forces splitting t_2ba5f0c9 across releases: B.1 is a contract *tightening* (previously-accepted garbage โ†’ `INVALID_PARAMS`) that belongs behind a minor, so the card can't land atomically โ€” violates card atomicity.
+- Two reviewable surfaces are each smaller, but the stacked-PR dependency and cross-release version reconciliation add coordination risk.
+- Best operational safety for the data-loss class; worst fit for the "drive cards one at a time, atomically" framing.
+
+**Complexity:** large
+**Risk:** medium
+
+## Variant 3 โ€” Contract-surface-partitioned single minor (1.24.0)
+
+**Approach:** Regroup the 13 fixes by whether they touch a frozen contract or require reindex, ignoring card boundaries for commit shaping. Land every contract-neutral fix first (t_f27d80fe lock, B.2/B.3/B.4, C.1/C.3โ€“C.6), then a single terminal "contract + schema" commit that bumps `LATEST_SCHEMA_VERSION`, adds the `basename` column, tightens `created_at` (B.1), folds in the tokenizer reindex, expands the self-heal set, and carries the version bump โ€” one reindex, one concentrated contract-review target.
+
+**Trade-offs:**
+- Concentrates the entire frozen-contract blast radius into one clearly-labeled commit โ€” ideal review focus for a frozen-contract project.
+- Fuses unrelated fixes (schema column, timestamp validation, tokenizer, self-heal) into one fat commit: sacrifices per-fix revertability and violates "each unit independently revertable / atomic conventional commits."
+- Couples the `INDEX_UNREADABLE` self-heal safety change to the reindex-forcing schema bump โ€” a rollback of one drags the other.
+- Cross-cutting commit ordering diverges from the "one card at a time" driving model, complicating per-card TDD attribution and merge-conflict reasoning against siblings.
+
+**Complexity:** medium
+**Risk:** high
+
+**Recommended: Variant 1** โ€” it is the only shaping that honors every stated convention simultaneously: cards land one at a time as contiguous, independently-revertable commit clusters with per-fix TDD, the P4 data-loss fix stabilizes the base before perf work rebases onto it, and the single unavoidable reindex is anchored to C.2 as its own auditable schema commit โ€” keeping V3's contract-review clarity (schema bump is isolated) without fusing unrelated fixes or coupling the self-heal safety change to the reindex. The split in V2 buys faster data-loss delivery at the cost of doubled release ceremony on a protected `main` and a non-atomic t_2ba5f0c9, neither of which this repo's bump-in-PR discipline and card-atomic framing tolerate cleanly.
diff --git a/docs/brainstorm/pr121-hardening-correctness/cli-output/prompt.md b/docs/brainstorm/pr121-hardening-correctness/cli-output/prompt.md
new file mode 100644
index 00000000..0ee6ea37
--- /dev/null
+++ b/docs/brainstorm/pr121-hardening-correctness/cli-output/prompt.md
@@ -0,0 +1,145 @@
+You are a senior software architect consulting on an Open Second Brain hardening release.
+
+# Project
+
+**Open Second Brain** โ€” an Obsidian-native memory layer for AI agents. Plain Markdown the user owns, in the vault they already use. The agent reads/writes a `Brain/` directory of `.md` files through deterministic CLI / MCP tools. No daemon, no hidden state outside the vault.
+
+- **Runtime:** Bun + TypeScript (`"type": "module"`, ESM), targets Node. Tests run via `bun test` (script `scripts/test`).
+- **Lint/format:** oxlint + oxfmt. Typecheck: `tsc --noEmit`. `bun run validate` = typecheck + lint + test.
+- **Versioning:** SemVer with **frozen public contracts** since 1.0.0 (MCP tool surface: tool names + input schemas + response-field meanings; CLI verb tree; documented config keys/env mirrors). New optional params and new response fields are additive (minor); renaming/removing/restricting is breaking (major).
+- **Stability policy:** search schema changes carry a `LATEST_SCHEMA_VERSION` bump and require a reindex. The kernel calls no LLM; embedding providers are pluggable.
+- **Current version:** 1.23.0. This work targets a follow-up minor/patch release.
+
+## Conventions (from prior suites, must be respected)
+
+- TDD: failing test first, then implementation; characterization tests pin behavior before refactors.
+- One feature branch, atomic conventional commits, full suite green between commits. Each unit independently revertable.
+- SOLID / KISS / DRY. No misleading fallbacks (a degraded path must log that it degraded, never silently fake success). No hardcoding of paths, sizes, or magic numbers.
+- All repo-facing strings (code, docs, commits, CHANGELOG, config) are **English-only**; logic is abstracted to be multi-language (CJK correctness matters โ€” there are dedicated Han/kana tests).
+- NUL-byte sentinel files exist in `src/core/search/` (e.g. `search.ts`); plain grep silently skips them โ€” workers use `command grep -a` or codegraph.
+- Existing patterns for file locks: a dedicated lockfile path (the in-flight fix for t_f27d80fe uses `acquireReindexLock`/`isReindexInProgress` โ€” a separate lockfile from the per-`Store.open` writer lock).
+
+# Git context (recent history)
+
+```
+67bcb71c refactor: DRY and decomposition (Phases 0-2) (#121) <-- merge commit, branch base
+b9bbcb16 feat(brain): semantic entity dedup and cross-encoder rerank (1.23.0) (#120)
+b8d709ee fix: keep full MCP status output and normalize codegraph paths (#116)
+a98bed1d feat(brain): retrieval precision and quality loop (v1.22.0) (#118)
+42816058 feat(brain): integrity & safety hardening suite (1.21.0) (#115)
+313d061e feat: configurable skills_dir + trigger-keyword auto-attach scoring (#114)
+```
+
+A validated local fix for **t_f27d80fe** already exists on branch `fix/index-swap-hardening` (commit `ec3bad27`, not yet pushed/PR'd) โ€” a dedicated reindex-level lock. The implementation phase may rebase/merge that onto this release branch rather than redo it.
+
+# Scope: 4 tasks shipping together in one release
+
+These four cards are driven **one at a time on a shared branch** `feat/pr121-hardening-correctness`. Each later card's worker must build on the commits the earlier cards landed and must not duplicate or conflict with siblings.
+
+---
+
+## TASK t_f27d80fe โ€” Index-swap hardening: fix reindex/crash-restore data-loss races (P4)
+
+Two interlocking races around the search-index rebuild. **Silent data-loss class โ€” highest priority.**
+
+**A.1 Concurrent `reindexVault` runs leave the live index unreadable.**
+- Where: `src/core/search/indexer.ts:659,664` (pre-lock `tryUnlink(newPath)` + staging-db seed) and `src/core/search/store.ts:356-359`.
+- Failure: run B deletes run A's in-progress staging DB before any lock, seeds a fresh empty staging DB at the same path. A finishes into the orphaned inode and its final `renameSync(newPath, dbPath)` renames B's EMPTY seed over the live index. An empty DB has no `index_state` table โ†’ every subsequent open throws `INDEX_UNREADABLE`, which `openReadOrSelfHeal` (search.ts:210-226) deliberately does NOT self-heal. Search broken until manual `o2b search reindex`.
+- Trigger: a schema-bump upgrade bumps `LATEST_SCHEMA_VERSION`; every concurrent search surface (CLI + long-lived MCP server) hits `SCHEMA_MISMATCH` at once and each invokes `reindexVault`.
+- Fix direction: acquire the writer lock BEFORE touching `newPath` (unlink + seed move inside the lock); second contender waits-or-bails. Consider `INDEX_UNREADABLE` joining the self-heal set once the swap is safe.
+
+**A.2 Crash-restore can clobber a freshly built index with a stale copy.**
+- Where: `src/core/search/indexer.ts:680-682` (swap: `tryUnlink(bak); tryRename(db->bak); renameSync(new->db)`) vs `src/core/search/store.ts:295-306` (every `Store.open` runs a lockless crash-restore preamble: `existsSync(db)==false && existsSync(bak) -> renameSync(bak, db)`).
+- Failure: a concurrent open in the window between `rename(db->bak)` and `rename(new->db)` restores the STALE pre-reindex snapshot, silently discarding the whole rebuild (incl. paid embeddings), with a misleading "restored" note.
+- Fix direction: hold the writer lock across the swap AND make the crash-restore preamble probe the same lock; alternatively gate the restore on a marker file the swap removes last. Must keep genuine crash-recovery working (pin with a test).
+
+**In-flight validated fix (commit ec3bad27):** dedicated reindex-level lock (`acquireReindexLock`/`isReindexInProgress` in store.ts, separate lockfile from per-Store.open writer lock) held across the entire reindexVault sequence โ€” staging build through final rename. Second concurrent reindexVault fails fast with existing typed `INDEX_LOCKED`. `Store.open`'s crash-restore preamble checks `isReindexInProgress` (non-blocking) and skips restoring while a reindex is actively running. 3 new regression tests: second reindexVault rejected while one in progress; Store.open does not restore `.bak` mid-reindex; genuine crash-restore still works.
+
+## TASK t_2ba5f0c9 โ€” Validation/output hardening (P4): four small independent fixes
+
+**B.1 `created_at` accepted with only a `YYYY-MM` prefix check.**
+- Where: `src/mcp/brain/generation-tools.ts:88-90` โ†’ `src/core/brain/generation-reports.ts:88,106-111` โ†’ `src/core/brain/continuity/store.ts:42-45,176`. Latent hole via `host-memory-write.ts:103` and `session-summary.ts:95`.
+- Failure: `"2026-13-01T00:00:00Z"` passes (junk `2026-13.jsonl` shard); `"2026-07-06T15:00:00+03:00"` passes then mis-sorts/mis-filters vs the fleet of `Z` timestamps (lexical compare at store.ts:213,222-223), shards into local-time month across boundaries.
+- Fix: validate full timestamp at the continuity-store boundary (canonical UTC `Z` form, real calendar date), reject with structured error. Contract tightening: previously-accepted garbage โ†’ INVALID_PARAMS.
+
+**B.2 Card snippet truncation can split a surrogate pair.**
+- Where: `src/core/search/search.ts:1121-1126` (`cardSnippet`: `collapsed.slice(0, CARD_SNIPPET_CHARS)`, `CARD_SNIPPET_CHARS=240`). File has a NUL-byte sentinel.
+- Failure: UTF-16 units 239/240 straddling an astral char (emoji, rare CJK) โ†’ lone surrogate rendered as U+FFFD, then "...". Flows into MCP `cards` disclosure surface.
+- Fix: truncate on code points (`[...collapsed]` or `Intl.Segmenter`). Changes output bytes for affected inputs.
+
+**B.3 Query-demand `since`/`until` precision-inconsistent filtering.**
+- Where: `src/core/brain/query-demand.ts:238-239` (lexical compare) vs `:448-453` (`normalizeDemandTimestamp`, exists to fix this class); raw filter strings from `knowledge-gaps.ts:23` and `recall-tools.ts:291`.
+- Failure: second-precision `--since 2026-07-01T00:00:00Z` lexically excludes every stored ms-precision record in `[00:00:00.000Z, 00:00:00.999Z]` because `.` sorts before `Z` โ€” up to one second of records wrongly dropped at each boundary.
+- Fix: run filter bounds through `normalizeDemandTimestamp` before comparing.
+
+**B.4 Secret-exec audit trail can persist a foreign bare secret.**
+- Where: `src/core/brain/secrets/exec.ts:88,96-108` โ€” audit records log `command = argv.join(" ")`; `appendAuditRecord` (`reliability/audit.ts:22`) redacts with default key=value-shaped patterns only.
+- Failure: agent passing some OTHER plaintext credential as a bare positional (`["mytool", "sk-abc123"]`) lands it in `Brain/log/secret-custody/*.jsonl` in cleartext; shaped patterns don't match bare tokens. Severity low but audit file is long-lived.
+- Fix: redact the logged `command` with `redactInfra: true` and/or a high-entropy-token pass before persisting.
+
+## TASK t_1224c740 โ€” Hot-path performance (P4): six items, each reviewable
+
+**C.1 Entity registry re-read from disk on every search query.**
+- Where: `src/core/search/entity-alias.ts:45` โ†’ `src/core/brain/entities/index-builder.ts:80-106` (NUL file); second payer `src/core/search/query-expansion.ts:106`. Per-write rebuilds in `entities/registry.ts` are tolerable; per-QUERY rebuild is not.
+- Cost: `readdirSync` + `readFileSync` + frontmatter parse of every file under `Brain/entities/` per query that names an entity. At 500-2000 entities that dwarfs the FTS/vector lookup.
+- Fix: mtime-keyed memo (dir mtime probe per query, full rebuild on change). Staleness-window trade-off (write within mtime granularity โ†’ one stale read).
+- Test: invalidation test (write โ†’ next query sees the new entity).
+
+**C.2 `documents.basename` column to kill SUBSTR full scans.**
+- Where: `src/core/search/store.ts:1064-1086` (`resolvedDocLinkPairs`) โ€” per unresolved link a COALESCE falls through to a `SUBSTR(d.path, ...)` suffix match plus an identical COUNT(*) uniqueness scan; neither can use `idx_documents_path`.
+- Cost: dangling wikilinks never materialize, so every `brain_bridges`/cluster call re-pays 2 full `documents` scans per dangling link โ€” at 5k docs / 2-5k dangling links, tens of millions of row visits per call.
+- Fix: add a `basename` column (populated at index time) with an index; rewrite suffix matches as equality joins. **Schema version bump + reindex required.**
+
+**C.3 `buildBacklinkIndex` cross-call cost model.**
+- Where: `src/core/brain/backlinks.ts:268-272` and per-tool-call callers (`query-tools.ts:250`, `resources.ts:369`, `doctor.ts:930`, `explorer.ts:88`, moc-audit, concept-cluster). Every call reads+parses the ENTIRE log history; grows unboundedly with brain age. Per-date dir rescan and digest double-build were fixed in PR #121 (no-behavior-change); this is the remaining cross-call recompute.
+- Fix: mtime/generation-keyed memo (staleness trade-off, like C.1) OR incremental log cursor. "Recompute on demand" is a documented design choice โ€” revisit deliberately, not drive-by.
+
+**C.4 Heal-enrich per-page regex rebuild.**
+- Where: `src/core/brain/heal-run.ts:76-88` + `src/core/brain/heal-enrich.ts:42-58`. Per page, re-sorts all K known phrases and recompiles a K-branch alternation regex; at 5k pages ร— 5k phrases = 5k compiles of a ~100-500KB pattern per dream heal run. Opt-in flag + dream-time-only already mitigate.
+- Fix: sort once, compile once, post-filter self-matches per page. Also drop the duplicate frontmatter parse at heal-run:80.
+
+**C.5 `links normalize --mode short` quadratic resolution.**
+- Where: `src/cli/brain/verbs/links.ts:72-86`, `format-wikilink.ts:60-69,76-90`. O(files ร— links ร— pages) `endsWith` scans + fresh P-sized array allocation per link; minutes of wall clock at 5k pages / 100k links.
+- Fix: build a basenameโ†’paths Map once per run; resolve+disambiguate against it. **Output must stay byte-identical โ€” pin with a fixture before optimizing.**
+
+**C.6 Continuity store full-history reads for "find latest".**
+- Where: `src/core/brain/continuity/store.ts:190-215` (`readAllRecords` parses every shard then filters); "latest"-seeking consumers at `session-summary.ts:182`, `pre-compact-extract.ts:148`, recall-telemetry/memory-cost/context-receipts full reads per call.
+- Fix: date-named shards allow a `since`-aware file skip and reverse-chronological early exit for latest-record queries.
+
+Sequencing note from the card: C.1 + C.2 are the biggest per-query wins; C.2 rides a schema bump which pairs naturally with a minor release. Remaining items opportunistic, each with a before/after measurement on a synthetic large vault.
+
+## TASK t_3ccb1512 โ€” Han-bigram tokenizer: restrict bigrams to contiguous Han spans (P3)
+
+- `tokenize()` in `src/core/surface/lexical-score.ts` emits overlapping 2-char bigrams for any token containing a Han char with length > 2. For a *mixed* token (`gbrainๅฎž็Žฐ`, no separator) windows span the ASCII/Han boundary โ†’ bigrams like `nๅฎž`: inert noise (match nothing real) but inflate term frequency/document length.
+- A Han run of exactly 2 chars in a longer mixed token (`abๅฎž็Žฐcd`) is gated by total length not run length โ†’ inconsistent vs a standalone 2-char Han token.
+- Desired: extract bigrams only from maximal contiguous Han spans, leaving non-Han segments untouched; full token still emitted. Removes cross-script noise; makes "Han" scope precise (docstring already says Han-only).
+- Acceptance (TDD): `tokenize("gbrainๅฎž็Žฐ")` emits `ๅฎž็Žฐ` + full token, NO `nๅฎž`; `tokenize("abๅฎž็Žฐcd")` emits `ๅฎž็Žฐ`; pure-Han and pure-ASCII unchanged; no duplicate emission for standalone 2-char `ๅฎž็Žฐ`.
+- Out of scope: widening beyond Han (kana/Hangul/CJK extensions).
+
+# Related source files (already inspected)
+
+- `src/core/search/indexer.ts` (reindex swap sequence ~659-682), `src/core/search/store.ts` (crash-restore preamble ~293-306, schema-version gate ~340, `resolvedDocLinkPairs` ~1064-1086)
+- `src/core/search/search.ts` (`cardSnippet` ~1121-1126, `openReadOrSelfHeal` ~210-226) โ€” NUL-byte sentinel
+- `src/core/brain/continuity/store.ts` (created_at handling ~42-45,176,213,222-223; `readAllRecords` ~190-215)
+- `src/core/brain/query-demand.ts` (since/until ~238-239, `normalizeDemandTimestamp` ~448-453)
+- `src/core/brain/secrets/exec.ts` (~84-112), `src/core/brain/reliability/audit.ts` (~22)
+- `src/core/brain/entities/index-builder.ts` (~80-106, NUL file), `src/core/search/entity-alias.ts:45`, `src/core/search/query-expansion.ts:106`
+- `src/core/brain/backlinks.ts` (~268-272), `src/core/brain/heal-run.ts` (~76-88), `src/core/brain/heal-enrich.ts` (~42-58)
+- `src/cli/brain/verbs/links.ts` (~72-86), `src/core/brain/link-graph/format-wikilink.ts` (~60-90)
+- `src/core/surface/lexical-score.ts` (`tokenize`)
+
+# Your task
+
+Produce exactly THREE distinct architectural variants for delivering this four-task hardening+correctness+performance release on a single shared branch, then exactly one recommendation.
+
+A "variant" is a *release-shaping* decision: how the 13 atomic fixes (2 in t_f27d80fe, 4 in t_2ba5f0c9, 6 in t_1224c740, 1 in t_3ccb1512) are grouped/sequenced/committed, how the schema-bump items (C.2) interact with the frozen search schema contract and the reindex requirement, how the in-flight validated fix (ec3bad27) is reconciled with a fresh implementation on the release branch, how staleness-window memoization (C.1, C.3) is exposed/contained, and how the output-byte-changing fixes (B.2, C.5 byte-identical pin) are de-risked. Consider: should everything ship as one minor, or split? Should the schema bump ride with t_f27d80fe's self-heal change or stay independent? Are the low-risk correctness fixes (B.1-B.4, t_3ccb1512) better landed first to stabilize, or last to avoid merge churn against the bigger perf work?
+
+For EACH variant give:
+- **Approach** (2-3 sentences)
+- **Trade-offs** (bullets)
+- **Complexity** (small | medium | large)
+- **Risk** (low | medium | high)
+
+Then exactly one line: **Recommended: Variant N** followed by a concise rationale (why it best fits the project's frozen-contract stability bar, TDD discipline, and reviewability).
+
+Variants and recommendation ONLY. No code. No sections outside the three variants + the recommendation.
diff --git a/docs/brainstorm/pr121-hardening-correctness/design.md b/docs/brainstorm/pr121-hardening-correctness/design.md
new file mode 100644
index 00000000..3019c116
--- /dev/null
+++ b/docs/brainstorm/pr121-hardening-correctness/design.md
@@ -0,0 +1,99 @@
+# PR #121 hardening + correctness โ€” design
+
+**Status:** draft
+**Author:** product-tech-lead (via Phase 0 brainstorm)
+**Audience:** implementation
+**Branch:** `feat/pr121-hardening-correctness` (from `main` @ `67bcb71c`, PR #121 merge)
+**Scope source:** three-lens discovery pass (correctness/security/performance) on PR #121, 2026-07-06
+
+## Problem statement
+
+PR #121 (`refactor: DRY and decomposition`) landed as a pure no-behavior-change refactor. A three-lens discovery pass over its diff surfaced 13 concrete fixes across four classes that were deliberately deferred so they would not block the refactor:
+
+- **Silent data loss** โ€” two interlocking races around the search-index rebuild (`reindexVault`): a concurrent run can swap an empty staging DB over the live index, and the crash-restore preamble can clobber a freshly built index with a stale `.bak`. Highest priority; realistic trigger is a schema-bump upgrade firing concurrent reindexes across every search surface.
+- **Validation/output holes** โ€” `created_at` accepted with only a `YYYY-MM` prefix (junk shards, mis-sorted records); card snippet truncation splitting a UTF-16 surrogate pair (lone U+FFFD on the MCP `cards` surface); `since`/`until` precision-inconsistent filtering dropping up to a second of records at each boundary; the secret-exec audit trail persisting a foreign bare credential in cleartext.
+- **Hot-path regressions** โ€” the entity index re-read from disk on every query; dangling-wikilink resolution doing full `documents` SUBSTR scans per link; unbounded backlink/heal/links-normalize/continuity recompute costs that scale with brain age.
+- **CJK tokenizer noise** โ€” the Han-bigram pass emits cross-script bigrams (`nๅฎž`) for mixed tokens, inflating term frequencies despite the docstring promising Han-only scope.
+
+All four are independently implementable; none were meta/umbrella; none have unmet parent dependencies.
+
+## Scope
+
+Four cards driven one at a time on the shared branch, each as a contiguous, independently-revertable commit cluster with per-fix TDD:
+
+- `t_f27d80fe` (P4) โ€” index-swap hardening (2 fixes: concurrent-reindex race + crash-restore clobber). **Rebases the validated local fix `ec3bad27` (`acquireReindexLock`/`isReindexInProgress`) onto this branch, then extends it with the `INDEX_UNREADABLE`โ†’self-heal change as a follow-on commit.**
+- `t_2ba5f0c9` (P4) โ€” validation/output hardening (4 fixes: B.1 `created_at` tightening, B.2 surrogate-safe snippet, B.3 since/until precision, B.4 secret-exec audit redaction).
+- `t_1224c740` (P4) โ€” hot-path performance (6 items: C.1 entity-index memo, C.2 `basename` column + schema bump, C.3 backlink memo, C.4 heal regex compile-once, C.5 links-normalize Map, C.6 continuity reverse-chron early exit).
+- `t_3ccb1512` (P3) โ€” Han-bigram tokenizer restricted to contiguous Han spans.
+
+Release target: **1.24.0** (minor) โ€” C.2 forces a `LATEST_SCHEMA_VERSION` bump + reindex, and B.1 is a contract tightening (previously-accepted garbage โ†’ `INVALID_PARAMS`).
+
+## Out of scope
+
+- Widening the CJK tokenizer beyond Han (kana / Hangul / CJK extensions) โ€” separate enhancement, explicitly excluded by `t_3ccb1512`.
+- The dependency-blocked high-value card `t_4678a91a` (blocked on out-of-scope `t_62363378`) โ€” deliberately excluded from this release.
+- Registry/manifest-driven command frameworks or any new public MCP tools / CLI verbs / config keys beyond the schema bump.
+- Auto-merging of semantic entity dedup (1.23.0's proposal-only surface stays proposal-only).
+- Any change to the `o2b search reindex` trigger semantics; the reindex requirement itself is unchanged, only ridden once at release.
+
+## Chosen approach
+
+**Consultant Variant 1 โ€” Card-sequential single minor (1.24.0), risk-ordered.** Ship all 13 fixes as one minor on the shared branch, committed one card at a time in risk order:
+
+1. `t_f27d80fe` first โ€” rebase `ec3bad27` (reindex-level lock) in as-is, then extend with the `INDEX_UNREADABLE` self-heal as a follow-on commit.
+2. The low-risk correctness cluster โ€” `t_2ba5f0c9` (B.1โ€“B.4) and `t_3ccb1512` (Han-bigram tokenizer).
+3. Contract-neutral perf items โ€” C.1, C.3, C.4, C.5, C.6 of `t_1224c740`.
+4. `t_1224c740` C.2 last โ€” the release-defining commit: `LATEST_SCHEMA_VERSION` bump, `basename` column + index, version + CHANGELOG bump. One reindex at release absorbs both C.2 and the tokenizer's re-tokenization.
+
+It is the only shaping that honors every stated convention simultaneously: cards land one at a time as contiguous, independently-revertable commit clusters with per-fix TDD; the P4 data-loss fix stabilizes the base before perf work rebases onto it; and the single unavoidable reindex is anchored to C.2 as its own auditable schema commit without fusing unrelated fixes or coupling the self-heal safety change to the reindex.
+
+## Design decisions
+
+- **Reconcile `ec3bad27`, don't redo it.** The local fix branch already implements `acquireReindexLock`/`isReindexInProgress` (a separate lockfile from the per-`Store.open` writer lock) with 3 regression tests. The implementation phase rebases/merges it onto this release branch rather than reimplementing. The self-heal expansion (`INDEX_UNREADABLE` joining `INDEX_MISSING`/`SCHEMA_MISMATCH` in `openReadOrSelfHeal`) is a *separate follow-on commit* on top, gated on the swap being safe โ€” so the safety change stays decoupled from the lock and independently revertable.
+- **Self-heal expansion stays decoupled from the schema bump.** `INDEX_UNREADABLE`โ†’self-heal rides with `t_f27d80fe` (its natural home, reviewable as a safety fix), NOT bundled into C.2's reindex-forcing commit. A rollback of the schema bump must not drag the self-heal safety change with it.
+- **One reindex, anchored to C.2.** C.2 is the only item that bumps `LATEST_SCHEMA_VERSION` (forcing a reindex). The Han-bigram tokenizer (`t_3ccb1512`) changes tokenization but does not bump the schema โ€” its effect on already-indexed content is absorbed by the single C.2-triggered reindex at release, so users pay one reindex, not two.
+- **B.1 (`created_at`) is the contract-tightening gate for minor vs patch.** Previously-accepted garbage timestamps become `INVALID_PARAMS` โ€” a behavior change that belongs behind a minor, which is why this release is 1.24.0 and not 1.23.1. The other correctness fixes (B.2/B.3/B.4) are byte-changing but not contract-breaking and could in principle patch; they ride the minor because splitting the card across releases would violate card atomicity (Variant 2 rejected).
+- **Staleness-window memos (C.1, C.3) are opt-in by construction, not config.** mtime/generation-keyed memoization introduces a bounded staleness window (a write within mtime granularity can serve one stale read). This is a *deliberate behavior trade-off*, documented at the call site, not a hidden flag โ€” matching the project's "no misleading fallbacks" rule (a memo hit must not silently lie about freshness; invalidation tests pin writeโ†’read-visibility).
+- **C.5 output is pinned byte-identical before optimization.** The `links normalize --mode short` rewrite (basenameโ†’paths Map) must not change user-facing output; a fixture captures current output first, then the optimization must reproduce it exactly.
+- **B.2 snippet truncation changes output bytes for affected inputs โ€” by design.** Truncating on code points (`[...collapsed]` or `Intl.Segmenter`) fixes a lone-surrogate defect; the new bytes are correct where the old were corrupt. This is a fix, not a regression, and is covered by an emoji-at-boundary fixture.
+- **No new dependencies.** All fixes use stdlib (`Array` spread, `Intl.Segmenter` where available, SQLite column + index, `Map`). The schema bump uses the existing migration path.
+- **NUL-byte sentinel files respected.** `search.ts` and `index-builder.ts` contain NUL bytes; workers use `command grep -a` or codegraph, never plain grep (which silently skips matches).
+
+## File changes
+
+**`t_f27d80fe` (rebase `ec3bad27` + extend):**
+- `src/core/search/store.ts` โ€” `acquireReindexLock`/`isReindexInProgress` (separate lockfile from per-`Store.open` writer lock); crash-restore preamble probes `isReindexInProgress` non-blocking.
+- `src/core/search/indexer.ts` โ€” `reindexVault` acquires the reindex lock before touching `newPath` (unlink + seed move inside the lock).
+- `src/core/search/search.ts` (NUL file) โ€” `openReadOrSelfHeal` self-heal set extended to include `INDEX_UNREADABLE`.
+- `tests/core/search/` โ€” 3 regression tests from `ec3bad27` (second reindex rejected mid-flight; `Store.open` does not restore `.bak` mid-reindex; genuine crash-restore still works) + self-heal test for `INDEX_UNREADABLE`.
+
+**`t_2ba5f0c9` (4 fixes):**
+- `src/core/brain/continuity/store.ts` โ€” full timestamp validation at the boundary (canonical UTC `Z`, real calendar date); reject with structured `INVALID_PARAMS`.
+- `src/core/search/search.ts` (NUL file) โ€” `cardSnippet` truncates on code points.
+- `src/core/brain/query-demand.ts` โ€” filter bounds run through `normalizeDemandTimestamp` before lexical compare.
+- `src/core/brain/secrets/exec.ts` โ€” logged `command` redacted with `redactRawOutput`/`redactInfra` and/or a high-entropy-token pass before it is handed to `appendAuditRecord`. The audit sink itself (`src/core/reliability/audit.ts`, imported via `../../reliability/audit.ts`) is unchanged; the fix lives at the call site in `exec.ts`, not in the audit module.
+- `tests/` โ€” table-driven accept/reject for `created_at`; emoji-at-boundary snippet fixture; ms- vs second-precision since/until fixture; bare-`sk-` audit-redaction fixture.
+
+**`t_1224c740` (6 items):**
+- `src/core/brain/entities/index-builder.ts` (NUL file) + `src/core/search/entity-alias.ts` / `query-expansion.ts` โ€” mtime-keyed memo of `buildEntityIndex` (dir mtime probe per query).
+- `src/core/search/store.ts` โ€” `documents.basename` column + index (populated at index time); `resolvedDocLinkPairs` rewritten as equality joins; `LATEST_SCHEMA_VERSION` bump.
+- `src/core/brain/backlinks.ts` โ€” mtime/generation-keyed memo or incremental log cursor for `buildBacklinkIndex`.
+- `src/core/brain/heal-run.ts` + `heal-enrich.ts` โ€” sort-once/compile-once regex; drop duplicate frontmatter parse.
+- `src/cli/brain/verbs/links.ts` + `src/core/brain/link-graph/format-wikilink.ts` โ€” basenameโ†’paths Map (byte-identical output pinned first).
+- `src/core/brain/continuity/store.ts` โ€” `since`-aware file skip + reverse-chron early exit for latest-record queries.
+- `tests/` โ€” entity invalidation; link-resolution + synthetic 5k-doc scale smoke; backlink memo freshness; heal regex compile-count; links-normalize byte-identity fixture; continuity latest-record early-exit.
+
+**`t_3ccb1512` (1 fix):**
+- `src/core/surface/lexical-score.ts` โ€” `tokenize` extracts bigrams only from maximal contiguous Han spans.
+- `tests/core/surface/lexical-score.test.ts` โ€” `gbrainๅฎž็Žฐ`โ†’`ๅฎž็Žฐ`+full (no `nๅฎž`); `abๅฎž็Žฐcd`โ†’`ๅฎž็Žฐ`; pure-Han/pure-ASCII unchanged; standalone 2-char `ๅฎž็Žฐ` no duplicate.
+
+**Release-wide:**
+- `package.json` + `CHANGELOG.md` โ€” version bump to 1.24.0 (lands with C.2, the schema-bump commit).
+
+## Risks
+
+- **Schema bump + reindex on upgrade.** C.2 bumps `LATEST_SCHEMA_VERSION`; every existing install reindexes on next open. Mitigated by `t_f27d80fe` landing first (the reindex path is now race-safe), and by the single-reindex anchor (users pay once). Risk: the race fix and the schema bump are in the same release โ€” if the race fix regressed, the forced reindex would exercise it under load. Mitigation: the 3 regression tests + the integration overlap test pin the path before C.2 lands.
+- **Staleness-window memos (C.1, C.3) can serve one stale read after a write.** Deliberate trade-off, pinned by invalidation tests (write โ†’ next query sees the new entity / backlinks). Documented at the call site; not hidden behind a flag.
+- **C.5 byte-identity regression.** The links-normalize rewrite must reproduce current output exactly. Mitigated by a characterization fixture captured before the rewrite.
+- **B.2 / B.1 change observable bytes / accepted inputs.** B.2 fixes a corrupt lone surrogate (new bytes are correct); B.1 rejects previously-accepted garbage (the minor-version gate). Both covered by fixtures; neither is a silent regression.
+- **`ec3bad27` rebase onto the release branch.** The fix was validated on `fix/index-swap-hardening` off an earlier base; rebasing onto `feat/pr121-hardening-correctness` (post-#121 merge) may surface context drift. Mitigation: rebased as the first commit cluster with the full suite green before any other card lands.
diff --git a/docs/brainstorm/pr121-hardening-correctness/plan.md b/docs/brainstorm/pr121-hardening-correctness/plan.md
new file mode 100644
index 00000000..4b0a5db5
--- /dev/null
+++ b/docs/brainstorm/pr121-hardening-correctness/plan.md
@@ -0,0 +1,103 @@
+# PR #121 hardening + correctness โ€” implementation plan
+
+One feature branch (`feat/pr121-hardening-correctness`), four cards driven **one at a time** in risk order, each a contiguous commit cluster with per-fix TDD (failing test first). Full suite green between cards. Each later card builds on the commits earlier cards landed โ€” do not duplicate or conflict with siblings.
+
+Drive order (matches `design.md` Chosen approach):
+1. `t_f27d80fe` โ€” index-swap hardening (stabilizes the base; the forced reindex from C.2 will exercise this path).
+2. `t_2ba5f0c9` โ€” validation/output hardening (low-risk correctness).
+3. `t_3ccb1512` โ€” Han-bigram tokenizer (low-risk CJK correctness; its tokenization change is absorbed by the C.2 reindex).
+4. `t_1224c740` โ€” hot-path performance, ending with C.2 (the schema bump + version bump, the release-defining commit).
+
+---
+
+## Task 1: Index-swap hardening โ€” fix reindex/crash-restore data-loss races (kanban t_f27d80fe, P4)
+
+Rebase the validated local fix `ec3bad27` (branch `fix/index-swap-hardening`) onto this release branch as the first commit cluster, then extend it with the `INDEX_UNREADABLE` self-heal as a separate follow-on commit.
+
+**Files:**
+- `src/core/search/store.ts` โ€” `acquireReindexLock` / `isReindexInProgress` (a dedicated lockfile, separate from the per-`Store.open` writer lock); `Store.open` crash-restore preamble (~293-306) probes `isReindexInProgress` non-blocking and skips restoring while a reindex is actively running.
+- `src/core/search/indexer.ts` โ€” `reindexVault` acquires the reindex lock BEFORE touching `newPath` (the `tryUnlink(newPath)` + staging seed at ~659,664 move inside the lock); second concurrent `reindexVault` fails fast with the existing typed `INDEX_LOCKED`.
+- `src/core/search/search.ts` (NUL-byte sentinel โ€” use `command grep -a`) โ€” `openReadOrSelfHeal` (~210-226) self-heal set extended to include `INDEX_UNREADABLE`.
+- `tests/core/search/` โ€” the 3 regression tests from `ec3bad27` (second `reindexVault` rejected while one in progress; `Store.open` does not restore `.bak` mid-reindex; genuine crash-restore still works) + a new self-heal test asserting `INDEX_UNREADABLE` is recovered.
+
+**Acceptance (a passing test):**
+- Integration test that starts two overlapping `reindexVault` runs (the second entering during the first's build phase) and asserts the surviving index is complete and readable โ€” NOT an empty DB, NOT `INDEX_UNREADABLE`.
+- `Store.open` does not restore `.bak` during an in-flight reindex (interleaved open-during-swap test).
+- Genuine crash-restore path still green (the existing crash-restore test stays green).
+- `INDEX_UNREADABLE` is now self-healed by `openReadOrSelfHeal` (new test).
+
+**Depends on:** none (first card; stabilizes the base for the rest).
+
+---
+
+## Task 2: Validation/output hardening โ€” four independent fixes (kanban t_2ba5f0c9, P4)
+
+Land as up to four commits (one per fix), each TDD. B.1 is the contract-tightening gate (previously-accepted garbage โ†’ `INVALID_PARAMS`) โ€” it is why the release is a minor.
+
+**B.1 โ€” `created_at` full-timestamp validation**
+- **Files:** `src/core/brain/continuity/store.ts` (boundary ~42-45,176; lexical compare at 213,222-223); possibly a shared validator extracted from `normalizeDemandTimestamp` (DRY โ€” see B.3).
+- **Acceptance:** table-driven accept/reject at the store boundary โ€” `"2026-13-01T00:00:00Z"` rejected (invalid month), `"2026-07-06T15:00:00+03:00"` rejected or canonicalized to `Z` (reject is simpler and matches the contract tightening); valid canonical `Z` timestamps accepted; one MCP-level case per exposed tool (`generation-tools.ts`, `host-memory-write.ts`, `session-summary.ts`).
+
+**B.2 โ€” surrogate-safe card snippet**
+- **Files:** `src/core/search/search.ts` (NUL file) โ€” `cardSnippet` (~1121-1126, `CARD_SNIPPET_CHARS=240`) truncates on code points (`[...collapsed].slice(0, N).join("")` or `Intl.Segmenter`).
+- **Acceptance:** snippet fixture with an emoji astral char at the truncation boundary asserts no lone surrogate / no U+FFFD; the snippet ends on a complete code point followed by `...`.
+
+**B.3 โ€” `since`/`until` precision-consistent filtering**
+- **Files:** `src/core/brain/query-demand.ts` โ€” filter bounds (~238-239) run through `normalizeDemandTimestamp` (~448-453) before the lexical compare. Consider extracting a shared canonical-UTC normalizer reused by B.1 (DRY).
+- **Acceptance:** boundary fixture โ€” a second-precision `--since 2026-07-01T00:00:00Z` does NOT exclude ms-precision records in `[00:00:00.000Z, 00:00:00.999Z]`.
+
+**B.4 โ€” secret-exec audit redaction**
+- **Files:** `src/core/brain/secrets/exec.ts` (~88,96-108) โ€” the logged `command = argv.join(" ")` is redacted (`redactInfra: true` and/or a high-entropy-token pass) before `appendAuditRecord` persists it.
+- **Acceptance:** audit fixture asserting a bare `sk-`-shaped token passed as a positional argument is masked in `Brain/log/secret-custody/*.jsonl`.
+
+**Depends on:** Task 1 (build on the stabilized base).
+
+---
+
+## Task 3: Han-bigram tokenizer โ€” restrict bigrams to contiguous Han spans (kanban t_3ccb1512, P3)
+
+TDD-first per the card's acceptance. Its tokenization change is absorbed by the single C.2 reindex at release (no separate schema bump here).
+
+**Files:**
+- `src/core/surface/lexical-score.ts` โ€” `tokenize` extracts bigrams only from maximal contiguous Han spans within each token, leaving non-Han segments untouched; the full token is still emitted as today.
+- `tests/core/surface/lexical-score.test.ts` โ€” the four acceptance cases below.
+
+**Acceptance (a passing test):**
+- `tokenize("gbrainๅฎž็Žฐ")` emits the Han bigram `ๅฎž็Žฐ` and the full token, and NO cross-boundary bigram (`nๅฎž`).
+- `tokenize("abๅฎž็Žฐcd")` emits `ๅฎž็Žฐ`.
+- Pure-Han and pure-ASCII tokenization unchanged (existing tests stay green).
+- No duplicate emission for a standalone 2-char Han token (`tokenize("ๅฎž็Žฐ")` stays `["ๅฎž็Žฐ"]`).
+
+**Depends on:** Task 2 (lands before the perf card; its tokenization change rides the C.2 reindex).
+
+---
+
+## Task 4: Hot-path performance โ€” six items (kanban t_1224c740, P4)
+
+Land the five contract-neutral items first (C.1, C.3, C.4, C.5, C.6), then C.2 as the release-defining commit (schema bump + version bump). Each item carries a before/after measurement on a synthetic large vault where the card calls for it.
+
+**C.1 โ€” entity-index mtime memo (biggest per-query win)**
+- **Files:** `src/core/brain/entities/index-builder.ts` (NUL file, ~80-106); consumers `src/core/search/entity-alias.ts:45` and `src/core/search/query-expansion.ts:106`.
+- **Acceptance:** an invalidation test โ€” write a new entity โ†’ the next query that names an entity sees it (mtime probe triggers rebuild). Existing entity suites green. Documented staleness-window trade-off at the call site.
+
+**C.2 โ€” `documents.basename` column (schema bump; release-defining commit)**
+- **Files:** `src/core/search/store.ts` โ€” add `basename` column + index (populated at index time); `resolvedDocLinkPairs` (~1064-1086) rewritten as equality joins (no `SUBSTR` full scan); bump `LATEST_SCHEMA_VERSION`; `package.json` + `CHANGELOG.md` โ†’ 1.24.0.
+- **Acceptance:** link-resolution suites green; a scale smoke test on a synthetic 5k-doc / multi-k-dangling-link fixture shows the suffix scans eliminated (equality joins). The single release reindex absorbs C.2 and the Task 3 tokenizer change.
+
+**C.3 โ€” `buildBacklinkIndex` cross-call memo**
+- **Files:** `src/core/brain/backlinks.ts` (~268-272); callers `query-tools.ts:250`, `resources.ts:369`, `doctor.ts:930`, `explorer.ts:88`, moc-audit, concept-cluster.
+- **Acceptance:** mtime/generation-keyed memo (or incremental log cursor) with a freshness test; full-history re-read eliminated on repeat calls within the same generation. "Recompute on demand" revisited deliberately, documented.
+
+**C.4 โ€” heal-enrich compile-once regex**
+- **Files:** `src/core/brain/heal-run.ts` (~76-88) + `src/core/brain/heal-enrich.ts` (~42-58). Sort once, compile once, post-filter self-matches per page; drop the duplicate frontmatter parse at `heal-run:80`.
+- **Acceptance:** existing heal tests green; a test asserting the alternation regex is compiled once per run, not once per page (compile-count assertion or equivalent).
+
+**C.5 โ€” `links normalize --mode short` Map-based resolution**
+- **Files:** `src/cli/brain/verbs/links.ts` (~72-86) + `src/core/brain/link-graph/format-wikilink.ts` (~60-90).
+- **Acceptance:** a characterization fixture captures current output BEFORE the rewrite; after the rewrite the output is byte-identical. Build a basenameโ†’paths Map once per run; resolve+disambiguate against it.
+
+**C.6 โ€” continuity reverse-chron early exit**
+- **Files:** `src/core/brain/continuity/store.ts` (~190-215, `readAllRecords`); consumers `session-summary.ts:182`, `pre-compact-extract.ts:148`, recall-telemetry/memory-cost/context-receipts.
+- **Acceptance:** a `since`-aware file-skip test (shards outside the window are not parsed) and a latest-record reverse-chron early-exit test (stops at the first match, not after parsing all shards).
+
+**Depends on:** Tasks 1โ€“3 (builds on the stabilized, correctness-fixed base; C.2's forced reindex exercises Task 1's race-safe reindex).
diff --git a/docs/brainstorm/pr121-hardening-correctness/variants.md b/docs/brainstorm/pr121-hardening-correctness/variants.md
new file mode 100644
index 00000000..5fa5b954
--- /dev/null
+++ b/docs/brainstorm/pr121-hardening-correctness/variants.md
@@ -0,0 +1,69 @@
+# PR #121 hardening + correctness โ€” consultant variants
+
+Verbatim output from the primary consultant (Claude, model `claude-opus-4-8`), 3 variants + recommendation. Primary succeeded (exit 0); no fallback run.
+
+Source: `docs/brainstorm/pr121-hardening-correctness/cli-output/claude.md`
+
+---
+
+## Variant 1 โ€” Card-sequential single minor (1.24.0), risk-ordered
+
+**Approach:** Ship all 13 fixes as one minor on the shared branch, committed one card at a time in risk order: land t_f27d80fe first (rebase `ec3bad27` in as-is, then extend it with the `INDEX_UNREADABLE`โ†’self-heal change as a follow-on commit), then the low-risk correctness cluster (B.1โ€“B.4 + the Han-bigram tokenizer), then the contract-neutral perf items (C.1, C.3โ€“C.6), and finally C.2 as the release-defining commit carrying the `LATEST_SCHEMA_VERSION` bump, the `basename` column, and the version/CHANGELOG bump. One reindex at release absorbs both C.2 and the tokenizer's re-tokenization.
+
+**Trade-offs:**
+- Each card stays a contiguous, independently-revertable commit cluster โ€” matches the "one card at a time / each unit revertable" convention exactly.
+- Data-loss fix reaches `main` first and stays green through every later commit; perf work rebases on a stabilized base, not vice-versa.
+- Self-heal expansion rides with t_f27d80fe (its natural home) and stays decoupled from the reindex-forcing schema bump โ€” two independent safety/contract surfaces reviewed separately.
+- The whole minor is gated behind one reindex because of C.2, so even pure-correctness fixes only take effect post-reindex for users โ€” acceptable for a minor, but not a fast path for the P4 data-loss fix.
+- Longest single branch lifetime; later cards carry the most rebase surface.
+
+**Complexity:** medium
+**Risk:** low
+
+---
+
+## Variant 2 โ€” Two-release split: correctness patch (1.23.1) then perf+schema minor (1.24.0)
+
+**Approach:** Cut two releases from the shared branch. First a patch: t_f27d80fe (data-loss) + the contract-neutral parts of t_2ba5f0c9 (B.2 surrogate, B.3 precision, B.4 audit redaction) โ€” no schema change, no reindex, ships immediately. Then a minor stacked on the patch: all of t_1224c740, C.2's schema bump, the Han-bigram tokenizer, and B.1's `created_at` tightening, all gated behind a single reindex.
+
+**Trade-offs:**
+- Gets the highest-priority silent-data-loss fix to users fastest, with zero reindex burden.
+- Two version bumps + two CHANGELOG headings + two PRs on a protected `main` โ€” doubles the release ceremony that CLAUDE.md mandates per PR.
+- Forces splitting t_2ba5f0c9 across releases: B.1 is a contract *tightening* (previously-accepted garbage โ†’ `INVALID_PARAMS`) that belongs behind a minor, so the card can't land atomically โ€” violates card atomicity.
+- Two reviewable surfaces are each smaller, but the stacked-PR dependency and cross-release version reconciliation add coordination risk.
+- Best operational safety for the data-loss class; worst fit for the "drive cards one at a time, atomically" framing.
+
+**Complexity:** large
+**Risk:** medium
+
+---
+
+## Variant 3 โ€” Contract-surface-partitioned single minor (1.24.0)
+
+**Approach:** Regroup the 13 fixes by whether they touch a frozen contract or require reindex, ignoring card boundaries for commit shaping. Land every contract-neutral fix first (t_f27d80fe lock, B.2/B.3/B.4, C.1/C.3โ€“C.6), then a single terminal "contract + schema" commit that bumps `LATEST_SCHEMA_VERSION`, adds the `basename` column, tightens `created_at` (B.1), folds in the tokenizer reindex, expands the self-heal set, and carries the version bump โ€” one reindex, one concentrated contract-review target.
+
+**Trade-offs:**
+- Concentrates the entire frozen-contract blast radius into one clearly-labeled commit โ€” ideal review focus for a frozen-contract project.
+- Fuses unrelated fixes (schema column, timestamp validation, tokenizer, self-heal) into one fat commit: sacrifices per-fix revertability and violates "each unit independently revertable / atomic conventional commits."
+- Couples the `INDEX_UNREADABLE` self-heal safety change to the reindex-forcing schema bump โ€” a rollback of one drags the other.
+- Cross-cutting commit ordering diverges from the "one card at a time" driving model, complicating per-card TDD attribution and merge-conflict reasoning against siblings.
+
+**Complexity:** medium
+**Risk:** high
+
+---
+
+**Recommended: Variant 1** โ€” it is the only shaping that honors every stated convention simultaneously: cards land one at a time as contiguous, independently-revertable commit clusters with per-fix TDD, the P4 data-loss fix stabilizes the base before perf work rebases onto it, and the single unavoidable reindex is anchored to C.2 as its own auditable schema commit โ€” keeping V3's contract-review clarity (schema bump is isolated) without fusing unrelated fixes or coupling the self-heal safety change to the reindex. The split in V2 buys faster data-loss delivery at the cost of doubled release ceremony on a protected `main` and a non-atomic t_2ba5f0c9, neither of which this repo's bump-in-PR discipline and card-atomic framing tolerate cleanly.
+
+---
+
+## Final decision rationale
+
+**Accepted Variant 1 (no override).** It satisfies all four project-specific constraints that the other variants strain against:
+
+1. **Card-atomic driving model** โ€” the orchestrator drives cards one at a time on a shared branch; Variant 1 is the only shaping whose commit clusters map 1:1 to cards, so per-card TDD attribution and merge-conflict reasoning stay clean. (Variant 3 ignores card boundaries; Variant 2 splits a card across releases.)
+2. **Frozen-contract reviewability** โ€” the schema bump (`LATEST_SCHEMA_VERSION`, `basename` column) is isolated as its own auditable commit, not fused with unrelated correctness fixes (Variant 3's failure) or scattered across releases (Variant 2).
+3. **Safety/contract decoupling** โ€” the `INDEX_UNREADABLE` self-heal rides with its natural home (t_f27d80fe) and stays decoupled from the reindex-forcing schema bump, so a rollback of one cannot drag the other.
+4. **Single-reindex economy** โ€” C.2's unavoidable reindex absorbs the Han-bigram tokenizer's re-tokenization at no extra cost; users pay one reindex for the whole release.
+
+The one real cost (the P4 data-loss fix is gated behind the C.2 reindex for users who upgrade to 1.24.0) is acceptable for a minor release and is itself mitigated by landing t_f27d80fe first โ€” the reindex path is race-safe before the schema bump forces load onto it.
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index b6d916ef..bbfc3e2f 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "1.23.1",
+ "version": "1.24.0",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index ccef5e27..a1cf287e 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.23.1",
+ "version": "1.24.0",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
"keywords": [
diff --git a/plugin.yaml b/plugin.yaml
index 4b312cc2..e803f447 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.23.1"
+version: "1.24.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index 421d0cf9..f0544400 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "1.23.1",
+ "version": "1.24.0",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index 4b312cc2..e803f447 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "1.23.1"
+version: "1.24.0"
description: "Open Second Brain - native Hermes memory provider backed by an Obsidian-compatible Markdown vault."
author: "Open Second Brain contributors"
memory_provider: true
diff --git a/pyproject.toml b/pyproject.toml
index eac77904..a29556b6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "1.23.1"
+version = "1.24.0"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/core/brain/continuity/store.ts b/src/core/brain/continuity/store.ts
index 802377b2..a1649717 100644
--- a/src/core/brain/continuity/store.ts
+++ b/src/core/brain/continuity/store.ts
@@ -39,6 +39,58 @@ export interface ContinuityPaginationOptions extends ContinuityRecordFilter {
const CONTINUITY_REL = `${BRAIN_LOG_REL}/continuity`;
const CURSOR_PREFIX = "offset:";
+/**
+ * Canonical UTC ISO-8601 shape: `YYYY-MM-DDTHH:MM:SS[.sss]Z`. A trailing
+ * `Z` is mandatory โ€” a numeric offset (`+03:00`) is rejected so every
+ * stored `createdAt` sorts and filters lexically against its siblings
+ * (readers compare the strings directly) and shards into the correct UTC
+ * month. Fractional seconds are optional (0โ€“3 digits) to accept both the
+ * second-precision and `toISOString()` millisecond forms already on disk.
+ */
+const CANONICAL_UTC_TIMESTAMP_RE =
+ /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.\d{1,3})?Z$/;
+
+/**
+ * True when `value` is a canonical UTC timestamp with a real calendar
+ * date/time. The regex enforces the `โ€ฆZ` shape; the round-trip check
+ * rejects out-of-range fields that `Date` silently rolls over (e.g.
+ * `2026-02-30` โ†’ Mar 2, `2026-13-โ€ฆ` โ†’ NaN), so `2026-13-01T00:00:00Z`
+ * and `2026-02-30T00:00:00Z` are both caught.
+ */
+export function isCanonicalUtcTimestamp(value: unknown): value is string {
+ if (typeof value !== "string") return false;
+ const match = CANONICAL_UTC_TIMESTAMP_RE.exec(value);
+ if (match === null) return false;
+ const ms = Date.parse(value);
+ if (!Number.isFinite(ms)) return false;
+ const date = new Date(ms);
+ return (
+ date.getUTCFullYear() === Number(match[1]) &&
+ date.getUTCMonth() + 1 === Number(match[2]) &&
+ date.getUTCDate() === Number(match[3]) &&
+ date.getUTCHours() === Number(match[4]) &&
+ date.getUTCMinutes() === Number(match[5]) &&
+ date.getUTCSeconds() === Number(match[6])
+ );
+}
+
+/**
+ * Guard every record's `createdAt` at the store boundary. Throws on any
+ * non-canonical value so a malformed timestamp can never reach disk as a
+ * junk shard (`2026-13.jsonl`) or a local-time record that mis-sorts
+ * against the fleet of `Z` timestamps. All append paths funnel through
+ * `buildRecord`, so this is the single choke point.
+ */
+export function assertCanonicalCreatedAt(createdAt: unknown): string {
+ if (!isCanonicalUtcTimestamp(createdAt)) {
+ throw new Error(
+ `invalid continuity createdAt: expected a canonical UTC ISO-8601 timestamp ` +
+ `(YYYY-MM-DDTHH:MM:SS[.sss]Z with a real calendar date), got ${JSON.stringify(createdAt)}`,
+ );
+ }
+ return createdAt;
+}
+
export function continuityLogPath(vault: string, month: string): string {
if (!/^\d{4}-\d{2}$/.test(month)) throw new Error(`invalid continuity month: ${month}`);
return ensureInsideVault(join(vault, CONTINUITY_REL, `${month}.jsonl`), vault);
@@ -130,7 +182,7 @@ export function listContinuityRecords(
vault: string,
filter: ContinuityRecordFilter = {},
): ReadonlyArray<ContinuityRecord> {
- return Object.freeze(readAllRecords(vault).filter((record) => matches(record, filter)));
+ return Object.freeze(readAllRecords(vault, filter).filter((record) => matches(record, filter)));
}
export function paginateContinuityRecords(
@@ -155,6 +207,7 @@ function buildRecord(
readonly payload: Readonly<Record<string, unknown>>;
},
): ContinuityRecord {
+ assertCanonicalCreatedAt(input.createdAt);
const payloadResult = safeContinuityPayload(input.payload ?? {});
const sourceRefs = Object.freeze([...(input.sourceRefs ?? [])]);
// `schema` stays OUT of recordId(): identical records must keep
@@ -187,12 +240,36 @@ function appendRecord(vault: string, record: ContinuityRecord): ContinuityRecord
return record;
}
-function readAllRecords(vault: string): ContinuityRecord[] {
+/**
+ * Read every stored record, optionally skipping whole month shards that
+ * a `since`/`until` filter proves cannot contribute a matching record.
+ *
+ * Shards are named `YYYY-MM.jsonl`, so the shard month is a 7-char prefix
+ * of every record's `createdAt` inside it. A shard whose month sorts
+ * strictly before `since`'s month can only hold records with
+ * `createdAt < since` (rejected by {@link matches}); one whose month sorts
+ * strictly after `until`'s month can only hold `createdAt > until`. Both
+ * are skipped without opening the file. The boundary months (equal to
+ * `since`/`until`'s month) are still read in full, so the returned set is
+ * byte-identical to reading everything and letting `matches` filter -
+ * this only avoids reading shards that would contribute nothing. Grows
+ * the read cost with the queried window, not the whole log history.
+ */
+function readAllRecords(vault: string, filter: ContinuityRecordFilter = {}): ContinuityRecord[] {
const dir = ensureInsideVault(join(vault, CONTINUITY_REL), vault);
if (!existsSync(dir)) return [];
+ const sinceMonth = filter.since ? filter.since.slice(0, 7) : undefined;
+ const untilMonth = filter.until ? filter.until.slice(0, 7) : undefined;
const records: ContinuityRecord[] = [];
for (const name of readdirSync(dir).toSorted()) {
if (!name.endsWith(".jsonl")) continue;
+ const month = name.slice(0, -".jsonl".length);
+ // Only a canonical YYYY-MM shard has a month prefix we can range-skip
+ // on; any other name is read in full to stay byte-identical.
+ if (/^\d{4}-\d{2}$/.test(month)) {
+ if (sinceMonth !== undefined && month < sinceMonth) continue;
+ if (untilMonth !== undefined && month > untilMonth) continue;
+ }
const path = ensureInsideVault(join(dir, name), vault);
let st;
try {
diff --git a/src/core/brain/entities/index-builder.ts b/src/core/brain/entities/index-builder.ts
index 2750d0ab..c6f90645 100644
Binary files a/src/core/brain/entities/index-builder.ts and b/src/core/brain/entities/index-builder.ts differ
diff --git a/src/core/brain/heal-enrich.ts b/src/core/brain/heal-enrich.ts
index 9261d11e..24712c35 100644
--- a/src/core/brain/heal-enrich.ts
+++ b/src/core/brain/heal-enrich.ts
@@ -34,21 +34,50 @@ export function deriveTitleFromContent(markdown: string): string | null {
}
/**
- * Wrap exact whole-token occurrences of any `known` title/alias in a
- * wikilink. Text inside existing wikilinks or inline code is left
- * untouched, so the function is idempotent. Returns the input unchanged
- * when `known` is empty.
+ * One known title/alias, prepared once: the raw form (for per-page
+ * self-link exclusion, which matches on the exact string the caller
+ * supplied) and its regex-escaped, trimmed form (for the alternation).
*/
-export function linkExactMentions(body: string, known: ReadonlyArray<string>): string {
- const phrases = known
- .map((k) => k.trim())
- .filter((k) => k.length > 0)
+interface PreparedPhrase {
+ readonly raw: string;
+ readonly escaped: string;
+}
+
+/**
+ * The full known title/alias set, trimmed / dropped-if-empty / sorted
+ * longest-first (lexicographic tie-break) / regex-escaped ONCE. The heal
+ * runner shares this across every page instead of re-sorting and
+ * re-escaping K phrases per page (the dominant per-page cost), then
+ * excludes each page's own few terms via {@link linkExactMentionsPrepared}.
+ */
+export interface PreparedHealPhrases {
+ readonly entries: ReadonlyArray<PreparedPhrase>;
+}
+
+/** Build the shared, sorted, escaped phrase set. Order-preserving under
+ * later per-page filtering, so output stays identical to sorting the
+ * already-filtered list. */
+export function prepareHealPhrases(known: ReadonlyArray<string>): PreparedHealPhrases {
+ const entries = known
+ .map((raw) => ({ raw, trimmed: raw.trim() }))
+ .filter((e) => e.trimmed.length > 0)
// Longest first so a multi-word title wins over a contained shorter
// one at the same position; lexicographic tie-break for determinism.
- .toSorted((a, b) => b.length - a.length || (a < b ? -1 : a > b ? 1 : 0));
- if (phrases.length === 0) return body;
+ .toSorted(
+ (a, b) =>
+ b.trimmed.length - a.trimmed.length ||
+ (a.trimmed < b.trimmed ? -1 : a.trimmed > b.trimmed ? 1 : 0),
+ )
+ .map((e) => Object.freeze({ raw: e.raw, escaped: escapeRegex(e.trimmed) }));
+ return Object.freeze({ entries: Object.freeze(entries) });
+}
- const alternation = phrases.map(escapeRegex).join("|");
+/** Compile the alternation from the ordered escaped forms and link every
+ * whole-token match outside protected spans. Shared core of both public
+ * entry points. */
+function applyPhrases(body: string, escaped: ReadonlyArray<string>): string {
+ if (escaped.length === 0) return body;
+ const alternation = escaped.join("|");
// Whole-token boundaries via Unicode letter/number lookarounds so the
// match is language-agnostic (works for any script).
const linkRe = new RegExp(`(?<![\\p{L}\\p{N}])(?:${alternation})(?![\\p{L}\\p{N}])`, "gu");
@@ -62,6 +91,38 @@ export function linkExactMentions(body: string, known: ReadonlyArray<string>): s
return parts.join("");
}
+/**
+ * Wrap exact whole-token occurrences of any `known` title/alias in a
+ * wikilink. Text inside existing wikilinks or inline code is left
+ * untouched, so the function is idempotent. Returns the input unchanged
+ * when `known` is empty.
+ */
+export function linkExactMentions(body: string, known: ReadonlyArray<string>): string {
+ return applyPhrases(
+ body,
+ prepareHealPhrases(known).entries.map((e) => e.escaped),
+ );
+}
+
+/**
+ * Prepared-set variant of {@link linkExactMentions}: link every phrase in
+ * `prepared` whose raw form is NOT in `exclude` (the page's own titles /
+ * aliases). Filtering the pre-sorted set preserves order, so the
+ * alternation - and therefore the output - is byte-identical to calling
+ * `linkExactMentions` with the pre-excluded list.
+ */
+export function linkExactMentionsPrepared(
+ body: string,
+ prepared: PreparedHealPhrases,
+ exclude: ReadonlySet<string>,
+): string {
+ const escaped: string[] = [];
+ for (const entry of prepared.entries) {
+ if (!exclude.has(entry.raw)) escaped.push(entry.escaped);
+ }
+ return applyPhrases(body, escaped);
+}
+
/** A page view the heal planner reads. */
export interface HealPageInput {
readonly frontmatter: Record<string, unknown>;
@@ -87,6 +148,27 @@ export function planHealEnrichment(
page: HealPageInput,
knownTitlesAndAliases: ReadonlyArray<string>,
): HealPlan {
+ return finishPlan(page, linkExactMentions(page.body, knownTitlesAndAliases));
+}
+
+/**
+ * Prepared-set variant of {@link planHealEnrichment} for the heal runner:
+ * links against the shared, pre-sorted phrase set minus this page's own
+ * titles/aliases. Output is byte-identical to
+ * `planHealEnrichment(page, [...known].filter((k) => !exclude.has(k)))`,
+ * but skips re-sorting and re-escaping the whole phrase set per page.
+ */
+export function planHealEnrichmentPrepared(
+ page: HealPageInput,
+ prepared: PreparedHealPhrases,
+ exclude: ReadonlySet<string>,
+): HealPlan {
+ return finishPlan(page, linkExactMentionsPrepared(page.body, prepared, exclude));
+}
+
+/** Shared tail: derive a missing title from the first H1 and package the
+ * (possibly) relinked body into a plan. */
+function finishPlan(page: HealPageInput, linked: string): HealPlan {
const existingTitle = page.frontmatter["title"];
const hasTitle = typeof existingTitle === "string" && existingTitle.trim().length > 0;
@@ -96,7 +178,6 @@ export function planHealEnrichment(
if (derived !== null) title = derived;
}
- const linked = linkExactMentions(page.body, knownTitlesAndAliases);
const body = linked !== page.body ? linked : undefined;
return {
diff --git a/src/core/brain/heal-run.ts b/src/core/brain/heal-run.ts
index 4cf897df..4eb4ef66 100644
--- a/src/core/brain/heal-run.ts
+++ b/src/core/brain/heal-run.ts
@@ -24,7 +24,7 @@ import {
writeFrontmatterAtomic,
} from "../vault.ts";
import { BRAIN_ROOT_REL } from "./paths.ts";
-import { planHealEnrichment } from "./heal-enrich.ts";
+import { planHealEnrichmentPrepared, prepareHealPhrases } from "./heal-enrich.ts";
export interface HealRunResult {
/** Pages scanned (outside the Brain root). */
@@ -72,6 +72,11 @@ export function runHealEnrichment(vault: string): HealRunResult {
ownTokens.set(p.path, own);
}
+ // Sort + regex-escape the whole known set ONCE; per page we only exclude
+ // that page's own few terms (see planHealEnrichmentPrepared) instead of
+ // re-sorting and recompiling the K-phrase set from scratch.
+ const prepared = prepareHealPhrases([...known]);
+
const changed: string[] = [];
for (const p of pages) {
let meta;
@@ -84,8 +89,7 @@ export function runHealEnrichment(vault: string): HealRunResult {
}
// Never link a page to its own title or aliases.
const own = ownTokens.get(p.path) ?? new Set<string>();
- const others = [...known].filter((k) => !own.has(k));
- const plan = planHealEnrichment({ frontmatter: meta, body }, others);
+ const plan = planHealEnrichmentPrepared({ frontmatter: meta, body }, prepared, own);
if (!plan.changed) continue;
const newMeta = plan.title !== undefined ? { ...meta, title: plan.title } : meta;
diff --git a/src/core/brain/link-graph/format-wikilink.ts b/src/core/brain/link-graph/format-wikilink.ts
index 1acb7b59..b22481b8 100644
--- a/src/core/brain/link-graph/format-wikilink.ts
+++ b/src/core/brain/link-graph/format-wikilink.ts
@@ -51,6 +51,45 @@ function stripMd(path: string): string {
return path.endsWith(".md") ? path.slice(0, -".md".length) : path;
}
+/**
+ * `/`-aligned-suffix index over the known pages: every trailing-segment
+ * suffix of every page maps to the (stripped) pages that carry it, in
+ * `knownPaths` order. A target side `t` resolves to exactly the pages
+ * where `page === t || page.endsWith("/" + t)` โ€” precisely the bucket
+ * `bySuffix.get(t)`, so per-link resolution drops from an O(pages) scan
+ * (plus a fresh pages-sized array per link in the old
+ * {@link shortestUniqueSuffix}) to a single Map lookup.
+ */
+type SuffixIndex = ReadonlyMap<string, ReadonlyArray<string>>;
+
+/**
+ * Built once per run and memoized on the `knownPaths` array identity:
+ * `links normalize` walks the vault once and passes the same array to
+ * every {@link normalizeWikilinks} call, so the index is built one time
+ * and reused across all files instead of O(files ร— links) rebuilds.
+ */
+const SUFFIX_INDEX_MEMO = new WeakMap<ReadonlyArray<string>, SuffixIndex>();
+
+function suffixIndex(knownPaths: ReadonlyArray<string>): SuffixIndex {
+ const cached = SUFFIX_INDEX_MEMO.get(knownPaths);
+ if (cached) return cached;
+ const bySuffix = new Map<string, string[]>();
+ for (const raw of knownPaths) {
+ const page = stripMd(raw);
+ const segments = page.split("/");
+ // Each `take` is a distinct-length key, so a page is inserted at
+ // most once per suffix (matching the old scan's single push).
+ for (let take = 1; take <= segments.length; take += 1) {
+ const key = segments.slice(segments.length - take).join("/");
+ const list = bySuffix.get(key);
+ if (list) list.push(page);
+ else bySuffix.set(key, [page]);
+ }
+ }
+ SUFFIX_INDEX_MEMO.set(knownPaths, bySuffix);
+ return bySuffix;
+}
+
/**
* The shortest trailing-segment suffix of `path` that no other known
* page shares. `Brain/notes/alpha` with no sibling `alpha` yields
@@ -58,11 +97,15 @@ function stripMd(path: string): string {
* `archive/beta`.
*/
export function shortestUniqueSuffix(path: string, knownPaths: ReadonlyArray<string>): string {
+ const index = suffixIndex(knownPaths);
const segments = path.split("/");
- const others = knownPaths.map(stripMd).filter((p) => p !== path);
for (let take = 1; take <= segments.length; take += 1) {
const suffix = segments.slice(segments.length - take).join("/");
- const clash = others.some((p) => p === suffix || p.endsWith("/" + suffix));
+ // `path`'s own bucket entry is always present; a clash is any OTHER
+ // page sharing the suffix โ€” identical to the old
+ // `others.filter(p => p !== path).some(...)`.
+ const bucket = index.get(suffix);
+ const clash = bucket !== undefined && bucket.some((p) => p !== path);
if (!clash) return suffix;
}
return path;
@@ -79,11 +122,8 @@ function resolveTarget(
): string | "ambiguous" | null {
const target = stripMd(targetSide.trim());
if (target === "") return null;
- const matches: string[] = [];
- for (const raw of knownPaths) {
- const page = stripMd(raw);
- if (page === target || page.endsWith("/" + target)) matches.push(page);
- }
+ const matches = suffixIndex(knownPaths).get(target);
+ if (matches === undefined) return null;
if (matches.length === 1) return matches[0]!;
if (matches.length > 1) return "ambiguous";
return null;
diff --git a/src/core/brain/query-demand.ts b/src/core/brain/query-demand.ts
index 4e749293..b2f9860d 100644
--- a/src/core/brain/query-demand.ts
+++ b/src/core/brain/query-demand.ts
@@ -224,13 +224,21 @@ export function readQueryDemand(
): ReadonlyArray<QueryDemandRecord> {
const path = queryDemandLogPath(vault);
if (!existsSync(path)) return Object.freeze([]);
+ // Normalize the filter bounds to the same millisecond-precision form the
+ // stored `ts` already uses (records are normalized at write time). The
+ // comparison below is lexical, so a second-precision `--since
+ // 2026-07-01T00:00:00Z` would otherwise drop every ms-precision record in
+ // `[00:00:00.000Z, 00:00:00.999Z]` โ€” `.` sorts before `Z`. Running the
+ // bounds through the same normalizer keeps both sides at one precision.
+ const since = filter.since !== undefined ? normalizeDemandTimestamp(filter.since) : undefined;
+ const until = filter.until !== undefined ? normalizeDemandTimestamp(filter.until) : undefined;
const out: QueryDemandRecord[] = [];
for (const line of readFileSync(path, "utf8").split("\n")) {
if (!line.trim()) continue;
const record = coerceRecord(line);
if (record === null) continue;
- if (filter.since !== undefined && record.ts < filter.since) continue;
- if (filter.until !== undefined && record.ts > filter.until) continue;
+ if (since !== undefined && record.ts < since) continue;
+ if (until !== undefined && record.ts > until) continue;
out.push(record);
}
out.sort((a, b) => (a.ts < b.ts ? -1 : a.ts > b.ts ? 1 : 0));
diff --git a/src/core/brain/secrets/exec.ts b/src/core/brain/secrets/exec.ts
index 3d5714fd..49765815 100644
--- a/src/core/brain/secrets/exec.ts
+++ b/src/core/brain/secrets/exec.ts
@@ -86,6 +86,17 @@ export async function runWithSecret(
): Promise<RunWithSecretResult> {
if (argv.length === 0) throw new Error("secret run: a command is required after --");
const command = argv.join(" ");
+ // The audit trail is long-lived. The custody store's own secret never
+ // reaches argv, but an agent may pass some OTHER credential as a bare
+ // positional argument; scrub the logged command through the bare-token
+ // and infra passes so a foreign secret is not persisted in cleartext.
+ // The raw `command` still drives the allowlist match and the denial error
+ // (the agent already holds its own input).
+ const auditCommand = redactRawOutput(command, {
+ redactTokens: true,
+ redactInfra: true,
+ maxInput: Number.POSITIVE_INFINITY,
+ });
const resolved = resolveSecretForExec(vault, name, ctx);
if (!matchesAllowlist(resolved.allow, command)) {
appendAuditRecord(join(brainDirs(vault).log, "secret-custody"), {
@@ -94,7 +105,7 @@ export async function runWithSecret(
action: "secret_exec_denied",
target: resolved.name,
ok: false,
- details: { command, allow: resolved.allow },
+ details: { command: auditCommand, allow: resolved.allow },
});
throw new SecretExecDeniedError(resolved.name, command, resolved.allow);
}
@@ -105,7 +116,7 @@ export async function runWithSecret(
action: "secret_exec_started",
target: resolved.name,
ok: true,
- details: { command, env_var: resolved.env_var },
+ details: { command: auditCommand, env_var: resolved.env_var },
});
const childEnv: Record<string, string> = {};
diff --git a/src/core/redactor.ts b/src/core/redactor.ts
index a9f41303..65648a03 100644
--- a/src/core/redactor.ts
+++ b/src/core/redactor.ts
@@ -224,6 +224,48 @@ function isPrivateOrReservedIPv6(ip: string): boolean {
return false;
}
+// ----- Bare high-entropy token detector (opt-in via `redactTokens`) ---------
+//
+// Credential tokens passed as bare positional values carry no key=value
+// shape for the assignment passes to latch onto (e.g. an argv like
+// `["mytool", "sk-abc123"]`). Two complementary shapes are scrubbed:
+// well-known vendor-prefixed keys, and long mixed-class runs that look
+// like an API key or hash. Every quantifier is bounded, so the pass stays
+// linear and cannot be driven into catastrophic backtracking.
+
+/**
+ * Vendor-prefixed credential tokens (OpenAI/Stripe `sk-`/`sk_`/`rk_`/`pk_`,
+ * GitHub `ghp_`/`gho_`/โ€ฆ/`github_pat_`, Slack `xox?-`, AWS `AKIAโ€ฆ`, Google
+ * `AIzaโ€ฆ`, GitLab `glpat-`). The recognizable prefix is what lets a short
+ * token like `sk-abc123` be caught without a length gate that would also
+ * hit ordinary words.
+ */
+const VENDOR_TOKEN_RE = new RegExp(
+ [
+ "\\b(?:sk|rk|pk)[-_][A-Za-z0-9._-]{3,200}",
+ "\\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{6,255}",
+ "\\bgithub_pat_[A-Za-z0-9_]{6,255}",
+ "\\bxox[baprs]-[A-Za-z0-9-]{6,200}",
+ "\\bAKIA[0-9A-Z]{16}\\b",
+ "\\bAIza[0-9A-Za-z._-]{10,100}",
+ "\\bglpat-[A-Za-z0-9_-]{6,100}",
+ ].join("|"),
+ "g",
+);
+
+/**
+ * A long bare run that mixes letters and digits โ€” the shape of an
+ * unprefixed API key, session id, or hash. Length-gated (โ‰ฅ 24) so ordinary
+ * words and short ids are never touched, with bounded repetition so the
+ * lookaheads stay linear.
+ */
+const HIGH_ENTROPY_TOKEN_RE =
+ /\b(?=[A-Za-z0-9_-]{24,200}\b)(?=[A-Za-z0-9_-]{0,199}[A-Za-z])(?=[A-Za-z0-9_-]{0,199}\d)[A-Za-z0-9_-]{24,200}\b/g;
+
+function redactBareTokens(text: string): string {
+ return text.replace(VENDOR_TOKEN_RE, PLACEHOLDER).replace(HIGH_ENTROPY_TOKEN_RE, PLACEHOLDER);
+}
+
function redactInfraTopology(text: string): string {
let out = text.replace(BASIC_AUTH_URL_RE, (_m, scheme: string) => `${scheme}${PLACEHOLDER}@`);
out = out.replace(IPV4_PORT_RE, PLACEHOLDER);
@@ -308,6 +350,15 @@ export interface RedactRawOutputOptions {
* where a bare coordinate is the likeliest topology leak.
*/
readonly redactInfra?: boolean;
+ /**
+ * When `true`, also scrub bare high-entropy credential tokens that carry
+ * no key=value shape โ€” vendor-prefixed keys (`sk-โ€ฆ`, `ghp_โ€ฆ`, `AKIAโ€ฆ`)
+ * and long mixed letter+digit runs. Off by default (over-redacting prose
+ * is worse than the narrow key/value passes). Enabled on the secret-exec
+ * audit trail, whose long-lived log records a full argv that may carry a
+ * foreign credential passed as a positional argument.
+ */
+ readonly redactTokens?: boolean;
}
export function redactRawOutput(text: string, opts: RedactRawOutputOptions = {}): string {
@@ -361,6 +412,8 @@ export function redactRawOutput(text: string, opts: RedactRawOutputOptions = {})
// Infra-topology pass last: it runs on values the key/value passes
// already left untouched (bare coordinates), and any value they redacted
// is now a placeholder with no IP/host shape left to match.
+ if (opts.redactTokens) out = redactBareTokens(out);
+
if (opts.redactInfra) out = redactInfraTopology(out);
return out;
diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.ts
index 6ccbf51e..bf98a97b 100644
--- a/src/core/search/indexer.ts
+++ b/src/core/search/indexer.ts
@@ -46,7 +46,7 @@ import { parseFrontmatterText } from "../vault.ts";
import { appendMetric } from "../brain/metrics.ts";
import { throwIfAborted } from "../brain/safeguard.ts";
import { extractEntities } from "./entities.ts";
-import { Store } from "./store.ts";
+import { acquireWriterLock, Store } from "./store.ts";
import { LATEST_SCHEMA_VERSION } from "./schema.ts";
import { SearchError } from "./types.ts";
import { walkVault } from "./walker.ts";
@@ -638,49 +638,67 @@ export async function reindexVault(
mkdirSync(dirname(config.dbPath), { recursive: true });
- // Build into the temp file with an override config.
- const tempConfig: ResolvedSearchConfig = Object.freeze({
- ...config,
- dbPath: newPath,
- });
+ // Hold the writer lock on the LIVE index path for the whole rebuild +
+ // swap. Keyed on `config.dbPath` (not the `.new` staging path), so a
+ // second concurrent reindex โ€” the double-reindex a schema-bump upgrade
+ // triggers when CLI and the long-lived MCP server both self-heal at
+ // once โ€” waits-or-bails on the SAME lock instead of unlinking this run's
+ // in-progress staging DB and then having its empty seed swapped over the
+ // live index (INDEX_UNREADABLE, silent data loss). The staging-DB opens
+ // below lock a different path (`.new`), so there is no self-deadlock.
+ const release = await acquireWriterLock(config.dbPath);
+ try {
+ // Build into the temp file with an override config.
+ const tempConfig: ResolvedSearchConfig = Object.freeze({
+ ...config,
+ dbPath: newPath,
+ });
- // Resumable staging (opt-in). A compatible in-progress `.new` from an
- // interrupted run is resumed via the incremental fastpath instead of
- // rebuilt from scratch; an incompatible or unreadable one is
- // discarded. With the flag OFF, the temp file is always rebuilt fresh
- // and no staging marker is written or read - byte-identical to the
- // pre-suite path (no extra staging-DB open/close cycles).
- let resume = false;
- if (config.resumeReindex) {
- const signature = reindexStagingSignature(config, opts?.embeddings === true);
- if (existsSync(newPath)) {
- resume = await stagingSignatureMatches(tempConfig, signature);
+ // Resumable staging (opt-in). A compatible in-progress `.new` from an
+ // interrupted run is resumed via the incremental fastpath instead of
+ // rebuilt from scratch; an incompatible or unreadable one is
+ // discarded. With the flag OFF, the temp file is always rebuilt fresh
+ // and no staging marker is written or read - byte-identical to the
+ // pre-suite path (no extra staging-DB open/close cycles).
+ let resume = false;
+ if (config.resumeReindex) {
+ const signature = reindexStagingSignature(config, opts?.embeddings === true);
+ if (existsSync(newPath)) {
+ resume = await stagingSignatureMatches(tempConfig, signature);
+ }
+ if (!resume) tryUnlink(newPath);
+ // Stamp the signature so an interruption leaves a build a later run
+ // can recognise and resume.
+ await withStagingStore(tempConfig, (store) =>
+ store.setState(REINDEX_SIGNATURE_KEY, signature),
+ );
+ } else {
+ tryUnlink(newPath);
}
- if (!resume) tryUnlink(newPath);
- // Stamp the signature so an interruption leaves a build a later run
- // can recognise and resume.
- await withStagingStore(tempConfig, (store) => store.setState(REINDEX_SIGNATURE_KEY, signature));
- } else {
- tryUnlink(newPath);
- }
- // `force: false` on resume lets the fastpath skip the files the
- // partial build already committed; a fresh build forces every file.
- const stats = await indexVault(tempConfig, { ...opts, force: !resume });
+ // `force: false` on resume lets the fastpath skip the files the
+ // partial build already committed; a fresh build forces every file.
+ const stats = await indexVault(tempConfig, { ...opts, force: !resume });
- // Clear the marker so the swapped-in live index carries no staging
- // state. Only present when resume was enabled.
- if (config.resumeReindex) {
- await withStagingStore(tempConfig, (store) => store.deleteState(REINDEX_SIGNATURE_KEY));
- }
+ // Clear the marker so the swapped-in live index carries no staging
+ // state. Only present when resume was enabled.
+ if (config.resumeReindex) {
+ await withStagingStore(tempConfig, (store) => store.deleteState(REINDEX_SIGNATURE_KEY));
+ }
- // Same-directory rename swap. The two renames are each atomic on
- // POSIX; the gap between them is the only crash window, which the
- // .bak restore on Store.open handles.
- tryUnlink(bakPath);
- tryRename(config.dbPath, bakPath); // no-op (ENOENT) on fresh reindex
- renameSync(newPath, config.dbPath); // must succeed โ€” `newPath` was just built
- return stats;
+ // Same-directory rename swap. The two renames are each atomic on
+ // POSIX; the gap between them is the only crash window. A read that
+ // opens in that window is held off by this same lock (see
+ // restoreFromBakIfMissing in store.ts), so it cannot restore the stale
+ // `.bak` over the freshly built index; a genuine crash leaves the lock
+ // stale and the .bak restore on the next Store.open recovers.
+ tryUnlink(bakPath);
+ tryRename(config.dbPath, bakPath); // no-op (ENOENT) on fresh reindex
+ renameSync(newPath, config.dbPath); // must succeed โ€” `newPath` was just built
+ return stats;
+ } finally {
+ await release();
+ }
}
/** index_state key carrying the staging-build compatibility signature. */
diff --git a/src/core/search/schema.ts b/src/core/search/schema.ts
index ca3a95df..b55c268c 100644
--- a/src/core/search/schema.ts
+++ b/src/core/search/schema.ts
@@ -16,7 +16,21 @@ import { SearchError } from "./types.ts";
* this raises `SCHEMA_MISMATCH` on open โ€” the operator must reindex
* with a newer binary.
*/
-export const LATEST_SCHEMA_VERSION = 7;
+export const LATEST_SCHEMA_VERSION = 8;
+
+/**
+ * The wikilink-resolution basename of a stored document path: the final
+ * `/`-segment with a trailing `.md` stripped. `notes/alpha.md` โ†’ `alpha`,
+ * top-level `alpha.md` โ†’ `alpha`. Persisted in `documents.basename` so
+ * dangling-link resolution can equality-join an index instead of
+ * `SUBSTR`-scanning every path. Must stay in lockstep with the resolution
+ * ladder in `store.ts` (`<target>.md` exact match, then basename suffix).
+ */
+export function documentBasename(path: string): string {
+ const slash = path.lastIndexOf("/");
+ const name = slash >= 0 ? path.slice(slash + 1) : path;
+ return name.endsWith(".md") ? name.slice(0, -".md".length) : name;
+}
const DDL_V1 = `
CREATE TABLE IF NOT EXISTS documents (
@@ -325,6 +339,34 @@ export const MIGRATIONS: ReadonlyArray<Migration> = Object.freeze([
`);
},
},
+ {
+ // v8 (hot-path-performance) - a persisted `documents.basename` column
+ // (final path segment, `.md` stripped) with an index, so
+ // dangling-wikilink resolution (`resolvedDocLinkPairs`,
+ // `resolveAliasTargets`) can equality-join `idx_documents_basename`
+ // instead of `SUBSTR`-scanning every `documents.path` twice per
+ // unresolved link. The column is backfilled from the existing paths
+ // in-place, so a migrated (not-yet-reindexed) index resolves links
+ // identically at once; a full reindex repopulates it via
+ // `upsertDocument`.
+ version: 8,
+ up(db) {
+ const docCols = db.query<{ name: string }, []>("PRAGMA table_info(documents)").all();
+ if (!docCols.some((c) => c.name === "basename")) {
+ db.exec("ALTER TABLE documents ADD COLUMN basename TEXT");
+ }
+ db.exec("CREATE INDEX IF NOT EXISTS idx_documents_basename ON documents(basename)");
+ const rows = db
+ .query<{ id: number; path: string }, []>(
+ "SELECT id, path FROM documents WHERE basename IS NULL",
+ )
+ .all();
+ const update = db.prepare<undefined, [string, number]>(
+ "UPDATE documents SET basename = ? WHERE id = ?",
+ );
+ for (const row of rows) update.run(documentBasename(row.path), row.id);
+ },
+ },
]);
/** Read the current schema version from `index_state`. Returns 0 if not yet recorded. */
diff --git a/src/core/search/search.ts b/src/core/search/search.ts
index 94b9e8b1..6a7e4d90 100644
--- a/src/core/search/search.ts
+++ b/src/core/search/search.ts
@@ -201,18 +201,22 @@ function addStructuredReasons(
}
/**
- * Open the index for reading, self-healing a stale or absent index. After a
- * plugin upgrade the on-disk index can be a different schema version
- * (`SCHEMA_MISMATCH`) or not yet built (`INDEX_MISSING`); rather than forcing
- * the user to run `o2b search reindex` / `o2b search index`, rebuild once and
- * retry. `reindexVault` is imported lazily so the hot path never pulls in the
- * indexer and there is no module cycle.
+ * Open the index for reading, self-healing a stale, absent, or unreadable
+ * index. After a plugin upgrade the on-disk index can be a different schema
+ * version (`SCHEMA_MISMATCH`), not yet built (`INDEX_MISSING`), or corrupt
+ * / truncated / non-OSB at the index path (`INDEX_UNREADABLE`); rather than
+ * forcing the user to run `o2b search reindex` / `o2b search index`, rebuild
+ * once and retry. `reindexVault` is imported lazily so the hot path never
+ * pulls in the indexer and there is no module cycle.
*/
async function openReadOrSelfHeal(config: ResolvedSearchConfig): Promise<Store> {
try {
return await Store.open(config, { mode: "read" });
} catch (e) {
- if (e instanceof SearchError && (e.code === "INDEX_MISSING" || e.code === "SCHEMA_MISMATCH")) {
+ if (
+ e instanceof SearchError &&
+ (e.code === "INDEX_MISSING" || e.code === "SCHEMA_MISMATCH" || e.code === "INDEX_UNREADABLE")
+ ) {
try {
const { reindexVault } = await import("./indexer.ts");
await reindexVault(config);
@@ -1150,11 +1154,16 @@ function toSearchCard(result: BrainSearchResult): SearchCard {
});
}
-function cardSnippet(content: string): string {
+export function cardSnippet(content: string): string {
const collapsed = content.replace(/\s+/g, " ").trim();
- return collapsed.length <= CARD_SNIPPET_CHARS
+ // Truncate on code points, not UTF-16 units: a raw `.slice` can cut an
+ // astral character (emoji, rare CJK) mid-surrogate-pair, shipping a lone
+ // surrogate that renders as U+FFFD. Spreading into an array iterates by
+ // code point, so the cap never splits a character.
+ const points = [...collapsed];
+ return points.length <= CARD_SNIPPET_CHARS
? collapsed
- : `${collapsed.slice(0, CARD_SNIPPET_CHARS)}...`;
+ : `${points.slice(0, CARD_SNIPPET_CHARS).join("")}...`;
}
/**
diff --git a/src/core/search/store.ts b/src/core/search/store.ts
index acddc569..649d1af8 100644
--- a/src/core/search/store.ts
+++ b/src/core/search/store.ts
@@ -25,6 +25,7 @@ import { SearchError } from "./types.ts";
import type { ResolvedSearchConfig } from "./types.ts";
import {
applyMigrations,
+ documentBasename,
dropVecTable,
ensureVecTable,
LATEST_SCHEMA_VERSION,
@@ -248,6 +249,75 @@ function acquireWriterLockSync(path: string): () => void {
throw new SearchError("INDEX_LOCKED", `another writer holds the search index lock: ${msg}`);
}
+/**
+ * Acquire the exclusive writer lock on the LIVE index path. Shared by
+ * `Store.open({ mode: "write" })` and `reindexVault`'s rebuild+swap so both
+ * serialise on the SAME lock (keyed on `dbPath`, not on the `.new` staging
+ * path). Fast-fails to `INDEX_LOCKED` after a few retries rather than
+ * blocking indefinitely, matching the module's fail-fast contention
+ * contract. `realpath: false` lets the lock be taken before `dbPath` exists
+ * (a fresh reindex has no live index yet); only the parent directory must
+ * already exist.
+ */
+export async function acquireWriterLock(dbPath: string): Promise<() => Promise<void>> {
+ try {
+ return await lockfile.lock(dbPath, {
+ retries: { retries: 3, factor: 1, minTimeout: 1000, maxTimeout: 1000 },
+ stale: WRITER_LOCK_STALE_MS,
+ // Explicit heartbeat: refresh the lock mtime mid-run so a long index
+ // is never mistaken for a stale lock and taken over.
+ update: WRITER_LOCK_HEARTBEAT_MS,
+ realpath: false,
+ });
+ } catch (e) {
+ const msg = e instanceof Error ? e.message : String(e);
+ throw new SearchError("INDEX_LOCKED", `another writer holds the search index lock: ${msg}`);
+ }
+}
+
+/**
+ * Crash-recovery preamble shared by every {@link Store.open}: if the live
+ * index file is absent but a `.bak` from a `reindex` rename swap is present,
+ * restore it. Guarded by the SAME writer lock `reindexVault` holds across
+ * its swap, so a read that opens during the swap's brief db-absent window
+ * cannot mistake it for a crashed reindex and clobber the freshly built
+ * index with the stale `.bak` (the silent-data-loss race, A.2).
+ *
+ * Lock semantics distinguish the two cases without a heuristic:
+ * - a live reindex mid-swap holds a heartbeated (non-stale) lock, so we
+ * block briefly, then find `dbPath` present and skip the restore;
+ * - a genuine crash leaves a stale lock (no heartbeat) that is taken over
+ * within the stale window, after which we restore.
+ * If the lock cannot be taken at all we skip the restore rather than risk a
+ * clobber; the caller's open path then re-evaluates existence and reports
+ * honestly (INDEX_MISSING). The `.bak` restore of a genuine crash is
+ * unaffected: with no live holder the lock is acquired immediately.
+ */
+function restoreFromBakIfMissing(dbPath: string): void {
+ const bak = dbPath + ".bak";
+ if (existsSync(dbPath) || !existsSync(bak)) return;
+ let release: (() => void) | null = null;
+ try {
+ release = acquireWriterLockSync(dbPath);
+ } catch {
+ // A live writer holds the lock (mid-swap). Do not restore โ€” the swap
+ // will place the fresh index; the open path re-checks existence.
+ return;
+ }
+ try {
+ // Re-check under the lock: the swap may have completed while we waited,
+ // or another opener may have already restored.
+ if (existsSync(dbPath) || !existsSync(bak)) return;
+ renameSync(bak, dbPath);
+ // eslint-disable-next-line no-console
+ console.error(`restored search index from ${bak} (previous reindex crash)`);
+ } catch {
+ /* fall through โ€” open path below will report INDEX_MISSING */
+ } finally {
+ release();
+ }
+}
+
/**
* Canonical alias/lookup-key normalisation for `doc_aliases` (v7):
* trim, NFC-normalise, lower-case - the exact rule
@@ -290,20 +360,11 @@ export class Store {
const loadVec = opts.loadVec !== false;
// Crash recovery: if the main index is missing but `.bak` is present
- // from a failed `reindex` rename window, restore it. Stderr notice
- // (not a thrown error) so existing tooling keeps working.
- if (!existsSync(config.dbPath)) {
- const bak = config.dbPath + ".bak";
- if (existsSync(bak)) {
- try {
- renameSync(bak, config.dbPath);
- // eslint-disable-next-line no-console
- console.error(`restored search index from ${bak} (previous reindex crash)`);
- } catch {
- /* fall through โ€” open path below will report INDEX_MISSING */
- }
- }
- }
+ // from a failed `reindex` rename window, restore it. Lock-guarded so a
+ // read opening during a live reindex's swap window cannot clobber the
+ // freshly built index with the stale `.bak`. Stderr notice (not a thrown
+ // error) so existing tooling keeps working.
+ restoreFromBakIfMissing(config.dbPath);
if (opts.mode === "read") {
if (!existsSync(config.dbPath)) {
@@ -358,20 +419,7 @@ export class Store {
seed.close();
}
- let release: () => Promise<void>;
- try {
- release = await lockfile.lock(config.dbPath, {
- retries: { retries: 3, factor: 1, minTimeout: 1000, maxTimeout: 1000 },
- stale: WRITER_LOCK_STALE_MS,
- // Explicit heartbeat: refresh the lock mtime mid-run so a long
- // index is never mistaken for a stale lock and taken over.
- update: WRITER_LOCK_HEARTBEAT_MS,
- realpath: false,
- });
- } catch (e) {
- const msg = e instanceof Error ? e.message : String(e);
- throw new SearchError("INDEX_LOCKED", `another writer holds the search index lock: ${msg}`);
- }
+ const release = await acquireWriterLock(config.dbPath);
let db: Database;
try {
@@ -566,11 +614,23 @@ export class Store {
const row = this.db
.query<
{ id: number },
- [string, string | null, string, number, number, string | null, string, string, string]
+ [
+ string,
+ string,
+ string | null,
+ string,
+ number,
+ number,
+ string | null,
+ string,
+ string,
+ string,
+ ]
>(
- "INSERT INTO documents(path, title, content_hash, mtime, size, page_type, created_at, updated_at, indexed_at) " +
- "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) " +
+ "INSERT INTO documents(path, basename, title, content_hash, mtime, size, page_type, created_at, updated_at, indexed_at) " +
+ "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) " +
"ON CONFLICT(path) DO UPDATE SET " +
+ " basename = excluded.basename, " +
" title = excluded.title, " +
" content_hash = excluded.content_hash, " +
" mtime = excluded.mtime, " +
@@ -582,6 +642,7 @@ export class Store {
)
.get(
doc.path,
+ documentBasename(doc.path),
doc.title,
doc.contentHash,
doc.mtime,
@@ -1021,9 +1082,12 @@ export class Store {
"JOIN documents d ON d.id = a.document_id " +
"WHERE a.alias = ? ORDER BY d.path ASC LIMIT 1",
);
- const basenameExists = this.db.prepare<{ id: number }, [string, string, string]>(
- "SELECT id FROM documents WHERE path = ? " +
- "OR SUBSTR(path, -(LENGTH(?) + 4)) = '/' || ? || '.md' LIMIT 1",
+ // A real document basename owns the target (top-level `target.md` or
+ // any nested `.../target.md`): both collapse to `basename = target`,
+ // an index lookup instead of the old `path = target.md OR SUBSTR(...)`
+ // full scan.
+ const basenameExists = this.db.prepare<{ id: number }, [string]>(
+ "SELECT id FROM documents WHERE basename = ? LIMIT 1",
);
const update = this.db.prepare<undefined, [number, string]>(
"UPDATE links SET target_document_id = ? " +
@@ -1037,7 +1101,7 @@ export class Store {
const key = normalizeAlias(row.target_path);
if (key.length === 0) continue;
// Skip targets a real document basename owns.
- if (basenameExists.get(`${row.target_path}.md`, row.target_path, row.target_path)) {
+ if (basenameExists.get(row.target_path)) {
continue;
}
const owner = aliasOwner.get(key);
@@ -1064,14 +1128,21 @@ export class Store {
resolvedDocLinkPairs(): Array<{ readonly source: number; readonly target: number }> {
const rows = this.db
.query<{ source: number; target: number | null }, []>(
+ // Resolution ladder: materialized id, then `<target>.md` exact
+ // (path is UNIQUE + indexed), then an UNAMBIGUOUS nested-basename
+ // match. The basename branch equality-joins idx_documents_basename
+ // and mirrors the old `SUBSTR(path) = '/'||target||'.md'` scan
+ // exactly: a `/`-aligned basename suffix is precisely
+ // `basename = target AND path is nested` (top-level `target.md`
+ // has no leading slash and is owned by the exact branch above).
"SELECT DISTINCT l.source_document_id AS source, " +
" COALESCE(" +
" l.target_document_id, " +
" (SELECT d.id FROM documents d WHERE d.path = l.target_path || '.md'), " +
" (SELECT d.id FROM documents d " +
- " WHERE SUBSTR(d.path, -(LENGTH(l.target_path) + 4)) = '/' || l.target_path || '.md' " +
+ " WHERE d.basename = l.target_path AND instr(d.path, '/') > 0 " +
" AND 1 = (SELECT COUNT(*) FROM documents d2 " +
- " WHERE SUBSTR(d2.path, -(LENGTH(l.target_path) + 4)) = '/' || l.target_path || '.md'))" +
+ " WHERE d2.basename = l.target_path AND instr(d2.path, '/') > 0))" +
" ) AS target " +
"FROM links l WHERE l.target_path IS NOT NULL",
)
diff --git a/src/core/surface/lexical-score.ts b/src/core/surface/lexical-score.ts
index 9d56cd05..c6c7ab70 100644
--- a/src/core/surface/lexical-score.ts
+++ b/src/core/surface/lexical-score.ts
@@ -40,17 +40,29 @@ const DEFAULTS = Object.freeze({ nameWeight: 3, tagWeight: 2, k1: 1.2, b: 0.75 }
* is provably unchanged, and the only consumer of scoreDescriptors today
* is the skill-attach path. The range covers Han Unified Ideographs only;
* kana / Hangul / CJK extensions are not bigram-split.
+ *
+ * Bigrams are extracted only from each maximal contiguous Han span within a
+ * token, so a mixed token like "gbrain..." yields the inner Han bigram but
+ * never a cross-script window (e.g. an ASCII+Han pair) that would match
+ * nothing yet inflate term frequency / document length. Scope is thus the
+ * Han run, not the total token length: an embedded 2-char Han run still
+ * emits its bigram.
*/
+const HAN_SPAN = /[ไธ€-้ฟฟ]+/gu;
+
export function tokenize(text: string): string[] {
const out: string[] = [];
for (const raw of text.toLowerCase().split(/[^\p{L}\p{N}]+/u)) {
if (raw.length < 2) continue;
- // Han bigram pass: a token with a Han character and length > 2 also
- // emits overlapping 2-char windows so "ๅฎž็Žฐๆ–นๅผ" and "็š„ๅฎž็Žฐๆ–นๅผ"
- // share the bigram "ๅฎž็Žฐ".
- if (/[\u4e00-\u9fff]/.test(raw) && raw.length > 2) {
- for (let i = 0; i < raw.length - 1; i++) {
- out.push(raw.slice(i, i + 2));
+ // Han bigram pass: emit overlapping 2-char windows from each maximal Han
+ // span so a spaceless run and a query prefixed run share their inner
+ // bigrams, without crossing into adjacent ASCII.
+ for (const span of raw.match(HAN_SPAN) ?? []) {
+ for (let i = 0; i < span.length - 1; i++) {
+ const bigram = span.slice(i, i + 2);
+ // Skip a bigram equal to the whole token (a standalone 2-char Han
+ // token) so it is not emitted twice.
+ if (bigram !== raw) out.push(bigram);
}
}
out.push(raw);
diff --git a/src/mcp/brain/generation-tools.ts b/src/mcp/brain/generation-tools.ts
index 84895f8a..6a0d1e89 100644
--- a/src/mcp/brain/generation-tools.ts
+++ b/src/mcp/brain/generation-tools.ts
@@ -26,6 +26,7 @@ import {
type GenerationReportFilter,
type GenerationUsage,
} from "../../core/brain/generation-reports.ts";
+import { isCanonicalUtcTimestamp } from "../../core/brain/continuity/store.ts";
import type { ContinuitySourceRef } from "../../core/brain/continuity/types.ts";
import { INVALID_PARAMS, MCPError } from "../protocol.ts";
import type { ServerContext, ToolDefinition } from "../tools.ts";
@@ -56,6 +57,19 @@ function recordAction(ctx: ServerContext, args: Record<string, unknown>): Record
const agent = requiredStringArg(TOOL, args, "agent");
const prompt = requiredStringArg(TOOL, args, "prompt");
+ // Validate `created_at` here rather than relying on the store guard:
+ // `emitGenerationReport` is fail-open (a throwing build returns null),
+ // so a malformed timestamp would silently report `recorded: false`
+ // instead of the INVALID_PARAMS the caller needs.
+ const createdAt = optionalStringArg(TOOL, args, "created_at");
+ if (createdAt !== undefined && !isCanonicalUtcTimestamp(createdAt)) {
+ throw new MCPError(
+ INVALID_PARAMS,
+ `${TOOL}: created_at must be a canonical UTC ISO-8601 timestamp ` +
+ `(e.g. 2026-07-06T15:00:00Z or 2026-07-06T15:00:00.000Z)`,
+ );
+ }
+
// Opt-in: per-call enable wins, else the config gate. Off => nothing written.
const enabled =
args["enable"] === true || resolveGenerationTraceEnabled(ctx.configPath ?? undefined);
diff --git a/tests/core/brain/continuity-store.test.ts b/tests/core/brain/continuity-store.test.ts
index efe38ce7..4e848548 100644
--- a/tests/core/brain/continuity-store.test.ts
+++ b/tests/core/brain/continuity-store.test.ts
@@ -9,6 +9,7 @@ import {
appendContinuityRecords,
appendContinuitySourceInvalidation,
continuityLogPath,
+ isCanonicalUtcTimestamp,
listContinuityRecords,
paginateContinuityRecords,
} from "../../../src/core/brain/continuity/store.ts";
@@ -123,6 +124,33 @@ describe("continuity store", () => {
});
expect(records.map((record) => record.kind)).toEqual(["session_turn", "source_invalidation"]);
});
+
+ test("since/until shard-skip returns the same records as an unfiltered window", () => {
+ const months = ["2026-03", "2026-05", "2026-07", "2026-09"];
+ for (const month of months) {
+ appendContinuityRecord(vault, {
+ kind: "context_receipt",
+ createdAt: `${month}-15T12:00:00Z`,
+ sourceRefs: [{ id: `src-${month}` }],
+ payload: { query: month },
+ });
+ }
+
+ // A bounded window skips the 2026-03 and 2026-09 shards but returns
+ // exactly the in-window records, in ascending createdAt order.
+ const windowed = listContinuityRecords(vault, {
+ since: "2026-05-01T00:00:00Z",
+ until: "2026-07-31T23:59:59Z",
+ });
+ expect(windowed.map((r) => r.payload["query"])).toEqual(["2026-05", "2026-07"]);
+
+ // Boundary months are read in full: since exactly on a record's month.
+ const openEnded = listContinuityRecords(vault, { since: "2026-05-15T12:00:00Z" });
+ expect(openEnded.map((r) => r.payload["query"])).toEqual(["2026-05", "2026-07", "2026-09"]);
+
+ // No filter still reads every shard.
+ expect(listContinuityRecords(vault)).toHaveLength(4);
+ });
});
describe("continuity batch append", () => {
@@ -212,3 +240,53 @@ describe("continuity batch append", () => {
expect(existsSync(continuityLogPath(vault, "2026-06"))).toBe(false);
});
});
+
+describe("createdAt validation at the store boundary", () => {
+ const ACCEPT: string[] = [
+ "2026-05-31T12:00:00Z", // second precision
+ "2026-06-15T10:00:00.000Z", // millisecond precision
+ "2026-06-15T10:00:00.5Z", // sub-second, โ‰ค3 digits
+ "2024-02-29T00:00:00Z", // real leap day
+ "2026-12-31T23:59:59.999Z",
+ ];
+ const REJECT: Array<[string, unknown]> = [
+ ["month out of range shards into junk", "2026-13-01T00:00:00Z"],
+ ["day out of range for the month", "2026-02-30T00:00:00Z"],
+ ["Feb 29 on a non-leap year", "2026-02-29T00:00:00Z"],
+ ["numeric offset instead of Z mis-sorts", "2026-07-06T15:00:00+03:00"],
+ ["no zone designator", "2026-07-06T15:00:00"],
+ ["date only", "2026-07-06"],
+ ["month prefix only", "2026-07"],
+ ["free text", "not-a-timestamp"],
+ ["empty string", ""],
+ ["lowercase z", "2026-07-06T15:00:00z"],
+ ["non-string number", 1_783_350_000_000],
+ ["null", null],
+ ];
+
+ test.each(ACCEPT)("accepts %s", (value) => {
+ expect(isCanonicalUtcTimestamp(value)).toBe(true);
+ const record = appendContinuityRecord(vault, {
+ kind: "session_turn",
+ createdAt: value,
+ sourceRefs: [{ id: "ok" }],
+ payload: { snippet: "valid" },
+ });
+ expect(record.createdAt).toBe(value);
+ });
+
+ test.each(REJECT)("rejects %s", (_label, value) => {
+ expect(isCanonicalUtcTimestamp(value)).toBe(false);
+ expect(() =>
+ appendContinuityRecord(vault, {
+ kind: "session_turn",
+ // eslint-disable-next-line @typescript-eslint/no-explicit-any
+ createdAt: value as any,
+ sourceRefs: [{ id: "bad" }],
+ payload: { snippet: "invalid" },
+ }),
+ ).toThrow(/invalid continuity createdAt/);
+ // Nothing was written: no shard exists for a rejected timestamp.
+ expect(listContinuityRecords(vault, {})).toHaveLength(0);
+ });
+});
diff --git a/tests/core/brain/entities.registry.test.ts b/tests/core/brain/entities.registry.test.ts
index 321b5397..ab9a7dc7 100644
--- a/tests/core/brain/entities.registry.test.ts
+++ b/tests/core/brain/entities.registry.test.ts
@@ -337,4 +337,44 @@ describe("buildEntityIndex", () => {
const index = buildEntityIndex(vault);
expect(index.entities).toHaveLength(1);
});
+
+ test("memoizes: an unchanged registry returns the same index object", () => {
+ seedAda();
+ const first = buildEntityIndex(vault);
+ const second = buildEntityIndex(vault);
+ expect(second).toBe(first); // same reference: no rebuild when nothing changed
+ });
+
+ test("invalidates the memo when a new entity is written", () => {
+ seedAda();
+ const before = buildEntityIndex(vault);
+ expect(before.byKey.has("projects:open second brain")).toBe(false);
+ upsertEntity(vault, {
+ category: "projects",
+ name: "Open Second Brain",
+ agent: "claude-dev-agent",
+ now: LATER,
+ });
+ const after = buildEntityIndex(vault);
+ expect(after).not.toBe(before); // rebuilt after the write
+ expect(after.byKey.get("projects:open second brain")?.id).toBe(
+ "ent-projects-open-second-brain",
+ );
+ });
+
+ test("invalidates the memo when an entity file is edited in place", () => {
+ seedAda();
+ expect(buildEntityIndex(vault).byAlias.has("newnym")).toBe(false);
+ // Idempotent upsert rewrites the SAME file (same path) with an added
+ // alias; the memo must notice via mtime/size and not serve the stale
+ // index despite the unchanged file set.
+ upsertEntity(vault, {
+ category: "people",
+ name: "Ada",
+ aliases: ["ะะดะฐ", "S.E.", "Newnym"],
+ agent: "claude-dev-agent",
+ now: LATER,
+ });
+ expect(buildEntityIndex(vault).byAlias.get("newnym")?.id).toBe("ent-people-ada");
+ });
});
diff --git a/tests/core/brain/format-wikilink.test.ts b/tests/core/brain/format-wikilink.test.ts
index 8665dbdf..a6054d23 100644
--- a/tests/core/brain/format-wikilink.test.ts
+++ b/tests/core/brain/format-wikilink.test.ts
@@ -33,6 +33,22 @@ test("shortestUniqueSuffix grows until the suffix is unambiguous", () => {
expect(shortestUniqueSuffix("Brain/archive/beta", PAGES)).toBe("archive/beta");
});
+test("shortestUniqueSuffix: memoized index stays correct across repeated calls and a 3-way basename clash", () => {
+ const pages = [
+ "a/x/beta",
+ "b/y/beta",
+ "c/z/beta", // three pages share basename `beta`; each needs 2 segments
+ "solo/gamma",
+ ];
+ // Repeated calls reuse the memoized suffix index built from `pages`.
+ expect(shortestUniqueSuffix("a/x/beta", pages)).toBe("x/beta");
+ expect(shortestUniqueSuffix("b/y/beta", pages)).toBe("y/beta");
+ expect(shortestUniqueSuffix("a/x/beta", pages)).toBe("x/beta"); // no corruption on reuse
+ expect(shortestUniqueSuffix("solo/gamma", pages)).toBe("gamma");
+ // A fresh array with identical content rebuilds the index and agrees.
+ expect(shortestUniqueSuffix("c/z/beta", [...pages])).toBe("z/beta");
+});
+
// โ”€โ”€ formatWikilinkBody โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
test("full mode rewrites a unique basename to its full key path", () => {
diff --git a/tests/core/brain/heal-enrich.test.ts b/tests/core/brain/heal-enrich.test.ts
index 2530b0b3..e5a6a27c 100644
--- a/tests/core/brain/heal-enrich.test.ts
+++ b/tests/core/brain/heal-enrich.test.ts
@@ -14,7 +14,10 @@ import { describe, expect, test } from "bun:test";
import {
deriveTitleFromContent,
linkExactMentions,
+ linkExactMentionsPrepared,
planHealEnrichment,
+ planHealEnrichmentPrepared,
+ prepareHealPhrases,
} from "../../../src/core/brain/heal-enrich.ts";
describe("deriveTitleFromContent", () => {
@@ -101,3 +104,51 @@ describe("planHealEnrichment", () => {
expect(plan.body).toBeUndefined();
});
});
+
+describe("prepared-set path is byte-identical to the excluded-list path", () => {
+ // Full known set; several phrases where one page's own multi-word title
+ // ("Foo Bar") contains another known phrase ("Bar") as a whole sub-token
+ // - the exact case where a naive compile-once-then-post-filter would
+ // diverge. The prepared path must match the excluded-list reference.
+ const known = ["Bar", "Foo Bar", "Acme Corp", "Acme", "Widget"];
+ const bodies = [
+ "Foo Bar mentions Bar and Widget here",
+ "Acme Corp and Acme both appear, plus `Widget` in code",
+ "[[Foo Bar]] already linked; Bar again",
+ "nothing relevant",
+ "Bar Foo Bar Bar",
+ ];
+ const excludeSets: ReadonlyArray<ReadonlyArray<string>> = [
+ ["Foo Bar"], // page owns the multi-word title containing "Bar"
+ ["Acme Corp"],
+ [],
+ ["Widget"],
+ ["Bar"],
+ ];
+
+ const prepared = prepareHealPhrases(known);
+
+ for (const body of bodies) {
+ for (const exclude of excludeSets) {
+ test(`link "${body.slice(0, 20)}" excluding [${exclude.join(",")}]`, () => {
+ const reference = linkExactMentions(
+ body,
+ known.filter((k) => !new Set(exclude).has(k)),
+ );
+ const viaPrepared = linkExactMentionsPrepared(body, prepared, new Set(exclude));
+ expect(viaPrepared).toBe(reference);
+ });
+ }
+ }
+
+ test("planHealEnrichmentPrepared matches planHealEnrichment on the excluded list", () => {
+ const page = { frontmatter: { title: "Foo Bar" }, body: "Foo Bar links Bar and Widget" };
+ const exclude = new Set(["Foo Bar"]);
+ const reference = planHealEnrichment(
+ page,
+ known.filter((k) => !exclude.has(k)),
+ );
+ const viaPrepared = planHealEnrichmentPrepared(page, prepared, exclude);
+ expect(viaPrepared).toEqual(reference);
+ });
+});
diff --git a/tests/core/brain/query-demand.test.ts b/tests/core/brain/query-demand.test.ts
index 0c373e23..d8e5d1f4 100644
--- a/tests/core/brain/query-demand.test.ts
+++ b/tests/core/brain/query-demand.test.ts
@@ -112,6 +112,38 @@ describe("readQueryDemand", () => {
test("empty when the log does not exist", () => {
expect(readQueryDemand(vault)).toEqual([]);
});
+
+ test("second-precision since/until bounds compare consistently against ms records", () => {
+ // Records are stored at millisecond precision. A raw lexical compare
+ // against a second-precision bound misfires because `.` (0x2E) sorts
+ // before `Z` (0x5A): `since 00:00:00Z` wrongly DROPS the whole
+ // [.000, .999] window, and `until 00:00:00Z` wrongly KEEPS it.
+ recordQueryDemand(vault, { terms: ["before"], resultCount: 1, at: "2026-06-30T23:59:59.500Z" });
+ recordQueryDemand(vault, {
+ terms: ["lowedge"],
+ resultCount: 1,
+ at: "2026-07-01T00:00:00.000Z",
+ });
+ recordQueryDemand(vault, { terms: ["midlow"], resultCount: 1, at: "2026-07-01T00:00:00.500Z" });
+ recordQueryDemand(vault, {
+ terms: ["highedge"],
+ resultCount: 1,
+ at: "2026-07-01T00:00:00.999Z",
+ });
+ recordQueryDemand(vault, { terms: ["after"], resultCount: 1, at: "2026-07-01T00:00:01.000Z" });
+
+ // `since 00:00:00Z` (normalized to .000Z) keeps the whole second and the
+ // record after it; the pre-fix lexical compare dropped lowedge/midlow/
+ // highedge because `.NNNZ` < `Z`.
+ const fromSecond = readQueryDemand(vault, { since: "2026-07-01T00:00:00Z" });
+ expect(fromSecond.map((r) => r.terms[0])).toEqual(["lowedge", "midlow", "highedge", "after"]);
+
+ // `until 00:00:00Z` (normalized to .000Z) is inclusive of exactly .000Z
+ // and excludes the rest of the second; the pre-fix lexical compare
+ // wrongly kept midlow/highedge (`.NNNZ` < `Z` โ‡’ not `> until`).
+ const untilSecond = readQueryDemand(vault, { until: "2026-07-01T00:00:00Z" });
+ expect(untilSecond.map((r) => r.terms[0])).toEqual(["before", "lowedge"]);
+ });
});
describe("aggregateQueryDemand", () => {
diff --git a/tests/core/brain/secrets/exec.test.ts b/tests/core/brain/secrets/exec.test.ts
index 740dba74..056bf48c 100644
--- a/tests/core/brain/secrets/exec.test.ts
+++ b/tests/core/brain/secrets/exec.test.ts
@@ -121,6 +121,42 @@ describe("runWithSecret", () => {
}
});
+ test("a foreign bare secret in argv is redacted in the persisted audit trail", async () => {
+ setSecret(vault, {
+ name: "api-key",
+ value: "own-custody-secret",
+ envVar: "MY_API_KEY",
+ allow: ["bun *"],
+ agent: "tester",
+ now: NOW,
+ });
+ // The agent passes some OTHER credential as a bare positional argument โ€”
+ // it never enters the custody store, so it is not a known literal, but it
+ // must not land in the long-lived audit log in cleartext.
+ const foreignToken = "sk-foreign-abc123def456";
+ const result = await runWithSecret(
+ vault,
+ "api-key",
+ ["bun", "-e", "process.exit(0)", foreignToken],
+ CTX,
+ );
+ expect(result.exitCode).toBe(0);
+
+ const auditDir = join(vault, "Brain", "log", "secret-custody");
+ const raw = readdirSync(auditDir)
+ .map((f) => readFileSync(join(auditDir, f), "utf8"))
+ .join("");
+ expect(raw).not.toContain(foreignToken);
+ const started = raw
+ .split("\n")
+ .filter((l) => l.trim().length > 0)
+ .map((l) => JSON.parse(l) as { action: string; details?: { command?: string } })
+ .find((l) => l.action === "secret_exec_started");
+ expect(started).toBeDefined();
+ expect(started!.details!.command).toContain("***REDACTED***");
+ expect(started!.details!.command).not.toContain(foreignToken);
+ });
+
test("the subprocess exit code propagates", async () => {
setSecret(vault, {
name: "api-key",
diff --git a/tests/core/search/card-snippet.test.ts b/tests/core/search/card-snippet.test.ts
new file mode 100644
index 00000000..83aca10f
--- /dev/null
+++ b/tests/core/search/card-snippet.test.ts
@@ -0,0 +1,49 @@
+/**
+ * `cardSnippet` truncates a layer-1 card snippet on CODE POINTS, not
+ * UTF-16 units. A raw `.slice(0, 240)` can cut an astral character (emoji,
+ * rare CJK) mid-surrogate-pair, shipping a lone surrogate that renders as
+ * U+FFFD to the MCP `cards` disclosure surface. These fixtures place an
+ * astral character exactly on the boundary and assert it survives whole.
+ */
+
+import { describe, expect, test } from "bun:test";
+
+import { cardSnippet } from "../../../src/core/search/search.ts";
+
+const CARD_SNIPPET_CHARS = 240;
+const REPLACEMENT_CHAR = "๏ฟฝ";
+
+describe("cardSnippet", () => {
+ test("does not split a surrogate pair straddling the char boundary", () => {
+ // 239 ASCII chars, then an emoji whose two UTF-16 units are at index
+ // 239 and 240 โ€” a raw UTF-16 slice(0, 240) would keep only the high
+ // surrogate. The emoji is followed by more text so truncation fires.
+ const content = "a".repeat(CARD_SNIPPET_CHARS - 1) + "๐Ÿ˜€" + "b".repeat(50);
+ const snippet = cardSnippet(content);
+
+ expect(snippet.endsWith("...")).toBe(true);
+ expect(snippet).not.toContain(REPLACEMENT_CHAR);
+ // The whole emoji is retained (both code units present, none dangling).
+ expect(snippet).toContain("๐Ÿ˜€");
+ // Round-tripping through the code-point iterator is lossless โ€” no lone
+ // surrogate survived (a lone surrogate re-encodes to U+FFFD).
+ expect([...snippet].every((cp) => cp !== REPLACEMENT_CHAR)).toBe(true);
+ // Kept 240 code points of body + the "..." marker.
+ expect([...snippet].length).toBe(CARD_SNIPPET_CHARS + 3);
+ });
+
+ test("returns short content unchanged (no marker, whitespace collapsed)", () => {
+ expect(cardSnippet(" hello world ")).toBe("hello world");
+ expect(cardSnippet("plain ๐Ÿ˜€ emoji")).toBe("plain ๐Ÿ˜€ emoji");
+ });
+
+ test("measures the cap in code points, not UTF-16 units", () => {
+ // 240 emoji = 480 UTF-16 units but exactly 240 code points, so it is at
+ // the cap and must pass through untruncated.
+ const content = "๐Ÿ˜€".repeat(CARD_SNIPPET_CHARS);
+ const snippet = cardSnippet(content);
+ expect(snippet.endsWith("...")).toBe(false);
+ expect([...snippet].length).toBe(CARD_SNIPPET_CHARS);
+ expect(snippet).not.toContain(REPLACEMENT_CHAR);
+ });
+});
diff --git a/tests/core/search/reindex-swap-races.test.ts b/tests/core/search/reindex-swap-races.test.ts
new file mode 100644
index 00000000..370f68c4
--- /dev/null
+++ b/tests/core/search/reindex-swap-races.test.ts
@@ -0,0 +1,140 @@
+/**
+ * Index-swap hardening (t_f27d80fe). Two interlocking races around the
+ * `reindexVault` rebuild + rename swap, both silent-data-loss class:
+ *
+ * A.1 โ€” two concurrent reindex runs. The pre-fix code unlinked and
+ * re-seeded the shared `.new` staging path with NO lock held, so a
+ * second run could destroy the first's in-progress staging DB and
+ * then have its empty seed swapped over the live index
+ * (INDEX_UNREADABLE, search stays broken until a manual reindex).
+ *
+ * A.2 โ€” a read opening in the swap window between `rename(db -> bak)` and
+ * `rename(new -> db)`. The lockless crash-restore preamble on
+ * `Store.open` saw "db missing, bak present", concluded a prior
+ * reindex crashed, and restored the STALE `.bak` over the fresh
+ * index โ€” discarding the whole rebuild.
+ *
+ * The fix serialises reindexes on the LIVE db-path writer lock and gates
+ * the crash-restore on that same lock. The genuine crash-recovery path
+ * (no live holder) is exercised by indexer.test.ts's ".bak auto-restore".
+ */
+import { afterEach, beforeEach, expect, test } from "bun:test";
+import { cpSync, existsSync, rmSync } from "node:fs";
+
+import { indexVault, reindexVault } from "../../../src/core/search/indexer.ts";
+import { acquireWriterLock, Store } from "../../../src/core/search/store.ts";
+import { SearchError } from "../../../src/core/search/types.ts";
+import type { IndexStats } from "../../../src/core/search/types.ts";
+import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
+
+let vault: string;
+let dbPath: string;
+let cleanup: () => void;
+
+beforeEach(() => {
+ const v = createTempVault("reindex-swap-races");
+ vault = v.vault;
+ dbPath = v.dbPath;
+ cleanup = v.cleanup;
+});
+
+afterEach(() => {
+ cleanup();
+});
+
+test("A.1: two overlapping reindex runs leave a complete, readable index", async () => {
+ const docs = 8;
+ for (let i = 0; i < docs; i++) writeMd(vault, `n${i}.md`, `# N${i}\n\nbody number ${i} words`);
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg); // seed a live index so the swap / .bak path engages
+
+ // Launch a second reindex DURING the first's build phase (while it holds
+ // the live-path writer lock). With the fix it serialises on that lock;
+ // without it, its synchronous pre-lock `tryUnlink(.new)` destroyed the
+ // first run's staging DB mid-build.
+ //
+ // The contender promise is held in a single-cell container rather than a
+ // bare `let` because TypeScript's control-flow analysis does not track
+ // mutations performed inside the `onFile` closure โ€” a bare `let second =
+ // null` is narrowed to `null` at the post-`await` read site, breaking the
+ // downstream `instanceof Error` narrowing (TS2358).
+ const second: Array<Promise<IndexStats | Error>> = [];
+ const firstStats = await reindexVault(cfg, {
+ onFile: () => {
+ if (second.length === 0) {
+ second.push(
+ reindexVault(cfg).catch((e: unknown) => (e instanceof Error ? e : new Error(String(e)))),
+ );
+ }
+ },
+ });
+ const secondResult: IndexStats | Error | null =
+ second.length > 0 ? ((await second[0]) ?? null) : null;
+
+ expect(firstStats.added).toBeGreaterThanOrEqual(docs);
+
+ // The second contender either rebuilt cleanly (took the lock once the
+ // first released) or fast-failed with INDEX_LOCKED โ€” never corruption.
+ if (secondResult instanceof Error) {
+ expect(secondResult).toBeInstanceOf(SearchError);
+ expect((secondResult as SearchError).code).toBe("INDEX_LOCKED");
+ }
+
+ // The surviving live index is complete and READABLE โ€” no empty seed was
+ // swapped in, so no INDEX_UNREADABLE.
+ const store = await Store.open(cfg, { mode: "read" });
+ try {
+ expect(store.listDocuments().size).toBe(docs);
+ } finally {
+ await store.close();
+ }
+ // Staging file cleaned up by the swap.
+ expect(existsSync(`${dbPath}.new`)).toBe(false);
+});
+
+test("A.2: a read during the swap window does not restore stale .bak over the fresh index", async () => {
+ writeMd(vault, "a.md", "# A\n\nalpha content");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg); // live index present
+ // Fabricate a `.bak` snapshot, as a prior reindex swap would leave.
+ cpSync(dbPath, `${dbPath}.bak`);
+
+ // Simulate a reindex mid-swap: hold the LIVE-path writer lock with the
+ // live db momentarily absent โ€” the exact window between rename(db->bak)
+ // and rename(new->db).
+ const release = await acquireWriterLock(dbPath);
+ try {
+ rmSync(dbPath);
+
+ // A concurrent read must NOT restore `.bak` while a writer holds the
+ // lock: doing so would clobber the about-to-be-swapped-in fresh index.
+ // It skips the restore and reports the transient missing state honestly.
+ await expect(Store.open(cfg, { mode: "read" })).rejects.toMatchObject({
+ code: "INDEX_MISSING",
+ });
+
+ // Crucially, `.bak` was left intact โ€” not consumed by a spurious restore.
+ expect(existsSync(`${dbPath}.bak`)).toBe(true);
+ expect(existsSync(dbPath)).toBe(false);
+ } finally {
+ await release();
+ }
+});
+
+test("A.2: crash-restore still fires when no writer holds the lock (genuine crash)", async () => {
+ writeMd(vault, "a.md", "# A\n\nalpha content");
+ const cfg = makeConfig({ vault, dbPath });
+ await indexVault(cfg);
+ cpSync(dbPath, `${dbPath}.bak`);
+
+ // No live writer: db absent + bak present is a genuine crashed-reindex
+ // signature, so the next open restores immediately.
+ rmSync(dbPath);
+ const store = await Store.open(cfg, { mode: "read" });
+ try {
+ expect(existsSync(dbPath)).toBe(true);
+ expect(existsSync(`${dbPath}.bak`)).toBe(false); // consumed by the restore
+ } finally {
+ await store.close();
+ }
+});
diff --git a/tests/core/search/schema-migration-v8.test.ts b/tests/core/search/schema-migration-v8.test.ts
new file mode 100644
index 00000000..c0d8410a
--- /dev/null
+++ b/tests/core/search/schema-migration-v8.test.ts
@@ -0,0 +1,92 @@
+/**
+ * Schema migration v7 -> v8: the `documents.basename` column (hot-path
+ * performance). v8 adds a `basename` column + `idx_documents_basename`
+ * and backfills it from the existing `path` values so a migrated (not
+ * yet reindexed) index resolves dangling wikilinks immediately.
+ */
+
+import { test, expect, beforeEach, afterEach } from "bun:test";
+import { Database } from "bun:sqlite";
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+import {
+ applyMigrations,
+ documentBasename,
+ LATEST_SCHEMA_VERSION,
+ readSchemaVersion,
+} from "../../../src/core/search/schema.ts";
+
+let tmp: string;
+let dbPath: string;
+
+beforeEach(() => {
+ tmp = mkdtempSync(join(tmpdir(), "osb-mig-v8-"));
+ dbPath = join(tmp, "test.sqlite");
+});
+afterEach(() => {
+ rmSync(tmp, { recursive: true, force: true });
+});
+
+function hasColumn(db: Database, table: string, name: string): boolean {
+ return db
+ .query<{ name: string }, []>(`PRAGMA table_info(${table})`)
+ .all()
+ .some((c) => c.name === name);
+}
+
+function hasIndex(db: Database, name: string): boolean {
+ return (
+ db
+ .query<{ name: string }, [string]>(
+ "SELECT name FROM sqlite_master WHERE type='index' AND name = ?",
+ )
+ .get(name) !== null
+ );
+}
+
+test("documentBasename strips the final path segment and .md", () => {
+ expect(documentBasename("Brain/notes/alpha.md")).toBe("alpha");
+ expect(documentBasename("alpha.md")).toBe("alpha");
+ expect(documentBasename("a/b/c.md")).toBe("c");
+ expect(documentBasename("no-extension")).toBe("no-extension");
+});
+
+test("fresh migration reaches latest with basename column and index", () => {
+ const db = new Database(dbPath);
+ expect(applyMigrations(db)).toBe(LATEST_SCHEMA_VERSION);
+ expect(hasColumn(db, "documents", "basename")).toBe(true);
+ expect(hasIndex(db, "idx_documents_basename")).toBe(true);
+ db.close();
+});
+
+test("a v7 index upgrades to v8 and backfills basename from path", () => {
+ const db = new Database(dbPath);
+ applyMigrations(db); // -> latest
+ // Seed rows, then simulate an older v7 index: drop the column + index
+ // and rewind the version so the v8 migration re-runs the backfill.
+ db.run(
+ "INSERT INTO documents(id, path, basename, title, content_hash, mtime, size, created_at, updated_at, indexed_at) " +
+ "VALUES (1, 'notes/alpha.md', NULL, 'A', 'h', 1, 1, 't', 't', 't')",
+ );
+ db.run(
+ "INSERT INTO documents(id, path, basename, title, content_hash, mtime, size, created_at, updated_at, indexed_at) " +
+ "VALUES (2, 'top.md', NULL, 'T', 'h', 1, 1, 't', 't', 't')",
+ );
+ db.run("DROP INDEX idx_documents_basename");
+ db.run("UPDATE documents SET basename = NULL");
+ db.run("UPDATE index_state SET value = '7' WHERE key = 'schema_version'");
+ expect(readSchemaVersion(db)).toBe(7);
+
+ expect(applyMigrations(db)).toBe(LATEST_SCHEMA_VERSION);
+ expect(hasIndex(db, "idx_documents_basename")).toBe(true);
+ const rows = db
+ .query<{ id: number; basename: string }, []>("SELECT id, basename FROM documents ORDER BY id")
+ .all();
+ expect(rows).toEqual([
+ { id: 1, basename: "alpha" },
+ { id: 2, basename: "top" },
+ ]);
+ db.close();
+});
diff --git a/tests/core/search/self-heal.test.ts b/tests/core/search/self-heal.test.ts
index 89a02f0b..7474acf2 100644
--- a/tests/core/search/self-heal.test.ts
+++ b/tests/core/search/self-heal.test.ts
@@ -1,8 +1,10 @@
/**
* Lazy self-heal on the search read path: after a plugin upgrade the on-disk
- * index can be a stale schema (SCHEMA_MISMATCH) or absent (INDEX_MISSING). A
- * search must transparently rebuild once and return results instead of forcing
- * the user to run `o2b search reindex` / `o2b search index`.
+ * index can be a stale schema (SCHEMA_MISMATCH), absent (INDEX_MISSING), or
+ * unreadable (INDEX_UNREADABLE โ€” a corrupt or truncated file, or a non-OSB
+ * sqlite db at the index path). A search must transparently rebuild once and
+ * return results instead of forcing the user to run `o2b search reindex` /
+ * `o2b search index`.
*/
import { test, expect, beforeEach, afterEach } from "bun:test";
import { Database } from "bun:sqlite";
@@ -10,6 +12,7 @@ import { existsSync, rmSync } from "node:fs";
import { indexVault } from "../../../src/core/search/indexer.ts";
import { search } from "../../../src/core/search/search.ts";
+import { Store } from "../../../src/core/search/store.ts";
import { createTempVault, makeConfig, writeMd } from "../../helpers/search-fixtures.ts";
let vault: string;
@@ -51,6 +54,33 @@ test("INDEX_MISSING on read self-heals: search builds the index on first use", a
expect(existsSync(dbPath)).toBe(true); // built by the self-heal
});
+test("INDEX_UNREADABLE on read self-heals: search rebuilds a corrupt index", async () => {
+ const config = makeConfig({ vault, dbPath });
+ await indexVault(config); // seed a real index so the path exists
+
+ // Corrupt the index's metadata so a read open fails AFTER the file opens
+ // successfully but DURING schema probing: drop the `index_state` table,
+ // which makes `readSchemaVersion` throw a raw sqlite error that the read
+ // path wraps as INDEX_UNREADABLE (the typed code for "index present but
+ // cannot be read"). This is the realistic corruption shape โ€” a partial
+ // write or a half-migrated file โ€” as opposed to total byte-level damage.
+ const corruptor = new Database(dbPath);
+ corruptor.run("DROP TABLE index_state");
+ corruptor.close();
+
+ // Sanity: the corruption really does surface as INDEX_UNREADABLE on a
+ // raw read open (the pre-self-heal condition).
+ await expect(Store.open(config, { mode: "read" })).rejects.toMatchObject({
+ code: "INDEX_UNREADABLE",
+ });
+
+ // Without self-heal this INDEX_UNREADABLE escapes to the caller; with it,
+ // the read path rebuilds once and returns results.
+ const outcome = await search(config, { query: "fox" });
+ expect(outcome.results.length).toBeGreaterThan(0);
+ expect(existsSync(dbPath)).toBe(true); // rebuilt by the self-heal
+});
+
test("a genuinely empty vault returns empty results, not an error", async () => {
rmSync(`${vault}/Notes/foo.md`, { force: true });
const config = makeConfig({ vault, dbPath });
diff --git a/tests/core/search/store-aliases.test.ts b/tests/core/search/store-aliases.test.ts
index 67780358..30f31791 100644
--- a/tests/core/search/store-aliases.test.ts
+++ b/tests/core/search/store-aliases.test.ts
@@ -53,8 +53,8 @@ function targetDocId(store: Store, sourceId: number): number | null {
return row?.target_document_id ?? null;
}
-test("schema version is 7", () => {
- expect(LATEST_SCHEMA_VERSION).toBe(7);
+test("schema version is 8", () => {
+ expect(LATEST_SCHEMA_VERSION).toBe(8);
});
test("normalizeAlias lower-cases, trims, and NFC-normalises", () => {
@@ -191,3 +191,36 @@ describe("resolveAliasTargets", () => {
}
});
});
+
+describe("resolvedDocLinkPairs basename resolution (v8)", () => {
+ test("exact top-level path wins over a nested basename; unique nested resolves; ambiguous nested is dropped", async () => {
+ const store = await Store.open(config, { mode: "write" });
+ try {
+ // Top-level `alpha.md` and nested `notes/alpha.md` both exist: the
+ // exact `<target>.md` branch must pick the top-level doc, not treat
+ // `alpha` as an ambiguous basename.
+ const topAlpha = doc(store, "alpha.md");
+ doc(store, "notes/alpha.md");
+ // Unique nested basename.
+ const beta = doc(store, "notes/beta.md");
+ // Ambiguous nested basename: two `.../gamma.md`.
+ doc(store, "x/gamma.md");
+ doc(store, "y/gamma.md");
+
+ const sAlpha = doc(store, "src/s-alpha.md");
+ const sBeta = doc(store, "src/s-beta.md");
+ const sGamma = doc(store, "src/s-gamma.md");
+ link(store, sAlpha, "alpha");
+ link(store, sBeta, "beta");
+ link(store, sGamma, "gamma");
+
+ const pairs = store.resolvedDocLinkPairs();
+ const bySource = new Map(pairs.map((p) => [p.source, p.target]));
+ expect(bySource.get(sAlpha)).toBe(topAlpha); // exact top-level wins
+ expect(bySource.get(sBeta)).toBe(beta); // unique nested resolves
+ expect(bySource.has(sGamma)).toBe(false); // ambiguous nested dropped
+ } finally {
+ await store.close();
+ }
+ });
+});
diff --git a/tests/core/surface/lexical-score.test.ts b/tests/core/surface/lexical-score.test.ts
index ccb0bef4..b2beaabd 100644
--- a/tests/core/surface/lexical-score.test.ts
+++ b/tests/core/surface/lexical-score.test.ts
@@ -39,6 +39,19 @@ test("a 2-char CJK token is kept whole without extra bigrams", () => {
expect(tokenize("ๅฎž็Žฐ")).toEqual(["ๅฎž็Žฐ"]);
});
+test("bigrams come only from the Han span of a mixed token, not across the boundary", () => {
+ // A spaceless mixed token like "gbrainๅฎž็Žฐ" should share the Han bigram
+ // "ๅฎž็Žฐ" with a query, but must NOT emit the cross-script window "nๅฎž"
+ // (inert noise that inflates term frequency / document length).
+ expect(tokenize("gbrainๅฎž็Žฐ")).toEqual(["ๅฎž็Žฐ", "gbrainๅฎž็Žฐ"]);
+});
+
+test("a 2-char Han run embedded in a longer mixed token still emits its bigram", () => {
+ // Gated by the Han run, not the total token length: "abๅฎž็Žฐcd" yields "ๅฎž็Žฐ"
+ // even though the token as a whole is > 2 chars of mostly ASCII.
+ expect(tokenize("abๅฎž็Žฐcd")).toEqual(["ๅฎž็Žฐ", "abๅฎž็Žฐcd"]);
+});
+
test("ASCII tokenization is unaffected by the CJK bigram pass", () => {
expect(tokenize("Hello World")).toEqual(["hello", "world"]);
});
diff --git a/tests/mcp/generation-reports-tool.test.ts b/tests/mcp/generation-reports-tool.test.ts
index 9e4270ab..8a31bad9 100644
--- a/tests/mcp/generation-reports-tool.test.ts
+++ b/tests/mcp/generation-reports-tool.test.ts
@@ -140,6 +140,32 @@ describe("brain_generation_reports tool", () => {
).toEqual([report["id"]]);
});
+ test("rejects a malformed created_at with INVALID_PARAMS", async () => {
+ const server = new MCPServer({ vault, configPath });
+ await initialize(server);
+
+ const bad = (await server.handleRequest({
+ jsonrpc: JSONRPC_VERSION,
+ id: 11,
+ method: "tools/call",
+ params: {
+ name: "brain_generation_reports",
+ arguments: {
+ action: "record",
+ handoff_kind: "write_session",
+ ref: "ws-bad",
+ agent: "tester",
+ enable: true,
+ prompt: "do the thing",
+ created_at: "2026-13-01T00:00:00Z",
+ },
+ },
+ })) as { error?: { message?: string }; result?: { isError?: boolean } };
+ expect(bad.error ?? bad.result?.isError).toBeTruthy();
+ // Nothing was written โ€” the invalid timestamp is rejected before the emit.
+ expect(listGenerationReports(vault)).toHaveLength(0);
+ });
+
test("rejects an unknown action and an unknown handoff_kind", async () => {
const server = new MCPServer({ vault, configPath });
await initialize(server);