Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
feat(hnsw): persistence + MCP tuning surface + onto_align prefilter
FABIOTESS06d42fc4abf6
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e9d69e6f..a3e5f15a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,7 @@ All notable changes to Open Ontologies are documented here.## [Unreleased]### Added+- **HNSW persistence + tuning + onto_align prefilter** (completes the moat scaffold). Three follow-on changes wire the prior HNSW scaffold into the rest of the system: (1) the built HNSW index now persists across process restarts via a new `hnsw_index_cache` SQLite table; `VecStore::load_from_db` automatically reinstates the cached index when its entries-fingerprint (deterministic FNV-1a 64-bit hash of sorted iri+text-vec bytes) matches the just-loaded vectors, and rejects stale caches when vectors changed — so a process startup over a populated DB skips the full rebuild. (2) New `onto_hnsw_build` MCP tool exposes the HNSW `ef_construction` and `ef_search` parameters so the connected orchestrator can tune index quality vs build/query time on larger ontologies; the tool optionally persists the rebuilt index. New `OntoHnswBuildInput` in `src/inputs.rs`. (3) `onto_align`'s candidate loop now transparently uses the HNSW index as a pre-filter when both source and target IRIs have embeddings in the vecstore: for each source class, a top-50 cosine shortlist of target candidates is computed once via HNSW, and the inner loop skips pairs not in the shortlist. The optimisation degrades gracefully — sources without embeddings fall back to the full target scan, preserving correctness on partially-embedded inputs. Two new persistence tests in `tests/vecstore_test.rs` cover the round-trip (vectors + index reload on a fresh `VecStore` over the same DB) and the cache-invalidation path (mutated vectors + unmutated index → cache rejected, rebuild on next search).- **HNSW-accelerated cosine search scaffold** (the "vector-index moat"). New optional `instant-distance` dependency (gated behind the existing `embeddings` feature, no impact on default builds) and a new `src/hnsw_index.rs` module wrapping the HNSW algorithm (Malkov & Yashunin, TPAMI 2020). `VecStore` grows a `search_cosine_hnsw(query, top_k)` method that builds the index lazily on first call and rebuilds whenever the store is mutated; the existing `search_cosine` brute-force linear scan is unchanged and continues to work without HNSW (zero regression risk). Strategic context: per the May 2026 ecosystem research, no Rust knowledge-graph engine ships native HNSW alongside its triple store — the de-facto stack is `Neo4j + Qdrant + a Python adapter`. This module is the foundation for Open Ontologies to fill that gap as a Rust-native MCP server with first-class semantic search inside the same process. Scaffold scope: the core index + integration + tests; follow-up work (persistence layer for the built index, MCP-tool surface for tuning HNSW `ef_search` / `ef_construction`, wiring into `onto_align`'s embedding-similarity signal) is tracked in the project notes. New tests: 3 unit tests in `src/hnsw_index.rs` + 3 integration tests in `tests/vecstore_test.rs` (round-trip top-1 agreement with brute-force, mutation-invalidation behaviour, empty-store edge case).- **GenOM-style description-based embedding enrichment for `onto_embed`**. New optional `descriptions: HashMap<String, String>` field on `OntoEmbedInput`. When the map is supplied, each class IRI in the map is embedded from its description text instead of from its `rdfs:label`; IRIs absent from the map fall back to the existing label-based embedding. Returns a new `enriched: <count>` field alongside the existing `embedded: <count>` so callers can see how many classes used descriptions vs. labels. This is the MCP-native form of the GenOM pattern (Mensa et al. 2025, accepted World Wide Web Journal, which showed Qwen-32B-generated descriptions lift alignment F1 substantially over raw-label embedding): the server doesn't generate descriptions, the connected orchestrator (Claude) authors them in-conversation using its own reasoning, then passes them in via this field. Net new dependencies: zero. Behaviour with no `descriptions` map is identical to before (back-compat). New inline unit tests in `src/inputs.rs` cover the deserialization both with and without the field present.- **`ies-4.3.1` marketplace preset — frozen MIT baseline** (#25). New marketplace catalogue entry pointing at the archived `dstl/IES4` repo at tag `v4.3.1` (3 Mar 2025, MIT-licensed, last public release before the IES governance transition to DBT / IES-Org). Distinct from the existing `ies` preset (which tracks `IES-Org/ont-ies` main and shifts as upstream evolves) — use `ies-4.3.1` when you need a reproducible compliance baseline that won't drift. Source: 5,375-line Turtle artefact `ies4.ttl`, baseURI `http://ies.data.gov.uk/ontology/ies4`, the same namespace used by the existing `boro` and `ies4` enforce rule packs. Install via `onto_marketplace install ies-4.3.1`. Inline unit tests in `src/marketplace.rs` verify the URL pins to the `v4.3.1` tag (not `main`) and that the live `ies` and frozen `ies-4.3.1` presets coexist with distinct IDs and URLs.diff --git a/Cargo.lock b/Cargo.lockindex 5e6f6bac..80a5d0f3 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -607,6 +607,15 @@ version = "0.22.1"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"+[[package]]+name = "bincode"+version = "1.3.3"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad"+dependencies = [+ "serde",+]+[[package]]name = "bindgen"version = "0.71.1"@@ -2086,6 +2095,8 @@ dependencies = ["parking_lot","rand 0.8.5","rayon",+ "serde",+ "serde-big-array",][[package]]@@ -2730,6 +2741,7 @@ dependencies = ["anyhow","arrow 54.3.1","axum",+ "bincode","calamine","chrono","clap",@@ -3935,6 +3947,15 @@ dependencies = ["serde_derive",]+[[package]]+name = "serde-big-array"+version = "0.5.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "11fc7cc2c76d73e0f27ee52abbd64eec84d46f370c88371120433196934e4b7f"+dependencies = [+ "serde",+]+[[package]]name = "serde_core"version = "1.0.228"diff --git a/Cargo.toml b/Cargo.tomlindex f7573d6f..f5e45bc7 100644--- a/Cargo.toml+++ b/Cargo.toml@@ -40,7 +40,8 @@ sqlx = { version = "0.8", default-features = false, features = ["runtime-tokio",duckdb = { version = "1", optional = true, features = ["bundled"] }tract-onnx = { version = "0.21", optional = true }tokenizers = { version = "0.21", optional = true, default-features = false, features = ["onig"] }-instant-distance = { version = "0.6", optional = true }+instant-distance = { version = "0.6", optional = true, features = ["with-serde"] }+bincode = { version = "1.3", optional = true }dirs = "6"shell-words = "1"@@ -49,7 +50,7 @@ default = []postgres = ["sqlx"]duckdb = ["dep:duckdb"]sql = ["postgres", "duckdb"]-embeddings = ["tract-onnx", "tokenizers", "instant-distance"]+embeddings = ["tract-onnx", "tokenizers", "instant-distance", "bincode"][dev-dependencies]tempfile = "3"diff --git a/src/align.rs b/src/align.rsindex 384d1054..e419a526 100644--- a/src/align.rs+++ b/src/align.rs@@ -394,11 +394,66 @@ impl AlignmentEngine {// Get learned weights (or defaults)let weights = self.get_learned_weights();- // Compute candidates: cartesian product of source × target classes+ // HNSW prefilter (optional, transparent): when both source and target class+ // IRIs have embeddings in the vecstore, use HNSW to compute a per-source+ // shortlist of candidate target IRIs and skip pairs not in the shortlist.+ // The default top-k is 50, which is well above the typical # of high-label-+ // similarity matches per source class but tight enough to cut the Cartesian+ // product cost on large ontologies. Gracefully degrades to full Cartesian+ // when embeddings are missing.+ #[cfg(feature = "embeddings")]+ let hnsw_prefilter: Option<std::collections::HashMap<String, std::collections::HashSet<String>>> = {+ const HNSW_TOP_K: usize = 50;+ if let Some(ref vs_arc) = self.vecstore {+ let target_iri_set: std::collections::HashSet<&str> =+ target_classes.iter().map(|c| c.iri.as_str()).collect();+ let mut vs = vs_arc.lock().unwrap();+ let mut by_source: std::collections::HashMap<String, std::collections::HashSet<String>> =+ std::collections::HashMap::new();+ let mut any_used = false;+ for sc in &source_classes {+ if let Some(src_vec) = vs.get_text_vec(&sc.iri).map(|v| v.to_vec()) {+ let shortlist = vs.search_cosine_hnsw(&src_vec, HNSW_TOP_K);+ if !shortlist.is_empty() {+ let filtered: std::collections::HashSet<String> = shortlist+ .into_iter()+ .filter(|(iri, _)| target_iri_set.contains(iri.as_str()))+ .map(|(iri, _)| iri)+ .collect();+ if !filtered.is_empty() {+ by_source.insert(sc.iri.clone(), filtered);+ any_used = true;+ }+ }+ }+ }+ if any_used { Some(by_source) } else { None }+ } else {+ None+ }+ };+ #[cfg(not(feature = "embeddings"))]+ let hnsw_prefilter: Option<std::collections::HashMap<String, std::collections::HashSet<String>>> = None;++ // Compute candidates: cartesian product of source × target classes,+ // optionally pruned to the HNSW shortlist when available.let mut candidates: Vec<serde_json::Value> = Vec::new();for sc in &source_classes {+ // HNSW prefilter: if we computed a shortlist for this source, skip+ // targets outside the shortlist. If the prefilter map exists but has+ // no entry for THIS source (e.g. source had no embedding), fall back+ // to scanning all targets — preserves correctness on partially-+ // embedded inputs.+ let shortlist_for_source: Option<&std::collections::HashSet<String>> =+ hnsw_prefilter.as_ref().and_then(|m| m.get(&sc.iri));+for tc in &target_classes {+ if let Some(shortlist) = shortlist_for_source+ && !shortlist.contains(&tc.iri)+ {+ continue;+ }// Skip self-matches (same IRI)if sc.iri == tc.iri {continue;diff --git a/src/hnsw_index.rs b/src/hnsw_index.rsindex b5f05697..460e5e1f 100644--- a/src/hnsw_index.rs+++ b/src/hnsw_index.rs@@ -42,11 +42,23 @@//! `VecStore::get_text_vec` + direct cosine — still works, no regression)use instant_distance::{Builder, HnswMap, Point, Search};+use serde::{Deserialize, Serialize};++/// Tunable HNSW build parameters. `None` means "use instant-distance default".+#[derive(Clone, Copy, Debug, Default)]+pub struct BuildParams {+ /// Size of the dynamic candidate list during graph construction. Higher+ /// values yield better recall at the cost of slower build time.+ pub ef_construction: Option<usize>,+ /// Size of the dynamic candidate list during query. Higher values yield+ /// better recall at the cost of slower search.+ pub ef_search: Option<usize>,+}/// A point in the HNSW index. Wraps an L2-normalised vector and implements the/// `instant-distance::Point` trait using `1.0 - dot_product` as the distance/// (cosine distance for L2-normalised vectors).-#[derive(Clone, Debug)]+#[derive(Clone, Debug, Serialize, Deserialize)]pub struct CosinePoint(pub Vec<f32>);impl Point for CosinePoint {@@ -79,6 +91,21 @@ impl CosineIndex {/// vectors must already be L2-normalised (as the VecStore guarantees on/// `upsert`).pub fn build<I, S>(entries: I) -> Self+ where+ I: IntoIterator<Item = (S, Vec<f32>)>,+ S: Into<String>,+ {+ Self::build_with_params(entries, BuildParams::default())+ }++ /// Build an HNSW index with explicit HNSW parameters.+ ///+ /// `ef_construction` controls the size of the dynamic candidate list during+ /// graph construction (higher = better recall, slower build).+ /// `ef_search` controls the candidate list during query (higher = better+ /// recall, slower search). The instant-distance defaults work well for+ /// ontologies up to ~10k classes; tune upward for higher-recall workloads.+ pub fn build_with_params<I, S>(entries: I, params: BuildParams) -> SelfwhereI: IntoIterator<Item = (S, Vec<f32>)>,S: Into<String>,@@ -89,13 +116,36 @@ impl CosineIndex {points.push(CosinePoint(vec));iris.push(iri.into());}- let inner = Builder::default().build(points, iris);+ let mut builder = Builder::default();+ if let Some(efc) = params.ef_construction {+ builder = builder.ef_construction(efc);+ }+ if let Some(efs) = params.ef_search {+ builder = builder.ef_search(efs);+ }+ let inner = builder.build(points, iris);Self {inner,search: Search::default(),}}+ /// Serialise the index to a byte buffer for persistence in SQLite.+ /// Round-trip via [`Self::from_bytes`] reconstructs an equivalent index+ /// without rebuilding from raw vectors.+ pub fn to_bytes(&self) -> anyhow::Result<Vec<u8>> {+ Ok(bincode::serialize(&self.inner)?)+ }++ /// Reconstruct an index from a byte buffer produced by [`Self::to_bytes`].+ pub fn from_bytes(bytes: &[u8]) -> anyhow::Result<Self> {+ let inner: HnswMap<CosinePoint, String> = bincode::deserialize(bytes)?;+ Ok(Self {+ inner,+ search: Search::default(),+ })+ }+/// Approximate top-k cosine search. Returns `(iri, similarity)` pairs/// sorted by similarity descending. Similarity is `1.0 - distance`, so/// the brute-force `search_cosine` and this method use the same scale.diff --git a/src/inputs.rs b/src/inputs.rsindex bbcef6fa..2d57ee1f 100644--- a/src/inputs.rs+++ b/src/inputs.rs@@ -447,6 +447,25 @@ pub struct OntoEmbedInput {pub descriptions: Option<std::collections::HashMap<String, String>>,}+#[derive(Deserialize, JsonSchema)]+pub struct OntoHnswBuildInput {+ /// HNSW `ef_construction` — size of the dynamic candidate list during+ /// graph construction. Higher values yield better recall at the cost of+ /// slower build. instant-distance's default (100) is a sensible starting+ /// point; tune upward (e.g. 200-400) for high-recall workloads.+ pub ef_construction: Option<usize>,+ /// HNSW `ef_search` — size of the dynamic candidate list during query.+ /// Higher values yield better recall per query at the cost of slower+ /// search. instant-distance's default works for most ontologies; raise+ /// (e.g. 100-200) when search recall matters more than latency.+ pub ef_search: Option<usize>,+ /// When true (default), persist the built index to SQLite so subsequent+ /// process restarts can skip the rebuild via `VecStore::load_cosine_index`.+ /// Set false for ephemeral one-shot builds.+ #[serde(default)]+ pub persist: Option<bool>,+}+#[derive(Deserialize, JsonSchema)]pub struct OntoSearchInput {/// Natural language querydiff --git a/src/server.rs b/src/server.rsindex 22c7602d..46ab0c98 100644--- a/src/server.rs+++ b/src/server.rs@@ -1537,6 +1537,38 @@ impl OpenOntologiesServer {} // cfg(feature = "embeddings")}+ #[tool(name = "onto_hnsw_build", description = "Build (or rebuild) the HNSW cosine index over the loaded text embeddings with explicit `ef_construction` and `ef_search` parameters. Persists the index to SQLite by default so subsequent process restarts skip the rebuild. Use after onto_embed when you want to tune index quality vs. build/query time on larger ontologies. Default builder parameters are sensible for ontologies up to ~10k classes.")]+ async fn onto_hnsw_build(&self, Parameters(input): Parameters<OntoHnswBuildInput>) -> String {+ #[cfg(not(feature = "embeddings"))]+ { let _ = input; return r#"{"error":"Compiled without embeddings feature. Rebuild with --features embeddings"}"#.to_string(); }+ #[cfg(feature = "embeddings")]+ {+ let persist = input.persist.unwrap_or(true);+ let params = crate::hnsw_index::BuildParams {+ ef_construction: input.ef_construction,+ ef_search: input.ef_search,+ };+ let mut vecstore = self.vecstore.lock().unwrap();+ vecstore.rebuild_cosine_index(params);+ let count = vecstore.len();+ let persisted = if persist {+ match vecstore.persist_cosine_index() {+ Ok(()) => true,+ Err(e) => return format!(r#"{{"error":"persist failed: {}"}}"#, e),+ }+ } else {+ false+ };+ serde_json::json!({+ "ok": true,+ "entries_indexed": count,+ "persisted": persisted,+ "ef_construction": input.ef_construction,+ "ef_search": input.ef_search,+ }).to_string()+ }+ }+#[tool(name = "onto_search", description = "Semantic search over the loaded ontology using natural language. Returns the most similar classes by text meaning, structural position, or both. Requires onto_embed to have been run first.")]async fn onto_search(&self, Parameters(input): Parameters<OntoSearchInput>) -> String {#[cfg(not(feature = "embeddings"))]diff --git a/src/state.rs b/src/state.rsindex b1415d61..4aa05167 100644--- a/src/state.rs+++ b/src/state.rs@@ -105,6 +105,19 @@ CREATE TABLE IF NOT EXISTS embeddings (updated_at TEXT NOT NULL DEFAULT (datetime('now')));+-- Cached HNSW cosine index over the embeddings.text_vec column. Single-row+-- table keyed on `kind` (currently only the cosine variant) so future+-- index variants (Poincare, product) can coexist. `entries_hash` is a+-- fingerprint of the (iri, text_vec) set the index was built from; if it+-- changes we know the cached index is stale and must be rebuilt.+CREATE TABLE IF NOT EXISTS hnsw_index_cache (+ kind TEXT PRIMARY KEY,+ entries_hash BLOB NOT NULL,+ entry_count INTEGER NOT NULL,+ serialised BLOB NOT NULL,+ updated_at TEXT NOT NULL DEFAULT (datetime('now'))+);+-- Compile cache for loaded ontology files. One row per ontology `name`.-- See src/cache.rs for the validity policy.CREATE TABLE IF NOT EXISTS ontology_cache (diff --git a/src/vecstore.rs b/src/vecstore.rsindex 7a6aaec0..28206bfa 100644--- a/src/vecstore.rs+++ b/src/vecstore.rs@@ -126,6 +126,107 @@ impl VecStore {scores}+ /// Deterministic FNV-1a 64-bit fingerprint of the entry set. Stable across+ /// processes; used to detect when a cached HNSW index is stale because the+ /// underlying vectors have changed. Includes both keys and text-vec bytes+ /// in the hash so re-embedding the same IRI with a new vector triggers a+ /// rebuild.+ fn entries_fingerprint(&self) -> Vec<u8> {+ let mut keys: Vec<&String> = self.entries.keys().collect();+ keys.sort();+ let mut hash: u64 = 0xcbf29ce484222325;+ for k in keys {+ for byte in k.as_bytes() {+ hash ^= *byte as u64;+ hash = hash.wrapping_mul(0x100000001b3);+ }+ let v = &self.entries[k];+ for f in &v.text_vec {+ for byte in f.to_le_bytes() {+ hash ^= byte as u64;+ hash = hash.wrapping_mul(0x100000001b3);+ }+ }+ }+ hash.to_le_bytes().to_vec()+ }++ /// Force-rebuild the HNSW cosine index using explicit HNSW parameters.+ /// Drops any previously-built index. The new index is held in memory; call+ /// [`Self::persist_cosine_index`] to save it.+ pub fn rebuild_cosine_index(&mut self, params: crate::hnsw_index::BuildParams) {+ if self.entries.is_empty() {+ self.cosine_index = None;+ return;+ }+ let points: Vec<(String, Vec<f32>)> = self+ .entries+ .iter()+ .map(|(iri, e)| (iri.clone(), e.text_vec.clone()))+ .collect();+ self.cosine_index = Some(crate::hnsw_index::CosineIndex::build_with_params(+ points, params,+ ));+ }++ /// Persist the current HNSW cosine index to SQLite (table `hnsw_index_cache`).+ /// Builds the index first if it isn't built. Subsequent `load_cosine_index()`+ /// calls (e.g. at process startup via `load_from_db`) read it back and skip+ /// the rebuild as long as the entry fingerprint matches.+ pub fn persist_cosine_index(&mut self) -> anyhow::Result<()> {+ if self.entries.is_empty() {+ return Ok(());+ }+ if self.cosine_index.is_none() {+ let points: Vec<(String, Vec<f32>)> = self+ .entries+ .iter()+ .map(|(iri, e)| (iri.clone(), e.text_vec.clone()))+ .collect();+ self.cosine_index = Some(CosineIndex::build(points));+ }+ let bytes = match self.cosine_index.as_ref() {+ Some(idx) => idx.to_bytes()?,+ None => return Ok(()),+ };+ let fp = self.entries_fingerprint();+ let count = self.entries.len() as i64;+ let conn = self.db.conn();+ conn.execute(+ "INSERT OR REPLACE INTO hnsw_index_cache (kind, entries_hash, entry_count, serialised) \+ VALUES ('cosine', ?1, ?2, ?3)",+ rusqlite::params![fp, count, bytes],+ )?;+ Ok(())+ }++ /// Try to load a previously-persisted HNSW cosine index. If the stored+ /// fingerprint matches the current entries' fingerprint, the index is+ /// deserialised in-place and subsequent `search_cosine_hnsw` calls skip+ /// the rebuild. If the fingerprint mismatches (or no cache exists), this+ /// is a no-op and the next `search_cosine_hnsw` rebuilds normally.+ pub fn load_cosine_index(&mut self) -> anyhow::Result<bool> {+ let conn = self.db.conn();+ let row: Option<(Vec<u8>, Vec<u8>)> = conn+ .query_row(+ "SELECT entries_hash, serialised FROM hnsw_index_cache WHERE kind = 'cosine'",+ [],+ |row| Ok((row.get::<_, Vec<u8>>(0)?, row.get::<_, Vec<u8>>(1)?)),+ )+ .ok();+ let (stored_hash, bytes) = match row {+ Some(x) => x,+ None => return Ok(false),+ };+ let current_hash = self.entries_fingerprint();+ if stored_hash != current_hash {+ // Stale — let the rebuild path handle it next time.+ return Ok(false);+ }+ self.cosine_index = Some(CosineIndex::from_bytes(&bytes)?);+ Ok(true)+ }+pub fn persist(&self) -> anyhow::Result<()> {let conn = self.db.conn();let tx = conn.unchecked_transaction()?;@@ -151,24 +252,31 @@ impl VecStore {}pub fn load_from_db(&mut self) -> anyhow::Result<()> {- let conn = self.db.conn();- let mut stmt = conn.prepare("SELECT iri, text_vec, struct_vec FROM embeddings")?;- let rows = stmt.query_map([], |row| {- let iri: String = row.get(0)?;- let text_bytes: Vec<u8> = row.get(1)?;- let struct_bytes: Vec<u8> = row.get(2)?;- Ok((iri, text_bytes, struct_bytes))- })?;-- for row in rows {- let (iri, text_bytes, struct_bytes) = row?;- self.entries.insert(iri, VecEntry {- text_vec: bytes_to_f32_vec(&text_bytes),- struct_vec: bytes_to_f32_vec(&struct_bytes),- });+ // Scope the connection + statement so the conn MutexGuard is dropped+ // before we call `load_cosine_index` (which re-acquires it).+ {+ let conn = self.db.conn();+ let mut stmt = conn.prepare("SELECT iri, text_vec, struct_vec FROM embeddings")?;+ let rows = stmt.query_map([], |row| {+ let iri: String = row.get(0)?;+ let text_bytes: Vec<u8> = row.get(1)?;+ let struct_bytes: Vec<u8> = row.get(2)?;+ Ok((iri, text_bytes, struct_bytes))+ })?;++ for row in rows {+ let (iri, text_bytes, struct_bytes) = row?;+ self.entries.insert(iri, VecEntry {+ text_vec: bytes_to_f32_vec(&text_bytes),+ struct_vec: bytes_to_f32_vec(&struct_bytes),+ });+ }}- // Invalidate any previously-built HNSW index.+ // Invalidate any previously-built HNSW index; try to load a persisted+ // one. If the persisted fingerprint matches the entries we just loaded,+ // the next `search_cosine_hnsw` skips the rebuild.self.cosine_index = None;+ let _ = self.load_cosine_index()?;Ok(())}diff --git a/tests/vecstore_test.rs b/tests/vecstore_test.rsindex bf167b23..55ecd1f4 100644--- a/tests/vecstore_test.rs+++ b/tests/vecstore_test.rs@@ -173,3 +173,96 @@ fn search_cosine_hnsw_on_empty_store_returns_empty() {let results = store.search_cosine_hnsw(&[1.0, 0.0, 0.0], 5);assert!(results.is_empty());}++#[test]+fn hnsw_index_persists_across_process_restart() {+ // Round-trip: upsert entries -> persist BOTH vectors and the built HNSW+ // index -> reopen a fresh VecStore on the same DB -> load_from_db should+ // restore both the entries and the cached index -> search_cosine_hnsw+ // works immediately without rebuilding.+ use tempfile::NamedTempFile;+ let tmp = NamedTempFile::new().unwrap();+ let path = tmp.path().to_path_buf();+ std::mem::forget(tmp);++ // Phase 1: populate, build, persist.+ {+ let db = StateDb::open(&path).unwrap();+ let mut store = VecStore::new(db);+ for (iri, v) in [+ ("http://ex.org/Cat", [1.0_f32, 0.05, 0.0]),+ ("http://ex.org/Kitten", [0.98, 0.1, 0.0]),+ ("http://ex.org/Dog", [0.9, 0.3, 0.0]),+ ("http://ex.org/Bird", [0.4, 0.7, 0.0]),+ ("http://ex.org/Car", [0.0, 0.1, 1.0]),+ ] {+ store.upsert(iri, &v, &[0.0]);+ }+ // Warm the index, then persist both vectors and the index.+ let _ = store.search_cosine_hnsw(&[1.0, 0.0, 0.0], 3);+ store.persist().expect("persist vectors");+ store.persist_cosine_index().expect("persist index");+ }++ // Phase 2: fresh store, load from same DB, query immediately.+ let db = StateDb::open(&path).unwrap();+ let mut store = VecStore::new(db);+ store.load_from_db().expect("load_from_db");+ assert_eq!(store.len(), 5, "all 5 vectors should reload");++ let results = store.search_cosine_hnsw(&[1.0, 0.0, 0.0], 1);+ assert_eq!(results.len(), 1);+ assert!(+ results[0].0.contains("Cat"),+ "after persistence round-trip, expected Cat as top-1 nearest neighbour; got {:?}",+ results+ );+}++#[test]+fn hnsw_index_cache_invalidated_when_entries_change() {+ // If the SQLite cache holds a stale index (different entry fingerprint),+ // load_cosine_index must NOT install it; the next search_cosine_hnsw+ // rebuilds from the new entries.+ use tempfile::NamedTempFile;+ let tmp = NamedTempFile::new().unwrap();+ let path = tmp.path().to_path_buf();+ std::mem::forget(tmp);++ // Persist an index built from one entry set.+ {+ let db = StateDb::open(&path).unwrap();+ let mut store = VecStore::new(db);+ store.upsert("http://ex.org/Cat", &[1.0, 0.0, 0.0], &[0.0]);+ store.upsert("http://ex.org/Dog", &[0.9, 0.1, 0.0], &[0.0]);+ let _ = store.search_cosine_hnsw(&[1.0, 0.0, 0.0], 1);+ store.persist().expect("persist vectors");+ store.persist_cosine_index().expect("persist index");+ }++ // Mutate the underlying embeddings table (simulate: a fresh embed run+ // produced different vectors for the same IRIs) — but DON'T re-persist+ // the index. The fingerprint should mismatch on the next load.+ {+ let db = StateDb::open(&path).unwrap();+ let mut store = VecStore::new(db);+ // Different vectors than the persisted set.+ store.upsert("http://ex.org/Cat", &[0.0, 1.0, 0.0], &[0.0]);+ store.upsert("http://ex.org/Dog", &[0.0, 0.9, 0.1], &[0.0]);+ store.persist().expect("persist new vectors only — NOT the index");+ }++ // Load fresh. The cached index's fingerprint won't match the new vectors,+ // so load_cosine_index should refuse it; search_cosine_hnsw must rebuild+ // and return the NEW nearest-neighbour, not the stale one.+ let db = StateDb::open(&path).unwrap();+ let mut store = VecStore::new(db);+ store.load_from_db().expect("load_from_db");+ let results = store.search_cosine_hnsw(&[0.0, 1.0, 0.0], 1);+ assert_eq!(results.len(), 1);+ assert!(+ results[0].0.contains("Cat"),+ "after vectors mutated and stale-cache rejected, expected Cat (now aligned with [0,1,0]); got {:?}",+ results+ );+}