AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
test(autoindex): pin #360 on the shape it was reported against, and log it
xerj-org11e66745b74b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f81c3c688..8dcf0875e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,7 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]-_Nothing yet._+### Fixed++- **`xerj autoindex` aborted a whole run when one file declared two or more SQL+ tables** ([#360](https://github.com/xerj-org/xerj/issues/360)). One file can+ feed several datasets — a SQL dump is one file and N tables — but the sealed+ record count was a single whole-file total. Finalisation charged that total to+ *each* of the file's datasets in turn and compared it against that one+ dataset's read-back, so any multi-table file disagreed with itself by+ construction and exited `1` with `dataset t1 exact read-back count 2 disagrees+ with sealed count 4`. The rows were always indexed correctly; only the+ reconciliation arithmetic was wrong, so `exit=1` discarded a complete corpus.+ Reported against a real `unum-cloud/usearch` file — `sqlite/README.md`,+ ordinary prose with three ` ```sql ` blocks in it, which the sniffer routes to+ the `sqldump` family.++ The sealed ledger is now keyed by the dataset identity each record was written+ under (`records_by_dataset` on the prepared artifact,+ `expected_records_by_dataset` on the manifest group), the read-back is scoped+ by `ax_dataset` as well as `ax_file`, and `validate_groups` rejects a ledger+ that names a dataset the group does not feed or that does not sum to the group+ total. Measured against a local node with the fixed binary: the two-table and+ three-table repros from the issue exit `0`, every table's rows are present+ (`_count` 2 per table), and each dataset's catalog document reports+ `record_count: 2` instead of the whole file's total.++ Existing state directories survive the upgrade: both new fields are omitted+ from serialization when empty, so a snapshot or manifest sealed by an older+ version re-serializes to the same bytes and keeps its digest. A pre-upgrade+ group that fanned out has no recoverable split, so it withdraws from that one+ equality rather than failing it — the state directory left behind by the+ aborted repro run resumes and commits on the fixed binary.## [1.0.0-rc.17] - 2026-08-15diff --git a/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs b/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rsindex 660cafde4..2eee88018 100644--- a/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs+++ b/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs@@ -1409,6 +1409,71 @@ fn a_two_table_sqlite_file_commits_its_generation() {assert_eq!(journal_events(state_dir.path(), "sync_commit"), 1);}+/// The shape #360 was actually reported against, and the one that reaches the+/// fan-out through the *sniffer* rather than through an extension: ordinary+/// markdown prose with SQL blocks in it, as in `unum-cloud/usearch`'s+/// `sqlite/README.md`. `sniff` routes any text containing `CREATE TABLE` and a+/// `;` to the `sqldump` family, so a README documenting two tables is one+/// content group feeding two datasets — no dump and no database file involved.+/// Worth pinning separately: the reporter's corpus contained neither.+#[test]+fn a_readme_documenting_two_tables_is_not_a_fatal_condition() {+ let _guard = HTTP_E2E_LOCK.lock().unwrap_or_else(|p| p.into_inner());+ let _replay_guard = sync_executor::REPLAY_FAILPOINT_TEST_LOCK+ .lock()+ .unwrap_or_else(|p| p.into_inner());+ let corpus = tempfile::tempdir().unwrap();+ let state_dir = tempfile::tempdir().unwrap();+ fs::write(+ corpus.path().join("README.md"),+ "# vectors\n\nStore them like this:\n\n```sql\n\+ CREATE TABLE t1 (id INTEGER PRIMARY KEY, v JSON NOT NULL);\n\+ INSERT INTO t1 (id, v) VALUES (10, '[1.0]'), (11, '[2.0]');\n\+ CREATE TABLE t2 (id INTEGER PRIMARY KEY, v JSON NOT NULL);\n\+ INSERT INTO t2 (id, v) VALUES (20, '[1.0]'), (21, '[2.0]');\n\+ ```\n",+ )+ .unwrap();+ let endpoint = HttpEndpoint::start();+ let config = cfg(corpus.path(), state_dir.path(), &endpoint.url, false);++ let (code, summary) = run_index_report(config).unwrap();+ let summary = summary.expect("a README describing two tables still commits");+ assert_eq!(code, 0, "a valid README is not a failed run");+ assert_eq!(summary["records_total"], 4);+ assert_eq!(journal_events(state_dir.path(), "sync_commit"), 1);++ let mut counts: Vec<(String, u64)> = endpoint+ .state+ .lock()+ .unwrap()+ .docs+ .iter()+ .filter(|((index, _), doc)| {+ index == catalog::CATALOG_INDEX+ && doc.get("doc_kind").and_then(Value::as_str) == Some("dataset")+ })+ .map(|(_, doc)| {+ (+ doc["slug"].as_str().unwrap_or_default().to_owned(),+ doc["record_count"]+ .as_u64()+ .expect("a dataset catalog document reports its record count"),+ )+ })+ .collect();+ counts.sort();+ assert_eq!(+ counts.len(),+ 2,+ "one dataset per declared table: {counts:?}"+ );+ assert!(+ counts.iter().all(|(_, records)| *records == 2),+ "each dataset carries only its own rows: {counts:?}"+ );+}+/// Where this branch and #241 meet.////// The generated `--no-graph` route returns from inside `run_index_report`,