Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(autoindex): close the three verification items on the #360 cross-corpus fix

xerj-org96a720c493bf
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 10a0ce012..65f2cbd63 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -43,13 +43,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
already indexed a byte-identical file** — the second of the three aborts
[#360](https://github.com/xerj-org/xerj/issues/360) reports, and the one that
killed the reporter's second corpus. A file's identity is derived from its
- CONTENT alone (`ids::file_key`) and the catalog is one global index
+ CONTENT alone (`content::full_digest`) and the catalog is one global index
(`autoindex-catalog`) that no `--prefix` namespaces, so two unrelated
checkouts sharing one file — an Apache-2.0 `LICENSE` is the everyday case —
share that file's catalog document IDs. A corpus holding the content five
times publishes a canonical document plus four alias documents; a corpus
holding it once republishes only the canonical, and the other corpus's four
- alias documents survive under their own `run_id`. The generation-wide barrier
+ alias documents survive under their own `run_id`. Be clear about what that
+ does *not* mean: the shared file's canonical catalog document
+ (`file:<content_id>`) belongs to whichever run published it last, so after the
+ second corpus runs, the first corpus's alias documents point at a
+ `duplicate_of` whose canonical entry is now the other corpus's. That is
+ pre-existing — the binary before this change overwrites it too, then aborts —
+ and this change removes the false abort without repairing it. The catalog
+ identity collision itself is tracked in
+ [#416](https://github.com/xerj-org/xerj/issues/416). The generation-wide barrier
counted every catalog document carrying the group's `file_key` — across every
run on the node — against `1 + aliases.len()` for the group in front of it, so
the second run exited `1` with `catalog canonical/alias count disagrees with
@@ -81,7 +89,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
documents carry `"time_field": null`, `"time_min": null`, `"time_max": null`
and `"semantic_field": null`. The exact read-back is the tripwire that caught
the engine rewriting a document, so it is deliberately left armed rather than
- taught to ignore the difference.
+ taught to ignore the difference. Tracked as
+ [#415](https://github.com/xerj-org/xerj/issues/415).
- **A lexical query on a `semantic_text` field now says so** (#363). `match`,
`match_phrase`, `multi_match` and `query_string` against a `semantic_text`
diff --git a/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs b/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs
index 36b85a168..a3ea1d310 100644
--- a/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs
+++ b/engine/crates/xerj-autoindex/src/incremental_reconcile_http_tests.rs
@@ -1477,7 +1477,7 @@ fn a_readme_documenting_two_tables_is_not_a_fatal_condition() {
/// The second abort #360 reported — the one that killed both of the corpora
/// the reporter pointed autoindex at, on one node, one after the other.
///
-/// `ids::file_key` derives a file's identity from its CONTENT alone, and
+/// `content::full_digest` derives a file's identity from its CONTENT alone, and
/// `catalog::CATALOG_INDEX` is a single global index that no `--prefix`
/// namespaces. Two unrelated checkouts that happen to share one byte-identical
/// file — an Apache-2.0 `LICENSE`, a `.gitignore`, an empty `__init__.py` —
diff --git a/engine/crates/xerj-autoindex/src/sync_executor.rs b/engine/crates/xerj-autoindex/src/sync_executor.rs
index 0ae4df236..e7080d826 100644
--- a/engine/crates/xerj-autoindex/src/sync_executor.rs
+++ b/engine/crates/xerj-autoindex/src/sync_executor.rs
@@ -666,7 +666,7 @@ impl SyncOperationBackend for EsSyncBackend<'_> {
);
// Scoped to this generation's `run_id`, the same way the run-summary
// read-back is (`lib.rs`). `file_key` is derived from CONTENT alone
- // (`ids::file_key`) and the catalog is one global index that no
+ // (`content::full_digest`) and the catalog is one global index that no
// `--prefix` namespaces, so an unscoped count also sees the
// canonical and alias documents that ANOTHER corpus on this node
// published for byte-identical content — two Apache-2.0 checkouts