Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(wal_tap): snapshot/restore only seeds a XERJ target, not cross-engine (#400)

xerj-org2aab06dac64c
diff --git a/engine/crates/xerj-engine/src/wal_tap.rs b/engine/crates/xerj-engine/src/wal_tap.rs
index 74fbe7d0a..099737148 100644
--- a/engine/crates/xerj-engine/src/wal_tap.rs
+++ b/engine/crates/xerj-engine/src/wal_tap.rs
@@ -84,7 +84,11 @@
//! included. On an index that has been running long enough to prune, at the
//! end: the surviving generations are an arbitrary recent slice, and shipping
//! them would look like a backfill without being one. **There is no backfill.**
-//! Seed the target with snapshot/restore if it needs the existing documents.
+//! Seed the target with the existing documents first if it needs them.
+//! snapshot/restore only seeds another XERJ node — XERJ's snapshot format
+//! (WAL + segments) is engine-specific and cannot be restored into an
+//! Elasticsearch/OpenSearch target; seed a cross-engine target with an
+//! external scan-and-push of the current documents (`_search`/PIT -> `_bulk`).
//!
//! **4b. What falling behind costs.** WAL retention is *not* coupled to the
//! tap, deliberately: `IndexStore::wal_maintain_all_verified` deletes a
diff --git a/engine/xerj.default.toml b/engine/xerj.default.toml
index 0b1b5bf8e..ca822b63e 100644
--- a/engine/xerj.default.toml
+++ b/engine/xerj.default.toml
@@ -518,8 +518,11 @@ access_log = false
# Before turning it on, three behaviours you are agreeing to:
#
# * No backfill. A new index (nothing pruned from its WAL yet) is shipped
-# whole; an established one starts from the moment it is allowlisted. Seed
-# the target with snapshot/restore first if it needs the existing documents.
+# whole; an established one starts from the moment it is allowlisted. If it
+# needs the pre-existing documents, seed the target first: snapshot/restore
+# only works when the target is another XERJ node (the snapshot format is
+# engine-specific). For an Elasticsearch/OpenSearch target, seed with an
+# external scan-and-push of the current documents (_search/PIT -> _bulk).
# * At-least-once. Each action carries `version_type=external` with the WAL
# seq_no, so redelivery is a no-op on a target that honours it.
# * Retention never waits for the target. WAL generations are pruned once they