Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(engine): a reopened index runs no catch-up merge until it changes (#2710)

xerj-org8bd229d272a4
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ca7af8113..0c2c5cab2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -126,6 +126,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
longer matches through an `index: false` field. `query_string` with
`["*"]` is unchanged.
+- A restarted node that receives no writes no longer runs catch-up merges
+ when its indexes open. Inherited merge debt is now paid at the first
+ segment-set change (a flush, refresh or delete) or on `_forcemerge`. Before,
+ a search-only node in the 1 GB gate started merging straight after the
+ restart, while it was serving reads under the memory breaker. This
+ supersedes the 120 s open quiet period in #2492, which only delayed the
+ same merges (Refs #2427).
+
- `_bulk` requests of ordinary text and code are no longer refused with a
429 while the node has memory to spare. Each raw bulk reserved 28 bytes
per input byte for analysis, a rate sized for the densest text, so a bulk
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index 26baa34a6..20fe721a9 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -6039,9 +6039,8 @@ mod merge_publication_transaction_tests {
/// stopped.
///
/// Same contract as `hand_driven_index`: the restart half of these tests
- /// asserts on a recovered segment set, and `Index::open` schedules its
- /// own open-time merge check too (`request_merge_check`, #871 — an
- /// opened index may carry pre-existing merge debt).
+ /// asserts on a recovered segment set, and any flush after the reopen
+ /// arms a background merge check (#871).
fn reopen_hand_driven(dir: &TempDir, name: &str) -> (Engine, Arc<Index>) {
let engine = Engine::new(config(dir)).unwrap();
let index = engine.get_index(name).unwrap();
@@ -7146,6 +7145,70 @@ mod merge_publication_transaction_tests {
}
}
+ async fn seed_segments_for_reopen(dir: &TempDir, name: &str, segments: usize) {
+ let (engine, index) = hand_driven_index(dir, name, Schema::empty());
+ for n in 0..segments {
+ index
+ .index_document(Some(format!("d{n}")), serde_json::json!({"value": n}))
+ .await
+ .unwrap();
+ index.flush().await.unwrap();
+ }
+ assert_eq!(index.store.snapshot().segments.len(), segments);
+ drop(index);
+ drop(engine);
+ }
+
+ /// #2427 — a restarted node that receives no writes runs no catch-up
+ /// merges. In the 1 GB release gate every index on the restarted
+ /// search-only node merged its inherited segments a few seconds after
+ /// boot, while it served reads above the breaker watermark on a
+ /// 3,072 MB cap. On the old code the open-time
+ /// `request_merge_check` evaluates (and merges) within the 5 s
+ /// debounce, so the first assertion fails.
+ #[tokio::test(start_paused = true)]
+ async fn a_reopened_index_without_writes_runs_no_catch_up_merge() {
+ const SEGMENTS: usize = 6;
+ let dir = TempDir::new().unwrap();
+ seed_segments_for_reopen(&dir, "no-catchup", SEGMENTS).await;
+
+ // Background scheduling stays running: it is the subject.
+ let engine = Engine::new(config(&dir)).unwrap();
+ let index = engine.get_index("no-catchup").unwrap();
+ for _ in 0..30 {
+ tokio::time::sleep(std::time::Duration::from_secs(60)).await;
+ tokio::task::yield_now().await;
+ }
+ assert_eq!(
+ index.merge_evaluation_count(),
+ 0,
+ "a reopened index with no writes evaluated a merge"
+ );
+ assert!(!index.is_merge_in_progress());
+ assert_eq!(index.store.snapshot().segments.len(), SEGMENTS);
+ let docs = index
+ .search(&SearchRequest::default())
+ .await
+ .unwrap()
+ .total
+ .value;
+ assert_eq!(docs, SEGMENTS as u64);
+
+ // The first write pays the inherited debt.
+ index
+ .index_document(Some("w".into()), serde_json::json!({"value": "w"}))
+ .await
+ .unwrap();
+ index.flush().await.unwrap();
+ settle_merge_scheduler(&index).await;
+ assert!(index.merge_evaluation_count() >= 1);
+ assert!(
+ index.store.snapshot().segments.len() < SEGMENTS,
+ "the first write did not pay the inherited merge debt"
+ );
+ index.abort_background_tasks();
+ }
+
/// #871 — merge scheduling must be event-driven: after an index's merge
/// work settles, ZERO further merge-policy evaluations may run until the
/// segment set changes again.
@@ -11863,12 +11926,20 @@ impl Index {
idx.on_segments_changed();
}
})));
- // An opened index can carry pre-existing merge debt — a crash
- // mid-cascade, or merge config lowered since the segments were
- // written — that no future event would pay off on a read-only
- // index. Schedule one debounced check now; it quiesces after a
- // single evaluation when there is nothing to do.
- index.request_merge_check();
+ // #2427 — no open-time merge check. An opened index can carry
+ // merge debt (a crash mid-cascade, or merge config lowered since
+ // the segments were written). Paying it at open made every index
+ // on a restarted node evaluate (and merge) a few seconds after
+ // boot: in the 1 GB release gate the search-only node, which
+ // received no writes, started merging its inherited segments while
+ // it served reads above the breaker watermark. Merging exists to
+ // bound the segment count that writes create; the debt does not
+ // affect correctness, and the index's first segment-set change
+ // (a flush, a tombstone-only segment, orphan recovery) fires the
+ // segments-changed hook above, which evaluates it then. An
+ // operator who wants it paid on a read-only index runs
+ // `_forcemerge`. This supersedes #2492's open-time quiet period,
+ // which only delayed the same burst by 120 s.
// RC4 W2 item 17: a flush-time-stale HNSW snapshot used to stay
// stale for the rest of the process lifetime (ingest kept paying
// full graph-maintenance cost while the ANN path never served).