AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
fix(engine): background merges skip the zstd-19 stored encode (ingest ~5x regression since rc.78) (#2441)
xerj-org12e36e31c016
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2cc1d521c..20fa3b994 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -121,6 +121,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0queries on stray digits. Re-index to drop them from existing indices.(#2423)+- **Indexing is fast again under a memory cap** (issue+ [#2438](https://github.com/xerj-org/xerj/issues/2438), part of+ [#2423](https://github.com/xerj-org/xerj/issues/2423)). Since rc.78 every+ background merge also tried a very slow, very small compression (zstd level+ 19) on the stored documents. That made each merge ~35x slower in that step,+ kept memory at the limit for the whole merge, and made `_bulk` answer 429+ until it ended, so indexing was about 5x slower and a loader could give up+ and drop documents. Background merges now use the normal compression level.+ An explicit `_forcemerge` still uses the slow, small one, so the force-merged+ size is unchanged; set `XERJ_MERGE_MAX_EFFORT=true` to use it for every+ merge. No data format change.+- **The helper thread that frees memory now starts when memory is high.**+ `xerj-drain-free` was refused with "memory is at the governor watermark",+ which is exactly when it is needed.+- **`_forcemerge?max_num_segments=1` now reaches one segment** when every+ segment is larger than half the per-merge size cap (it used to stop at two).+- Server startup now reports the resolved data directory, its configurationsource and whether this launch created it. New-directory guidance pointsoperators back to their previous `--data-dir` or `--config` when they expecteddiff --git a/engine/crates/xerj-common/src/thread_spawn.rs b/engine/crates/xerj-common/src/thread_spawn.rsindex d7fe2ab04..bcc02e4b9 100644--- a/engine/crates/xerj-common/src/thread_spawn.rs+++ b/engine/crates/xerj-common/src/thread_spawn.rs@@ -325,6 +325,22 @@ wheretry_spawn_with(&StdSpawn, true, name, f)}+/// Spawn a helper whose only job is to *free* memory (e.g. the memtable's+/// `xerj-drain-free`, which drops drained generations off the publish path).+///+/// Ignores the watermark gate on purpose: the gate closes exactly when RSS+/// is at the governor watermark, which is exactly when a memory-relief+/// helper should run. Gating it meant the drain's frees ran inline on the+/// publish thread under pressure (logged as "best-effort helper thread was+/// not started"). A real OS failure is still an `Err` with the job handed+/// back, and the caller drops it inline.+pub fn try_spawn_memory_relief<F>(name: &str, f: F) -> Result<SpawnedThread, SpawnFailure<F>>+where+ F: FnOnce() + Send + 'static,+{+ try_spawn_with(&StdSpawn, true, name, f)+}+/// [`try_spawn`] with an explicit stack size.////// Used by the in-process shard call, whose handler does not fit on thediff --git a/engine/crates/xerj-common/tests/memory_relief_spawn_at_watermark.rs b/engine/crates/xerj-common/tests/memory_relief_spawn_at_watermark.rsnew file mode 100644index 000000000..446c31e86--- /dev/null+++ b/engine/crates/xerj-common/tests/memory_relief_spawn_at_watermark.rs@@ -0,0 +1,33 @@+//! A memory-relief helper (`xerj-drain-free`) must start when the governor+//! watermark gate is closed — that is when it is needed. Its own test binary+//! because the spawn gate is process-global (first install wins).++use std::sync::mpsc;+use std::time::Duration;++use xerj_common::thread_spawn::{+ install_spawn_gate, spawn_allowed, try_spawn, try_spawn_memory_relief,+};++fn at_watermark() -> bool {+ false+}++#[test]+fn drain_free_helper_starts_at_the_watermark() {+ assert!(install_spawn_gate(at_watermark));+ assert!(!spawn_allowed(), "gate simulates RSS at the watermark");++ // An ordinary best-effort helper is refused at the watermark...+ assert!(try_spawn("xerj-best-effort", || {}).is_err());++ // ...but the memory-relief helper starts and runs its job.+ let (tx, rx) = mpsc::channel();+ let handle = try_spawn_memory_relief("xerj-drain-free", move || {+ tx.send(()).unwrap();+ })+ .unwrap_or_else(|f| panic!("relief spawn refused at the watermark: {}", f.error));+ rx.recv_timeout(Duration::from_secs(10))+ .expect("relief job ran");+ handle.join().unwrap();+}diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rsindex f6b4b5a71..649720a29 100644--- a/engine/crates/xerj-engine/src/index.rs+++ b/engine/crates/xerj-engine/src/index.rs@@ -105,6 +105,128 @@ async fn await_publication_barrier<T>(rx: tokio::sync::oneshot::Receiver<T>, wha}}+/// Forcemerge batch selection over segments sorted smallest first: chunks+/// of at most `chunk_len` segments whose input stays under `cap` bytes+/// (`0` = no cap, #948). Returns index lists into `sizes`.+///+/// Convergence: when every segment is already larger than half the cap,+/// no chunk can hold two, and the old selection returned nothing — an+/// explicit `max_num_segments=1` stopped at 2+ segments (seen on a 2 GiB+/// cap: two 16-27 MB segments vs a 32 MiB batch cap). In that case the two+/// smallest segments are merged anyway: one minimal batch is the least an+/// explicit force-merge must do to make progress, and the caller loops+/// passes until it reaches its target.+pub(crate) fn forcemerge_chunks(sizes: &[u64], chunk_len: usize, cap: u64) -> Vec<Vec<usize>> {+ let chunk_len = chunk_len.max(2);+ let mut chunks: Vec<Vec<usize>> = Vec::new();+ let mut batch: Vec<usize> = Vec::new();+ let mut batch_bytes = 0u64;+ for (i, &size) in sizes.iter().enumerate() {+ if !batch.is_empty()+ && (batch.len() >= chunk_len || (cap > 0 && batch_bytes.saturating_add(size) > cap))+ {+ chunks.push(std::mem::take(&mut batch));+ batch_bytes = 0;+ }+ batch_bytes = batch_bytes.saturating_add(size);+ batch.push(i);+ }+ if !batch.is_empty() {+ chunks.push(batch);+ }+ chunks.retain(|c| c.len() >= 2);+ if chunks.is_empty() && sizes.len() >= 2 {+ chunks.push(vec![0, 1]);+ }+ chunks+}++#[cfg(test)]+mod forcemerge_chunks_tests {+ use super::forcemerge_chunks;+ const MIB: u64 = 1024 * 1024;++ #[test]+ fn segments_over_the_cap_still_converge_to_one() {+ // The corpus200 case: two segments, each over half a 32 MiB cap.+ assert_eq!(+ forcemerge_chunks(&[16 * MIB, 27 * MIB], 16, 32 * MIB),+ vec![vec![0, 1]]+ );+ // Three oversized segments: merge the two smallest this pass.+ assert_eq!(+ forcemerge_chunks(&[40 * MIB, 50 * MIB, 60 * MIB], 16, 32 * MIB),+ vec![vec![0, 1]]+ );+ }++ #[test]+ fn cap_and_count_still_split_batches() {+ let sizes = [MIB; 5];+ assert_eq!(+ forcemerge_chunks(&sizes, 2, 0),+ vec![vec![0, 1], vec![2, 3]]+ );+ assert_eq!(+ forcemerge_chunks(&[10 * MIB, 10 * MIB, 10 * MIB, 10 * MIB], 16, 25 * MIB),+ vec![vec![0, 1], vec![2, 3]]+ );+ }++ #[test]+ fn one_segment_is_nothing_to_do() {+ assert!(forcemerge_chunks(&[MIB], 16, 32 * MIB).is_empty());+ assert!(forcemerge_chunks(&[], 16, 32 * MIB).is_empty());+ }+}++/// Whether a merge runs the level-19 long-effort stored encode+/// (`encode_stored_v2_forcemerge_at_level`).+///+/// An explicit `_forcemerge` always does: the operator asked for the+/// smallest index. A background (tiered) merge does only when the+/// operator opts in with `XERJ_MERGE_MAX_EFFORT=true|1|yes|on`. Since+/// rc.78 every background merge ran it, which made the stored encode ~35x+/// slower and held the breaker latched (429 to ingest) for the whole merge.+pub(crate) fn merge_uses_long_effort_stored_encode(+ explicit_forcemerge: bool,+ opt_in: Option<&str>,+) -> bool {+ if explicit_forcemerge {+ return true;+ }+ matches!(+ opt_in.map(|v| v.trim().to_ascii_lowercase()).as_deref(),+ Some("1" | "true" | "yes" | "on")+ )+}++#[cfg(test)]+mod merge_long_effort_tests {+ use super::merge_uses_long_effort_stored_encode as uses;++ #[test]+ fn background_merge_uses_fast_encode_by_default() {+ assert!(!uses(false, None));+ assert!(!uses(false, Some("")));+ assert!(!uses(false, Some("false")));+ assert!(!uses(false, Some("0")));+ }++ #[test]+ fn forcemerge_always_uses_long_effort_encode() {+ assert!(uses(true, None));+ assert!(uses(true, Some("false")));+ }++ #[test]+ fn background_long_effort_is_opt_in() {+ for v in ["1", "true", "TRUE", " yes ", "on"] {+ assert!(uses(false, Some(v)), "{v}");+ }+ }+}+#[cfg(test)]mod collection_publication_fail_closed_tests {use super::*;@@ -13620,24 +13742,10 @@ impl Index {let sizes: Vec<u64> = segs.iter().map(|s| s.size_bytes).collect();let ids: Vec<SegmentId> = segs.into_iter().map(|s| s.id.clone()).collect();let chunk_len = (mc.max_merge_count as usize).max(2);- let mut chunks: Vec<Vec<SegmentId>> = Vec::new();- let mut batch: Vec<SegmentId> = Vec::new();- let mut batch_bytes = 0u64;- for (id, size) in ids.into_iter().zip(sizes) {- if !batch.is_empty()- && (batch.len() >= chunk_len || (cap > 0 && batch_bytes + size > cap))- {- chunks.push(std::mem::take(&mut batch));- batch_bytes = 0;- }- batch_bytes = batch_bytes.saturating_add(size);- batch.push(id);- }- if !batch.is_empty() {- chunks.push(batch);- }- chunks.retain(|c| c.len() >= 2);- chunks+ forcemerge_chunks(&sizes, chunk_len, cap)+ .into_iter()+ .map(|chunk| chunk.into_iter().map(|i| ids[i].clone()).collect())+ .collect()}Some(_) => Vec::new(),None => policy.select_merges(&segments_snapshot_init),@@ -13858,6 +13966,15 @@ impl Index {// is precisely why raising the effort here is safe and raising it// on flush was not.let merge_zstd_level = self.compression_config.level.zstd_level();+ // Ingest-regression fix: only an explicit `_forcemerge` (or the+ // `XERJ_MERGE_MAX_EFFORT` opt-in) pays for the level-19 stored+ // chooser. A background merge holds RSS at the breaker+ // watermark for as long as it runs, so its encode speed is+ // ingest throughput.+ let long_effort_merge = merge_uses_long_effort_stored_encode(+ force_max_segments.is_some(),+ std::env::var("XERJ_MERGE_MAX_EFFORT").ok().as_deref(),+ );let batch_for_task = batch;let metas_for_task = metas;let failed_for_task = Arc::clone(&failed_batches);@@ -14450,11 +14567,19 @@ impl Index {};let stored_timer = std::time::Instant::now();// The merge entry also runs the per-column long-effort- // zstd chooser (#1038) — background-only by contract.- let encoded = xerj_storage::stored_codec::encode_stored_v2_merge_at_level(- &merged_json_buf,- merge_zstd_level,- );+ // zstd chooser (#1038) only on an explicit `_forcemerge`+ // or the opt-in; background merges use the plain level.+ let encoded = if long_effort_merge {+ xerj_storage::stored_codec::encode_stored_v2_forcemerge_at_level(+ &merged_json_buf,+ merge_zstd_level,+ )+ } else {+ xerj_storage::stored_codec::encode_stored_v2_merge_at_level(+ &merged_json_buf,+ merge_zstd_level,+ )+ };// #948 attribution: the compressed output buffer is an// exact owned length; it coexists with the drained buffer// for one encode call.diff --git a/engine/crates/xerj-engine/src/memtable.rs b/engine/crates/xerj-engine/src/memtable.rsindex 7e83671a2..1faea93fe 100644--- a/engine/crates/xerj-engine/src/memtable.rs+++ b/engine/crates/xerj-engine/src/memtable.rs@@ -2825,9 +2825,10 @@ impl FrozenClaim {removed_bytes,)});- // Free the dead generations off the publish critical path. If- // the thread cannot be created — the governor watermark is already- // engaged, `pthread_create` fails, or (the #1122 failure) the new+ // Free the dead generations off the publish critical path. The+ // spawn ignores the watermark gate: at the watermark is exactly when+ // this memory-relief helper must run. If the thread cannot be+ // created — `pthread_create` fails, or (the #1122 failure) the new// thread's stack cannot be reserved — the Arcs drop inline: same// correctness, we only lose the async-free optimisation. The spawn// itself must not panic: std panics *inside* the child when its@@ -2838,7 +2839,9 @@ impl FrozenClaim {// flush storm under pressure does not emit a line per generation.static DRAIN_FREE_DEGRADED: AtomicBool = AtomicBool::new(false);if let Err(failure) =- xerj_common::thread_spawn::try_spawn("xerj-drain-free", move || drop(dead))+ xerj_common::thread_spawn::try_spawn_memory_relief("xerj-drain-free", move || {+ drop(dead)+ }){xerj_common::thread_spawn::warn_best_effort_degraded(&DRAIN_FREE_DEGRADED,diff --git a/engine/crates/xerj-storage/src/stored_codec.rs b/engine/crates/xerj-storage/src/stored_codec.rsindex dc00b129e..da98d6fe3 100644--- a/engine/crates/xerj-storage/src/stored_codec.rs+++ b/engine/crates/xerj-storage/src/stored_codec.rs@@ -70,7 +70,8 @@//! * form `0` — the payload is a zstd frame written at the encode-time level//! (the only form ZBS2 segments can carry);//! * form `1` — `[0x01][level: u8][window_log: u8][zstd frame]`, written by-//! the merge-path effort chooser (see [`RAW_LONG_LEVEL`]): level 19 ++//! the merge-path effort chooser (see [`RAW_LONG_LEVEL`]): level 19 on an+//! explicit `_forcemerge` (background merges do not run it), plus//! long-distance matching + an explicit window, which finds repeats tens//! of MB apart that a default window cannot. `level`/`window_log` are//! informational — the zstd frame is self-describing.@@ -269,6 +270,17 @@ impl ColCodec {/// column; the chooser keeps whichever candidate is smaller, so a column/// that level 3 already answers (e.g. short ids) is stored exactly as/// before.+///+/// Level 19 is reserved for an explicit `_forcemerge`+/// ([`encode_stored_v2_forcemerge_at_level`]) or an operator opt-in.+/// Background merges skip this candidate: zstd 19 encodes at ~2 MB/s per+/// thread, and on a 1M-doc / 200 MB corpus at a 2 GiB cap the stored+/// encode was 98% of every background merge (`XERJ_PROF merge-batch`:+/// stored_us 94-107 s vs 2.4-3.3 s before this chooser existed, rc.77).+/// While a merge runs its working set keeps RSS above the governor+/// watermark, so the parent breaker answered 429 to ingest for the whole+/// merge: bulk throughput fell from ~41k docs/s (rc.77) to ~5-14k docs/s+/// (rc.78-rc.99).const RAW_LONG_LEVEL: i32 = 19;const RAW_LONG_WINDOW_LOG_MAX: u32 = 24;@@ -294,23 +306,32 @@ pub fn encode_stored_v2(stored_docs_json: &[u8]) -> Vec<u8> {/// operator changes `compression.level` interleave with no migration and no/// format flag.pub fn encode_stored_v2_at_level(stored_docs_json: &[u8], level: i32) -> Vec<u8> {- encode_stored_v2_at_level_inner(stored_docs_json, level, false)+ encode_stored_v2_at_level_inner(stored_docs_json, level, None)}-/// Merge-path variant of [`encode_stored_v2_at_level`]: in addition to the-/// operator's level, each JSON fallback column is also compressed at-/// [`RAW_LONG_LEVEL`] with long-distance matching, and the smaller payload is-/// kept (the same measured-chooser rule the `.dv` ZNV2 codec uses). Merges-/// are background work, so the extra encode pass is not on any-/// back-pressure-critical path; flush never calls this.+/// Background-merge entry. Encodes exactly like [`encode_stored_v2_at_level`]+/// at the operator's level: **no** long-effort candidate. Background merges+/// are on the back-pressure path — their working set holds RSS at the+/// breaker watermark while they run, so their encode speed is ingest+/// throughput (see [`RAW_LONG_LEVEL`]). Use+/// [`encode_stored_v2_forcemerge_at_level`] for the max-effort encode.pub fn encode_stored_v2_merge_at_level(stored_docs_json: &[u8], level: i32) -> Vec<u8> {- encode_stored_v2_at_level_inner(stored_docs_json, level, true)+ encode_stored_v2_at_level_inner(stored_docs_json, level, None)+}++/// Explicit-`_forcemerge` (or operator opt-in) variant: in addition to the+/// operator's level, each JSON fallback column is also compressed at+/// [`RAW_LONG_LEVEL`] (19) with long-distance matching, and the smaller+/// payload is kept (the same measured-chooser rule the `.dv` ZNV2 codec+/// uses). A force-merge is an operator request for the smallest index.+pub fn encode_stored_v2_forcemerge_at_level(stored_docs_json: &[u8], level: i32) -> Vec<u8> {+ encode_stored_v2_at_level_inner(stored_docs_json, level, Some(RAW_LONG_LEVEL))}fn encode_stored_v2_at_level_inner(stored_docs_json: &[u8],level: i32,- long_effort_chooser: bool,+ long_effort_level: Option<i32>,) -> Vec<u8> {// Parse the JSON array of documents. Failure → v1 fallback.let docs: Vec<serde_json::Value> = match serde_json::from_slice(stored_docs_json) {@@ -381,7 +402,7 @@ fn encode_stored_v2_at_level_inner(&columns,Some(stored_docs_json),level,- long_effort_chooser,+ long_effort_level,),present_nulls,)@@ -545,7 +566,7 @@ fn encode_stored_v2_from_values_inner(&columns,stored_docs_json,STORED_ZSTD_LEVEL,- false,+ None,),present_nulls,)@@ -559,15 +580,16 @@ fn encode_stored_v2_from_values_inner(/// fast path on large segments where the columnar form always wins and/// serialising a multi-MB JSON array per flush is measurable CPU).///-/// `long_effort_chooser` adds a second, higher-effort zstd candidate for-/// each JSON fallback column (see [`RAW_LONG_LEVEL`]) — merge only.+/// `long_effort_level` adds a second, higher-effort zstd candidate (LDM ++/// explicit window) at that level for each JSON fallback column (see+/// [`RAW_LONG_LEVEL`]) — force-merge only.fn encode_v2_columns(num_docs: usize,col_order: &[String],columns: &[Vec<&serde_json::Value>],stored_docs_json: Option<&[u8]>,level: i32,- long_effort_chooser: bool,+ long_effort_level: Option<i32>,) -> Vec<u8> {// THROWAWAY prof (XERJ_PROF): encode-phase breakdown.let prof = std::env::var_os("XERJ_PROF").is_some();@@ -731,8 +753,8 @@ fn encode_v2_columns(let col_json = serde_json::to_vec(col).unwrap_or_default();let zstd_payload =zstd::encode_all(Cursor::new(&col_json), level).unwrap_or_else(|_| col_json.clone());- let (zstd_payload, zstd_form) = if long_effort_chooser {- try_long_effort_zstd(&col_json, zstd_payload)+ let (zstd_payload, zstd_form) = if let Some(long_level) = long_effort_level {+ try_long_effort_zstd_at(&col_json, zstd_payload, long_level)} else {(zstd_payload, 0u8)};@@ -2713,10 +2735,16 @@ fn zigzag_varint_decode(cur: &mut Cursor<&[u8]>) -> Result<i64> {/// [`RAW_LONG_WINDOW_LOG_MAX`] and the input's own size (a window larger/// than the payload cannot match anything and only inflates the decode/// buffer).+#[cfg(test)]fn compress_long_effort(input: &[u8]) -> Option<(Vec<u8>, u32)> {+ compress_long_effort_at(input, RAW_LONG_LEVEL)+}++/// [`compress_long_effort`] at a caller-chosen zstd level.+fn compress_long_effort_at(input: &[u8], long_level: i32) -> Option<(Vec<u8>, u32)> {let size_log = 64 - (input.len() as u64).saturating_sub(1).leading_zeros();let window_log = RAW_LONG_WINDOW_LOG_MAX.min(size_log);- let mut compressor = zstd::bulk::Compressor::new(RAW_LONG_LEVEL).ok()?;+ let mut compressor = zstd::bulk::Compressor::new(long_level).ok()?;use zstd::zstd_safe::CParameter;compressor.set_parameter(CParameter::WindowLog(window_log))@@ -2732,12 +2760,19 @@ fn compress_long_effort(input: &[u8]) -> Option<(Vec<u8>, u32)> {/// payload is smaller. Form 1 carries the 3-byte `[1][level][window_log]`/// prefix; when the plain form wins, the caller's original payload is/// returned untouched so the segment can stay byte-identical ZBS2.+#[cfg(test)]fn try_long_effort_zstd(col_json: &[u8], plain: Vec<u8>) -> (Vec<u8>, u8) {- match compress_long_effort(col_json) {+ try_long_effort_zstd_at(col_json, plain, RAW_LONG_LEVEL)+}++/// [`try_long_effort_zstd`] at a caller-chosen level; the level byte of+/// the form-1 prefix records it (informational — decode ignores it).+fn try_long_effort_zstd_at(col_json: &[u8], plain: Vec<u8>, long_level: i32) -> (Vec<u8>, u8) {+ match compress_long_effort_at(col_json, long_level) {Some((long, window_log)) if long.len() + 3 < plain.len() => {let mut payload = Vec::with_capacity(3 + long.len());payload.push(1u8);- payload.push(RAW_LONG_LEVEL as u8);+ payload.push(long_level.clamp(0, u8::MAX as i32) as u8);payload.push(window_log as u8);payload.extend_from_slice(&long);(payload, 1u8)@@ -5103,6 +5138,72 @@ mod tests {assert_eq!(kept, incumbent);}+ /// Ingest-regression fix: a background merge encodes at the operator's+ /// level with no long-effort candidate (byte-identical to the plain+ /// encode), an explicit force-merge runs the level-19 candidate (form-1+ /// level byte 19 and a smaller section on LDM-friendly text), and both+ /// decode to exactly the documents the flush encoding holds.+ #[test]+ fn background_merge_skips_long_effort_forcemerge_keeps_level_19() {+ let corpus: Vec<u8> = "the quick brown fox jumps over the lazy dog. "+ .repeat(5000)+ .into_bytes();+ let (fm, form) = try_long_effort_zstd_at(&corpus, corpus.clone(), RAW_LONG_LEVEL);+ assert_eq!(form, 1);+ assert_eq!(fm[1], RAW_LONG_LEVEL as u8);+ assert_eq!(zstd::decode_all(&fm[3..]).unwrap(), corpus);++ // A free-text column whose sentences repeat at long distance (what+ // the LDM candidate exists for).+ let pool: Vec<String> = (0..400u64)+ .map(|i| {+ format!(+ "sentence {i} about shard {} replica {} and segment merge {} with payload {}",+ i * 7919 % 1013,+ i * 104_729 % 997,+ i * 31 % 89,+ i.wrapping_mul(0x9E37_79B9_7F4A_7C15) >> 20+ )+ })+ .collect();+ // More distinct bodies than DICT_MAX_CARDINALITY, so `body` is a+ // JSON fallback column rather than a dict.+ let docs: Vec<serde_json::Value> = (0..(DICT_MAX_CARDINALITY as u64 + 2000))+ .map(|i| {+ let a = &pool[(i * 37 % 400) as usize];+ let b = &pool[(i * 101 % 400) as usize];+ let c = &pool[(i * 211 % 400) as usize];+ json!({+ "_id": format!("doc-{i}"),+ "_seq_no": i,+ "_source": { "body": format!("{a} {b} {c} doc {i}"), "n": i }+ })+ })+ .collect();+ let raw = serde_json::to_vec(&docs).unwrap();+ let level = 6;+ let plain = encode_stored_v2_at_level(&raw, level);+ let background = encode_stored_v2_merge_at_level(&raw, level);+ let forcemerge = encode_stored_v2_forcemerge_at_level(&raw, level);+ assert_eq!(+ background, plain,+ "a background merge must not run the long-effort candidate"+ );+ assert!(+ forcemerge.len() < background.len(),+ "force-merge keeps the max-effort encode ({} vs {})",+ forcemerge.len(),+ background.len()+ );+ let want: serde_json::Value =+ serde_json::from_slice(&decode_stored(&encode_stored_v2(&raw)).unwrap()).unwrap();+ for (name, enc) in [("background", &background), ("forcemerge", &forcemerge)] {+ let got: serde_json::Value =+ serde_json::from_slice(&decode_stored(enc).unwrap()).unwrap();+ assert_eq!(got, want, "{name} merge encode must round-trip");+ }+ }+/// A handcrafted form-1 `RAW_JSON` column (what the merge-path chooser/// writes) must decode everywhere a form-0 column does: full decode,/// column projection, the kNN typed-vector path, and the estimator.