Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(engine): resolve ES meta-field sort keys instead of nulling them (#401)

xerj-orgbf8eca801e1a
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c3711e8f8..aa6b599e9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
+- **`sort` on an ES meta-field other than `_score` / `_doc` / `_id` resolved to
+ `null` on every hit** ([#401](https://github.com/xerj-org/xerj/issues/401)).
+ `compute_sort_values` special-cased exactly those three keys and sent
+ everything else to `get_field_value(source, field)` — a lookup *inside*
+ `_source`. `_seq_no`, `_version`, `_primary_term` and `_index` are engine
+ metadata and are never literal `_source` keys, so the lookup missed on every
+ document and the whole result set tied on `[null]`. The damage lands on
+ keyset pagination: `search_after: [null]` is never strictly greater than the
+ next hit's `[null]`, so `sort: [{"_seq_no":"asc"}]` + `search_after` — the
+ consistent full-scan pattern used for migrations — either re-reads page one
+ forever or stops after it, with no error either way. Measured on a 30-document
+ index before the fix: 4 ids collected out of 30, page two empty.
+
+ These fields now resolve through the same version-map accessors that populate
+ the response meta-fields (`lookup_seq_no` / `lookup_version`), so a hit's
+ `sort` value and its `_seq_no` / `_version` agree. `_primary_term` is the
+ constant `1` the hit meta-field already emits and `_index` is the index name;
+ both are constant within one index, so they tie like any other constant sort
+ key and a client paginating on them still needs a tie-breaker. Sorting on
+ `_seq_no` under `index.disable_sequence_numbers` is unchanged — es_compat
+ rejects it at the API edge before the request reaches the engine.
+
+ Real sort values are only half of it: the memtable's bounded sort-candidate
+ path narrows the heap's input using a doc-values column keyed by the sort
+ field, and no such column exists for a meta-field, so it classified every
+ buffered document as "missing the field" and returned an arbitrary
+ `materialisation_limit`-sized prefix. A correctly ranked page over a wrong
+ candidate set is still a silently wrong page, so that path — and the
+ segment-side sort shadow — now decline meta-fields and take the full walk.
+ The same gate fixes an independent instance of the identical defect on `_id`,
+ which already resolved to a real sort value: over a 2 000-document memtable
+ `sort: [{"_id":"desc"}]` returned
+ `["d1995","d1989","d1981","d1973","d1961"]` instead of
+ `["d1999","d1998","d1997","d1996","d1995"]`.
+
- **`xerj autoindex` aborted a whole run when one file declared two or more SQL
tables** ([#360](https://github.com/xerj-org/xerj/issues/360)). One file can
feed several datasets — a SQL dump is one file and N tables — but the sealed
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index c1403c1ce..ef4728ead 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -12683,6 +12683,61 @@ impl Index {
)
}
+ /// Resolve an ES meta-field named as a SORT key (#401).
+ ///
+ /// These fields are engine metadata: they are never literal keys in
+ /// `_source`, so `compute_sort_values`' generic `get_field_value` path
+ /// resolved every one of them to `Null` on every document — a sort that
+ /// silently ties the whole result set, and a `search_after` cursor that
+ /// can never advance past page one (`[null]` is never strictly greater
+ /// than `[null]`). Lucene has no such failure mode because a sort key
+ /// is read from the field's DOC VALUES column rather than from the
+ /// stored document — `SortedNumericSortField` selects the document's
+ /// representative sort value out of `SortedNumericDocValues`
+ /// (`lucene/core/src/java/org/apache/lucene/search/SortedNumericSortField.java:49-59`)
+ /// and `NumericComparator` compares those column values directly
+ /// (`lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java:47-63`).
+ /// xerj has no column for these fields, so it resolves them through the
+ /// SAME accessors that populate the response meta-fields — a hit's
+ /// `sort` value and its `_seq_no` / `_version` therefore agree.
+ ///
+ /// Returns `None` for every other field name, so the caller falls
+ /// through to the ordinary `_source` lookup.
+ fn meta_sort_value(&self, field: &str, id: &str) -> Option<Value> {
+ // Hot path: this runs per candidate doc per sort field inside the
+ // bounded collectors, and an ordinary field name is the overwhelming
+ // case — settle it with one byte compare instead of four `str` eq.
+ if !field.starts_with('_') {
+ return None;
+ }
+ match field {
+ // `_seq_no`: the ES-sortable one (real ES gives it doc values
+ // and rejects the sort only under
+ // `index.disable_sequence_numbers`, which es_compat.rs already
+ // rejects at the API edge before the request reaches here).
+ // Unresolvable (unknown / tombstoned) stays `Null` so the
+ // request's `missing` policy still applies.
+ "_seq_no" => Some(
+ self.lookup_seq_no(id)
+ .map_or(Value::Null, |s| Value::Number(s.into())),
+ ),
+ "_version" => Some(
+ self.lookup_version(id)
+ .map_or(Value::Null, |v| Value::Number(v.into())),
+ ),
+ // xerj is single-shard with `primary_term` fixed at 1 — the same
+ // constant the hit meta-field emits (es_compat.rs:13250). Constant
+ // for every doc, so it ties like any other constant sort key;
+ // callers paginating on it still need a tie-breaker field.
+ "_primary_term" => Some(Value::Number(1.into())),
+ // Constant within one index; it discriminates only in the
+ // cross-index merge, which re-sorts on these same `sort` arrays
+ // (es_compat.rs:10855).
+ "_index" => Some(Value::String(self.name.as_str().to_string())),
+ _ => None,
+ }
+ }
+
/// Retrieve a document by its string ID.
///
/// Checks the memtable first, then searches on-disk segments.
@@ -14431,7 +14486,7 @@ impl Index {
}
seen_ids.insert(hit.id.clone());
let seq = self.lookup_seq_no(&hit.id).unwrap_or(u64::MAX);
- topk.offer(hit, seq);
+ topk.offer(hit, seq, self);
}
} else if all_hits.len() < materialisation_limit && !seen_ids.contains(&hit.id)
{
@@ -14480,6 +14535,7 @@ impl Index {
passage: None,
},
seq,
+ self,
);
}
}
@@ -14686,6 +14742,16 @@ impl Index {
let sf = &request.sort[0];
if sf.is_score()
|| sf.is_doc_order()
+ || sf.field == "_id"
+ // An ES meta-field has no memtable dv column, so
+ // `sort_candidates_numeric` would classify EVERY
+ // buffered doc as "missing this field" and hand
+ // back an arbitrary `cap`-sized prefix as the
+ // candidate set. With #401's real `_seq_no`
+ // values that prefix is no longer a superset of
+ // the true top-cap — it would rank the wrong
+ // documents correctly. Take the full walk.
+ || is_meta_sort_field(&sf.field)
|| sf.mode != SortMode::default()
|| sf.missing != SortMissing::default()
{
@@ -16302,7 +16368,7 @@ impl Index {
if let Some(topk) = sort_topk.as_mut() {
let seq =
self.lookup_seq_no(&hit.id).unwrap_or(u64::MAX);
- topk.offer(hit, seq);
+ topk.offer(hit, seq, self);
} else {
all_hits.push(hit);
// #179 — hold the collector at
@@ -17096,7 +17162,7 @@ impl Index {
// ordering agree on the exact total order (incl. date-string
// normalisation).
for hit in &mut final_hits {
- hit.sort = compute_sort_values(&hit.source, hit.score, &hit.id, sort_fields);
+ hit.sort = compute_sort_values(&hit.source, hit.score, &hit.id, sort_fields, self);
}
// Sort by the populated sort keys. Key ties break by seq_no
// ASC (insertion order == ES internal doc-id order post-B1),
@@ -22996,6 +23062,16 @@ impl Index {
field: &str,
seg_doc_count: u64,
) -> Option<Resident<Option<Arc<Vec<(i64, u32)>>>>> {
+ // An ES meta-field (`_seq_no`, `_version`, …) is engine metadata, not
+ // a `_source` key, so no segment ever writes a dv column under that
+ // name. Bail BEFORE the registry insert so the publish-time warm
+ // doesn't burn one of its 16 slots probing a column that cannot
+ // exist, and so a future column that happened to share the name
+ // could never silently drive the candidate cut for a key the
+ // engine resolves from the version map instead (#401).
+ if is_meta_sort_field(field) {
+ return None;
+ }
// Register the field so the publish-time warm pre-builds this
// shadow for every FUTURE segment (bounded registry).
if self.sort_shadow_fields.len() < 16 && !self.sort_shadow_fields.contains_key(field) {
@@ -23496,7 +23572,7 @@ impl Index {
} else {
scorer(source_ref, id_ref)
};
- let key = compute_sort_values(source_ref, score, id_ref, topk.fields.as_slice());
+ let key = compute_sort_values(source_ref, score, id_ref, topk.fields.as_slice(), self);
if !topk.would_admit(&key) {
continue;
}
@@ -24192,7 +24268,8 @@ impl Index {
} else {
scorer(source_ref, id_ref)
};
- let key = compute_sort_values(source_ref, score, id_ref, topk.fields.as_slice());
+ let key =
+ compute_sort_values(source_ref, score, id_ref, topk.fields.as_slice(), self);
if !topk.would_admit(&key) {
continue;
}
@@ -35908,9 +35985,10 @@ impl SortTopK {
}
/// Offer a fully-sourced hit. Computes its sort key from `_source`,
/// then delegates to `offer_keyed`. `seq` is the hit's insertion-order
- /// tie-break (see `SortHeapEntry::seq`).
- fn offer(&mut self, mut hit: Hit, seq: u64) {
- hit.sort = compute_sort_values(&hit.source, hit.score, &hit.id, &self.fields);
+ /// tie-break (see `SortHeapEntry::seq`). `idx` resolves the ES
+ /// meta-fields that live outside `_source` (see `meta_sort_value`).
+ fn offer(&mut self, mut hit: Hit, seq: u64, idx: &Index) {
+ hit.sort = compute_sort_values(&hit.source, hit.score, &hit.id, &self.fields, idx);
self.offer_keyed(hit, seq);
}
/// Offer a hit whose `sort` key is already computed. Applies the
@@ -36969,11 +37047,27 @@ fn sort_epoch_memo(s: &str) -> Option<f64> {
v
}
+/// ES meta-fields whose sort value comes from engine metadata rather than
+/// from `_source` or from the hit itself (#401). `_score` / `_doc` / `_id`
+/// are deliberately NOT here: `compute_sort_values` resolves those from the
+/// hit it is already holding, with no index lookup.
+///
+/// Also the gate for the columnar sort-candidate prefilters: those narrow the
+/// candidate set with a doc-values column keyed by the sort field name, and
+/// no such column exists for a meta-field — a memtable/segment column probe
+/// would report "every doc is missing this field" and cut the candidate set
+/// to an arbitrary `cap` docs, which is the same silently-wrong page the
+/// null sort values produced.
+fn is_meta_sort_field(field: &str) -> bool {
+ matches!(field, "_seq_no" | "_version" | "_primary_term" | "_index")
+}
+
fn compute_sort_values(
source: &Value,
score: f32,
id: &str,
sort_fields: &[xerj_query::sort::SortField],
+ idx: &Index,
) -> Vec<Value> {
let mut sort_vals: Vec<Value> = Vec::with_capacity(sort_fields.len());
for sf in sort_fields {
@@ -36991,6 +37085,12 @@ fn compute_sort_values(
// ordering and `search_after: [last_id]` keyset paging (used by
// reindex) compare correctly. Matches ES, which echoes the id.
Value::String(id.to_string())
+ } else if let Some(meta) = idx.meta_sort_value(&sf.field, id) {
+ // The rest of the ES meta-fields (`_seq_no`, `_version`,
+ // `_primary_term`, `_index`) — engine metadata, resolved from
+ // the version map instead of from `_source` (#401). Everything
+ // else falls through to the `_source` lookup below.
+ meta
} else {
// `.keyword` is ES's conventional multi-field sub-type on
// dynamic/keyword fields: sorting on `foo.keyword` reads the
diff --git a/engine/crates/xerj-engine/tests/sort_on_meta_fields.rs b/engine/crates/xerj-engine/tests/sort_on_meta_fields.rs
new file mode 100644
index 000000000..8cf4a6212
--- /dev/null
+++ b/engine/crates/xerj-engine/tests/sort_on_meta_fields.rs
@@ -0,0 +1,305 @@
+//! Regression test for #401 — `sort` on an ES meta-field other than
+//! `_score` / `_doc` / `_id` resolved to `null` on EVERY hit.
+//!
+//! `compute_sort_values` special-cased exactly `_score`, `_doc` and `_id`;
+//! everything else went to `get_field_value(source, field)`, a lookup INSIDE
+//! `_source`. `_seq_no` / `_version` / `_primary_term` / `_index` are engine
+//! metadata and are never literal `_source` keys, so the lookup missed on
+//! every document and the whole result set tied on `[null]`. The visible
+//! damage is keyset pagination: `search_after: [null]` is never strictly
+//! greater than the next hit's `[null]`, so a client paging on
+//! `sort: [{"_seq_no":"asc"}]` — the consistent-full-scan pattern ES
+//! documents, and the one this repo's own `wal_tap` shape uses — either
+//! re-reads page one forever or stops after it, with no error either way.
+//!
+//! Lucene cannot express this bug: a sort key is read from the field's DOC
+//! VALUES column, not from the stored document. `SortedNumericSortField`
+//! selects the document's representative value out of `SortedNumericDocValues`
+//! (`lucene/core/src/java/org/apache/lucene/search/SortedNumericSortField.java:49-59`)
+//! and `NumericComparator` compares those column values directly
+//! (`lucene/core/src/java/org/apache/lucene/search/comparators/NumericComparator.java:47-63`).
+//! xerj has no column for a meta-field, so `Index::meta_sort_value` resolves
+//! it through the same version-map accessors that populate the response
+//! meta-fields (`lookup_seq_no` / `lookup_version`).
+//!
+//! Two collectors had to agree for that to be enough, and the second one is
+//! the trap: the memtable's bounded sort-candidate path
+//! (`sort_candidates_numeric`) cuts the candidate set with a dv column keyed
+//! by the sort field. There is no `_seq_no` column, so it classified every
+//! buffered doc as "missing the field" and handed back an arbitrary
+//! `materialisation_limit`-sized prefix. Real sort values over a wrong
+//! candidate set is still a silently wrong page — the same defect wearing
+//! the other hat — so `is_meta_sort_field` also gates that path. The same
+//! gate is what fixes `_id` at the cap: `_id` already resolved to a real
+//! sort value, and `sort: [{"_id":"desc"}]` over a >cap memtable STILL
+//! returned the wrong five documents on main.
+
+use serde_json::{json, Value};
+use tempfile::TempDir;
+use xerj_common::config::Config;
+use xerj_common::types::Schema;
+use xerj_engine::Engine;
+use xerj_query::parse_request;
+
+fn make_engine(dir: &TempDir) -> Engine {
+ let mut config = Config::default();
+ config.server.data_dir = dir.path().to_str().unwrap().to_string();
+ Engine::new(config).expect("engine::new")
+}
+
+fn sorted_req(field: &str, order: &str, size: usize) -> xerj_query::ast::SearchRequest {
+ parse_request(&json!({
+ "query": {"match_all": {}},
+ "size": size,
+ "sort": [{field: order}],
+ }))
+ .expect("parse_request")
+}
+
+/// Every hit's `sort` value for a meta-field must be the document's REAL
+/// metadata, not `null` — and `_seq_no` must agree with `lookup_seq_no`,
+/// the accessor the `_seq_no` response field is built from.
+#[tokio::test]
+async fn meta_field_sort_values_are_real_not_null() {
+ let dir = TempDir::new().unwrap();
+ let engine = make_engine(&dir);
+ engine.create_index("meta_sort", Schema::empty()).unwrap();
+ let idx = engine.get_index("meta_sort").unwrap();
+
+ for i in 0..8u64 {
+ idx.index_document(Some(format!("d{i}")), json!({ "n": i }))
+ .await
+ .unwrap();
+ }
+ // A second write on one doc so `_version` is not constant across the
+ // corpus: a fix that resolved `_version` to a hard-coded 1 would pass a
+ // "not null" assertion but still not be reading per-doc metadata.
+ idx.index_document(Some("d3".to_string()), json!({ "n": 3, "touched": true }))
+ .await
+ .unwrap();
+
+ let res = idx.search(&sorted_req("_seq_no", "asc", 50)).await.unwrap();
+ assert_eq!(res.hits.len(), 8, "corpus size");
+ for hit in &res.hits {
+ let expected = idx
+ .lookup_seq_no(&hit.id)
+ .unwrap_or_else(|| panic!("{} has no seq_no", hit.id));
+ assert_eq!(
+ hit.sort.first(),
+ Some(&Value::from(expected)),
+ "hit {} sort value must be its real _seq_no, got {:?}",
+ hit.id,
+ hit.sort
+ );
+ }
+ // …and the page must actually be ordered by it.
+ let seqs: Vec<u64> = res
+ .hits
+ .iter()
+ .map(|h| h.sort[0].as_u64().expect("numeric _seq_no sort value"))
+ .collect();
+ let mut ascending = seqs.clone();
+ ascending.sort_unstable();
+ assert_eq!(seqs, ascending, "_seq_no asc page must be sorted by seq_no");
+ assert_eq!(
+ seqs.iter().collect::<std::collections::HashSet<_>>().len(),
+ seqs.len(),
+ "_seq_no is unique per live doc, so no two hits may tie"
+ );
+
+ // `_version`: real per-doc write counter (d3 was written twice).
+ let res = idx
+ .search(&sorted_req("_version", "desc", 50))
+ .await
+ .unwrap();
+ assert_eq!(res.hits.len(), 8, "corpus size under _version sort");
+ for hit in &res.hits {
+ let expected = idx.lookup_version(&hit.id).expect("version");
+ assert_eq!(
+ hit.sort.first(),
+ Some(&Value::from(expected)),
+ "hit {} sort value must be its real _version",
+ hit.id
+ );
+ }
+ assert_eq!(
+ res.hits[0].sort.first().and_then(Value::as_u64),
+ Some(2),
+ "the twice-written doc's _version sort value must be 2, not a constant"
+ );
+ assert_eq!(
+ res.hits[0].id,
+ "d3",
+ "_version desc must put the twice-written doc first, got {:?}",
+ res.hits.iter().map(|h| &h.id).collect::<Vec<_>>()
+ );
+
+ // `_primary_term` / `_index` are constant in a single-shard, single-index
+ // engine, but they must be the CONSTANT the response meta-fields emit,
+ // never `null` — a null key silently ties the whole result set.
+ for (field, expected) in [
+ ("_primary_term", Value::from(1u64)),
+ ("_index", Value::from("meta_sort")),
+ ] {
+ let res = idx.search(&sorted_req(field, "asc", 50)).await.unwrap();
+ // Non-vacuity: an empty page would satisfy the loop below.
+ assert_eq!(res.hits.len(), 8, "sort on {field} must return the corpus");
+ for hit in &res.hits {
+ assert_eq!(
+ hit.sort.first(),
+ Some(&expected),
+ "sort on {field} must resolve, got {:?} for {}",
+ hit.sort,
+ hit.id
+ );
+ }
+ }
+}
+
+/// The reported impact: keyset pagination on `_seq_no`. With `[null]` sort
+/// values the cursor never advances, so this loop either re-reads page one
+/// forever (caught by the iteration budget) or returns fewer than the whole
+/// corpus.
+#[tokio::test]
+async fn search_after_on_seq_no_pages_the_whole_index() {
+ let dir = TempDir::new().unwrap();
+ let engine = make_engine(&dir);
+ engine.create_index("sa_seq", Schema::empty()).unwrap();
+ let idx = engine.get_index("sa_seq").unwrap();
+
+ const N: usize = 25;
+ for i in 0..N {
+ idx.index_document(Some(format!("d{i:03}")), json!({ "n": i }))
+ .await
+ .unwrap();
+ }
+ // Flush these to a segment, then buffer five more, so the paged walk
+ // spans both the segment and the memtable source.
+ idx.flush().await.unwrap();
+ for i in N..(N + 5) {
+ idx.index_document(Some(format!("d{i:03}")), json!({ "n": i }))
+ .await
+ .unwrap();
+ }
+ let total = N + 5;
+
+ let mut collected: Vec<String> = Vec::new();
+ let mut after: Option<Vec<Value>> = None;
+ // Budget: enough pages to drain the corpus, few enough that a stuck
+ // cursor trips it instead of hanging the suite.
+ for _ in 0..(total + 5) {
+ let mut body = json!({
+ "query": {"match_all": {}},
+ "size": 4,
+ "sort": [{"_seq_no": "asc"}],
+ });
+ if let Some(ref a) = after {
+ body["search_after"] = Value::Array(a.clone());
+ }
+ let res = idx.search(&parse_request(&body).unwrap()).await.unwrap();
+ if res.hits.is_empty() {
+ break;
+ }
+ after = res.hits.last().map(|h| h.sort.clone());
+ collected.extend(res.hits.iter().map(|h| h.id.clone()));
+ assert!(
+ collected.len() <= total,
+ "cursor is not advancing: {} ids collected from a {}-doc index \
+ (first repeat page starts at {:?})",
+ collected.len(),
+ total,
+ &collected[total.min(collected.len().saturating_sub(1))]
+ );
+ }
+
+ let mut unique = collected.clone();
+ unique.sort();
+ unique.dedup();
+ assert_eq!(
+ unique.len(),
+ total,
+ "_seq_no keyset pagination must visit every doc exactly once; \
+ collected {} ids, {} unique",
+ collected.len(),
+ unique.len()
+ );
+ assert_eq!(collected.len(), total, "no doc may be returned twice");
+}
+
+/// The candidate-cut half. `materialisation_limit` is
+/// `max(from + size + 100, 256)`, so a memtable with more docs than that
+/// exercises the bounded candidate path — which has no dv column to cut on
+/// for a meta-field (nor for `_id`) and must therefore not be used.
+#[tokio::test]
+async fn meta_field_sort_is_exact_past_the_memtable_candidate_cap() {
+ let dir = TempDir::new().unwrap();
+ let engine = make_engine(&dir);
+ engine.create_index("cap_sort", Schema::empty()).unwrap();
+ let idx = engine.get_index("cap_sort").unwrap();
+
+ // > materialisation_limit (256) buffered docs, none flushed.
+ const N: u64 = 2000;
+ for i in 0..N {
+ idx.index_document(Some(format!("d{i:04}")), json!({ "n": i }))
+ .await
+ .unwrap();
+ }
+
+ // Ids are lexicographically ordered with insertion order, so the
+ // expected page is the same for `_seq_no` and `_id`.
+ let newest: Vec<String> = (N - 5..N).rev().map(|i| format!("d{i:04}")).collect();
+ let oldest: Vec<String> = (0..5).map(|i| format!("d{i:04}")).collect();
+
+ for field in ["_seq_no", "_id"] {
+ let res = idx.search(&sorted_req(field, "desc", 5)).await.unwrap();
+ let ids: Vec<String> = res.hits.iter().map(|h| h.id.clone()).collect();
+ assert_eq!(
+ ids, newest,
+ "sort {field} desc over a {N}-doc memtable must return the true \
+ top 5, not an arbitrary prefix of the candidate cut"
+ );
+
+ let res = idx.search(&sorted_req(field, "asc", 5)).await.unwrap();
+ let ids: Vec<String> = res.hits.iter().map(|h| h.id.clone()).collect();
+ assert_eq!(ids, oldest, "sort {field} asc over a {N}-doc memtable");
+ }
+
+ // Control: a real `_source` field DOES have a memtable dv column, so the
+ // bounded candidate path stays enabled for it and must stay correct.
+ let res = idx.search(&sorted_req("n", "desc", 5)).await.unwrap();
+ let ids: Vec<String> = res.hits.iter().map(|h| h.id.clone()).collect();
+ assert_eq!(ids, newest, "control: plain numeric field sort");
+}
+
+/// Same guarantee once the docs live in a segment rather than the memtable —
+/// the segment side resolves `_seq_no` through the version map too.
+#[tokio::test]
+async fn seq_no_sort_is_exact_after_flush() {
+ let dir = TempDir::new().unwrap();
+ let engine = make_engine(&dir);
+ engine
+ .create_index("flushed_sort", Schema::empty())
+ .unwrap();
+ let idx = engine.get_index("flushed_sort").unwrap();
+
+ const N: u64 = 600;
+ for i in 0..N {
+ idx.index_document(Some(format!("d{i:04}")), json!({ "n": i }))
+ .await
+ .unwrap();
+ }
+ idx.flush().await.unwrap();
+
+ let res = idx.search(&sorted_req("_seq_no", "desc", 5)).await.unwrap();
+ let ids: Vec<String> = res.hits.iter().map(|h| h.id.clone()).collect();
+ let newest: Vec<String> = (N - 5..N).rev().map(|i| format!("d{i:04}")).collect();
+ assert_eq!(ids, newest, "flushed _seq_no desc page");
+ for hit in &res.hits {
+ assert_eq!(
+ hit.sort.first().and_then(Value::as_u64),
+ idx.lookup_seq_no(&hit.id),
+ "flushed hit {} must carry its real _seq_no",
+ hit.id
+ );
+ }
+}