AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
fix(engine): `_count` stops answering a `term` on a text field from the analysed term dictionary (#362)
xerj-orgeaeab4f8eb56
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c2591c4ef..513e693f8 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -204,6 +204,98 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0the candidate vectors and removes a lock acquisition; both are small againstthe encode/decode/similarity work the scan already does.+- **`_count` no longer reports documents `_search` cannot return**+ ([#362](https://github.com/xerj-org/xerj/issues/362)). `POST /{index}/_count`+ is `_search` at `size: 0`, so the two are supposed to answer the same+ question. On a `text` field they did not. A `text` mapping carries no+ doc-values, so the term-count shortcut had no column to read and answered+ from the segment's FTS term dictionary instead — analysed tokens, tokenised+ and lowercased — while `_search` resolves a `term` on such a field against+ the whole `_source` value. Two oracles, two questions, wrong in **both**+ directions on a flushed segment:++ | `term` query on a `text` field | old `_count` | `_search` |+ |---|---|---|+ | `{"term":{"title":"testsegmentreader.java"}}` | `1` | 0 hits |+ | `{"term":{"title":"quick"}}`, value `"the quick brown fox"` | `1` | 0 hits |+ | `{"term":{"title":"Doc7.java the quick brown fox"}}` — the byte-exact `_source` value | `0` | **1 hit** |++ The shortcut now abandons when a segment has no doc-values column for the+ field, and the ordinary delete-aware scan — the code that produces the hits —+ answers. That shortcut is not `_count`-only: its result also authorises the+ bounded stored scan at `size > 0`, so `hits.total` was affected too. On a+ 200,000-document index, with a `term` on a `text` field matching 10,000+ documents, `_search` at `size: 10` returned its 10 hits under+ `"total": {"value": 130}`, and `_count` for the same query answered `0`. Both+ now answer `10000`.++ **This costs real time, and the bill is not only "the wrong spellings".**+ Scan cost does not depend on how the term is spelled, so every `term` count on+ a `text` field with no doc-values now pays a full scan — the byte-exact+ spelling included. Its cost changes exactly as much as a misspelling's does,+ and per the table above its *value* changes too, from an undercount to the+ truth. Measured on this branch in the `ci-test` profile, one segment, 20+ distinct query values per shape to defeat the request cache (median of 20,+ same box, same corpus, only `index.rs` swapped):++ | `_count` shape, 200,000 docs | before | after |+ |---|---|---|+ | `term` on `text`, no doc-values | 4.2 ms | 356 ms |+ | same, byte-exact `_source` value | 4.1 ms (and answered `0`) | 356 ms (answers `1`) |+ | `term` on `keyword`, `doc_values: false` | 21 µs | 45 µs |+ | `term` on `keyword` with doc-values (control) | 347 µs | 216 µs |+ | `term` on `long`, `doc_values: false` | 383 ms | 353 ms |+ | `_search` at `size: 10`, `term` on `text` matching 10,000 docs | 9.4 ms (total `130`) | 225 ms (total `10000`) |++ At 50,000 documents the first row is 2.3 ms → 81 ms. That is roughly 35× at+ 50 k and 85× at 200 k, but the box carried load average 70–85 from unrelated+ work throughout, so treat the multiple as order-of-magnitude; the number to+ plan around is the absolute one, about a third of a second per `_count` at+ 200,000 documents.++ The middle rows are there to bound the blast radius, and they are the+ reassuring ones. A `keyword` field with `doc_values: false` reaches the same+ abandoned branch, but a `term` on a declared `keyword` field still projects+ into the FTS tree, so the segment scan's own `term_doc_freq` shortcut answers+ it: its **route** changed, its cost class did not, and it stays sub-linear+ rather than becoming a scan. A `long` with `doc_values: false` is unchanged+ because it was already scanning — a field with no FTS dictionary never reached+ the removed fallback in the first place. And every count already served from a+ doc-values column is untouched, which is what the control row shows.++ **One shape's `_search` page changes, not just its total.** With the shortcut+ gone the stored scan no longer stops after materialising `from + size`; it+ walks every match and then picks the page. For an *unsorted* `term` on a+ `text` field with many equally-scoring matches, that changes which documents+ come back — measured over 2,000 identically-scoring documents at `size: 10`,+ the ten ids differ before and after (and the total goes from `80` to `2000`).+ Every score involved is identical, so neither page is more correct than the+ other, but the change is observable. Queries with an explicit `sort` return+ byte-identical pages and identical totals before and after.++ **ES compatibility, stated rather than buried.** On the repaired shape this+ moves `_count` *away* from the Elasticsearch answer while making it+ self-consistent: ES answers `{"term":{"body":"quick"}}` with `1` against+ `"the quick brown fox"`, XERJ used to answer `1` as well — but with zero+ retrievable hits — and now answers `0`, which is what its own `_search`+ returns. A count no page of hits can reproduce is a coincidence, not+ compatibility. What `_search` *should* answer for a `term` on an analysed+ field is the separate, hit-set-moving decision tracked in+ [#397](https://github.com/xerj-org/xerj/issues/397); closing that is what+ makes both properties true at once, and is also the only route back to a+ sub-linear count for this shape.++ Two things this change does **not** do. It does not touch the other half of+ #362's thread — `case_insensitive` accepted and silently dropped on+ `prefix`/`wildcard` — so the issue's headline symptom survives:+ `{"term":{"title":"TestSegmentReader.java"}}` still answers `1` while+ `{"prefix":{"title":"TestSeg"}}` answers `0`. And it is not the narrower fix+ the issue thread proposed (dropping the lowercase retry the shortcut did on a+ dictionary miss); that was tried and measured, and it leaves the first two+ rows of the first table wrong while additionally making the byte-exact+ `{"term":{"title":"TestSegmentReader.java"}}` count `0` where `_search`+ returns `1`.+## [1.0.0-rc.17] - 2026-08-15### Addeddiff --git a/engine/crates/xerj-api/tests/count_agrees_with_search.rs b/engine/crates/xerj-api/tests/count_agrees_with_search.rsnew file mode 100644index 000000000..22730cc99--- /dev/null+++ b/engine/crates/xerj-api/tests/count_agrees_with_search.rs@@ -0,0 +1,198 @@+//! Issue #362, from the outside: `_count` must never report documents that+//! `_search` cannot return.+//!+//! `POST /{index}/_count` is implemented as `search({query, size: 0}).total`+//! (`es_compat::count_docs`), so the two endpoints are supposed to answer the+//! same question. On a `text` field they did not. A `text` mapping carries+//! `doc_values: false`, so the engine's term-count shortcut had no column to+//! read and fell back to the segment's FTS term dictionary — which holds+//! ANALYZED tokens (tokenised and lowercased by the standard analyzer), while+//! `_search` resolves a `term` on that field against the whole `_source`+//! value. Both spellings below counted a document that no `_search`, at any+//! `size`, could produce:+//!+//! ```text+//! {"term":{"title":"testsegmentreader.java"}} count 1, hits 0+//! {"term":{"title":"quick"}} count 1, hits 0+//! ```+//!+//! The second row has no case component at all, which is why it is here: it is+//! the case that proves the disagreement is not about casing.++use axum::body::Body;+use axum::http::{Request, StatusCode};+use serde_json::{json, Value};+use tower::ServiceExt;++async fn app() -> (axum::Router, tempfile::TempDir) {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = xerj_common::config::Config::default();+ config.server.data_dir = dir.path().to_string_lossy().into_owned();+ config.storage.wal_sync = xerj_common::config::WalSync::Async;+ let metrics = xerj_common::metrics::Metrics::new().expect("metrics");+ let engine = xerj_engine::Engine::new(config.clone()).expect("engine");+ let state = xerj_api::state::AppState::new(config, engine, metrics);+ (xerj_api::router::build_es_compat_router(state), dir)+}++async fn json_req(+ app: &axum::Router,+ method: &str,+ path: &str,+ body: Value,+) -> (StatusCode, Value) {+ let response = app+ .clone()+ .oneshot(+ Request::builder()+ .method(method)+ .uri(path)+ .header("content-type", "application/json")+ .body(Body::from(body.to_string()))+ .expect("request"),+ )+ .await+ .expect("response");+ let status = response.status();+ let bytes = axum::body::to_bytes(response.into_body(), usize::MAX)+ .await+ .expect("body");+ (+ status,+ serde_json::from_slice(&bytes).unwrap_or(Value::Null),+ )+}++/// Two documents, one `text` field, flushed to a segment — the shape the+/// issue was filed against (a source-file corpus with CamelCase names).+async fn seeded() -> (axum::Router, tempfile::TempDir) {+ let (app, dir) = app().await;+ let (status, body) = json_req(+ &app,+ "PUT",+ "/files",+ json!({+ "mappings": { "properties": {+ "title": { "type": "text" },+ "path": { "type": "keyword" }+ }}+ }),+ )+ .await;+ assert_eq!(status, StatusCode::OK, "create index failed: {body}");++ for (id, doc) in [+ (+ "java",+ json!({"title": "TestSegmentReader.java", "path": "lucene/TestSegmentReader.java"}),+ ),+ (+ "fox",+ json!({"title": "the quick brown fox", "path": "docs/fox.txt"}),+ ),+ ] {+ let (status, body) = json_req(&app, "PUT", &format!("/files/_doc/{id}"), doc).await;+ assert!(status.is_success(), "index {id} failed: {status} {body}");+ }+ // The count shortcut under test only runs over flushed segments.+ let (status, body) = json_req(&app, "POST", "/files/_flush", json!({})).await;+ assert!(status.is_success(), "flush failed: {status} {body}");+ (app, dir)+}++/// `_count` and `_search` must agree on both the total AND the number of+/// documents actually retrievable.+async fn assert_agrees(app: &axum::Router, query: Value, label: &str) {+ let (status, count_body) =+ json_req(app, "POST", "/files/_count", json!({ "query": query })).await;+ assert_eq!(+ status,+ StatusCode::OK,+ "{label}: _count failed {count_body}"+ );+ let count = count_body["count"].as_u64().expect("count is a number");++ let (status, search_body) = json_req(+ app,+ "POST",+ "/files/_search",+ json!({ "query": query, "size": 50 }),+ )+ .await;+ assert_eq!(+ status,+ StatusCode::OK,+ "{label}: _search failed {search_body}"+ );+ let total = search_body["hits"]["total"]["value"]+ .as_u64()+ .expect("total is a number");+ let hits = search_body["hits"]["hits"]+ .as_array()+ .expect("hits is an array")+ .len() as u64;++ assert_eq!(+ count, total,+ "{label}: _count said {count}, _search total said {total} for {query}"+ );+ assert_eq!(+ count, hits,+ "{label}: _count said {count} but _search returned {hits} document(s) for {query}"+ );+}++#[tokio::test]+async fn count_does_not_overreport_a_lowercased_term_on_a_text_field() {+ let (app, _dir) = seeded().await;+ assert_agrees(+ &app,+ json!({"term": {"title": "testsegmentreader.java"}}),+ "lowercased spelling",+ )+ .await;+}++#[tokio::test]+async fn count_does_not_overreport_an_analyzed_token_on_a_text_field() {+ let (app, _dir) = seeded().await;+ assert_agrees(&app, json!({"term": {"title": "quick"}}), "analyzed token").await;+}++/// The spelling that does match keeps working: no undercount in exchange.+#[tokio::test]+async fn count_still_agrees_on_the_exact_text_term() {+ let (app, _dir) = seeded().await;+ let query = json!({"term": {"title": "TestSegmentReader.java"}});+ assert_agrees(&app, query.clone(), "exact spelling").await;+ let (_, body) = json_req(&app, "POST", "/files/_count", json!({ "query": query })).await;+ assert_eq!(+ body["count"], 1,+ "exact-spelling term must still count its document: {body}"+ );+}++/// A `keyword` field has doc-values and never took the FTS fallback; it must+/// keep its exact, case-sensitive count.+#[tokio::test]+async fn keyword_term_count_is_unchanged() {+ let (app, _dir) = seeded().await;+ let query = json!({"term": {"path": "lucene/TestSegmentReader.java"}});+ assert_agrees(&app, query.clone(), "keyword field").await;+ let (_, body) = json_req(&app, "POST", "/files/_count", json!({ "query": query })).await;+ assert_eq!(body["count"], 1, "keyword term must count 1: {body}");++ let wrong_case = json!({"term": {"path": "LUCENE/TestSegmentReader.java"}});+ assert_agrees(&app, wrong_case.clone(), "keyword field, wrong case").await;+ let (_, body) = json_req(+ &app,+ "POST",+ "/files/_count",+ json!({ "query": wrong_case }),+ )+ .await;+ assert_eq!(+ body["count"], 0,+ "keyword term must stay case-sensitive: {body}"+ );+}diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rsindex fcc9bf55b..80311f9b4 100644--- a/engine/crates/xerj-engine/src/index.rs+++ b/engine/crates/xerj-engine/src/index.rs@@ -19399,14 +19399,15 @@ impl Index {////// Supported shapes:/// * `MatchAll` — sum of segment `doc_count` + memtable doc count.- /// * `Term { field, value }` — for a keyword field:+ /// * `Term { field, value }` — for a field with doc-values:/// - Counts matches in the memtable via `doc_values_keyword_column`.- /// - For each on-disk segment, calls `FtsIndexReader::term_doc_freq`- /// on the raw value; if the segment analyzer lowercased terms,- /// falls back to the lowercased form.- /// - If the segment has no FTS data for the field at all, the- /// shortcut is abandoned (return `None`) because the segment- /// would otherwise undercount.+ /// - For each on-disk segment, counts through that segment's+ /// doc-values column — the same values `_search` compares against.+ /// - If a segment has no column for the field, the shortcut is+ /// abandoned (return `None`) and the scan answers. It used to+ /// consult the FTS term dictionary instead, whose ANALYZED terms+ /// are not what `_search` matches a `term` against, so the count+ /// could exceed the retrievable hit set (#362).////// Exact match count for a `bool` that carries `should` and/or `must_not`/// clauses — the shapes the fused must/filter-intersection arm bails on.@@ -20265,15 +20266,14 @@ impl Index {// 2. Keyword column → `doc_freq(term)` is an O(n) ord scan.// 3. Numeric column → `range_count(v, v, true, true)` bisects// the sorted index in O(log n).- // 4. Fall back to FTS `term_doc_freq` if both columns are- // missing for this segment.- // 5. If FTS is missing too, abandon — we can't answer safely.+ // 4. Neither column present for this segment → abandon (#362).+ // The FTS term dictionary used to answer here and it is the+ // wrong oracle; see the comment on that branch below.let segments_dir = self.data_dir.join("segments");let raw = match value {Value::String(s) => s.as_str().to_string(),other => other.to_string(),};- let lowered = raw.to_ascii_lowercase();// Same bool coercion as the memtable side above — `value.as_f64()`// alone returns `None` for a JSON boolean, which used to make// `served_by_dv` false for every segment whose on-disk column for@@ -20316,25 +20316,63 @@ impl Index {continue;}- // Fallback: FTS term dictionary. This is the slow path that- // parses meta.json; we only take it when the field isn't in- // the segment's doc-values (e.g. text fields with positions).- let reader = match FtsIndexReader::open(&segments_dir, &meta.id, &[field]) {- Ok(r) => r,- Err(_) => return None, // FST missing — abandon- };- reader.field_stats(field)?;- let df = reader- .term_doc_freq(field, &raw)- .or_else(|| {- if raw == lowered {- None- } else {- reader.term_doc_freq(field, &lowered)- }- })- .unwrap_or(0);- seg_matches = seg_matches.saturating_add(df as u64);+ // No doc-values column for this field in this segment: ABANDON+ // (#362). This used to fall back to the segment's FTS term+ // dictionary, which answers a DIFFERENT question than the one+ // `_search` answers, so `_count` published totals no page of hits+ // could reproduce:+ //+ // term title=testsegmentreader.java _count 1, _search total 0+ // term title=quick, "the quick brown fox"+ // _count 1, _search total 0+ //+ // A field only reaches here because it has no column, which on+ // this engine means an analyzed `text` field (`es_compat`'s+ // `doc_values_default`) or an explicit `doc_values: false`. Its+ // term dictionary holds ANALYZED tokens — `build_fts_field_configs`+ // gives `Text` the standard analyzer, so the entries are+ // tokenized and lowercased — while `query_node_to_fts_with_+ // keyword_fields` declines the FTS projection for a bare `Term`+ // and `_search` resolves it through `doc_matches_query`, comparing+ // the WHOLE `_source` value. A dictionary hit therefore says+ // nothing about whether the scan will produce a document.+ //+ // The lowercase re-spelling this replaces (`term_doc_freq(raw)`+ // `.or_else(term_doc_freq(lowered))`) was scaffolding over that:+ // it retried with a term the caller never wrote, which is what+ // made `term` look case-normalising next to `prefix`/`wildcard`+ // and is what #362 was filed as. Deleting only the retry does+ // not fix this — the `quick` row above carries no casing at all,+ // its dictionary hit is genuine, and the byte-exact spelling+ // would start counting 0 where `_search` still answers 1. The+ // oracle is wrong, not its spelling.+ //+ // Lucene's counts cannot drift this way: the sub-linear answer is+ // a method on the same `Weight` that builds the scorer, and its+ // default is "I cannot answer" — `Weight::count`+ // (lucene/core/src/java/org/apache/lucene/search/Weight.java:198)+ // returns -1, documented as the count not being computable in+ // sub-linear time, so `IndexSearcher::count`+ // (lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java:495)+ // collects for real. The specialisation that does answer,+ // `TermQuery::TermWeight::count`+ // (lucene/core/src/java/org/apache/lucene/search/TermQuery.java:260),+ // reads `termsEnum.docFreq()` off the query's OWN term state and+ // hands straight back to `super.count` once deletions make that+ // stop being the live answer. Returning `None` here is that+ // hand-back: the caller's delete-aware scan — the code that+ // materialises the hits — answers instead.+ //+ // Cost: a `term` count on a text field is a scan again. That is+ // the price of the number being true, and it is paid only by the+ // shape that was wrong.+ //+ // (What `_search` SHOULD do for a `term` on an analyzed field is+ // a separate question — ES matches the analyzed dictionary, so+ // `quick` would find "the quick brown fox" there. Changing that+ // moves the hit set and needs its own issue; this change only+ // makes `_count` agree with whatever `_search` does today.)+ return None;}Some(mem_matches + seg_matches)diff --git a/engine/crates/xerj-engine/tests/count_matches_search_on_text_term.rs b/engine/crates/xerj-engine/tests/count_matches_search_on_text_term.rsnew file mode 100644index 000000000..259989ac6--- /dev/null+++ b/engine/crates/xerj-engine/tests/count_matches_search_on_text_term.rs@@ -0,0 +1,176 @@+//! Regression tests for issue #362: `_count` must answer the same question+//! `_search` answers.+//!+//! `POST /{index}/_count` is `search({query, size: 0}).total`, so a `size: 0`+//! total that disagrees with the `size > 0` hit set is the HTTP-visible bug:+//! a count describing documents nobody can retrieve.+//!+//! Pre-fix, `try_shortcut_count`'s Term arm fell back to the segment FTS term+//! dictionary whenever the field had no doc-values column — which is exactly+//! what an analyzed `text` field is. The dictionary holds ANALYZED tokens+//! (standard analyzer: tokenised, lowercased), while `_search` resolves a+//! `term` on such a field against the whole `_source` value. Two documents,+//! two spellings, both counted and neither retrievable.++use serde_json::{json, Value};+use tempfile::TempDir;+use xerj_common::config::Config;+use xerj_common::types::{FieldConfig, FieldType, Schema};+use xerj_engine::{Engine, Index};+use xerj_query::parse_request;++fn make_engine(dir: &TempDir) -> Engine {+ let mut config = Config::default();+ config.server.data_dir = dir.path().to_str().unwrap().to_string();+ Engine::new(config).expect("engine::new")+}++/// What `POST /_count` runs: `size: 0`, total only (`es_compat::count_docs`).+async fn count_total(idx: &Index, q: &Value) -> u64 {+ let req = parse_request(&json!({ "query": q, "size": 0, "from": 0 })).expect("parse_request");+ idx.search(&req).await.expect("count search").total.value+}++/// What `POST /_search` runs: a real page of hits.+async fn search_hits(idx: &Index, q: &Value) -> (u64, usize) {+ let req = parse_request(&json!({ "query": q, "size": 50 })).expect("parse_request");+ let r = idx.search(&req).await.expect("search");+ (r.total.value, r.hits.len())+}++async fn seed() -> (TempDir, std::sync::Arc<Index>) {+ let dir = TempDir::new().unwrap();+ let engine = make_engine(&dir);+ let mut schema = Schema::empty();+ // `"type": "text"` from an ES mapping lands here with `doc_values: false`+ // (`es_compat::es_properties_to_fields`), which is precisely what sends a+ // term count for this field into the FTS term-dictionary fallback.+ let mut title = FieldConfig::new("title", FieldType::Text);+ title.options.doc_values = false;+ schema.fields.push(title);+ schema+ .fields+ .push(FieldConfig::new("path", FieldType::Keyword));+ // A keyword field with `doc_values: false` reaches the same+ // no-column branch as `title`, so the abandon must not cost it its+ // exact count — it is the shape that would turn an overcount into an+ // undercount if the branch were narrowed the other way.+ let mut code = FieldConfig::new("code", FieldType::Keyword);+ code.options.doc_values = false;+ schema.fields.push(code);+ engine.create_index("count-parity", schema).unwrap();+ let idx = engine.get_index("count-parity").unwrap();++ idx.index_document(+ Some("java".into()),+ json!({+ "title": "TestSegmentReader.java",+ "path": "lucene/TestSegmentReader.java",+ "code": "AB-1234"+ }),+ )+ .await+ .unwrap();+ idx.index_document(+ Some("fox".into()),+ json!({"title": "the quick brown fox", "path": "docs/fox.txt", "code": "ab-1234"}),+ )+ .await+ .unwrap();+ // The count shortcut only runs against flushed segments; the memtable+ // arm is a different code path and already agrees with the scan.+ idx.flush().await.unwrap();+ (dir, idx)+}++/// The `_count` / `_search` disagreement, one query per row.+async fn assert_count_agrees_with_search(idx: &Index, q: Value, label: &str) {+ let count = count_total(idx, &q).await;+ let (total, hits) = search_hits(idx, &q).await;+ assert_eq!(+ count, total,+ "{label}: _count total {count} != _search total {total} for {q}"+ );+ assert_eq!(+ count, hits as u64,+ "{label}: _count says {count} but _search can materialise only {hits} document(s) for {q}"+ );+}++/// Issue #362's own repro: a term whose spelling differs from `_source`+/// only in case. The dictionary retry made `_count` find it; `_search`+/// compares against the raw `_source` value and cannot.+#[tokio::test]+async fn count_does_not_overreport_lowercased_term_on_text_field() {+ let (_dir, idx) = seed().await;+ assert_count_agrees_with_search(+ &idx,+ json!({"term": {"title": "testsegmentreader.java"}}),+ "lowercased spelling",+ )+ .await;+}++/// The half with no case component at all: an ANALYZED token of a+/// multi-word text value. The dictionary hit here is genuine, which is why+/// deleting the lowercase retry alone would not have fixed it.+#[tokio::test]+async fn count_does_not_overreport_analyzed_token_on_text_field() {+ let (_dir, idx) = seed().await;+ assert_count_agrees_with_search(+ &idx,+ json!({"term": {"title": "quick"}}),+ "analyzed token of a multi-word value",+ )+ .await;+}++/// The exact spelling still counts, and still returns its document — the+/// fix must not trade the overcount for an undercount.+#[tokio::test]+async fn count_still_finds_the_exact_text_term() {+ let (_dir, idx) = seed().await;+ let q = json!({"term": {"title": "TestSegmentReader.java"}});+ assert_count_agrees_with_search(&idx, q.clone(), "exact spelling").await;+ assert_eq!(+ count_total(&idx, &q).await,+ 1,+ "exact-spelling term must still count its document"+ );+}++/// A `keyword` field keeps its exact, byte-comparing count: that field has+/// a doc-values column and never took the FTS fallback.+#[tokio::test]+async fn keyword_term_count_is_unchanged() {+ let (_dir, idx) = seed().await;+ let q = json!({"term": {"path": "lucene/TestSegmentReader.java"}});+ assert_count_agrees_with_search(&idx, q.clone(), "keyword field").await;+ assert_eq!(count_total(&idx, &q).await, 1, "keyword term must count 1");+ assert_eq!(+ count_total(+ &idx,+ &json!({"term": {"path": "LUCENE/TestSegmentReader.java"}})+ )+ .await,+ 0,+ "keyword term must stay case-sensitive"+ );+}++/// The abandon is not a licence to undercount: a `keyword` field with+/// `doc_values: false` takes the very same no-column branch and must still+/// count exactly one document per spelling, case-sensitively.+#[tokio::test]+async fn keyword_without_doc_values_still_counts_exactly() {+ let (_dir, idx) = seed().await;+ for (spelling, expected) in [("AB-1234", 1), ("ab-1234", 1), ("zz-9", 0)] {+ let q = json!({"term": {"code": spelling}});+ assert_count_agrees_with_search(&idx, q.clone(), "keyword, doc_values:false").await;+ assert_eq!(+ count_total(&idx, &q).await,+ expected,+ "keyword without doc-values: wrong count for {spelling}"+ );+ }+}