Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(es-compat): let _explain and _msearch carry the index:false rejection

xerj-orge64d81edd5a8
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fe6c2650e..5c38f4ab1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -25,9 +25,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
is unsearchable, and a query naming one is now rejected with ES's own
sentence, `Cannot search on field [f] since it is not indexed nor has doc
values.`, as a 400 `search_phase_execution_exception` /
- `query_shard_exception`. `_search`, `_count`, `_msearch` and `_explain` all
- inherit the check. `exists` is unaffected and still returns zero hits rather
- than an error, which is also what ES does.
+ `query_shard_exception`. The check lives in one place — `search_inner` — and
+ `_search`, `_count`, `_msearch` and `_explain` were each measured carrying it
+ all the way to the wire: `_explain` answers the 400 instead of a confident
+ `"matched": false` for a query it never ran, and `_msearch` renders it as a
+ per-response 400 envelope with `type` and `root_cause` instead of a bare 500.
+ `exists` is unaffected and still returns zero hits rather than an error, which
+ is also what ES does.
+
+ The multi-field query types follow ES's own split rather than a blanket rule.
+ An explicitly *named* field is rejected like any other named field —
+ `multi_match` / `simple_query_string` with `"fields": ["note"]`, `query_string`
+ with `"default_field": "note"` — while a *wildcard* spec such as
+ `"fields": ["*"]` is never an error and goes on being answered exactly as
+ before, because ES expands a pattern over the searchable fields and silently
+ drops the rest (`QueryParserHelper.resolveMappingField`). Before this,
+ the same user intent got two different answers: the `simple_query_string` form
+ was rejected (the parser lowers it to a `match`) while the `multi_match` form
+ returned the document through the unsearchable field.
The field is dropped from the full-text index at flush and merge only where
the stored-doc fallback is *equivalent* — a non-indexed `text` field. An
@@ -37,9 +52,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
change: for those types the footprint the option implies is knowingly forgone
rather than bought with wrong answers.
- Known gap, unchanged by this release: the field-less `query_string` arm of the
- stored-document scan is schema-free and walks every `_source` key, so a token
- living only in an unsearchable field can still match there.
+ Known gaps, measured and left open rather than claimed shut. **Aggregations
+ and sort do not inherit the check**: only the `query` clause is walked, so
+ `{"size":0,"aggs":{"a":{"terms":{"field":"note"}}}}` still answers 200 with a
+ bucket built from `_source`, and `{"sort":[{"note":"asc"}]}` still answers 200.
+ ES rejects those too, but for an unrelated reason and with an unrelated
+ sentence — `Fielddata is disabled on [f] in [i]…`, which it raises for *any*
+ `text` field whether or not `index: false` was declared — so reusing this
+ error there would misreport a wider divergence as this one. **The field-less
+ arms of the stored-document scan are schema-free**, walking every `_source`
+ key, so a token living only in an unsearchable field can still match when
+ **no** field is named: `query_string {"query":"…"}` and `more_like_this` with
+ no `fields` both still return the document. (Their *named* forms are
+ rejected — `more_like_this` with `fields` lowers to a `bool.should` of
+ `match` at parse time and inherits the check.) Both gaps are pinned by
+ assertions in
+ `index_false_is_honoured::documented_gaps_are_still_open_and_still_documented`,
+ so closing one fails the test that says it is open.
- **`autoindex` no longer leaves immortal catalog entries for skipped files**
([#238](https://github.com/xerj-org/xerj/issues/238)). A file added after the
diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rs
index 315358f9d..552788657 100644
--- a/engine/crates/xerj-api/src/es_compat.rs
+++ b/engine/crates/xerj-api/src/es_compat.rs
@@ -18889,11 +18889,12 @@ async fn msearch_impl(
.map_err(|e| xerj_common::XerjError::invalid_query(e.to_string()))
{
Ok(r) => r,
+ // Rendered through the same `ApiError` path the single-request
+ // `_search` uses, so a sub-response carries ES's full envelope
+ // (`root_cause`, `type`, `reason`) and not a bare `reason`. See
+ // the `search_error` arm below for why that matters.
Err(e) => {
- responses.push(json!({
- "error": { "reason": e.to_string() },
- "status": 400
- }));
+ responses.push(ApiError::new(e).into_value());
continue;
}
};
@@ -18929,14 +18930,14 @@ async fn msearch_impl(
let mut total_count: u64 = 0;
let mut total_relation = "eq";
let mut merged_aggs: Option<Value> = None;
- let mut search_error: Option<String> = None;
+ let mut search_error: Option<xerj_common::XerjError> = None;
let mut script_failure: Option<String> = None;
for idx_name in &index_names {
let idx = match state.engine.get_index(idx_name) {
Ok(i) => i,
Err(e) => {
- search_error = Some(e.to_string());
+ search_error = Some(xerj_common::XerjError::from(e));
break;
}
};
@@ -18964,7 +18965,7 @@ async fn msearch_impl(
}
}
Err(e) => {
- search_error = Some(e.to_string());
+ search_error = Some(xerj_common::XerjError::from(e));
break;
}
}
@@ -18975,11 +18976,16 @@ async fn msearch_impl(
continue;
}
+ // The sub-response carries the SAME body the single-request `_search`
+ // would have produced for this error, status and all: `ApiError`
+ // already maps every `XerjError` to its ES exception type, status and
+ // `root_cause` (see `api/error.rs`). It used to be flattened to a bare
+ // `{"reason": …}` under a hardcoded 500, which turned a plain user
+ // mapping declaration — a query naming an `"index": false` field, a
+ // 400 `search_phase_execution_exception` everywhere else — into a
+ // server error with no `type` for a client to switch on.
if let Some(err) = search_error {
- responses.push(json!({
- "error": { "reason": err },
- "status": 500
- }));
+ responses.push(ApiError::new(err).into_value());
continue;
}
@@ -27945,13 +27951,22 @@ pub async fn explain_doc(
// Reporting the fault alongside the verdict satisfies both: the caller
// still gets the answer, and never gets it without being told a limit
// tripped.
- let mut script_failure: Option<String> = None;
- let matched = match idx.search(&ids_req).await {
- Ok(result) => {
- script_failure = result.script_failure.clone();
- !result.hits.is_empty()
- }
- Err(_) => false,
+ let (matched, script_failure) = match idx.search(&ids_req).await {
+ Ok(result) => (!result.hits.is_empty(), result.script_failure),
+ // The engine REFUSED to run the query — e.g. it names a field the
+ // mapping declared `"index": false` with no doc values, which ES
+ // rejects with `Cannot search on field [f] since it is not indexed
+ // nor has doc values.` Publishing `matched: false` here would be a
+ // confident negative for a question that was never asked, which is
+ // the accepted-and-ignored failure of #204 wearing a diagnostic hat.
+ // Every other error path in this handler already returns `ApiError`;
+ // so does this one, giving `_explain` the same 400 envelope
+ // `_search` and `_count` return for the identical query.
+ //
+ // Deliberately NOT the same as `script_failure` above: there the
+ // engine DID produce a hit list and only scoring was degraded, so the
+ // verdict is real and the fault is reported alongside it.
+ Err(e) => return ApiError::new(xerj_common::XerjError::from(e)).into_response(),
};
let score = if matched { 1.0_f64 } else { 0.0_f64 };
diff --git a/engine/crates/xerj-api/tests/index_false_is_honoured.rs b/engine/crates/xerj-api/tests/index_false_is_honoured.rs
index d6ffae8e1..20ed82aaa 100644
--- a/engine/crates/xerj-api/tests/index_false_is_honoured.rs
+++ b/engine/crates/xerj-api/tests/index_false_is_honoured.rs
@@ -318,3 +318,327 @@ async fn an_alias_to_an_unsearchable_field_is_rejected_too() {
"the error names the resolved target, as ES does: {reason:?}"
);
}
+
+// ─────────────────────────────────────────────────────────────────────────────
+// The other read surfaces that route through `search_inner`
+//
+// The check lives in one place, but "one place" is only worth claiming if the
+// endpoints on top of it actually let the rejection through. Two of them did
+// not, and both failures were the same silent-answer shape #204 is about:
+// `_explain` swallowed the engine's refusal and published `matched: false`,
+// and `_msearch` flattened it into a bare 500 with no `type` to switch on.
+// ─────────────────────────────────────────────────────────────────────────────
+
+async fn ndjson_req(app: &axum::Router, path: &str, body: &str) -> (StatusCode, Value) {
+ send(
+ app,
+ Request::builder()
+ .method("POST")
+ .uri(path)
+ .header("content-type", "application/x-ndjson")
+ .body(Body::from(body.to_string()))
+ .expect("request"),
+ )
+ .await
+}
+
+/// `_count` runs the same query with no hits to return, so a swallowed
+/// rejection would show up as `count: 0` — a confident wrong number.
+#[tokio::test]
+async fn count_inherits_the_rejection() {
+ let (app, _dir) = app_with_mapping().await;
+
+ let (status, body) = json_req(
+ &app,
+ "POST",
+ "/t/_count",
+ json!({ "query": { "match": { "note": "pelicans" } } }),
+ )
+ .await;
+ assert_eq!(status, StatusCode::BAD_REQUEST, "{body}");
+ assert_eq!(
+ body.pointer("/error/root_cause/0/type")
+ .and_then(Value::as_str),
+ Some("query_shard_exception"),
+ "{body}"
+ );
+}
+
+/// `_explain` must not answer `matched: false` for a query the engine refused
+/// to run. That is a *confident negative* — indistinguishable, to the caller,
+/// from "we ran it and your document does not match" — and it is what this
+/// endpoint returned until the `Err(_) => false` arm in `explain_doc` was
+/// replaced by the same `ApiError` every other arm of that handler uses.
+#[tokio::test]
+async fn explain_reports_the_rejection_instead_of_a_confident_no_match() {
+ let (app, _dir) = app_with_mapping().await;
+
+ let (status, body) = json_req(
+ &app,
+ "POST",
+ "/t/_explain/1",
+ json!({ "query": { "match": { "note": "pelicans" } } }),
+ )
+ .await;
+ assert_eq!(
+ status,
+ StatusCode::BAD_REQUEST,
+ "_explain must inherit the rejection, got {status}: {body}"
+ );
+ assert_eq!(
+ body.pointer("/matched"),
+ None,
+ "no verdict is published: {body}"
+ );
+ assert_eq!(
+ body.pointer("/error/type").and_then(Value::as_str),
+ Some("search_phase_execution_exception"),
+ "{body}"
+ );
+ let reason = body
+ .pointer("/error/root_cause/0/reason")
+ .and_then(Value::as_str)
+ .unwrap_or_default();
+ assert!(
+ reason
+ .contains("Cannot search on field [note] since it is not indexed nor has doc values."),
+ "got {reason:?}: {body}"
+ );
+
+ // An answerable query on the same document still gets its verdict — the
+ // change must not turn `_explain` into a refusal machine.
+ let (status, body) = json_req(
+ &app,
+ "POST",
+ "/t/_explain/1",
+ json!({ "query": { "match": { "body": "pelicans" } } }),
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert_eq!(body.pointer("/matched"), Some(&json!(true)), "{body}");
+}
+
+/// `_msearch` answers 200 overall and puts per-request outcomes in
+/// `responses[]`, so the rejection has to arrive there in ES's shape: status
+/// 400, a `type`, and a `root_cause`. It used to arrive as
+/// `{"error":{"reason":"…"},"status":500}` — a server error for a mapping the
+/// user declared, with nothing structured for a client to branch on.
+#[tokio::test]
+async fn msearch_renders_the_rejection_as_a_per_response_400_envelope() {
+ let (app, _dir) = app_with_mapping().await;
+
+ let body_ndjson = concat!(
+ "{\"index\":\"t\"}\n",
+ "{\"query\":{\"match\":{\"note\":\"pelicans\"}}}\n",
+ "{\"index\":\"t\"}\n",
+ "{\"query\":{\"match\":{\"body\":\"pelicans\"}}}\n"
+ );
+ let (status, body) = ndjson_req(&app, "/_msearch", body_ndjson).await;
+ assert_eq!(status, StatusCode::OK, "_msearch envelope is 200: {body}");
+
+ assert_eq!(
+ body.pointer("/responses/0/status").and_then(Value::as_u64),
+ Some(400),
+ "the failing sub-request must be a 400, not a 500: {body}"
+ );
+ assert_eq!(
+ body.pointer("/responses/0/error/type")
+ .and_then(Value::as_str),
+ Some("search_phase_execution_exception"),
+ "{body}"
+ );
+ assert_eq!(
+ body.pointer("/responses/0/error/root_cause/0/type")
+ .and_then(Value::as_str),
+ Some("query_shard_exception"),
+ "{body}"
+ );
+ let reason = body
+ .pointer("/responses/0/error/root_cause/0/reason")
+ .and_then(Value::as_str)
+ .unwrap_or_default();
+ assert!(
+ reason
+ .contains("Cannot search on field [note] since it is not indexed nor has doc values."),
+ "got {reason:?}: {body}"
+ );
+
+ // The sibling sub-request is unaffected: one bad query in a batch must not
+ // cost the others their answers.
+ assert_eq!(
+ body.pointer("/responses/1/hits/total/value")
+ .and_then(Value::as_u64),
+ Some(1),
+ "{body}"
+ );
+}
+
+/// ES's split for the multi-field types: an explicitly NAMED field is kept
+/// whatever its type and fails in the per-field builder, while a WILDCARD spec
+/// silently drops the fields that are not searchable
+/// (`QueryParserHelper.resolveMappingField`, semantics only). Before this,
+/// `multi_match {"fields":["note"]}` returned the document through the
+/// unsearchable field while `simple_query_string {"fields":["note"]}` — the
+/// same intent, lowered to `Match` by the parser — was rejected.
+#[tokio::test]
+async fn multi_field_queries_reject_a_named_unsearchable_field_but_not_a_pattern() {
+ let (app, _dir) = app_with_mapping().await;
+
+ for query in [
+ json!({ "multi_match": { "query": "pelicans", "fields": ["note"] } }),
+ json!({ "multi_match": { "query": "pelicans", "fields": ["note", "body"] } }),
+ // A boost suffix is part of the spec, not part of the field name.
+ json!({ "multi_match": { "query": "pelicans", "fields": ["note^3"] } }),
+ json!({ "simple_query_string": { "query": "pelicans", "fields": ["note"] } }),
+ json!({ "simple_query_string": { "query": "pelicans", "fields": ["note", "body"] } }),
+ json!({ "query_string": { "query": "pelicans", "default_field": "note" } }),
+ ] {
+ let (status, body) = search(&app, query.clone()).await;
+ assert_eq!(
+ status,
+ StatusCode::BAD_REQUEST,
+ "query {query} names an unsearchable field, got {status}: {body}"
+ );
+ let reason = body
+ .pointer("/error/root_cause/0/reason")
+ .and_then(Value::as_str)
+ .unwrap_or_default();
+ assert!(
+ reason.contains("Cannot search on field [note]"),
+ "got {reason:?} for {query}: {body}"
+ );
+ }
+
+ // …while a pattern is never an error, even though `note` matches it: ES
+ // drops the unsearchable fields out of a wildcard expansion instead of
+ // failing the request, and turning `fields: ["*"]` into a hard error for
+ // the whole index is exactly what this arm exists to avoid.
+ for query in [
+ json!({ "multi_match": { "query": "pelicans", "fields": ["*"] } }),
+ json!({ "multi_match": { "query": "pelicans", "fields": ["note*"] } }),
+ json!({ "multi_match": { "query": "pelicans", "fields": ["*", "body"] } }),
+ json!({ "simple_query_string": { "query": "pelicans", "fields": ["*"] } }),
+ json!({ "query_string": { "query": "pelicans", "default_field": "*" } }),
+ ] {
+ let (status, body) = search(&app, query.clone()).await;
+ assert_eq!(
+ status,
+ StatusCode::OK,
+ "a wildcard spec must not be an error: {query} → {status}: {body}"
+ );
+ }
+
+ // Where a concrete searchable field is named alongside the pattern, the
+ // document still comes back — the arm drops clauses, it does not disarm
+ // the query. (A *bare* `["*"]` answers 0 hits in XERJ today; that is a
+ // pre-existing gap in wildcard field expansion, unrelated to this change
+ // — an index with no `index: false` field at all answers it the same way
+ // — so it is not asserted as a hit here.)
+ for query in [
+ json!({ "multi_match": { "query": "pelicans", "fields": ["*", "body"] } }),
+ json!({ "query_string": { "query": "pelicans", "default_field": "*" } }),
+ ] {
+ let (status, body) = search(&app, query.clone()).await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert_eq!(
+ body.pointer("/hits/total/value").and_then(Value::as_u64),
+ Some(1),
+ "the searchable `body` field still answers {query}: {body}"
+ );
+ }
+}
+
+/// The gaps this change does NOT close, pinned so the code and the claim
+/// cannot drift apart. These assertions describe present behaviour that is
+/// wrong-by-ES and is documented as open in the CHANGELOG — a failure here
+/// means a gap was closed and the disclosure needs deleting, not that the
+/// build is broken.
+#[tokio::test]
+async fn documented_gaps_are_still_open_and_still_documented() {
+ let (app, _dir) = app_with_mapping().await;
+
+ // GAP 1 — aggregations do not inherit the check. `request.aggs` is opaque
+ // JSON below the API layer and only the `query` clause is walked. (ES
+ // rejects this too, but for an unrelated reason and with an unrelated
+ // sentence: `Fielddata is disabled on [note] in [t]…`, which it raises for
+ // ANY text field, `index: false` or not.)
+ let (status, body) = json_req(
+ &app,
+ "POST",
+ "/t/_search",
+ json!({ "size": 0, "aggs": { "a": { "terms": { "field": "note" } } } }),
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "GAP 1 changed: {body}");
+ assert!(
+ body.pointer("/aggregations/a/buckets/0/key").is_some(),
+ "GAP 1 changed — aggregations now inherit the check, update the CHANGELOG: {body}"
+ );
+
+ // …and sorting on the same field is likewise not checked.
+ let (status, body) = json_req(
+ &app,
+ "POST",
+ "/t/_search",
+ json!({ "sort": [ { "note": "asc" } ] }),
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "GAP 1 (sort half) changed: {body}");
+
+ // GAP 2 — the FIELD-LESS arms of the stored-document scan are schema-free
+ // and walk every `_source` key, so a token living only in an unsearchable
+ // field still matches when NO field is named.
+ for query in [
+ json!({ "query_string": { "query": "pelicans" } }),
+ json!({ "more_like_this": { "like": ["pelicans"], "min_term_freq": 1, "max_query_terms": 5 } }),
+ ] {
+ let (status, body) = search(&app, query.clone()).await;
+ assert_eq!(status, StatusCode::OK, "GAP 2 changed for {query}: {body}");
+ }
+}
+
+/// `more_like_this` needs no arm of its own: with an explicit `fields` list the
+/// parser lowers it to a `bool.should` of `match` clauses, so it inherits the
+/// leaf check. Asserted because the CHANGELOG says so — the claim and the code
+/// have to be checkable against each other.
+#[tokio::test]
+async fn a_named_more_like_this_field_inherits_the_check() {
+ let (app, _dir) = app_with_mapping().await;
+
+ for fields in [json!(["note"]), json!(["note", "body"])] {
+ let query = json!({
+ "more_like_this": {
+ "fields": fields, "like": ["pelicans"],
+ "min_term_freq": 1, "max_query_terms": 5
+ }
+ });
+ let (status, body) = search(&app, query.clone()).await;
+ assert_eq!(status, StatusCode::BAD_REQUEST, "{query} → {body}");
+ let reason = body
+ .pointer("/error/root_cause/0/reason")
+ .and_then(Value::as_str)
+ .unwrap_or_default();
+ assert!(
+ reason.contains("Cannot search on field [note]"),
+ "got {reason:?}: {body}"
+ );
+ }
+
+ // The searchable control field still answers it.
+ let (status, body) = search(
+ &app,
+ json!({
+ "more_like_this": {
+ "fields": ["body"], "like": ["pelicans"],
+ "min_term_freq": 1, "max_query_terms": 5
+ }
+ }),
+ )
+ .await;
+ assert_eq!(status, StatusCode::OK, "{body}");
+ assert_eq!(
+ body.pointer("/hits/total/value").and_then(Value::as_u64),
+ Some(1),
+ "{body}"
+ );
+}
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index 8771f1777..de0cc8a64 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -26925,25 +26925,63 @@ fn declared_field<'a>(schema: &'a Schema, field: &str) -> Option<&'a FieldConfig
/// unchanged: such a field keeps its whole-value postings, because they are
/// what answers it *exactly* — see `memtable::fts_excluded_fields`.
///
-/// Deliberately NOT covered: the multi-field query types (`multi_match`,
-/// `query_string`, `simple_query_string`, `more_like_this`). ES throws when
-/// any *named* field is unsearchable, but XERJ expands wildcard field specs
-/// against the schema, so rejecting here would turn `fields: ["*"]` into a
-/// hard error for the whole index.
+/// The multi-field query types (`multi_match`, `simple_query_string`,
+/// `query_string`) follow ES's own split, not a blanket rule: an
+/// explicitly *named* field is kept whatever its type and left to fail in the
+/// per-field builder, while a *wildcard* spec silently drops the fields that
+/// are not searchable (`acceptAllTypes = isSimpleMatchPattern(spec) == false`
+/// in `server/src/main/java/org/elasticsearch/index/search/QueryParserHelper.java:115-150`,
+/// read for semantics only — ES is AGPL/SSPL/Elastic-2.0 and no code from it
+/// is reproduced here). So `fields: ["note"]` is rejected exactly like
+/// `match` on `note`, and `fields: ["*"]` keeps working over the searchable
+/// fields. Without this, the same user intent got two different answers:
+/// `simple_query_string {"fields":["note"]}` and `query_string
+/// {"default_field":"note"}` already 400ed because the parser lowers a
+/// single-field form to `Match`, while `multi_match {"fields":["note"]}` —
+/// and either of those with two fields — returned the document *through* the
+/// unsearchable field.
///
-/// The FTS route for those queries no longer projects a clause onto an
-/// unsearchable field (`text_fields`/`exact_fields` in `search_inner` apply
-/// ES's `isSearchable()` rule), but the stored-doc scan's field-less
-/// `query_string` arm is schema-free: it walks every non-`_` key of
-/// `_source`, so a token living only in an unsearchable field still matches
-/// there. Measured, not assumed. That arm's divergence from the FTS
+/// `more_like_this` needs no arm here: with an explicit `fields` list the
+/// parser lowers it to a `bool.should` of `match` clauses, which the leaf arms
+/// already reject (measured: `fields: ["note"]` and `["note","body"]` both
+/// 400). Without one it stays a `MoreLikeThis` node, which is the field-less
+/// gap below.
+///
+/// One gap remains open here, stated rather than papered over. The FTS route
+/// for these queries no longer projects a clause onto an unsearchable field
+/// (`text_fields`/`exact_fields` in `search_inner` apply ES's `isSearchable()`
+/// rule), but the stored-doc scan's FIELD-LESS arms are schema-free: they walk
+/// every non-`_` key of `_source`, so a token living only in an unsearchable
+/// field still matches when no field is named at all — `query_string
+/// {"query": "…"}` and a `more_like_this` with no `fields` both still return
+/// the document. Measured, not assumed. That divergence from the FTS
/// projection predates this change and is called out in its own comment in
/// `doc_matches_query`; making it schema-aware means threading a schema
/// through ~79 call sites and is left as its own change.
+///
+/// Also not covered, and not claimed: aggregation and sort field targets. Only
+/// `request.query` reaches this function; `request.aggs` is opaque JSON at this
+/// layer, and both `{"aggs":{"a":{"terms":{"field":"note"}}}}` and
+/// `{"sort":[{"note":"asc"}]}` still answer 200 (measured). ES rejects those
+/// too, but for an unrelated reason with an unrelated sentence
+/// (`Fielddata is disabled on [f] in [i]…`,
+/// `server/src/main/java/org/elasticsearch/index/mapper/TextFieldMapper.java:1534-1546`)
+/// which it raises whether or not `index: false` was declared — a wider
+/// divergence than this change, and one this error message would misreport.
fn unsearchable_query_field(q: &QueryNode, schema: &Schema) -> Option<String> {
- let check = |field: &String| -> Option<String> {
+ let check = |field: &str| -> Option<String> {
let fc = declared_field(schema, field)?;
- (!fc.options.indexed && !fc.options.doc_values).then(|| field.clone())
+ (!fc.options.indexed && !fc.options.doc_values).then(|| field.to_string())
+ };
+ // One entry of a `fields: [...]` list. `"title^3"` carries a boost;
+ // `"*"` / `"body.*"` is a pattern, which ES expands over searchable
+ // fields only and therefore must never be an error.
+ let check_spec = |spec: &String| -> Option<String> {
+ let (name, _boost) = parse_field_boost(spec);
+ if name.contains('*') {
+ return None;
+ }
+ check(name)
};
match q {
// ── Leaves that name exactly one field ────────────────────────────
@@ -26964,6 +27002,19 @@ fn unsearchable_query_field(q: &QueryNode, schema: &Schema) -> Option<String> {
| QueryNode::GeoPolygon { field, .. }
| QueryNode::GeoShape { field, .. } => check(field),
+ // ── Multi-field types: named fields fail, patterns expand ─────────
+ // ES's split as a rule, not as code (see this function's doc
+ // comment). `simple_query_string` with 2+ fields lowers to
+ // `MultiMatch` in the parser, so this covers that too, as well as
+ // `combined_fields`, which is rewritten to `multi_match:
+ // cross_fields`.
+ QueryNode::MultiMatch { fields, .. } => fields.iter().find_map(check_spec),
+ QueryNode::SimpleQueryString { fields, .. } => fields.iter().find_map(check_spec),
+ QueryNode::QueryString {
+ default_field: Some(f),
+ ..
+ } => check_spec(f),
+
// `exists` is NOT an error in ES: an unsearchable field is simply
// absent from `_field_names`, so the clause matches nothing. The
// API layer already rewrites it to `match_none`
@@ -27013,10 +27064,11 @@ fn unsearchable_query_field(q: &QueryNode, schema: &Schema) -> Option<String> {
.or_else(|| unsearchable_query_field(big, schema))
}
- // Everything else names no single mapped field to validate:
- // MatchAll/MatchNone/Ids/Script, the multi-field types documented
- // above, `percolate` (the field holds stored queries, not data), and
- // `more_like_this`.
+ // Everything else names no mapped data field to validate:
+ // MatchAll/MatchNone/Ids/Script, a field-less `query_string`, a
+ // field-less `more_like_this` (with `fields` the parser lowers it to
+ // `bool.should` of `match`, which the leaf arms above catch), and
+ // `percolate` (the field holds stored queries, not data).
_ => None,
}
}