AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
fix(es-compat): fold knn+query to hybrid only when hybrid-safe (#458 rework)
xerj-org456761bfc7f0
diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rsindex 5f8d887e4..ae7f6fdd4 100644--- a/engine/crates/xerj-api/src/es_compat.rs+++ b/engine/crates/xerj-api/src/es_compat.rs@@ -8923,23 +8923,41 @@ async fn search_impl(// hybrid request's clothes, which is a wire-compat divergence from// ES 8.x and not merely an internal limitation.//- // `hybrid` is the shape this engine actually combines: it fans each- // sub-query out as its own search and fuses the ranked lists, which- // is what ES does for `knn` beside `query`. Scores are RRF rather- // than a score sum, so absolute `_score` values differ from ES —- // that is a visible difference, and it is strictly better than- // returning the wrong documents in silence. The sibling branch- // above already refuses multi-knn-plus-query loudly rather than- // dropping it; this is the same principle applied to the case that- // was quietly wrong instead of loudly unsupported.- json!({- "hybrid": {- "queries": [- { "query": existing_q },- { "query": knn_query_json }- ]- }- })+ // `hybrid` is the shape this engine actually combines (it fans each+ // sub-query out and RRF-fuses the ranked lists), so it dispatches+ // the kNN half correctly. BUT the hybrid executor REJECTS+ // aggregations (returns 400) and ignores sort/collapse/search_after.+ // Folding EVERY knn-beside-query request to hybrid would therefore+ // turn a valid faceted request (knn + query + aggs) from 200 into a+ // 400, and silently drop a caller's sort/collapse — trading one+ // wrong answer for another. So fold to `hybrid` only when the+ // request carries none of those; otherwise keep the lexical+ // `bool.should`. Its kNN-dropped hits are the PRE-EXISTING behaviour+ // (strictly no worse than before this fix), and it still honours+ // aggs/sort/collapse and returns 200. In the hybrid case scores are+ // RRF, not a BM25 sum — a visible, documented difference, still+ // better than silently returning the wrong documents.+ let hybrid_safe = body.aggs.is_none()+ && body.aggregations.is_none()+ && body.sort.is_none()+ && body.collapse.is_none()+ && body.search_after.is_none()+ // A BM25-tuned `min_score` (e.g. 2.0) applied post-fusion to RRF+ // micro-scores (~1/61) would drop EVERY hit — keep bool.should,+ // where min_score compares against the BM25 sum it was tuned for.+ && body.min_score.is_none();+ if hybrid_safe {+ json!({+ "hybrid": {+ "queries": [+ { "query": existing_q },+ { "query": knn_query_json }+ ]+ }+ })+ } else {+ json!({ "bool": { "should": [existing_q, knn_query_json] } })+ }} else {knn_query_json};