AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(bench): definitive full-91 closed-loop scorecard — 35W/36L/11T (replaces "81/91" mirage)
xerj-orge4cc313f74f3
diff --git a/demo/playbooks/DISK_SIZE_2026-07-09.md b/demo/playbooks/DISK_SIZE_2026-07-09.mdnew file mode 100644index 000000000..e977ac8d9--- /dev/null+++ b/demo/playbooks/DISK_SIZE_2026-07-09.md@@ -0,0 +1,195 @@+# XERJ On-Disk Footprint — Diagnosis + Merge-Level Compression Fix (2026-07-09)++**TL;DR** — On an identical 100k-doc representative corpus (keyword + text ++numeric + date + bool), force-merged to one segment on both engines:++| Engine | On-disk (index/shard dir) | vs ES |+|---|---:|---:|+| **XERJ before** | 10,415,477 B (9.93 MiB) | **0.382×** |+| **XERJ after** (this change) | **9,028,909 B (8.61 MiB)** | **0.331×** |+| Elasticsearch 8.13.4 | 27,279,805 B (26.0 MiB) | 1.0× |++**The premise did not hold on this corpus.** XERJ was *already ~2.6× smaller*+than ES here, not 2× larger. The old "XERJ ≈ 2× ES disk" claim came from a+different scenario (almost certainly the un-merged / many-small-segment state+during active ingest — XERJ is 16-shard so a freshly-flushed index has ≥16+segments each with per-field FST/meta/norms sidecars, whereas ES auto-merges;+and/or a text-heavier corpus). In *steady state* (force-merged) XERJ wins on+disk on this workload.++The safe fix still landed a real **13.3 % steady-state reduction** and widened+XERJ's lead, with byte-identical round-trip and every gate green.++---++## Phase 1 — Measurement + Diagnosis++### Corpus+- 100,000 deterministic docs cycled from `demo/data/extras/chat-events.ndjson`,+ enriched with a real analyzed `body` text field and a high-cardinality+ `doc_id` keyword. Mapping: 6 keyword, 1 text, 5 numeric (long/double), 1 date,+ 1 boolean. Identical bytes ingested into XERJ (:9200) and ES (:9201).+- Both force-merged to a single segment (`_forcemerge?max_num_segments=1`) ++ flush, so both are in steady state.++### On-disk layout of a XERJ segment+XERJ writes **more files per segment than ES** but they are individually+addressable (ES bundles everything into one compound `.cfs`):++| File | Section | Codec |+|---|---|---|+| `<seg>.seg` | stored `_source` (columnar V2) + tombstones | zstd |+| `<seg>.dv` | doc-values (numeric i64 columns + keyword ordinal columns) | zstd |+| `<seg>.<field>.post` | FTS positional postings | zstd (ZPS1) |+| `<seg>.<field>.meta` | per-term metadata (df/ttf/offset/len) | zstd (ZFM4) |+| `<seg>.<field>.fst` | FST term dictionary | fst crate |+| `<seg>.<field>.norms` | per-doc field length | raw u16 |+| `<seg>.ids` | external-id → internal-ord index | raw |+| `<seg>.sidx` | skip index (written, **never read** — 16 B) | raw |++### Where the bytes are (XERJ, before)++| Component | Bytes | % |+|---|---:|---:|+| `.post` (postings, 14 fields) | 5,570,604 | 53.5 % |+| → `body.post` alone | 3,928,612 | 37.7 % |+| → 6 numeric fields' `.post` | ~1,078,000 | 10.4 % |+| `.seg` (stored `_source`) | 2,072,355 | 19.9 % |+| `.dv` (doc-values) | 1,819,138 | 17.5 % |+| `.ids` (id index) | 552,154 | 5.3 % |+| `.meta` (term metadata) | 229,560 | 2.2 % |+| `.norms` | 105,486 | 1.0 % |+| `.fst` | 59,987 | 0.6 % |+| **Total (disksize index dir)** | **10,415,477** | 100 % |++ES stores the equivalent in a single compound file: `_3.cfs` = 27,278,335 B+(+ 697 B `.cfe` + 388 B `.si` + 380 B `segments_4`).++### Root causes of the top contributors+1. **`body.post` (3.93 MB, 38 %)** — FTS *positional* postings for the one+ text field: 100k docs × ~30 tokens ≈ 3 M positions. This is inherent to+ phrase-searchable text and already zstd-compressed; positions can't be+ dropped without breaking `match_phrase`.+2. **Numeric fields get a full inverted index too (~1.16 MB incl. fst/meta).**+ `context_tokens.post`, `cost_usd.post`, `latency_ms.post`, … each carry a+ postings list. ES does **not** do this for numerics — it uses BKD points ++ doc-values only. XERJ keyword-analyzes every non-text field into the FST so+ numeric `term` queries and the `size:0` count-shortcut can read `doc_freq`+ straight from the term dictionary. This is a genuine XERJ-vs-ES structural+ difference and a real redundancy vs. the `.dv` column — but it is+ **load-bearing** (numeric term/count paths read it), so it is not a safe+ one-pass removal.+3. **`.seg` (2.07 MB)** and **`.dv` (1.82 MB)** — already zstd, but at the fast+ *flush* level (3). The columnar `_source` and the numeric i64 columns are+ highly redundant and compress ~20–25 % tighter at a higher level.++Everything durable was already zstd level **3** (deliberately: level 19 on the+*flush* path was reverted in the past because it collapsed sustained-ingest+throughput 70–100× — see the ingest-perf regression report). So "just compress+`_source`" was already done; the remaining safe lever is **spending more+compression effort where it doesn't hurt ingest: the merge path.**++### Compression-level ceiling (empirical sweep, force-merged 100k)++| zstd level | total | seg | dv | post | meta |+|---|---:|---:|---:|---:|---:|+| 3 (flush default) | 10,415,477 | 2,072,355 | 1,819,138 | 5,570,604 | 229,560 |+| 9 | 9,789,136 | 1,726,778 | 1,733,544 | 5,406,891 | 198,112 |+| 12 | 9,737,700 | 1,725,519 | 1,698,279 | 5,394,763 | 195,328 |+| **19** | **9,022,248** | 1,561,830 | 1,436,158 | 5,119,175 | 181,281 |++Level 19 is the sweet spot (−13.4 %). The stored/`.dv`/`.meta` columns give+~20–25 %; the positional `body.post` only ~8 % (already high-entropy).++### Ranked fix options+| # | Option | Est. saving (this corpus) | Risk | Status |+|---|---|---:|---|---|+| 1 | **Merge-path zstd 3→19** for `.post`/`.meta`/`.seg`/`.dv` | −13.3 % | **Very low** (lossless, format-identical, flush untouched) | **DONE** |+| 2 | Drop redundant numeric-field inverted postings | ~11 % here, more on log/metric corpora | Medium — numeric `term`/count-shortcut must move fully to points/doc-values first | Follow-up |+| 3 | Delta+bit-pack small-integer `.dv` columns before zstd | ~2–4 % | Medium — new column format version | Follow-up |+| 4 | Compress/trim `.ids` id-index (0.55 MB, raw) | ~3–4 % | Low–medium | Follow-up |+| 5 | Stop emitting the never-read `.sidx`; coalesce 240 tiny WAL files | negligible bytes, inode win | Low | Follow-up |++---++## Phase 2 — Fix: merge-path zstd level 3 → 19++**Idea:** keep the fast level 3 on the **flush** path (ingest-critical,+back-pressure-bounded) and raise to level 19 on the **merge/forcemerge** path,+which is background and builds the long-lived steady-state segments. Because a+force-merge (and every background compaction) *re-encodes* the segment, the+durable on-disk bytes converge on the merge-level output regardless of flush+level.++**Why it is safe (correctness-paramount):**+- zstd decompression is **level-independent** — the on-disk frame format is+ identical at any level, so every reader is byte-for-byte unchanged and old+ and new segments interoperate. A plumbing mistake can at worst cost flush CPU;+ it can never corrupt or lose data.+- All existing **flush** call sites keep passing the fast level (3), so+ sustained-ingest throughput is untouched (the past zstd-19 regression was a+ flush-path regression).++### Files changed+- `engine/crates/xerj-storage/src/doc_values.rs`+ - `DV_FLUSH_ZSTD_LEVEL=3`, `DV_MERGE_ZSTD_LEVEL=19` (585, 592);+ `encode_columns` → thin wrapper over new `encode_columns_level(cols, level)` (597, 604).+- `engine/crates/xerj-storage/src/stored_codec.rs`+ - `STORED_MERGE_ZSTD_LEVEL=19` (129); `encode_stored_v2` → wrapper over new+ `encode_stored_v2_level(bytes, level)` (160, 167); `level` threaded through+ `encode_stored_v2_from_values_inner` / `encode_v2_columns` /+ `encode_dict_bitpack` / `encode_cross_dep`; the flush `from_values*` entries+ pin `STORED_ZSTD_LEVEL` (3).+- `engine/crates/xerj-fts/src/index.rs`+ - `MERGE_ZSTD_LEVEL=19` (204); `FtsIndexWriter.zstd_level` field (default 3) ++ `set_zstd_level()` (469); threaded through `finish` → `write_field_static`+ (720) and `encode_field_meta_v4` (241) for both `.post` and `.meta`.+- `engine/crates/xerj-engine/src/index.rs`+ - **Merge** path: `encode_stored_v2_level(…, STORED_MERGE_ZSTD_LEVEL)` (2841),+ `fts_writer.set_zstd_level(MERGE_ZSTD_LEVEL)` (2871),+ `write_doc_values_sidecar(…, DV_MERGE_ZSTD_LEVEL)` (2938).+ - **Flush** path: `write_doc_values_sidecar(…, DV_FLUSH_ZSTD_LEVEL)` (10698);+ flush FTS writer keeps the default level. `write_doc_values_sidecar` gained+ a `level` parameter (8070).++### Results (100k, force-merged)++| Component | before (L3) | after (L19) | Δ |+|---|---:|---:|---:|+| `.seg` | 2,072,355 | 1,561,105 | −24.7 % |+| `.dv` | 1,819,138 | 1,441,697 | −20.7 % |+| `.post` | 5,570,604 | 5,119,926 | −8.1 % |+| → `body.post` | 3,928,612 | 3,553,174 | −9.6 % |+| `.meta` | 229,560 | 182,361 | −20.6 % |+| **Total** | **10,415,477** | **9,028,909** | **−13.3 %** |++New ratio vs ES: **0.331×** (was 0.382×).++### Correctness evidence+- **Round-trip `_source`:** 300 sampled docs read back from the level-19 merged+ segment — every field value byte-identical to the generated doc. 0 value+ mismatches. (XERJ's columnar codec reorders `_source` keys; that is+ pre-existing and orthogonal to the lossless level change.)+- **Query parity vs ES:** 10/10 identical hit totals — term (keyword/numeric/+ bool), range (long/double), `match`, `match_phrase`, bool, exists.+- **Unit tests:** `cargo test -p xerj-engine -p xerj-storage -p xerj-fts` green+ (includes stored-codec parity + numeric/keyword round-trip tests).+- **ES-YAML suite:** **1360 passed · 0 failed · 3 skipped** (target held).+- **Read latency A/B** (level-19 :9200 vs level-3 :9202, identical data): search+ paths sub-ms on both; the decompression-sensitive `match_phrase body` is+ identical (0.35/1.17 vs 0.30/1.16 ms p50/p99); the `_doc/{id}` slow path+ (~918 ms, pre-existing) is identical on both levels → no decompression+ regression.++### Caveats / risks+- **Background merge CPU** rises ~8× per merge (level 19 ≈ 25 MB/s/core vs level+ 3 ≈ 250). Merges are background and not back-pressure-gated, but under+ *sustained max ingest* slower merges could let tier-0 segment count grow+ transiently. Flush throughput itself is unchanged. If this ever bites, drop+ the merge level to 12 (captures ~half the win at ~3× flush CPU) — it is a+ single constant per crate.+- **Forward/backward compatible:** no format/magic change; level is purely an+ encode-effort knob. Mixed-level segments coexist and read identically.+- The **numeric-postings redundancy (option 2)** is the larger structural lever,+ especially on log/metric corpora where numerics dominate — but it changes what+ the read path may rely on and must be done carefully; left as a follow-up.diff --git a/demo/playbooks/FULL_MATRIX_SCORECARD_2026-07-09.md b/demo/playbooks/FULL_MATRIX_SCORECARD_2026-07-09.mdnew file mode 100644index 000000000..fb5cae6ce--- /dev/null+++ b/demo/playbooks/FULL_MATRIX_SCORECARD_2026-07-09.md@@ -0,0 +1,331 @@+# XERJ vs Elasticsearch — DEFINITIVE Full-Matrix Scorecard (2026-07-09)++**This is a MEASUREMENT + compatibility audit. No engine code was changed; nothing was committed.**++This scorecard **replaces the discredited "81 W / 91" headline number.** That number was a+mirage: it was measured with the **open-loop, 200 req/s coordinated-omission driver**+(`timed()` in `bench-matrix.mjs`) with **XERJ's query cache ON**, so it measured+throughput-under-saturation + whole-result cache clones, not each engine's true per-query cost.+Every cell below is re-measured **closed-loop** (single client, sequential, one request at a+time — the true round-trip), with **XERJ's query cache OFF** and `request_cache=false` on every+search, using **identical query bodies and an identical corpus** on both engines.++## Method (fair, closed-loop, single-client)++- **Build:** committed working tree at **`bd3bb41`** (clean; only an untracked doc file present),+ release binary `engine/target/release/xerj` from `cargo build --release -p xerj-server`.+ No sources modified; nothing committed.+- **XERJ** on `:9200`, fresh temp data dir, `--insecure`, **`XERJ_DISABLE_QUERY_CACHE=1`**+ (whole-result cache OFF). **Real Elasticsearch 8.13.4** on `:9201`, default settings, untouched.+ (XERJ self-reports wire version 8.13.0.) `python :8080` untouched.+- **Corpus:** the flat LLM-telemetry corpus `demo/data/extras/chat-events.ndjson` (4,008 real docs)+ cycled to **100,000 docs** for the read surface, **identical** on both engines, with the exact+ `bench-matrix.mjs` mapping (keyword `model`/`intent`/`status`/`tenant`/`top_doc`, boolean+ `cache_hit`, integer token/latency fields, double `cost_usd`, date `@timestamp`).+ **Single segment** on both (`_forcemerge?max_num_segments=1` after seeding + 3 s settle).+ Both `/perf/_count` = 100,000.+- **Bodies:** the **query/agg/pipeline/feature bodies are copied VERBATIM** from+ `demo/playbooks/bench-matrix.mjs` (29 queries, 31 basic aggs, 12 pipeline aggs, 7 feature ops).+- **Measurement:** CLOSED-LOOP — feasibility probe, then **warm 8**, then **p50/p99 over 50 iters**,+ one request at a time. Lean keep-alive `http` client, applied identically to both engines.+- **Verdict rule (from XERJ's POV):** compare **p50**. `|Δp50| ≤ 0.30 ms` ⇒ **TIE**; otherwise+ **WIN** (XERJ faster) / **LOSS** (ES faster). If XERJ errors/unsupported while ES works ⇒+ **XERJ-UNSUPPORTED** (a compat gap, not a perf loss); if ES 4xx while XERJ works ⇒ **ES-REJECTS**.+- **Correctness guard:** every cell captures a hit-total / agg signal; a WIN/TIE whose result+ diverges from ES would be flagged **FAST-BUT-DIVERGENT**. A separate value-level spot-check+ (below) compares actual agg *values* vs ES so no "win" is a fast-but-wrong result.+- Harness: `/tmp/.../scratchpad/full_matrix.mjs`; raw JSON: `results.json`; run log: `fullrun.log`.++---++## SUMMARY COUNTS++### Table A — read / agg / feature matrix (79 cells)++| group | cells | WIN | LOSS | TIE | XERJ-UNSUPPORTED | ES-REJECTS | BOTH-ERROR |+|---|--:|--:|--:|--:|--:|--:|--:|+| queries | 29 | 0 | 24 | 3 | 0 | 2 | 0 |+| basic aggs | 31 | 17 | 8 | 6 | 0 | 0 | 0 |+| pipeline aggs | 12 | 12 | 0 | 0 | 0 | 0 | 0 |+| feature ops | 7 | 3 | 3 | 1 | 0 | 0 | 0 |+| **Total** | **79** | **32** | **35** | **10** | **0** | **2** | **0** |++- **Perf verdicts (both engines ran identical work):** **32 WIN / 35 LOSS / 10 TIE.**+- **Compat gaps:** **0 XERJ-UNSUPPORTED**, **2 ES-REJECTS** (both are XERJ being *more lenient*+ than ES — see Compatibility section), **0 BOTH-ERROR**.+- **Fast-but-wrong cells: 0.** No WIN/TIE returned results that diverge from ES.++### Table B — ingest, disk & vector (6 cells)++| dimension | XERJ | ES | verdict |+|---|--:|--:|:--:|+| ingest throughput (300k, 1 client, 10k/bulk) | **95,469 docs/s** | 62,154 docs/s | **WIN** (1.54×) |+| ingest per-bulk latency p50 / p99 | **84.0 / 168.6 ms** | 151.5 / 176.9 ms | **WIN** (p50 1.80×) |+| index on-disk size (`/perf` `_stats` store) | **9.85 MB** | 12.07 MB | **WIN** (0.82×) |+| kNN k=10 latency p50 / p99 | 309.55 / 1397.73 ms | **2.59 / 4.12 ms** | **LOSS** (ES 120× faster) |+| kNN recall@10 | 1.00 | 1.00 | **TIE** |+| hybrid lexical+vector RRF | **works, 4.84 ms** | unavailable on 8.13.4 | XERJ-only¹ |++¹ ES 8.13.4 cannot run the hybrid: the `retriever` syntax returns **400** (introduced in ES 8.14),+and the older `rank:{rrf}` form returns **403 — "current license is non-compliant for [Reciprocal+Rank Fusion (RRF)]"** (RRF is a commercial-license feature; this ES runs a basic license). XERJ+executes **both** RRF forms. XERJ's RRF result **could not be validated against ES** here because+ES refuses to run it — so this is a *capability* edge, not a verified correctness/latency win.++### Grand total (perf-comparable dimensions)++**35 WIN / 36 LOSS / 11 TIE**, plus **2 ES-REJECTS**, **1 XERJ-only (RRF)**, **0 XERJ-UNSUPPORTED**.++---++## DETAILED MATRIX++Verdict is XERJ's POV. p50/p99 in ms. `hits` is the shared result signal (size:0 shows the+`track_total_hits:true` total forced onto **both** engines; 100000 = full corpus).++### QUERIES (29) — 0 WIN / 24 LOSS / 3 TIE / 2 ES-REJECTS++| case | XERJ status | XERJ p50/p99 (ms) | ES status | ES p50/p99 (ms) | hits | verdict |+|---|---|--:|---|--:|--:|:--:|+| match_all | ok | 2.83 / 6.35 | ok | 0.73 / 1.73 | 100000 | LOSS |+| match_none | ok | 0.36 / 0.80 | ok | 0.62 / 1.40 | 0 | TIE |+| match(model) | ok | 4.23 / 6.20 | ok | 1.08 / 2.13 | 27744 | LOSS |+| match_phrase(top_doc) | ok | 2.65 / 4.14 | ok | 0.87 / 1.88 | 9082 | LOSS |+| match_phrase_prefix | ok | 4.44 / 8.79 | **err 400** | — | 25259 | **ES-REJECTS** |+| match_bool_prefix | ok | 4.16 / 4.81 | ok | 1.23 / 2.09 | 25259 | LOSS |+| multi_match | ok | 4.46 / 5.83 | ok | 1.64 / 2.65 | 27744 | LOSS |+| combined_fields | ok | 7.10 / 9.82 | **err 400** | — | 27744 | **ES-REJECTS** |+| query_string | ok | 14.95 / 17.00 | ok | 2.26 / 3.03 | 27371 | LOSS |+| simple_query_string | ok | 19.11 / 28.20 | ok | 0.52 / 0.96 | 98752 | LOSS |+| more_like_this | ok | **269.20 / 419.16** | ok | 0.93 / 1.90 | 9082 | LOSS |+| term(status) | ok | 2.27 / 3.45 | ok | 0.78 / 1.58 | 98752 | LOSS |+| terms(model) | ok | 3.91 / 5.27 | ok | 1.18 / 1.90 | 27744 | LOSS |+| range(latency_ms) | ok | 2.47 / 3.58 | ok | 1.33 / 1.62 | 29067 | LOSS |+| range(@timestamp) | ok | **339.67 / 386.87** | ok | 0.61 / 1.42 | 100000 | LOSS |+| range(cost_usd) | ok | 2.62 / 3.85 | ok | 1.61 / 2.51 | 41540 | LOSS |+| prefix(model) | ok | 9.74 / 13.84 | ok | 2.06 / 3.30 | 84731 | LOSS |+| wildcard(model) | ok | 12.30 / 17.12 | ok | 2.31 / 3.90 | 84731 | LOSS |+| regexp(model) | ok | 2.00 / 4.02 | ok | 2.21 / 4.06 | 84731 | TIE |+| fuzzy(model) | ok | 5.19 / 6.93 | ok | 1.27 / 2.14 | 27744 | LOSS |+| exists(cost_usd) | ok | 1.59 / 2.47 | ok | 0.52 / 0.76 | 100000 | LOSS |+| ids | ok | 0.26 / 0.52 | ok | 0.35 / 1.25 | 0 | TIE |+| term(cache_hit) | ok | 3.07 / 3.94 | ok | 0.53 / 1.05 | 42875 | LOSS |+| bool must+filter+should+must_not | ok | **164.83 / 246.33** | ok | 4.32 / 5.57 | 37369 | LOSS |+| constant_score | ok | 1.98 / 2.73 | ok | 0.69 / 1.61 | 98752 | LOSS |+| boosting | ok | **244.83 / 295.28** | ok | 5.37 / 7.56 | 98752 | LOSS |+| dis_max | ok | **247.27 / 370.48** | ok | 1.81 / 2.82 | 27744 | LOSS |+| function_score | ok | **240.59 / 303.97** | ok | 6.39 / 7.92 | 100000 | LOSS |+| pinned | ok | **247.61 / 371.93** | ok | 2.75 / 3.61 | 98752 | LOSS |++**Reads: XERJ loses every comparable raw-lookup / scored-query race.** ES's postings/BKD lookup+floor is 0.5–2 ms; XERJ sits above it even on fast paths (`match_all` 2.83 ms, `term` 2.27 ms).+Six query shapes are **O(N) brute-force cliffs on XERJ** (bold): `range(@timestamp)` 340 ms,+`more_like_this` 269 ms, and the scored-compound family `boosting`/`dis_max`/`function_score`/+`pinned`/`bool(must+should+must_not)` at 165–248 ms — all `<8 ms` on ES. `ids`/`_mget` match+nothing on either engine (corpus uses auto-`_id`s), so those are equal no-match races.++### BASIC AGGS (31) — 17 WIN / 8 LOSS / 6 TIE++| case | XERJ status | XERJ p50/p99 (ms) | ES status | ES p50/p99 (ms) | hits | verdict |+|---|---|--:|---|--:|--:|:--:|+| avg | ok | 0.22 / 0.70 | ok | 3.63 / 4.76 | 100000 | **WIN** |+| sum | ok | 0.26 / 0.53 | ok | 3.47 / 4.52 | 100000 | **WIN** |+| min | ok | 0.15 / 0.34 | ok | 0.32 / 0.54 | 100000 | TIE |+| max | ok | 0.15 / 0.28 | ok | 0.30 / 0.41 | 100000 | TIE |+| stats | ok | 0.12 / 0.28 | ok | 3.31 / 4.06 | 100000 | **WIN** |+| extended_stats | ok | 0.38 / 0.60 | ok | 4.28 / 4.88 | 100000 | **WIN** |+| value_count | ok | 0.14 / 0.42 | ok | 2.26 / 2.99 | 100000 | **WIN** |+| cardinality | ok | 0.51 / 0.70 | ok | 2.46 / 2.86 | 100000 | **WIN** |+| percentiles | ok | 0.38 / 1.15 | ok | 13.61 / 19.68 | 100000 | **WIN** |+| percentile_ranks | ok | 0.40 / 0.68 | ok | 14.07 / 18.27 | 100000 | **WIN**² |+| median_absolute_deviation | ok | **1004.45 / 1061.47** | ok | 18.89 / 26.00 | 100000 | LOSS |+| matrix_stats | ok | **1028.69 / 1092.68** | ok | 22.11 / 31.64 | 100000 | LOSS |+| scripted_metric | ok | **1086.31 / 1133.94** | ok | 6.79 / 8.57 | 100000 | LOSS |+| top_hits (sub) | ok | 3.70 / 6.66 | ok | 4.95 / 6.71 | 100000 | **WIN** |+| terms | ok | 0.50 / 1.95 | ok | 0.77 / 1.87 | 100000 | TIE |+| rare_terms | ok | **996.88 / 1037.74** | ok | 9.12 / 13.72 | 100000 | LOSS |+| significant_terms | ok | **1061.33 / 1125.62** | ok | 3.07 / 5.30 | 100000 | LOSS |+| histogram | ok | 3.51 / 41.61 | ok | 6.32 / 8.09 | 100000 | **WIN**³ |+| date_histogram | ok | 0.35 / 34.96 | ok | 0.80 / 1.62 | 100000 | **WIN**³ |+| auto_date_histogram | ok | **1212.42 / 1298.06** | ok | 4.93 / 6.84 | 100000 | LOSS |+| variable_width_histogram | ok | 1.12 / 43.27 | ok | 12.63 / 18.84 | 100000 | **WIN**³ |+| range | ok | 0.38 / 11.02 | ok | 0.63 / 0.88 | 100000 | TIE |+| date_range | ok | 0.34 / 1.64 | ok | 0.68 / 0.85 | 100000 | **WIN** |+| filter | ok | 1.55 / 17.36 | ok | 4.26 / 6.27 | 100000 | **WIN**³ |+| filters | ok | 0.34 / 1.56 | ok | 0.59 / 0.84 | 100000 | TIE |+| missing | ok | **992.05 / 1061.64** | ok | 1.98 / 3.45 | 100000 | LOSS |+| global | ok | 0.42 / 17.33 | ok | 3.73 / 5.16 | 1248 | **WIN**³ |+| adjacency_matrix | ok | 1.45 / 48.31 | ok | 4.13 / 5.70 | 100000 | **WIN**³ |+| composite | ok | 5.64 / 8.88 | ok | 4.29 / 5.79 | 100000 | LOSS |+| random_sampler | ok | 1.00 / 2.63 | ok | 1.13 / 1.92 | 100000 | TIE |+| terms+avg(cost) | ok | 1.51 / 1.95 | ok | 4.51 / 6.16 | 100000 | **WIN** |++² `percentile_ranks` — WIN and values ~correct, but XERJ keys the `values` map `"200"`/`"500"`+while ES uses `"200.0"`/`"500.0"` (a wire-format divergence); see Correctness section.+³ p50-WIN but **p99 tail is unstable** (histogram 41.6 ms, date_histogram 35.0 ms, vwh 43.3 ms,+adjacency_matrix 48.3 ms, filter 17.4 ms, global 17.3 ms) — first-touch lazy columnar/GC spikes.+Verdict is on p50 (per the rule), but the tail is honestly worse than ES on these rows.++**Aggs are XERJ's real strength — where the data is columnar.** The precomputed doc-value stat+aggs (`avg`/`sum`/`stats`/`extended_stats`/`value_count`/`percentiles`/`percentile_ranks`) beat+ES by 3–14 ms, all returning **identical values** (spot-check below). But **seven aggs are ~1-second+O(N) brute-force cliffs** on XERJ (`median_absolute_deviation`, `matrix_stats`, `scripted_metric`,+`rare_terms`, `significant_terms`, `auto_date_histogram`, `missing`) — 30–250× slower than ES.++### PIPELINE AGGS (12) — 12 WIN / 0 LOSS / 0 TIE++| case | XERJ status | XERJ p50/p99 (ms) | ES status | ES p50/p99 (ms) | hits | verdict |+|---|---|--:|---|--:|--:|:--:|+| sum_bucket | ok | 1.52 / 7.24 | ok | 4.23 / 6.91 | 100000 | **WIN** |+| avg_bucket | ok | 1.44 / 6.13 | ok | 4.26 / 6.20 | 100000 | **WIN** |+| max_bucket | ok | 1.40 / 5.16 | ok | 5.08 / 6.48 | 100000 | **WIN** |+| stats_bucket | ok | 1.37 / 2.73 | ok | 4.35 / 7.31 | 100000 | **WIN** |+| percentiles_bucket | ok | 1.39 / 7.77 | ok | 4.34 / 6.15 | 100000 | **WIN** |+| derivative | ok | 1.37 / 6.62 | ok | 4.57 / 6.10 | 100000 | **WIN** |+| cumulative_sum | ok | 1.41 / 3.58 | ok | 4.67 / 6.42 | 100000 | **WIN** |+| moving_fn | ok | 1.41 / 6.02 | ok | 4.41 / 6.27 | 100000 | **WIN** |+| serial_diff | ok | 1.54 / 18.76 | ok | 4.57 / 6.24 | 100000 | **WIN** |+| bucket_script | ok | 1.82 / 4.57 | ok | 5.83 / 8.63 | 100000 | **WIN** |+| bucket_selector | ok | 1.39 / 4.82 | ok | 4.24 / 7.52 | 100000 | **WIN** |+| bucket_sort | ok | 1.70 / 3.83 | ok | 4.26 / 7.11 | 100000 | **WIN** |++**Clean 12/12 XERJ sweep.** All pipeline aggs are built on a `date_histogram` + `sum` sub-agg,+which is XERJ's fast columnar path; XERJ finishes in ~1.4–1.8 ms vs ES's ~4–6 ms. Values verified+identical (`sum_bucket` and `cumulative_sum` last-bucket both match ES to <1e-9 rel).++### FEATURE OPS (7) — 3 WIN / 3 LOSS / 1 TIE++| case | XERJ status | XERJ p50/p99 (ms) | ES status | ES p50/p99 (ms) | hits | verdict |+|---|---|--:|---|--:|--:|:--:|+| sort-heavy (2-key desc/asc) | ok | 2.93 / 4.45 | ok | 4.00 / 5.64 | 100000 | **WIN** |+| deep from+size (from 500) | ok | 5.55 / 9.58 | ok | 1.29 / 1.53 | 100000 | LOSS |+| search_after | ok | 2.66 / 4.00 | ok | 3.82 / 5.29 | 100000 | **WIN** |+| highlight | ok | 2.53 / 4.50 | ok | 1.09 / 1.36 | 98752 | LOSS |+| _count | ok | 0.12 / 0.25 | ok | 0.53 / 0.98 | 100000 | **WIN**⁴ |+| _msearch | ok | 1.58 / 4.24 | ok | 0.68 / 2.36 | — | LOSS |+| _mget | ok | 0.12 / 0.50 | ok | 0.37 / 1.26 | — | TIE |++⁴ `_count` measured **without** the bogus `request_cache=false` param that `bench-matrix.mjs`+appends — ES legitimately rejects that param on `/_count` (valid only on `/_search`), a harness+bug not a `_count` gap. See the leniency note in the Compatibility section.++---++## CORRECTNESS VERIFICATION (so no "win" is fast-but-wrong)++A separate value-level spot-check compared actual XERJ vs ES agg **values** on the same 100k index:++```+OK avg=838.26063 (both) OK sum(cost_usd)=947.791318 (rel 0.000%)+OK stats.avg/.sum (both exact) OK extended_stats.variance/.std_dev (rel 0.000%)+OK value_count=100000 (both) OK cardinality(top_doc)=12 (both)+OK percentiles.p50 XERJ 826 vs ES 822.45 (rel 0.43%, TDigest-approx)+OK terms buckets/keys/counts (identical) OK date_histogram/histogram/vwh/range/date_range tot_doc=100000+OK filter.avg (identical) OK adjacency_matrix/composite bucket counts (identical)+OK global.value_count=100000 OK pipe.sum_bucket / cumulative_sum.last (rel <1e-9)+DIFF percentile_ranks: values ~correct (XERJ 200→0.45 vs ES 0.47) but keyed "200" not "200.0"+```++- **All XERJ agg WINs return correct values** — the wins are real, not empty/short-circuited.+- **`percentiles`**: XERJ returns an **exact** integer percentile (826) vs ES's TDigest-approx+ (822.45) — XERJ is *more* accurate here, and faster. Not a defect.+- **`percentile_ranks`**: the one value-shape divergence — values are approximately correct+ (TDigest-approx), but the `values` map is keyed **`"200"`/`"500"`** on XERJ vs ES's+ **`"200.0"`/`"500.0"`**. A client expecting the ES `"200.0"` key reads `undefined` from XERJ.+ This is a genuine **wire-format compat bug**, though it does not make the latency win invalid.++---++## COMPATIBILITY / XERJ-UNSUPPORTED++**XERJ-UNSUPPORTED count on the runnable matrix: 0.** Across all 29 queries, 31 basic aggs, 12+pipeline aggs and 7 feature ops, **XERJ executed every ES query/agg/pipeline/feature shape** — it+never errored or returned "unsupported" while ES succeeded. The compat divergences that *do* exist+run the **other** direction (XERJ is more permissive than ES) or are format nuances:++1. **`match_phrase_prefix` on a keyword field** → **ES-REJECTS** ("Can only use phrase prefix+ queries on text fields — not on [top_doc] which is of type [keyword]"). **XERJ runs it** and+ returns 25,259 hits. XERJ does not enforce ES's text-field restriction — a **lenient semantic+ divergence** (XERJ answering a query ES deems illegal), not a XERJ capability win.+2. **`combined_fields` on a keyword field** → **ES-REJECTS** ("Field [model] of type [keyword]+ does not support [combined_fields] queries"). **XERJ runs it** (27,744 hits). Same lenient+ divergence.+3. **`_count` extra-param leniency** — ES returns **400** for `_count?request_cache=false`+ (`illegal_argument_exception: unrecognized parameter [request_cache]`); XERJ ignores the unknown+ param and returns 200. XERJ is more permissive about unknown query-string params.+4. **`percentile_ranks` value-key format** — `"200"` vs ES `"200.0"` (see Correctness).+5. **Hybrid RRF** — XERJ supports both the `retriever:{rrf}` and `rank:{rrf}` syntaxes; ES 8.13.4+ supports neither on this license (retriever → 400, version-gated to 8.14+; rank → 403,+ RRF is a commercial-license feature). A XERJ capability edge, correctness unverified vs ES.++### Families NOT exercised (need field types the flat corpus lacks) — separate known surface++These are **not** covered by this matrix and remain a **separate, known compat surface** (see+`STUB_AUDIT.md`): `geo_*` queries + `geohash_grid`/`geotile_grid`/`geo_bounds`/`geo_centroid`,+`ip_range`/`ip_prefix`, `nested`/`has_child`/`has_parent` (join), `span_*`, `significant_text`,+`semantic`/hybrid retrievers on real embeddings, `percolate`. "0 XERJ-UNSUPPORTED" applies **only**+to the corpus-runnable matrix above, not to these skipped families.++---++## HONEST BOTTOM LINE++- **Where XERJ genuinely wins:**+ - **Columnar aggregations** — `avg`/`sum`/`stats`/`extended_stats`/`value_count`/`cardinality`/+ `percentiles`/`percentile_ranks` beat ES by 3–14 ms with **identical (or more exact) values**.+ - **Pipeline aggregations** — a clean **12/12 sweep** (~1.4 ms vs ES's ~4–6 ms).+ - **Ingest** — **95k vs 62k docs/s** (1.54×) and per-bulk p50 **84 vs 152 ms**.+ - **On-disk size** — **9.85 vs 12.07 MB** (0.82×).+ - **Some feature ops** — `sort-heavy`, `search_after`, `_count` (fast columnar sort/count).+ - **kNN recall** — perfect **1.00** (exact brute-force) tying ES's HNSW on this set.+ - **Hybrid RRF availability** — runs out-of-the-box where this ES build cannot (license/version).+- **Where XERJ loses:**+ - **Raw lookup latency** — ES's sub-2 ms postings/BKD floor beats XERJ on **every** comparable+ read (`match_all`/`term`/`range`/`match`/`prefix`/`wildcard`/…). This is the honest headline:+ **XERJ wins 0 of the raw-read races.**+ - **O(N) brute-force cliffs** — `range(@timestamp)` (340 ms), `more_like_this` (269 ms), the+ scored-compound family `boosting`/`dis_max`/`function_score`/`pinned`/`bool(must+should)`+ (165–248 ms), and **seven ~1-second aggs** (`median_absolute_deviation`, `matrix_stats`,+ `scripted_metric`, `rare_terms`, `significant_terms`, `auto_date_histogram`, `missing`).+ - **kNN latency** — exact brute-force is **120× slower** than ES HNSW (310 ms vs 2.6 ms) — correct+ but not scalable; recall parity only holds because this 50k random set is trivially separable.+ - **p99 tails on several WIN aggs** — histogram/date_histogram/vwh/adjacency_matrix show 35–48 ms+ p99 spikes even where p50 wins.+- **Compat:** **0 XERJ-UNSUPPORTED** on the runnable matrix (XERJ runs every ES shape here); the only+ gaps are XERJ being *too lenient* (2 ES-REJECTS + `_count` param), one `percentile_ranks` key-format+ bug, and the untested geo/ip/nested/join/span/semantic surface.++**Net: 35 WIN / 36 LOSS / 11 TIE** across all perf-comparable dimensions — a near-even split, NOT+"81/91". XERJ owns aggregations, pipelines, ingest and disk; ES owns raw-read latency and vector+search. The old headline collapsed the moment the cache was turned off and the driver went+closed-loop.++---++## RECONCILIATION vs `READ_SCORECARD_2026-07-09.md` (the ~38-shape core)++That scorecard reported **7 WIN / 22 LOSS / 9 TIE** on an **enriched** corpus (it added an analyzed+`body` text field + a high-cardinality `doc_id` keyword). This matrix uses the **flat keyword**+corpus from `bench-matrix.mjs`, so the overlapping shapes agree **directionally** with two+corpus-driven drifts to note:++- **Same story on reads:** both scorecards show XERJ winning **0** raw-read races; ES's sub-ms+ lookup floor is unbeaten in both. The O(N) scored-query cliffs (`boosting`/`function_score`)+ reproduce (~160 ms enriched → ~245 ms here at 100k).+- **Same story on stat aggs:** `avg`/`sum`/`stats`/`extended_stats`/`value_count`/`percentiles`/+ `percentile_ranks` are WINs in both.+- **DRIFT — `cardinality`:** READ_SCORECARD scored it a **LOSS** (9.52 vs 3.79 ms) on the+ **high-cardinality** `doc_id`; here it's a **WIN** (0.51 vs 2.46 ms) on `top_doc`, which has only+ **12** distinct values. Cardinality's verdict is field-cardinality-dependent — flag when quoting.+- **DRIFT — `histogram`/`date_histogram`:** TIE in READ_SCORECARD (both sub-2 ms), WIN here — but+ with the p99-tail caveat above. Boundary flips at the sub-ms floor, not a code change.+- **New cliffs this matrix surfaced** (absent from the 38-shape core): `range(@timestamp)` 340 ms+ and the seven ~1-second exotic aggs — because this matrix exercises the full agg/query surface,+ not just the 38 core shapes.++---++*Closed-loop, single-client, XERJ query-cache OFF, ES default, 100k identical single-segment corpus.+XERJ `:9200` (this run's server, stopped by exact PID after), ES `:9201` (v8.13.4, untouched),+python `:8080` untouched. Committed engine at `bd3bb41`. No engine code changed; nothing committed.+Harness `full_matrix.mjs`; raw `results.json`; value spot-check `verify_values.mjs`.*