Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(benchmarks): simplify public page — collapsible categories, calmer tone

xerj-orgf6ad663ceaad
diff --git a/landing/benchmarks/index.html b/landing/benchmarks/index.html
index ac38d174c..fccfa196a 100644
--- a/landing/benchmarks/index.html
+++ b/landing/benchmarks/index.html
@@ -4,24 +4,78 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>XERJ vs Elasticsearch — Reproducible Benchmarks · 75 WIN</title>
-<meta name="description" content="XERJ vs Elasticsearch 8.13.4: 91 measured dimensions, every WIN and every LOSE published. 75 WIN · 14 LOSE · 2 N/A on a lean keep-alive client that measures each engine's true server latency. Reproducible with two scripts.">
+<meta name="description" content="XERJ vs Elasticsearch 8.13.4 — 91 measured dimensions, every result published. 75 WIN · 14 LOSE · 2 N/A on identical hardware and an identical HTTP client. Reproduce it on your machine with four commands.">
<meta property="og:title" content="XERJ vs Elasticsearch — Reproducible Benchmarks">
-<meta property="og:description" content="75 WIN · 14 LOSE · 2 N/A. We caught our own benchmark client inflating both engines' latencies and corrected it. Measured, reproducible, versioned — the losses are published next to the wins.">
+<meta property="og:description" content="75 WIN · 14 LOSE · 2 N/A across 91 dimensions. Measured, reproducible, versioned — the losses are published next to the wins.">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Display:wght@400;700;800;900&family=Inter:wght@400;500;600;700&family=IBM+Plex+Sans:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><rect width='100' height='100' fill='%230b0b0d'/><text x='50' y='68' font-size='60' font-family='Big Shoulders Display,sans-serif' font-weight='900' fill='%23ffc400' text-anchor='middle'>X</text></svg>">
-<link rel="stylesheet" href="/style.css?v=run9-75w14l">
+<link rel="stylesheet" href="/style.css?v=run9-calm-75w14l">
<style>
/* page-local: 5-column variant of the product .bench grid + verdict tones */
- .bench-scroll { overflow-x: auto; max-width: 100%; margin-top: var(--sp-8); }
- .bench.matrix { grid-template-columns: minmax(300px, 1fr) 110px 110px 90px 80px; min-width: 760px; margin-top: 0; }
+ .bench-scroll { overflow-x: auto; max-width: 100%; }
+ .bench.matrix { grid-template-columns: minmax(220px, 1fr) 92px 92px 78px 70px; min-width: 620px; margin-top: 0; }
+ .bench.matrix .head { padding-top: 0; }
.bench.matrix .cell.win { color: var(--z-accent); font-weight: 700; }
.bench.matrix .cell.lose { color: #e05252; font-weight: 700; }
.bench.matrix .cell.na { color: var(--z-mute); font-weight: 400; }
- .bench.matrix .cell.test .note { color: var(--z-mute); font-size: var(--fs-11); }
- .callout { border-left: 1px solid var(--z-accent); padding: var(--sp-2) 0 var(--sp-2) var(--sp-6); margin: var(--sp-8) 0; max-width: 860px; }
- .callout p { max-width: none; }
+
+ /* collapsible scorecard categories — native <details>/<summary>, no JS */
+ .cat { border-bottom: 1px solid var(--z-line); }
+ .cat:first-of-type { border-top: 1px solid var(--z-line); }
+ .cat > summary {
+ list-style: none;
+ cursor: pointer;
+ display: flex;
+ align-items: baseline;
+ justify-content: space-between;
+ gap: var(--sp-6);
+ padding: var(--sp-4) 0;
+ font-family: var(--font-data);
+ }
+ .cat > summary::-webkit-details-marker { display: none; }
+ .cat > summary:hover .cat-label { color: var(--z-accent); }
+ .cat-label {
+ font-size: var(--fs-13);
+ letter-spacing: var(--track-label);
+ text-transform: uppercase;
+ color: var(--z-ink);
+ }
+ .cat-label::before {
+ content: "\25B8";
+ color: var(--z-mute);
+ margin-right: var(--sp-3);
+ font-size: var(--fs-11);
+ }
+ .cat[open] > summary .cat-label::before { content: "\25BE"; }
+ .cat-tally {
+ font-family: var(--font-mono);
+ font-size: var(--fs-13);
+ color: var(--z-mute);
+ white-space: nowrap;
+ font-variant-numeric: tabular-nums;
+ }
+ .cat-tally .w { color: var(--z-accent); font-weight: 700; }
+ .cat-tally .l { color: #e05252; font-weight: 700; }
+ .cat-tally .na { color: var(--z-mute); }
+ .cat[open] .bench-scroll { margin: 0 0 var(--sp-6); }
+
+ .cat-total {
+ display: flex;
+ justify-content: space-between;
+ align-items: baseline;
+ gap: var(--sp-6);
+ padding: var(--sp-4) 0;
+ margin-top: var(--sp-8);
+ border-top: 1px solid var(--z-line);
+ font-family: var(--font-data);
+ font-size: var(--fs-13);
+ letter-spacing: var(--track-label);
+ text-transform: uppercase;
+ color: var(--z-mute);
+ }
+ .cat-total .cat-tally { font-size: var(--fs-16); }
</style>
</head>
<body>
@@ -50,69 +104,41 @@
<div class="kicker">
<span class="accent">BENCHMARKS</span><span class="dash">·</span>
<span>XERJ V1.0.0-RC.1 VS ELASTICSEARCH 8.13.4</span><span class="dash">·</span>
- <span>RUN 9 · 2026-07-06</span>
+ <span>2026-07-06</span>
</div>
<h1 class="hero">EVERY CELL.<br><span class="accent">BOTH</span> DIRECTIONS.</h1>
<p class="lead">
- One harness, two engines, identical workload: 91 measured dimensions
- across ingest, every query / aggregation / pipeline family, mixed
- read-under-write, kNN, and disk. As of the
- <span class="mono">2026-07-06</span> run the honest score is
- <strong>75 WIN · 14 LOSE · 2 N/A</strong> — XERJ wins
- <strong>75 of 91 measured cells</strong>. This run corrects a
- measurement error we found in <em>our own</em> harness: earlier runs
- (run 8 was 53/36, and the 45–53-win runs before it) timed reads
- through Node's global <span class="mono">fetch</span>/undici client,
- which adds <strong>~1.5&nbsp;ms of pure client overhead to every
- request</strong> — enough to swamp the sub-millisecond server response
- of <em>both</em> engines and compress ~40 rows into a client-jitter
- noise band. We reworked the harness to a lean
- <strong>keep-alive HTTP client</strong> (Node core
- <span class="mono">http</span> + a shared per-host
- <span class="mono">http.Agent</span>, <span class="mono">maxSockets:256</span>),
- applied identically to both engines, and cross-checked fairness with
- an independent <span class="mono">curl</span> client. The corrected
- numbers reveal each engine's <strong>true server round-trip</strong> —
- XERJ ~0.1–0.5&nbsp;ms vs ES ~1.5&nbsp;ms — so the read wins are real,
- not a client artifact. Ingest wins <strong>3 of 4 cells</strong>
- (1.34–1.56× throughput); full-text, term, range and wildcard reads win
- outright (<span class="mono">query_string</span> 13.97×,
- <span class="mono">wildcard</span> 14.96×,
- <span class="mono">boosting</span> 49.64×,
- <span class="mono">function_score</span> 54.10×,
- <span class="mono">sort-heavy</span> 16.24×); kNN is
- <strong>5.28× faster at 100% / 100% recall@10 parity</strong>; and the
- on-disk index is <strong>1.26× smaller</strong> than ES
- (644.4&nbsp;MB vs 809.7&nbsp;MB). The 2 N/A rows are queries
- Elasticsearch 8.13 itself rejects
- (<span class="mono">match_phrase_prefix</span>,
- <span class="mono">combined_fields</span> sent through the shared
- harness body) while XERJ answers them. The 14 losses that remain are
- published below, scored against us, with named fixes in flight.
+ One harness, two engines, one machine, identical workload —
+ 91 measured dimensions across ingest, every query, aggregation and
+ pipeline family, mixed read-under-write, kNN and on-disk size. The
+ honest score is <strong>75 WIN · 14 LOSE · 2 N/A</strong>: XERJ wins
+ 75 of the 91 cells. Every loss is published below, next to the wins,
+ and each one fails our CI. Reproduce the whole thing on your own box
+ with four commands.
</p>
<div class="metrics">
<div class="metric">
<div class="eyebrow">XERJ WINS</div>
<div class="value">75</div>
- <div class="hint">Ratio &gt; 1× · lower latency or higher throughput</div>
+ <div class="hint">Lower latency, higher throughput, smaller disk</div>
</div>
<div class="metric">
<div class="eyebrow">XERJ LOSSES</div>
<div class="value">14</div>
- <div class="hint">Published, tracked, and each one fails CI</div>
+ <div class="hint">Published, tracked — each one fails CI</div>
</div>
<div class="metric">
<div class="eyebrow">NOT SCORED</div>
<div class="value">2</div>
- <div class="hint">Queries ES 8.13 itself rejects (400) — never scored in XERJ's favor</div>
+ <div class="hint">Queries ES 8.13 itself rejects (400)</div>
</div>
<div class="metric">
<div class="eyebrow">ES-YAML CONFORMANCE</div>
<div class="value">1326<span class="unit">/0</span></div>
- <div class="hint">Passed / failed · guardrail for every perf change</div>
+ <div class="hint">Passed / failed · the guardrail for every perf change</div>
</div>
</div>
</section>
@@ -124,17 +150,15 @@
<p class="wide">
Both engines run as a single node on the same machine, security off,
- queried over localhost. No containers, no network hop, no cluster
- coordination on either side. Whatever this box gives one engine, it
- gives the other — including the benchmark client, which is now the
- same lean keep-alive HTTP client for both.
+ over localhost — same hardware, same client, byte-for-byte the same
+ request bodies.
</p>
- <div class="row"><div class="k">MACHINE</div><div class="v">AMD Ryzen AI Max+ 395 (w/ Radeon 8060S) · 32 hardware threads (<span class="mono">nproc</span>) · 119&nbsp;GiB RAM (<span class="mono">free&nbsp;-g</span>) · Linux</div></div>
- <div class="row"><div class="k">XERJ</div><div class="v"><span class="mono">xerj v1.0.0-rc.1</span> · release build (<span class="mono">cargo build --release</span>) · <span class="mono">--insecure</span>, fresh data dir · port :9200</div></div>
- <div class="row"><div class="k">ELASTICSEARCH</div><div class="v"><span class="mono">8.13.4</span> official tarball · <span class="mono">xpack.security.enabled: false</span>, <span class="mono">discovery.type: single-node</span> · 4&nbsp;GB heap (<span class="mono">-Xms4g -Xmx4g</span>) · port :9201</div></div>
- <div class="row"><div class="k">CORPUS</div><div class="v">Real LLM-telemetry events (model, status, latency_ms, cost_usd, tokens, tenant, timestamps) — not synthetic filler. Reads run against 1M docs; ingest is measured at 100k and 1M docs with 1 and 8 concurrent clients.</div></div>
- <div class="row"><div class="k">TOPOLOGY</div><div class="v">1 node vs 1 node · same machine · localhost · identical request bodies, byte for byte, through an identical client</div></div>
+ <div class="row"><div class="k">MACHINE</div><div class="v">AMD Ryzen AI Max+ 395 · 32 hardware threads · 119&nbsp;GiB RAM · Linux</div></div>
+ <div class="row"><div class="k">XERJ</div><div class="v"><span class="mono">v1.0.0-rc.1</span> · release build · <span class="mono">--insecure</span>, fresh data dir · port :9200</div></div>
+ <div class="row"><div class="k">ELASTICSEARCH</div><div class="v"><span class="mono">8.13.4</span> official tarball · security off, single-node · 4&nbsp;GB heap · port :9201</div></div>
+ <div class="row"><div class="k">CORPUS</div><div class="v">Real LLM-telemetry events (model, status, latency, cost, tokens, tenant, timestamps) — not synthetic filler. Reads run against 1M docs; ingest is measured at 100k and 1M docs with 1 and 8 concurrent clients.</div></div>
+ <div class="row"><div class="k">TOPOLOGY</div><div class="v">1 node vs 1 node · same machine · localhost · identical request bodies through an identical client</div></div>
</section>
<!-- 02 — METHODOLOGY ----------------------------------------->
@@ -147,586 +171,673 @@
one file, Node builtins only, checked into the repo. Its rules:
</p>
- <div class="row"><div class="k">READ TRANSPORT</div><div class="v">Read / mixed / kNN latencies are timed through a lean <strong>keep-alive HTTP client</strong> — Node core <span class="mono">http</span> + a shared per-host <span class="mono">http.Agent</span> (<span class="mono">maxSockets:256</span>) keyed only by <span class="mono">host:port</span> — applied <strong>identically to both engines</strong>. We do <em>not</em> use Node's global <span class="mono">fetch</span>/undici: it adds ~1.5&nbsp;ms of pure client overhead per request that swamps both engines' sub-millisecond server times. Ingest / mixed / kNN bulk load uses <span class="mono">curl</span>, identically for both.</div></div>
- <div class="row"><div class="k">OPEN-LOOP LOAD</div><div class="v">Read requests fire on a fixed <strong>200&nbsp;req/s</strong> cadence at <span class="mono">t0&nbsp;+&nbsp;i/rate</span>, independent of when earlier responses return. A slow engine cannot slow the clock down and flatter its own tail.</div></div>
- <div class="row"><div class="k">SAMPLING</div><div class="v">Per family: 15 untimed warmup calls, then <strong>p50 over 120 timed iterations</strong>. Each request has a 15&nbsp;s bound — an engine that hangs is recorded as collapsed and scored, not silently dropped.</div></div>
- <div class="row"><div class="k">FEASIBILITY PROBE</div><div class="v">Every family is probed first. A 4xx classifies it <span class="mono">unsupported</span> for that engine; the probe's result signal (hit totals, agg shape) is captured for the correctness check.</div></div>
- <div class="row"><div class="k">RESULT-SIGNAL MISMATCH</div><div class="v">If the two engines return materially different results (e.g. 0 hits vs 277,449), the row is scored <span class="mono">N/A&nbsp;mismatch</span> — <strong>an engine that returns wrong or empty results cannot win on latency</strong>.</div></div>
- <div class="row"><div class="k">IDENTICAL WORK</div><div class="v"><span class="mono">track_total_hits: true</span> is injected into every <span class="mono">_search</span> body on both sides. ES caps hit totals at 10,000 by default; forcing exact totals means neither engine wins by short-circuiting the count.</div></div>
- <div class="row"><div class="k">VERDICTS</div><div class="v">Ratios are normalized so <strong>&gt;1× always means XERJ is better</strong> (lower latency, higher docs/s, smaller disk, higher recall). Any LOSE row makes the runner exit non-zero — the scorecard is a CI gate, not a brochure.</div></div>
-
- <div class="callout">
- <div class="kicker"><span class="accent">HOW WE CAUGHT OUR OWN CLIENT</span><span class="dash">·</span><span>THE UNDICI OVERHEAD BAND · 2026-07-06</span></div>
- <p class="wide">
- Runs 7 and 8 (47- and 53-win) timed reads through Node's global
- <span class="mono">fetch</span>, which is backed by undici. That
- client adds roughly <strong>1.5&nbsp;ms of fixed overhead to every
- request</strong> — connection bookkeeping, header parsing, promise
- plumbing — and both XERJ and ES answer most reads in well under a
- millisecond. The client overhead was larger than either engine's
- server time, so it drowned the signal: ~40 rows landed in a noise
- band where win/lose flipped on client jitter rather than engine
- behaviour. We A/B'd the two transports with the identical client
- hitting both engines (<span class="mono">size:0</span> average over
- 300k requests):
- </p>
- <p class="wide">
- <span class="mono">undici fetch → XERJ 1.61&nbsp;ms · ES 2.77&nbsp;ms</span><br>
- <span class="mono">http keep-alive → XERJ 0.126&nbsp;ms · ES 0.283&nbsp;ms</span>
- </p>
- <p class="wide">
- The keep-alive client reveals each engine's <strong>true server
- round-trip</strong> — and the ~2.2× gap the client overhead had been
- hiding. We verified the fix does not quietly favour XERJ by
- re-running the whole matrix through an independent
- <span class="mono">curl</span> client: XERJ was ≤ ES on every op, and
- ES's reused connections were not penalized. We found our own
- benchmark client inflating <em>both</em> engines' latencies, and we
- corrected it. That is the point of printing this section: a benchmark
- that can embarrass you is the only kind worth trusting when it
- doesn't.
- </p>
- </div>
-
- <div class="callout">
- <div class="kicker"><span class="accent">HOW WE ALMOST FOOLED OURSELVES BEFORE</span><span class="dash">·</span><span>THE QUERY-CACHE MIRAGE · 2026-07-01</span></div>
- <p class="wide">
- An earlier head-to-head showed XERJ winning reads 1.3–2.2× — and the
- numbers were a mirage. Those benchmarks repeated the <em>same</em>
- query against a <em>static</em> index, so XERJ's result cache served
- every call after the first. We were measuring cache hits, not query
- execution: uncached, a <span class="mono">match_all size:10</span>
- actually took <strong>2.28&nbsp;seconds</strong>, because hit
- materialization scanned every matching document instead of the top
- <span class="mono">from+size</span>. We published the finding
- (<span class="mono">demo/playbooks/CRITICAL_FINDING_read_perf_cache_mirage.md</span>),
- fixed the O(N) path to O(from+size), and hardened the harness with
- the mismatch detection and identical-work rules above. Two
- self-corrections, both published — because the scorecard is only
- worth anything if we treat our own errors the way we'd treat a
- competitor's.
- </p>
- </div>
+ <div class="row"><div class="k">READ TRANSPORT</div><div class="v">Read / mixed / kNN latencies are timed through a lean <strong>keep-alive HTTP client</strong> (Node core <span class="mono">http</span> + a shared per-host <span class="mono">http.Agent</span>), applied <strong>identically to both engines</strong> — we avoid Node's global <span class="mono">fetch</span>/undici because it adds ~1.5&nbsp;ms of client overhead per request that would swamp both engines' sub-millisecond server times. Ingest / kNN bulk load uses <span class="mono">curl</span>, identically for both.</div></div>
+ <div class="row"><div class="k">OPEN-LOOP LOAD</div><div class="v">Read requests fire on a fixed <strong>200&nbsp;req/s</strong> cadence, independent of when earlier responses return — a slow engine can't slow the clock down and flatter its own tail.</div></div>
+ <div class="row"><div class="k">SAMPLING</div><div class="v">Per family: 15 untimed warmups, then <strong>p50 over 120 timed iterations</strong>, with a 15&nbsp;s per-request bound. An engine that hangs is recorded and scored, not silently dropped.</div></div>
+ <div class="row"><div class="k">CORRECTNESS GATE</div><div class="v">Every family is probed first; a 4xx marks it <span class="mono">unsupported</span> for that engine. If the two engines return materially different results, the row is scored <span class="mono">N/A</span> — <strong>an engine that returns wrong or empty results can't win on latency</strong>. <span class="mono">track_total_hits: true</span> is forced on both sides so neither wins by short-circuiting the count.</div></div>
+ <div class="row"><div class="k">VERDICTS</div><div class="v">Ratios are normalized so <strong>&gt;1× always means XERJ is better</strong>. Any LOSE row makes the runner exit non-zero — the scorecard is a CI gate, not a brochure.</div></div>
</section>
<!-- 03 — FULL RESULTS ---------------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>FULL RESULTS · 2026-07-06 · NO CHERRY-PICKING</span></div>
+ <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>FULL RESULTS · NO CHERRY-PICKING</span></div>
<h2 class="scene">THE WHOLE<br><span class="accent">SCORECARD.</span></h2>
<p class="wide">
All 91 rows from <span class="mono">demo/playbooks/SCORECARD.md</span>,
- unedited. Latency rows are p50 in milliseconds on the lean keep-alive
- client; ingest rows are docs/s; the mixed rows are p99 under a
- concurrent bulk writer. <strong>The headlines this run:</strong> ingest
- is a WIN in 3 of 4 cells (100k×c1 <span class="mono">1.44×</span>,
- 100k×c8 <span class="mono">1.34×</span>, 1m×c1
- <span class="mono">1.56×</span>; only 1m×c8 loses at
- <span class="mono">0.93×</span>); full-text, term, range and wildcard
- reads win outright (<span class="mono">query_string</span> 13.97×,
- <span class="mono">wildcard</span> 14.96×,
- <span class="mono">regexp</span> 15.84×,
- <span class="mono">range(cost)</span> 10.19×,
- <span class="mono">boosting</span> 49.64×,
- <span class="mono">function_score</span> 54.10×,
- <span class="mono">sort-heavy</span> 16.24×,
- <span class="mono">search_after</span> 4.14×); kNN k=10 is
- <strong>5.28× faster at identical 100% recall@10</strong>; and the
- on-disk index is <strong>1.26× smaller</strong> than ES
- (644.4&nbsp;MB vs 809.7&nbsp;MB). The two rows marked N/A are queries
- ES 8.13 itself answers with a 400 through the shared harness body
- (<span class="mono">match_phrase_prefix</span>,
- <span class="mono">combined_fields</span>) while XERJ executes them —
- never scored in XERJ's favor. The red that remains falls into three
- honest groups, named below the table.
+ grouped by family. Categories that contain a loss are open by default;
+ the all-win families are collapsed — open any of them to see every row.
+ Latency rows are p50 in milliseconds, ingest is docs/s, and the mixed
+ rows are p99 under a concurrent bulk writer. Ratios are normalized so
+ <strong>&gt;1× means XERJ is faster / higher-throughput / smaller</strong>.
</p>
- <div class="bench-scroll">
- <div class="bench matrix">
- <div class="head">DIMENSION</div>
- <div class="head num">XERJ</div>
- <div class="head num">ELASTICSEARCH</div>
- <div class="head num">RATIO</div>
- <div class="head num">VERDICT</div>
- <div class="cell test">ingest 100k × c1 (docs/s)</div>
- <div class="cell num xy">103,126</div>
- <div class="cell num es">71,432</div>
- <div class="cell num ratio win">1.44×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">ingest 100k × c8 (docs/s)</div>
- <div class="cell num xy">355,363</div>
- <div class="cell num es">264,935</div>
- <div class="cell num ratio win">1.34×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">ingest 1m × c1 (docs/s)</div>
- <div class="cell num xy">109,543</div>
- <div class="cell num es">70,197</div>
- <div class="cell num ratio win">1.56×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">ingest 1m × c8 (docs/s)</div>
- <div class="cell num xy">338,555</div>
- <div class="cell num es">365,530</div>
- <div class="cell num ratio lose">0.93×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read q: match_all (p50 ms)</div>
- <div class="cell num xy">0.78</div>
- <div class="cell num es">1.08</div>
- <div class="cell num ratio win">1.38×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: match_none (p50 ms)</div>
- <div class="cell num xy">0.57</div>
- <div class="cell num es">0.77</div>
- <div class="cell num ratio win">1.35×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: match(model) (p50 ms)</div>
- <div class="cell num xy">0.54</div>
- <div class="cell num es">0.96</div>
- <div class="cell num ratio win">1.77×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: match_phrase(top_doc) (p50 ms)</div>
- <div class="cell num xy">0.53</div>
- <div class="cell num es">0.93</div>
- <div class="cell num ratio win">1.75×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: match_phrase_prefix (p50 ms)</div>
- <div class="cell num xy">0.46</div>
- <div class="cell num es">unsupported (400)</div>
- <div class="cell num ratio na">—</div>
- <div class="cell num verdict na">N/A</div>
- <div class="cell test">read q: match_bool_prefix (p50 ms)</div>
- <div class="cell num xy">0.56</div>
- <div class="cell num es">3.91</div>
- <div class="cell num ratio win">6.94×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: multi_match (p50 ms)</div>
- <div class="cell num xy">0.74</div>
- <div class="cell num es">5.17</div>
- <div class="cell num ratio win">7.03×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: combined_fields (p50 ms)</div>
- <div class="cell num xy">0.45</div>
- <div class="cell num es">unsupported (400)</div>
- <div class="cell num ratio na">—</div>
- <div class="cell num verdict na">N/A</div>
- <div class="cell test">read q: query_string (p50 ms)</div>
- <div class="cell num xy">0.61</div>
- <div class="cell num es">8.46</div>
- <div class="cell num ratio win">13.97×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: simple_query_string (p50 ms)</div>
- <div class="cell num xy">1.33</div>
- <div class="cell num es">1.18</div>
- <div class="cell num ratio lose">0.89×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read q: more_like_this (p50 ms)</div>
- <div class="cell num xy">0.46</div>
- <div class="cell num es">1.12</div>
- <div class="cell num ratio win">2.42×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: term(status) (p50 ms)</div>
- <div class="cell num xy">0.69</div>
- <div class="cell num es">1.29</div>
- <div class="cell num ratio win">1.86×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: terms(model) (p50 ms)</div>
- <div class="cell num xy">0.50</div>
- <div class="cell num es">4.33</div>
- <div class="cell num ratio win">8.57×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: range(latency_ms) (p50 ms)</div>
- <div class="cell num xy">0.51</div>
- <div class="cell num es">4.69</div>
- <div class="cell num ratio win">9.20×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: range(@timestamp) (p50 ms)</div>
- <div class="cell num xy">0.48</div>
- <div class="cell num es">0.75</div>
- <div class="cell num ratio win">1.56×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: range(cost_usd) (p50 ms)</div>
- <div class="cell num xy">0.57</div>
- <div class="cell num es">5.81</div>
- <div class="cell num ratio win">10.19×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: prefix(model) (p50 ms)</div>
- <div class="cell num xy">0.69</div>
- <div class="cell num es">8.50</div>
- <div class="cell num ratio win">12.24×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: wildcard(model) (p50 ms)</div>
- <div class="cell num xy">0.58</div>
- <div class="cell num es">8.65</div>
- <div class="cell num ratio win">14.96×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: regexp(model) (p50 ms)</div>
- <div class="cell num xy">0.55</div>
- <div class="cell num es">8.73</div>
- <div class="cell num ratio win">15.84×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: fuzzy(model) (p50 ms)</div>
- <div class="cell num xy">0.61</div>
- <div class="cell num es">1.53</div>
- <div class="cell num ratio win">2.49×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: exists(cost_usd) (p50 ms)</div>
- <div class="cell num xy">0.83</div>
- <div class="cell num es">1.44</div>
- <div class="cell num ratio win">1.74×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: ids (p50 ms)</div>
- <div class="cell num xy">0.46</div>
- <div class="cell num es">0.53</div>
- <div class="cell num ratio win">1.15×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: term(cache_hit) (p50 ms)</div>
- <div class="cell num xy">0.49</div>
- <div class="cell num es">0.92</div>
- <div class="cell num ratio win">1.88×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: bool must+filter+should+must_not (p50 ms)</div>
- <div class="cell num xy">1.33</div>
- <div class="cell num es">15.49</div>
- <div class="cell num ratio win">11.61×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: constant_score (p50 ms)</div>
- <div class="cell num xy">0.55</div>
- <div class="cell num es">0.90</div>
- <div class="cell num ratio win">1.64×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: boosting (p50 ms)</div>
- <div class="cell num xy">0.68</div>
- <div class="cell num es">33.95</div>
- <div class="cell num ratio win">49.64×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: dis_max (p50 ms)</div>
- <div class="cell num xy">0.61</div>
- <div class="cell num es">4.10</div>
- <div class="cell num ratio win">6.76×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: function_score (p50 ms)</div>
- <div class="cell num xy">0.72</div>
- <div class="cell num es">38.87</div>
- <div class="cell num ratio win">54.10×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read q: pinned (p50 ms)</div>
- <div class="cell num xy">0.41</div>
- <div class="cell num es">14.53</div>
- <div class="cell num ratio win">35.21×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: avg (p50 ms)</div>
- <div class="cell num xy">0.65</div>
- <div class="cell num es">1.18</div>
- <div class="cell num ratio win">1.80×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: sum (p50 ms)</div>
- <div class="cell num xy">0.30</div>
- <div class="cell num es">1.27</div>
- <div class="cell num ratio win">4.25×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: min (p50 ms)</div>
- <div class="cell num xy">0.50</div>
- <div class="cell num es">1.19</div>
- <div class="cell num ratio win">2.37×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: max (p50 ms)</div>
- <div class="cell num xy">0.44</div>
- <div class="cell num es">1.02</div>
- <div class="cell num ratio win">2.33×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: stats (p50 ms)</div>
- <div class="cell num xy">0.34</div>
- <div class="cell num es">1.30</div>
- <div class="cell num ratio win">3.79×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: extended_stats (p50 ms)</div>
- <div class="cell num xy">0.35</div>
- <div class="cell num es">1.39</div>
- <div class="cell num ratio win">4.00×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: value_count (p50 ms)</div>
- <div class="cell num xy">1.11</div>
- <div class="cell num es">1.17</div>
- <div class="cell num ratio win">1.05×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: cardinality (p50 ms)</div>
- <div class="cell num xy">0.32</div>
- <div class="cell num es">1.12</div>
- <div class="cell num ratio win">3.53×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: percentiles (p50 ms)</div>
- <div class="cell num xy">1.11</div>
- <div class="cell num es">1.31</div>
- <div class="cell num ratio win">1.18×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: percentile_ranks (p50 ms)</div>
- <div class="cell num xy">1.14</div>
- <div class="cell num es">1.28</div>
- <div class="cell num ratio win">1.12×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: median_absolute_deviation (p50 ms)</div>
- <div class="cell num xy">0.55</div>
- <div class="cell num es">1.51</div>
- <div class="cell num ratio win">2.77×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: matrix_stats (p50 ms)</div>
- <div class="cell num xy">1.22</div>
- <div class="cell num es">2.21</div>
- <div class="cell num ratio win">1.82×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: scripted_metric (p50 ms)</div>
- <div class="cell num xy">1.22</div>
- <div class="cell num es">2.14</div>
- <div class="cell num ratio win">1.75×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: top_hits (sub) (p50 ms)</div>
- <div class="cell num xy">0.54</div>
- <div class="cell num es">1.46</div>
- <div class="cell num ratio win">2.69×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: terms (p50 ms)</div>
- <div class="cell num xy">1.14</div>
- <div class="cell num es">1.48</div>
- <div class="cell num ratio win">1.29×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: rare_terms (p50 ms)</div>
- <div class="cell num xy">0.78</div>
- <div class="cell num es">2.02</div>
- <div class="cell num ratio win">2.59×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: significant_terms (p50 ms)</div>
- <div class="cell num xy">0.47</div>
- <div class="cell num es">2.07</div>
- <div class="cell num ratio win">4.40×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: histogram (p50 ms)</div>
- <div class="cell num xy">0.37</div>
- <div class="cell num es">0.69</div>
- <div class="cell num ratio win">1.87×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: date_histogram (p50 ms)</div>
- <div class="cell num xy">0.46</div>
- <div class="cell num es">2.00</div>
- <div class="cell num ratio win">4.33×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: auto_date_histogram (p50 ms)</div>
- <div class="cell num xy">0.47</div>
- <div class="cell num es">0.94</div>
- <div class="cell num ratio win">2.00×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: variable_width_histogram (p50 ms)</div>
- <div class="cell num xy">0.51</div>
- <div class="cell num es">2.17</div>
- <div class="cell num ratio win">4.22×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: range (p50 ms)</div>
- <div class="cell num xy">1.16</div>
- <div class="cell num es">0.67</div>
- <div class="cell num ratio lose">0.58×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read agg: date_range (p50 ms)</div>
- <div class="cell num xy">0.35</div>
- <div class="cell num es">1.53</div>
- <div class="cell num ratio win">4.39×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: filter (p50 ms)</div>
- <div class="cell num xy">1.26</div>
- <div class="cell num es">1.22</div>
- <div class="cell num ratio lose">0.97×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read agg: filters (p50 ms)</div>
- <div class="cell num xy">0.37</div>
- <div class="cell num es">1.08</div>
- <div class="cell num ratio win">2.89×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: missing (p50 ms)</div>
- <div class="cell num xy">0.43</div>
- <div class="cell num es">0.62</div>
- <div class="cell num ratio win">1.43×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: global (p50 ms)</div>
- <div class="cell num xy">1.15</div>
- <div class="cell num es">0.62</div>
- <div class="cell num ratio lose">0.54×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read agg: adjacency_matrix (p50 ms)</div>
- <div class="cell num xy">0.38</div>
- <div class="cell num es">2.02</div>
- <div class="cell num ratio win">5.36×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: composite (p50 ms)</div>
- <div class="cell num xy">1.21</div>
- <div class="cell num es">0.72</div>
- <div class="cell num ratio lose">0.60×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read agg: random_sampler (p50 ms)</div>
- <div class="cell num xy">0.36</div>
- <div class="cell num es">4.82</div>
- <div class="cell num ratio win">13.24×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read agg: terms+avg(cost) (p50 ms)</div>
- <div class="cell num xy">1.22</div>
- <div class="cell num es">0.76</div>
- <div class="cell num ratio lose">0.63×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read pipe: sum_bucket (p50 ms)</div>
- <div class="cell num xy">0.39</div>
- <div class="cell num es">0.74</div>
- <div class="cell num ratio win">1.91×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: avg_bucket (p50 ms)</div>
- <div class="cell num xy">1.20</div>
- <div class="cell num es">2.15</div>
- <div class="cell num ratio win">1.80×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: max_bucket (p50 ms)</div>
- <div class="cell num xy">0.37</div>
- <div class="cell num es">0.71</div>
- <div class="cell num ratio win">1.93×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: stats_bucket (p50 ms)</div>
- <div class="cell num xy">0.59</div>
- <div class="cell num es">2.15</div>
- <div class="cell num ratio win">3.66×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: percentiles_bucket (p50 ms)</div>
- <div class="cell num xy">1.19</div>
- <div class="cell num es">1.39</div>
- <div class="cell num ratio win">1.17×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: derivative (p50 ms)</div>
- <div class="cell num xy">0.34</div>
- <div class="cell num es">2.11</div>
- <div class="cell num ratio win">6.31×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: cumulative_sum (p50 ms)</div>
- <div class="cell num xy">1.19</div>
- <div class="cell num es">2.07</div>
- <div class="cell num ratio win">1.74×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: moving_fn (p50 ms)</div>
- <div class="cell num xy">0.36</div>
- <div class="cell num es">0.75</div>
- <div class="cell num ratio win">2.08×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: serial_diff (p50 ms)</div>
- <div class="cell num xy">1.19</div>
- <div class="cell num es">0.89</div>
- <div class="cell num ratio lose">0.75×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read pipe: bucket_script (p50 ms)</div>
- <div class="cell num xy">0.33</div>
- <div class="cell num es">0.65</div>
- <div class="cell num ratio win">1.97×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: bucket_selector (p50 ms)</div>
- <div class="cell num xy">1.17</div>
- <div class="cell num es">2.17</div>
- <div class="cell num ratio win">1.86×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read pipe: bucket_sort (p50 ms)</div>
- <div class="cell num xy">0.43</div>
- <div class="cell num es">0.70</div>
- <div class="cell num ratio win">1.62×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: sort-heavy (p50 ms)</div>
- <div class="cell num xy">1.04</div>
- <div class="cell num es">16.95</div>
- <div class="cell num ratio win">16.24×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: deep from+size (from 500) (p50 ms)</div>
- <div class="cell num xy">1.41</div>
- <div class="cell num es">1.21</div>
- <div class="cell num ratio lose">0.86×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">read feat: search_after (p50 ms)</div>
- <div class="cell num xy">4.34</div>
- <div class="cell num es">17.97</div>
- <div class="cell num ratio win">4.14×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: highlight (p50 ms)</div>
- <div class="cell num xy">1.35</div>
- <div class="cell num es">1.94</div>
- <div class="cell num ratio win">1.44×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: _count (p50 ms)</div>
- <div class="cell num xy">0.24</div>
- <div class="cell num es">1.73</div>
- <div class="cell num ratio win">7.30×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: _msearch (p50 ms)</div>
- <div class="cell num xy">0.98</div>
- <div class="cell num es">1.29</div>
- <div class="cell num ratio win">1.32×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">read feat: _mget (p50 ms)</div>
- <div class="cell num xy">0.21</div>
- <div class="cell num es">0.47</div>
- <div class="cell num ratio win">2.23×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">mixed match_all (p99 ms, under write)</div>
- <div class="cell num xy">76.55</div>
- <div class="cell num es">2.27</div>
- <div class="cell num ratio lose">0.03×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">mixed bool (p99 ms, under write)</div>
- <div class="cell num xy">82.14</div>
- <div class="cell num es">9.49</div>
- <div class="cell num ratio lose">0.12×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">mixed range (p99 ms, under write)</div>
- <div class="cell num xy">171.28</div>
- <div class="cell num es">4.71</div>
- <div class="cell num ratio lose">0.03×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">mixed terms (p99 ms, under write)</div>
- <div class="cell num xy">75.60</div>
- <div class="cell num es">2.76</div>
- <div class="cell num ratio lose">0.04×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">mixed cardinality (p99 ms, under write)</div>
- <div class="cell num xy">81.84</div>
- <div class="cell num es">17.33</div>
- <div class="cell num ratio lose">0.21×</div>
- <div class="cell num verdict lose">LOSE</div>
- <div class="cell test">kNN k=10 (p50 ms)</div>
- <div class="cell num xy">0.57</div>
- <div class="cell num es">3.02</div>
- <div class="cell num ratio win">5.28×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">kNN recall@10</div>
- <div class="cell num xy">100.0%</div>
- <div class="cell num es">100.0%</div>
- <div class="cell num ratio win">1.00×</div>
- <div class="cell num verdict win">WIN</div>
- <div class="cell test">index on-disk size</div>
- <div class="cell num xy">644.4 MB</div>
- <div class="cell num es">809.7 MB</div>
- <div class="cell num ratio win">1.26×</div>
- <div class="cell num verdict win">WIN</div>
- </div>
+ <div class="cat-total">
+ <span>Total · 91 dimensions</span>
+ <span class="cat-tally"><span class="w">75&nbsp;W</span> · <span class="l">14&nbsp;L</span> · <span class="na">2&nbsp;N/A</span></span>
</div>
- <p class="wide" style="margin-top: var(--sp-6);">
- The 14 losses cluster into three honest groups. <strong>(1) Mixed
- read-under-write — 5 rows.</strong>
- <span class="mono">match_all / bool / range / terms / cardinality</span>
- fired while a bulk writer runs land at p99
- <span class="mono">76–171&nbsp;ms</span> against ES's quiesced
- <span class="mono">2–17&nbsp;ms</span>. This is a
- <strong>real XERJ weakness</strong> — read tail latency under sustained
- flush/merge write pressure — and worth stating plainly that XERJ was
- also under heavier write load this run (~113k vs ~76k&nbsp;docs/s), so
- it was doing more work while being read. <strong>(2) ~8 marginal
- near-ties.</strong> <span class="mono">simple_query_string</span>, aggs
- <span class="mono">range / filter / global / composite /
- terms+avg</span>, pipe <span class="mono">serial_diff</span>, and deep
- <span class="mono">from+size</span> — sub-1.5&nbsp;ms rows where ES
- edges XERJ by tenths of a millisecond. Genuine, small.
- <strong>(3) Ingest 1m × 8 clients</strong> at
- <span class="mono">0.93×</span> (338k vs 366k docs/s): ES still wins
- high-concurrency bulk ingest at scale. All three are active work items,
- tracked below.
- </p>
+ <!-- Ingest -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Ingest</span>
+ <span class="cat-tally"><span class="w">3&nbsp;W</span> · <span class="l">1&nbsp;L</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">ingest 100k × c1</div>
+ <div class="cell num xy">103,126</div>
+ <div class="cell num es">71,432</div>
+ <div class="cell num ratio win">1.44×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">ingest 100k × c8</div>
+ <div class="cell num xy">355,363</div>
+ <div class="cell num es">264,935</div>
+ <div class="cell num ratio win">1.34×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">ingest 1m × c1</div>
+ <div class="cell num xy">109,543</div>
+ <div class="cell num es">70,197</div>
+ <div class="cell num ratio win">1.56×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">ingest 1m × c8</div>
+ <div class="cell num xy">338,555</div>
+ <div class="cell num es">365,530</div>
+ <div class="cell num ratio lose">0.93×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Full-text & phrase queries -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Full-text &amp; phrase queries</span>
+ <span class="cat-tally"><span class="w">8&nbsp;W</span> · <span class="l">1&nbsp;L</span> · <span class="na">2&nbsp;N/A</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">match_all</div>
+ <div class="cell num xy">0.78</div>
+ <div class="cell num es">1.08</div>
+ <div class="cell num ratio win">1.38×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">match_none</div>
+ <div class="cell num xy">0.57</div>
+ <div class="cell num es">0.77</div>
+ <div class="cell num ratio win">1.35×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">match(model)</div>
+ <div class="cell num xy">0.54</div>
+ <div class="cell num es">0.96</div>
+ <div class="cell num ratio win">1.77×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">match_phrase(top_doc)</div>
+ <div class="cell num xy">0.53</div>
+ <div class="cell num es">0.93</div>
+ <div class="cell num ratio win">1.75×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">match_phrase_prefix</div>
+ <div class="cell num xy">0.46</div>
+ <div class="cell num es">unsupported (400)</div>
+ <div class="cell num ratio na">—</div>
+ <div class="cell num verdict na">N/A</div>
+ <div class="cell test">match_bool_prefix</div>
+ <div class="cell num xy">0.56</div>
+ <div class="cell num es">3.91</div>
+ <div class="cell num ratio win">6.94×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">multi_match</div>
+ <div class="cell num xy">0.74</div>
+ <div class="cell num es">5.17</div>
+ <div class="cell num ratio win">7.03×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">combined_fields</div>
+ <div class="cell num xy">0.45</div>
+ <div class="cell num es">unsupported (400)</div>
+ <div class="cell num ratio na">—</div>
+ <div class="cell num verdict na">N/A</div>
+ <div class="cell test">query_string</div>
+ <div class="cell num xy">0.61</div>
+ <div class="cell num es">8.46</div>
+ <div class="cell num ratio win">13.97×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">simple_query_string</div>
+ <div class="cell num xy">1.33</div>
+ <div class="cell num es">1.18</div>
+ <div class="cell num ratio lose">0.89×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">more_like_this</div>
+ <div class="cell num xy">0.46</div>
+ <div class="cell num es">1.12</div>
+ <div class="cell num ratio win">2.42×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Term, range & filter queries -->
+ <details class="cat">
+ <summary>
+ <span class="cat-label">Term, range &amp; filter queries</span>
+ <span class="cat-tally"><span class="w">12&nbsp;W</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">term(status)</div>
+ <div class="cell num xy">0.69</div>
+ <div class="cell num es">1.29</div>
+ <div class="cell num ratio win">1.86×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">terms(model)</div>
+ <div class="cell num xy">0.50</div>
+ <div class="cell num es">4.33</div>
+ <div class="cell num ratio win">8.57×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">range(latency_ms)</div>
+ <div class="cell num xy">0.51</div>
+ <div class="cell num es">4.69</div>
+ <div class="cell num ratio win">9.20×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">range(@timestamp)</div>
+ <div class="cell num xy">0.48</div>
+ <div class="cell num es">0.75</div>
+ <div class="cell num ratio win">1.56×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">range(cost_usd)</div>
+ <div class="cell num xy">0.57</div>
+ <div class="cell num es">5.81</div>
+ <div class="cell num ratio win">10.19×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">prefix(model)</div>
+ <div class="cell num xy">0.69</div>
+ <div class="cell num es">8.50</div>
+ <div class="cell num ratio win">12.24×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">wildcard(model)</div>
+ <div class="cell num xy">0.58</div>
+ <div class="cell num es">8.65</div>
+ <div class="cell num ratio win">14.96×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">regexp(model)</div>
+ <div class="cell num xy">0.55</div>
+ <div class="cell num es">8.73</div>
+ <div class="cell num ratio win">15.84×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">fuzzy(model)</div>
+ <div class="cell num xy">0.61</div>
+ <div class="cell num es">1.53</div>
+ <div class="cell num ratio win">2.49×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">exists(cost_usd)</div>
+ <div class="cell num xy">0.83</div>
+ <div class="cell num es">1.44</div>
+ <div class="cell num ratio win">1.74×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">ids</div>
+ <div class="cell num xy">0.46</div>
+ <div class="cell num es">0.53</div>
+ <div class="cell num ratio win">1.15×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">term(cache_hit)</div>
+ <div class="cell num xy">0.49</div>
+ <div class="cell num es">0.92</div>
+ <div class="cell num ratio win">1.88×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Compound & relevance queries -->
+ <details class="cat">
+ <summary>
+ <span class="cat-label">Compound &amp; relevance queries</span>
+ <span class="cat-tally"><span class="w">6&nbsp;W</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">bool must+filter+should+must_not</div>
+ <div class="cell num xy">1.33</div>
+ <div class="cell num es">15.49</div>
+ <div class="cell num ratio win">11.61×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">constant_score</div>
+ <div class="cell num xy">0.55</div>
+ <div class="cell num es">0.90</div>
+ <div class="cell num ratio win">1.64×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">boosting</div>
+ <div class="cell num xy">0.68</div>
+ <div class="cell num es">33.95</div>
+ <div class="cell num ratio win">49.64×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">dis_max</div>
+ <div class="cell num xy">0.61</div>
+ <div class="cell num es">4.10</div>
+ <div class="cell num ratio win">6.76×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">function_score</div>
+ <div class="cell num xy">0.72</div>
+ <div class="cell num es">38.87</div>
+ <div class="cell num ratio win">54.10×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">pinned</div>
+ <div class="cell num xy">0.41</div>
+ <div class="cell num es">14.53</div>
+ <div class="cell num ratio win">35.21×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Metric aggregations -->
+ <details class="cat">
+ <summary>
+ <span class="cat-label">Metric aggregations</span>
+ <span class="cat-tally"><span class="w">14&nbsp;W</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">avg</div>
+ <div class="cell num xy">0.65</div>
+ <div class="cell num es">1.18</div>
+ <div class="cell num ratio win">1.80×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">sum</div>
+ <div class="cell num xy">0.30</div>
+ <div class="cell num es">1.27</div>
+ <div class="cell num ratio win">4.25×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">min</div>
+ <div class="cell num xy">0.50</div>
+ <div class="cell num es">1.19</div>
+ <div class="cell num ratio win">2.37×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">max</div>
+ <div class="cell num xy">0.44</div>
+ <div class="cell num es">1.02</div>
+ <div class="cell num ratio win">2.33×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">stats</div>
+ <div class="cell num xy">0.34</div>
+ <div class="cell num es">1.30</div>
+ <div class="cell num ratio win">3.79×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">extended_stats</div>
+ <div class="cell num xy">0.35</div>
+ <div class="cell num es">1.39</div>
+ <div class="cell num ratio win">4.00×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">value_count</div>
+ <div class="cell num xy">1.11</div>
+ <div class="cell num es">1.17</div>
+ <div class="cell num ratio win">1.05×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">cardinality</div>
+ <div class="cell num xy">0.32</div>
+ <div class="cell num es">1.12</div>
+ <div class="cell num ratio win">3.53×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">percentiles</div>
+ <div class="cell num xy">1.11</div>
+ <div class="cell num es">1.31</div>
+ <div class="cell num ratio win">1.18×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">percentile_ranks</div>
+ <div class="cell num xy">1.14</div>
+ <div class="cell num es">1.28</div>
+ <div class="cell num ratio win">1.12×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">median_absolute_deviation</div>
+ <div class="cell num xy">0.55</div>
+ <div class="cell num es">1.51</div>
+ <div class="cell num ratio win">2.77×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">matrix_stats</div>
+ <div class="cell num xy">1.22</div>
+ <div class="cell num es">2.21</div>
+ <div class="cell num ratio win">1.82×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">scripted_metric</div>
+ <div class="cell num xy">1.22</div>
+ <div class="cell num es">2.14</div>
+ <div class="cell num ratio win">1.75×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">top_hits (sub)</div>
+ <div class="cell num xy">0.54</div>
+ <div class="cell num es">1.46</div>
+ <div class="cell num ratio win">2.69×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Bucket aggregations -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Bucket aggregations</span>
+ <span class="cat-tally"><span class="w">12&nbsp;W</span> · <span class="l">5&nbsp;L</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">terms</div>
+ <div class="cell num xy">1.14</div>
+ <div class="cell num es">1.48</div>
+ <div class="cell num ratio win">1.29×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">rare_terms</div>
+ <div class="cell num xy">0.78</div>
+ <div class="cell num es">2.02</div>
+ <div class="cell num ratio win">2.59×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">significant_terms</div>
+ <div class="cell num xy">0.47</div>
+ <div class="cell num es">2.07</div>
+ <div class="cell num ratio win">4.40×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">histogram</div>
+ <div class="cell num xy">0.37</div>
+ <div class="cell num es">0.69</div>
+ <div class="cell num ratio win">1.87×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">date_histogram</div>
+ <div class="cell num xy">0.46</div>
+ <div class="cell num es">2.00</div>
+ <div class="cell num ratio win">4.33×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">auto_date_histogram</div>
+ <div class="cell num xy">0.47</div>
+ <div class="cell num es">0.94</div>
+ <div class="cell num ratio win">2.00×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">variable_width_histogram</div>
+ <div class="cell num xy">0.51</div>
+ <div class="cell num es">2.17</div>
+ <div class="cell num ratio win">4.22×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">range</div>
+ <div class="cell num xy">1.16</div>
+ <div class="cell num es">0.67</div>
+ <div class="cell num ratio lose">0.58×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">date_range</div>
+ <div class="cell num xy">0.35</div>
+ <div class="cell num es">1.53</div>
+ <div class="cell num ratio win">4.39×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">filter</div>
+ <div class="cell num xy">1.26</div>
+ <div class="cell num es">1.22</div>
+ <div class="cell num ratio lose">0.97×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">filters</div>
+ <div class="cell num xy">0.37</div>
+ <div class="cell num es">1.08</div>
+ <div class="cell num ratio win">2.89×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">missing</div>
+ <div class="cell num xy">0.43</div>
+ <div class="cell num es">0.62</div>
+ <div class="cell num ratio win">1.43×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">global</div>
+ <div class="cell num xy">1.15</div>
+ <div class="cell num es">0.62</div>
+ <div class="cell num ratio lose">0.54×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">adjacency_matrix</div>
+ <div class="cell num xy">0.38</div>
+ <div class="cell num es">2.02</div>
+ <div class="cell num ratio win">5.36×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">composite</div>
+ <div class="cell num xy">1.21</div>
+ <div class="cell num es">0.72</div>
+ <div class="cell num ratio lose">0.60×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">random_sampler</div>
+ <div class="cell num xy">0.36</div>
+ <div class="cell num es">4.82</div>
+ <div class="cell num ratio win">13.24×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">terms+avg(cost)</div>
+ <div class="cell num xy">1.22</div>
+ <div class="cell num es">0.76</div>
+ <div class="cell num ratio lose">0.63×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Pipeline aggregations -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Pipeline aggregations</span>
+ <span class="cat-tally"><span class="w">11&nbsp;W</span> · <span class="l">1&nbsp;L</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">sum_bucket</div>
+ <div class="cell num xy">0.39</div>
+ <div class="cell num es">0.74</div>
+ <div class="cell num ratio win">1.91×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">avg_bucket</div>
+ <div class="cell num xy">1.20</div>
+ <div class="cell num es">2.15</div>
+ <div class="cell num ratio win">1.80×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">max_bucket</div>
+ <div class="cell num xy">0.37</div>
+ <div class="cell num es">0.71</div>
+ <div class="cell num ratio win">1.93×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">stats_bucket</div>
+ <div class="cell num xy">0.59</div>
+ <div class="cell num es">2.15</div>
+ <div class="cell num ratio win">3.66×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">percentiles_bucket</div>
+ <div class="cell num xy">1.19</div>
+ <div class="cell num es">1.39</div>
+ <div class="cell num ratio win">1.17×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">derivative</div>
+ <div class="cell num xy">0.34</div>
+ <div class="cell num es">2.11</div>
+ <div class="cell num ratio win">6.31×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">cumulative_sum</div>
+ <div class="cell num xy">1.19</div>
+ <div class="cell num es">2.07</div>
+ <div class="cell num ratio win">1.74×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">moving_fn</div>
+ <div class="cell num xy">0.36</div>
+ <div class="cell num es">0.75</div>
+ <div class="cell num ratio win">2.08×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">serial_diff</div>
+ <div class="cell num xy">1.19</div>
+ <div class="cell num es">0.89</div>
+ <div class="cell num ratio lose">0.75×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">bucket_script</div>
+ <div class="cell num xy">0.33</div>
+ <div class="cell num es">0.65</div>
+ <div class="cell num ratio win">1.97×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">bucket_selector</div>
+ <div class="cell num xy">1.17</div>
+ <div class="cell num es">2.17</div>
+ <div class="cell num ratio win">1.86×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">bucket_sort</div>
+ <div class="cell num xy">0.43</div>
+ <div class="cell num es">0.70</div>
+ <div class="cell num ratio win">1.62×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Query features -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Query features</span>
+ <span class="cat-tally"><span class="w">6&nbsp;W</span> · <span class="l">1&nbsp;L</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">sort-heavy</div>
+ <div class="cell num xy">1.04</div>
+ <div class="cell num es">16.95</div>
+ <div class="cell num ratio win">16.24×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">deep from+size (from 500)</div>
+ <div class="cell num xy">1.41</div>
+ <div class="cell num es">1.21</div>
+ <div class="cell num ratio lose">0.86×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">search_after</div>
+ <div class="cell num xy">4.34</div>
+ <div class="cell num es">17.97</div>
+ <div class="cell num ratio win">4.14×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">highlight</div>
+ <div class="cell num xy">1.35</div>
+ <div class="cell num es">1.94</div>
+ <div class="cell num ratio win">1.44×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">_count</div>
+ <div class="cell num xy">0.24</div>
+ <div class="cell num es">1.73</div>
+ <div class="cell num ratio win">7.30×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">_msearch</div>
+ <div class="cell num xy">0.98</div>
+ <div class="cell num es">1.29</div>
+ <div class="cell num ratio win">1.32×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">_mget</div>
+ <div class="cell num xy">0.21</div>
+ <div class="cell num es">0.47</div>
+ <div class="cell num ratio win">2.23×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Mixed — read under write -->
+ <details class="cat" open>
+ <summary>
+ <span class="cat-label">Mixed — read under write</span>
+ <span class="cat-tally"><span class="l">5&nbsp;L</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION (p99, under write)</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">match_all</div>
+ <div class="cell num xy">76.55</div>
+ <div class="cell num es">2.27</div>
+ <div class="cell num ratio lose">0.03×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">bool</div>
+ <div class="cell num xy">82.14</div>
+ <div class="cell num es">9.49</div>
+ <div class="cell num ratio lose">0.12×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">range</div>
+ <div class="cell num xy">171.28</div>
+ <div class="cell num es">4.71</div>
+ <div class="cell num ratio lose">0.03×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">terms</div>
+ <div class="cell num xy">75.60</div>
+ <div class="cell num es">2.76</div>
+ <div class="cell num ratio lose">0.04×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ <div class="cell test">cardinality</div>
+ <div class="cell num xy">81.84</div>
+ <div class="cell num es">17.33</div>
+ <div class="cell num ratio lose">0.21×</div>
+ <div class="cell num verdict lose">LOSE</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Vector & kNN -->
+ <details class="cat">
+ <summary>
+ <span class="cat-label">Vector &amp; kNN</span>
+ <span class="cat-tally"><span class="w">2&nbsp;W</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">kNN k=10 (p50 ms)</div>
+ <div class="cell num xy">0.57</div>
+ <div class="cell num es">3.02</div>
+ <div class="cell num ratio win">5.28×</div>
+ <div class="cell num verdict win">WIN</div>
+ <div class="cell test">kNN recall@10</div>
+ <div class="cell num xy">100.0%</div>
+ <div class="cell num es">100.0%</div>
+ <div class="cell num ratio win">1.00×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
+
+ <!-- Storage -->
+ <details class="cat">
+ <summary>
+ <span class="cat-label">Storage</span>
+ <span class="cat-tally"><span class="w">1&nbsp;W</span></span>
+ </summary>
+ <div class="bench-scroll">
+ <div class="bench matrix">
+ <div class="head">DIMENSION</div>
+ <div class="head num">XERJ</div>
+ <div class="head num">ES</div>
+ <div class="head num">RATIO</div>
+ <div class="head num">VERDICT</div>
+ <div class="cell test">index on-disk size</div>
+ <div class="cell num xy">644.4 MB</div>
+ <div class="cell num es">809.7 MB</div>
+ <div class="cell num ratio win">1.26×</div>
+ <div class="cell num verdict win">WIN</div>
+ </div>
+ </div>
+ </details>
</section>
<!-- 04 — REPRODUCE IT ---------------------------------------->
@@ -735,7 +846,7 @@
<h2 class="scene">FOUR COMMANDS.<br><span class="accent">YOUR MACHINE.</span></h2>
<p class="wide">
- Everything on this page regenerates from the repo. No hosted harness,
+ Everything on this page regenerates from the repo — no hosted harness,
no private dataset, no hand-tuned engine flags.
</p>
@@ -744,15 +855,15 @@
<span class="prompt">$</span> bash scratchpad/es_up.sh
<span class="prompt">$</span> bash scratchpad/run_scorecard.sh --docs 100k,1m --clients 1,8 --knn --mixed</pre>
- <div class="row"><div class="k">es_up.sh</div><div class="v">Downloads the official Elasticsearch <span class="mono">8.13.4</span> linux-x86_64 tarball (cached after the first run), writes a single-node config with security off on port <span class="mono">:9201</span>, boots it with a 4&nbsp;GB heap, and polls until it answers 200. Idempotent — re-running is a no-op if ES is already up.</div></div>
- <div class="row"><div class="k">run_scorecard.sh</div><div class="v">Boots the release XERJ binary on <span class="mono">:9200</span> with a fresh data dir, keeps it alive for the whole run, then executes the matrix against both engines and shuts XERJ down. Exits non-zero if any row is a LOSE.</div></div>
- <div class="row"><div class="k">HARNESS</div><div class="v"><span class="mono">demo/playbooks/bench-matrix.mjs</span> — the matrix runner and scorecard generator (Node 24, no dependencies). Reads are timed through a lean <strong>keep-alive HTTP client</strong> (Node core <span class="mono">http</span> + a shared per-host <span class="mono">http.Agent</span>, <span class="mono">maxSockets:256</span>), applied identically to both engines; ingest / mixed / kNN bulk load uses <span class="mono">curl</span>, identically for both. Output lands in <span class="mono">demo/playbooks/SCORECARD.md</span>, which is the exact file this page's table is rendered from.</div></div>
+ <div class="row"><div class="k">es_up.sh</div><div class="v">Downloads the official Elasticsearch <span class="mono">8.13.4</span> tarball (cached after the first run), configures a single node with security off on <span class="mono">:9201</span>, and boots it with a 4&nbsp;GB heap. Idempotent.</div></div>
+ <div class="row"><div class="k">run_scorecard.sh</div><div class="v">Boots the release XERJ binary on <span class="mono">:9200</span> with a fresh data dir, runs the matrix against both engines, and shuts XERJ down. Exits non-zero if any row is a LOSE.</div></div>
+ <div class="row"><div class="k">HARNESS</div><div class="v"><span class="mono">demo/playbooks/bench-matrix.mjs</span> — the runner and scorecard generator (Node 24, no dependencies). Output lands in <span class="mono">demo/playbooks/SCORECARD.md</span>, the exact file this page's table is rendered from.</div></div>
<p class="wide">
- Your absolute numbers will differ with hardware; the ratios and
- verdicts are the claim. If your run disagrees with this page, file an
- issue with your <span class="mono">SCORECARD.md</span> — that is
- precisely what the harness is for.
+ Your absolute numbers will differ with hardware — the ratios and
+ verdicts are the claim. If your run disagrees, file an issue with your
+ <span class="mono">SCORECARD.md</span>; that is exactly what the harness
+ is for.
</p>
</section>
@@ -761,103 +872,26 @@
<div class="kicker"><span class="accent">05</span><span class="dash">·</span><span>KNOWN ISSUES &amp; WHAT'S NEXT</span></div>
<h2 class="scene">THE RED CELLS<br><span class="accent">ARE THE ROADMAP.</span></h2>
- <h3 class="section">Mixed read-under-write — 5 rows</h3>
- <p class="wide">
- Under a concurrent 8-client bulk-write stream, XERJ's read p99 still
- loses: <span class="mono">match_all / bool / range / terms /
- cardinality</span> fired while the writer runs land at
- <strong>76–171&nbsp;ms</strong> against ES's quiesced
- <strong>2–17&nbsp;ms</strong>. This is a genuine XERJ weakness, not a
- client artifact: read tail latency degrades under sustained
- flush/merge write pressure. It is worth noting XERJ was also carrying
- the heavier write load in this run (~113k vs ~76k&nbsp;docs/s) — it was
- doing more work while being read — but that does not excuse the tail.
- None of these 5 rows is scored in XERJ's favor, and shrinking the
- read-under-write tail to ES's steady-state is the top active work item.
- </p>
-
- <h3 class="section">The marginal near-tie band</h3>
- <p class="wide">
- A cluster of losses is a handful of aggregations and queries where both
- engines answer in <strong>well under 1.5&nbsp;ms</strong> and ES edges
- XERJ by tenths of a millisecond —
- <span class="mono">simple_query_string</span>, aggs
- <span class="mono">range</span>, <span class="mono">filter</span>,
- <span class="mono">global</span>, <span class="mono">composite</span>,
- <span class="mono">terms+avg(cost)</span>, pipe
- <span class="mono">serial_diff</span>, and deep
- <span class="mono">from+size</span>. These are per-request-overhead
- deltas, not algorithmic gaps, and shaving that fixed floor is under
- active profiling.
- </p>
-
- <h3 class="section">Ingest at 8 clients — one real cell left</h3>
- <p class="wide">
- Ingest is now a <strong>WIN in 3 of 4 cells</strong>: single-client
- throughput wins at both corpus sizes
- (100k <span class="mono">1.44×</span>, 1M <span class="mono">1.56×</span>)
- and 8-client at 100k wins <span class="mono">1.34×</span>. The one
- remaining ingest loss is 1M × 8 clients: 338k&nbsp;docs/s to ES's
- 366k (<span class="mono">0.93×</span>). Profiling points at a
- <strong>flush-finalize CPU ceiling</strong> — segment finalization
- serializes on a single core while writers wait. The fix is in
- flight (<span class="mono">demo/playbooks/BEAT_ES_MASTER_PLAN.md</span>).
- Definition of done is unchanged: every cell in the scorecard green,
- enforced by CI — any new LOSE fails the build.
- </p>
-
- <h3 class="section">Production hardening landed along the way</h3>
- <p class="wide">
- The perf work did not come at the expense of durability — the
- opposite: mappings now <strong>persist durably across
- restarts</strong>; a <strong>merge-race segment-lease</strong>
- mechanism keeps in-flight queries pinned to live segments (verified
- with 601k queries under continuous merge churn — zero mismatches);
- WAL and segment <strong>garbage is cleaned up</strong> instead of
- accumulating; and a <strong>critical merge data-loss defect</strong>
- was found and fixed along the way. This run adds a
- <strong>dedicated, deprioritised ingest pool</strong> so bulk writes
- can no longer occupy the query executor,
- <strong>cooperative query timeouts</strong> so a slow query yields
- instead of wedging its worker, and a fix for a
- <strong>quadratic FTS segment-write path</strong>.
- </p>
-
- <h3 class="section">Families this corpus can't measure</h3>
- <p class="wide">
- Skipped, not hidden — each needs a purpose-built index the flat
- telemetry corpus lacks: <span class="mono">geo_*</span> queries and
- aggregations (no geo_point/geo_shape field),
- <span class="mono">ip_range / ip_prefix</span> (no ip field),
- <span class="mono">nested / has_child / has_parent</span> (flat
- corpus, no join mapping), <span class="mono">span_*</span> (needs a
- positional text field), <span class="mono">significant_text</span>
- (corpus fields are keyword), <span class="mono">semantic / hybrid
- retriever</span> (needs a dense_vector field — kNN is covered
- separately by <span class="mono">--knn</span> on a purpose-built
- index), and <span class="mono">percolate</span> (parses but no-ops —
- not benchmarkable for correctness). Purpose-built corpora for these
- families are planned follow-ups.
- </p>
-
- <h3 class="section">The standing guardrail</h3>
- <p class="wide">
- Every performance change must hold ES-YAML REST conformance at
- <strong>1326 passed / 0 failed</strong>. Speed bought with correctness
- is not a win, and it does not merge.
- </p>
+ <div class="row"><div class="k">Mixed read-under-write · 5 rows</div><div class="v">The real weakness. With a concurrent bulk writer running, read p99 lands at <strong>76–171&nbsp;ms</strong> against ES's quiesced <strong>2–17&nbsp;ms</strong> — read tail latency degrades under sustained flush/merge pressure. Shrinking it to ES's steady-state is the top active work item.</div></div>
+ <div class="row"><div class="k">Marginal near-ties · 8 rows</div><div class="v">A handful of sub-1.5&nbsp;ms rows where ES edges XERJ by tenths of a millisecond (<span class="mono">simple_query_string</span>, aggs <span class="mono">range / filter / global / composite / terms+avg</span>, pipe <span class="mono">serial_diff</span>, deep <span class="mono">from+size</span>). Per-request overhead, not algorithmic gaps — under profiling.</div></div>
+ <div class="row"><div class="k">Ingest 1M × 8 clients · 1 row</div><div class="v">338k vs 366k&nbsp;docs/s (<span class="mono">0.93×</span>). ES still wins high-concurrency bulk ingest at scale; profiling points at a flush-finalize CPU ceiling. Fix in flight — the other 3 ingest cells already win.</div></div>
+ <div class="row"><div class="k">Hardening along the way</div><div class="v">Durable mappings across restarts, a merge-race segment lease (601k queries under merge churn, zero mismatches), WAL/segment garbage collection, a merge data-loss fix, a dedicated deprioritised ingest pool, and cooperative query timeouts.</div></div>
+ <div class="row"><div class="k">Not measured here</div><div class="v">Skipped, not hidden — each needs a purpose-built index the flat corpus lacks: <span class="mono">geo_*</span>, <span class="mono">ip_range / ip_prefix</span>, <span class="mono">nested / has_child / has_parent</span>, <span class="mono">span_*</span>, <span class="mono">significant_text</span>, <span class="mono">semantic / hybrid</span> retrieval, and <span class="mono">percolate</span>. Purpose-built corpora are planned follow-ups.</div></div>
+ <div class="row"><div class="k">The standing guardrail</div><div class="v">Every performance change must hold ES-YAML REST conformance at <strong>1326 passed / 0 failed</strong>. Speed bought with correctness is not a win, and it does not merge.</div></div>
</section>
<!-- 06 — CHANGELOG -------------------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">06</span><span class="dash">·</span><span>CHANGELOG</span></div>
- <div class="row"><div class="k">2026-07-06 (run 9)</div><div class="v">75 WIN: corrected the read-latency transport. Earlier runs timed reads through Node's global <span class="mono">fetch</span>/undici, which adds ~1.5&nbsp;ms of pure client overhead per request — enough to swamp both engines' sub-millisecond server times and compress ~40 rows into a client-jitter band that flipped win/lose on noise. Reworked the harness to a lean keep-alive HTTP client (Node core <span class="mono">http</span> + a shared per-host <span class="mono">http.Agent</span>, <span class="mono">maxSockets:256</span>), applied identically to both engines and cross-checked with an independent <span class="mono">curl</span> client (XERJ ≤ ES on every op). True server round-trips now measured: full-text / term / range / wildcard reads win outright (query_string 13.97×, wildcard 14.96×, boosting 49.64×, function_score 54.10×, sort-heavy 16.24×), kNN 5.28× at 100% recall parity, on-disk index 1.26× smaller (644.4 MB vs 809.7 MB), ingest wins 3 of 4 cells. 14 honest losses remain: mixed read-under-write p99 (76–171 ms vs ES 2–17 ms), ~8 sub-1.5 ms near-tie aggs/queries, and 1m×c8 ingest (0.93×) · 75 WIN / 14 LOSE / 2 N/A. Supersedes run 8.</div></div>
- <div class="row"><div class="k">2026-07-06 (run 8 — SUPERSEDED)</div><div class="v">Reported 53 WIN / 36 LOSE / 2 N/A. Read latencies were measured through Node's global <span class="mono">fetch</span>/undici client, which added ~1.5&nbsp;ms of fixed per-request overhead to both engines and hid their true sub-millisecond server times — roughly 40 rows sat in a client-jitter noise band. Transport corrected and re-run honestly as run 9 above · was 53 WIN / 36 LOSE / 2 N/A, superseded by run 9.</div></div>
- <div class="row"><div class="k">2026-07-06 (run 7 — SUPERSEDED)</div><div class="v">First honest read numbers after the query-cache fix, and first run with the harder mixed read-under-write cluster added to the matrix — but still timed through the undici client · was 47 WIN / 42 LOSE / 2 N/A, superseded by run 9.</div></div>
- <div class="row"><div class="k">2026-07-05 (run 6 — RETRACTED)</div><div class="v">Retracted as a query-cache measurement artifact. The 77-win headline was inflated: read wins were served from a result cache the harness was not invalidating under test, and the 6.22× / 114.5&nbsp;MB disk and kNN figures were post-forcemerge-quiesced, not steady-state. Measurement fixed (commit 2ff3e9a) and re-run honestly as runs 7–9 above · was 77 WIN / 12 LOSE / 2 N/A, superseded by run 9.</div></div>
- <div class="row"><div class="k">2026-07-05</div><div class="v">Run 5: agg-hang cluster eliminated, text queries fixed, board clean (no collapses / mismatches); mixed-tail + request-overhead + c8 flush fixes in flight · 45 WIN / 41 LOSE / 5 N/A.</div></div>
- <div class="row"><div class="k">2026-07-04 (late)</div><div class="v">Run 4: clean rerun after the round-1 fixes. c1 ingest and search_after flip to WIN; agg-hang cluster isolated (~27 rows, fix in flight) · 35 WIN / 47 LOSE / 9 N/A.</div></div>
- <div class="row"><div class="k">2026-07-04</div><div class="v">First public full-matrix publication. XERJ v1.0.0-rc.1 vs Elasticsearch 8.13.4 · 91 dimensions · 43 WIN / 33 LOSE / 15 N/A · run truncated by a search_after OOM defect, since fixed and verified in run 4 above.</div></div>
+ <div class="kicker"><span class="accent">06</span><span class="dash">·</span><span>TESTING CHANGELOG</span></div>
+ <div class="row"><div class="k">2026-07-06 (run 9)</div><div class="v">75 WIN. Corrected the read-latency transport: earlier runs timed reads through Node's global <span class="mono">fetch</span>/undici (~1.5&nbsp;ms of client overhead per request), which swamped both engines' sub-millisecond server times and pushed ~40 rows into a noise band. Re-ran on a lean keep-alive HTTP client, applied identically to both engines and cross-checked with <span class="mono">curl</span> (XERJ ≤ ES on every op). True server round-trips now show full-text / term / range / wildcard reads winning outright, kNN 5.28× at 100% recall parity, index 1.26× smaller, ingest 3 of 4 cells. 14 honest losses remain · 75 W / 14 L / 2 N/A. Supersedes run 8.</div></div>
+ <div class="row"><div class="k">2026-07-06 (run 8 — SUPERSEDED)</div><div class="v">Reported 53 WIN / 36 LOSE / 2 N/A. Reads were still timed through the undici client, whose fixed per-request overhead hid both engines' true sub-millisecond server times — roughly 40 rows sat in a client-jitter band. Transport fixed and re-run honestly as run 9.</div></div>
+ <div class="row"><div class="k">2026-07-06 (run 7 — SUPERSEDED)</div><div class="v">First honest read numbers after the query-cache fix, and first run with the harder mixed read-under-write cluster added — but still timed through undici · was 47 WIN / 42 LOSE / 2 N/A.</div></div>
+ <div class="row"><div class="k">2026-07-05 (run 6 — RETRACTED)</div><div class="v">Retracted as a measurement artifact: the 77-win headline leaned on post-forcemerge-quiesced disk/kNN figures and a result cache that wasn't being invalidated under test — steady-state it didn't hold. Fixed and re-run as runs 7–9 · was 77 WIN / 12 LOSE / 2 N/A.</div></div>
+ <div class="row"><div class="k">2026-07-05 (run 5)</div><div class="v">Agg-hang cluster eliminated, text queries fixed, board clean (no collapses / mismatches); mixed-tail + request-overhead + c8 flush fixes in flight · 45 WIN / 41 LOSE / 5 N/A.</div></div>
+ <div class="row"><div class="k">2026-07-04 (late, run 4)</div><div class="v">Clean rerun after the round-1 fixes. c1 ingest and search_after flip to WIN; agg-hang cluster isolated (~27 rows, fix in flight) · 35 WIN / 47 LOSE / 9 N/A.</div></div>
+ <div class="row"><div class="k">2026-07-04</div><div class="v">First public full-matrix publication. XERJ v1.0.0-rc.1 vs Elasticsearch 8.13.4 · 91 dimensions · 43 WIN / 33 LOSE / 15 N/A · run truncated by a search_after OOM defect, since fixed.</div></div>
+ <div class="row"><div class="k">2026-07-01</div><div class="v">The query-cache mirage — our favourite bug to have caught before anyone else did. An early head-to-head had XERJ winning reads by a suspiciously cheerful margin; turned out the harness was replaying the <em>same</em> query against a <em>static</em> index, and our result cache was happily serving every call after the first — so we were benchmarking the cache, not the engine. Uncached, a <span class="mono">match_all size:10</span> actually took ~2.28&nbsp;s, because hit materialization scanned every match instead of the top <span class="mono">from+size</span>. We fixed that path from O(N) to O(from+size), taught the harness to invalidate and to flag result-signal mismatches, and re-ran honestly. Filed under: if the numbers look too good, they are.</div></div>
+ <div class="row"><div class="k">earlier · pre-public</div><div class="v">Development runs. Many un-numbered iterations shaking out the harness and the engine before any of this was fit to publish — the usual loop of find a bug, fix it, and distrust the result until it holds up.</div></div>
</section>
<footer class="footer">