Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

test(perf): add the rc.12 measurement harness and capture the baseline

xerj-orga28302c60d61
diff --git a/demo/playbooks/rc12/BASELINE.md b/demo/playbooks/rc12/BASELINE.md
new file mode 100644
index 000000000..fb03f3941
--- /dev/null
+++ b/demo/playbooks/rc12/BASELINE.md
@@ -0,0 +1,119 @@
+# rc.12 baseline — measured 2026-08-07 on `main` @ 8a6fa38 (v1.0.0-rc.11)
+
+Produced by `demo/playbooks/rc12/measure_rc12.sh baseline-main 500000`.
+Raw numbers: `results-baseline-main.json`.
+
+**Corpus.** 500,000 documents, one index, force-merged to a single segment so we
+measure steady state. Mapping is explicit (6 keyword, 1 analyzed `text`, 4
+numeric, 1 date, 1 boolean). The `body` field is 40 tokens drawn **Zipfian from a
+60,000-term vocabulary** — this matters: an earlier version of this harness used a
+24-word vocabulary and produced a 0.0595x ratio and uniformly sub-0.5ms queries,
+which flattered both axes into meaninglessness. Realistic term diversity is a
+precondition for this measurement being worth anything.
+
+---
+
+## Axis 1 — disk
+
+| | value |
+|---|---:|
+| raw source bytes | 356,593,195 (340 MiB) |
+| index on disk | 154,413,168 (147 MiB) |
+| **index / raw** | **0.433x** |
+| ingest | 33.0 s (≈15.2k docs/s) |
+| force-merge to 1 segment | **116.52 s** |
+
+Where the bytes are:
+
+| artifact | bytes | share | what it is |
+|---|---:|---:|---|
+| **`.dv`** | **57,357,593** | **37.1%** | doc-values — full-value column for every string, *including the analyzed body* |
+| `.seg` | 51,449,817 | 33.3% | stored `_source`, zstd |
+| `.post` | 40,951,589 | 26.5% | postings incl. positions |
+| `.ids` | 3,057,388 | 2.0% | external-id → ordinal index, stored raw |
+| `.meta` | 1,464,327 | 0.9% | per-term metadata |
+| `.fst` | 64,485 | 0.0% | term dictionary |
+| `.norms` | 28,305 | 0.0% | per-doc field length |
+
+**`.dv` is the largest artifact at 37.1%**, independently corroborating discussion
+#148's 34% measurement on a completely different corpus (WooCommerce source). See
+`EVIDENCE-doc-values-on-text.md` for the isolated experiment proving doc-values are
+built for `text` fields at all, and that `"doc_values": false` is silently ignored.
+
+---
+
+## Axis 2 — query latency, and why "2x" cannot honestly be claimed here
+
+| query | p50 ms | p99 ms | engine `took` | hits |
+|---|---:|---:|---:|---:|
+| term_keyword_low_card | 0.30 | 0.49 | **0** | 10,000 |
+| term_keyword_high_card | 0.21 | 0.56 | **0** | 1 |
+| range_date_narrow | 0.33 | 0.58 | **0** | 50 |
+| range_long_wide | 0.30 | 0.46 | **0** | 10,000 |
+| match_text_common | 0.29 | 0.36 | **0** | 10,000 |
+| match_text_multi | 0.31 | 0.41 | **0** | 10,000 |
+| match_phrase | 0.28 | 0.54 | **0** | 10,000 |
+| bool_must_filter | 0.37 | 0.74 | **0** | 10,000 |
+| deep_page (from 9000) | 1.04 | 1.16 | **0** | 10,000 |
+| large_page (size 500) | **4.72** | 5.94 | **0** | 10,000 |
+| sort_by_numeric | 1.17 | 1.38 | **0** | 10,000 |
+| agg_terms_low_card | 0.21 | 0.30 | **0** | 10,000 |
+| agg_terms_high_card | 0.24 | 0.35 | **0** | 10,000 |
+| agg_stats_numeric | 0.19 | 0.31 | **0** | 10,000 |
+| agg_date_histogram | 0.33 | 0.46 | **0** | 10,000 |
+| agg_histogram_numeric | 0.23 | 0.38 | **0** | 10,000 |
+| agg_nested_terms_stats | 0.21 | 0.61 | **0** | 10,000 |
+
+### The engine reports `took: 0` for every single query.
+
+That is the headline, and it constrains what rc.12 may claim. At 500k documents on
+a warm single segment, XERJ's own timer cannot resolve any of these queries, and
+the 0.2–0.6 ms wall-clock is HTTP, JSON serialisation, and loopback — not engine
+work. **You cannot demonstrate a 2x speedup on a quantity that already measures
+zero.** Any "2x faster" claim built on these shapes would be measuring the harness,
+not the engine, which is exactly the saturation artifact that invalidated an
+earlier round of XERJ benchmarking.
+
+Only three shapes do measurable work, and they are dominated by *materialisation*
+rather than matching:
+
+- `large_page` (4.72 ms) — hydrating 500 documents from `_source`
+- `sort_by_numeric` (1.17 ms) — sorting 500k docs by a numeric column
+- `deep_page` (1.04 ms) — skipping 9,000 documents
+
+Note also that `hits` saturates at exactly **10,000** for every broad query; that is
+the scored-total cap, not a corpus property.
+
+### Where a real, measurable 2x actually lives
+
+On this evidence the honest performance targets for rc.12 are **not** point-query
+latency. They are:
+
+1. **Force-merge: 116.52 s for 500k docs.** Real, large, and directly measurable.
+ This is also the number that any merge-time compression lever must be scored
+ against — the 2026-07-09 playbook's merge-path zstd-19 proposal raises merge CPU
+ ~8x, and it must not be scheduled without measuring it here first.
+2. **Ingest: 33.0 s (15.2k docs/s).**
+3. **Hydration/paging cost**, visible in `large_page`.
+
+To make point-query latency measurable at all would require a substantially larger
+corpus or a closed-loop concurrent load generator. Until such a harness exists, the
+correct statement for rc.12 is that query latency is **at the measurement floor and
+must not regress** — a guard, not an improvement target.
+
+---
+
+## How to reproduce and compare
+
+```sh
+# baseline (already captured)
+demo/playbooks/rc12/measure_rc12.sh baseline-main 500000
+
+# after a change, on the same corpus and doc count
+demo/playbooks/rc12/measure_rc12.sh <change-label> 500000
+
+# then diff results-baseline-main.json against results-<change-label>.json
+```
+
+The corpus generator is seeded (`random.Random(20261207)`), so both runs index a
+byte-identical corpus and the A/B is valid.
diff --git a/demo/playbooks/rc12/EVIDENCE-doc-values-on-text.md b/demo/playbooks/rc12/EVIDENCE-doc-values-on-text.md
new file mode 100644
index 000000000..17fa6672d
--- /dev/null
+++ b/demo/playbooks/rc12/EVIDENCE-doc-values-on-text.md
@@ -0,0 +1,97 @@
+# Evidence: doc-values are built for `text` fields, and `"doc_values": false` is ignored
+
+Measured 2026-08-07 against `xerj v1.0.0-rc.11` (commit `8a6fa38`), live instance.
+Two independent experiments. Both are reproducible with the commands shown.
+
+This file records *empirical* evidence for two claims in discussion #148. It does
+not project a saving — see the rc.12 plan for that arithmetic.
+
+---
+
+## Experiment 1 — `"doc_values": false` is accepted, stored, and silently ignored
+
+This is a **correctness bug**, not only a footprint issue: XERJ accepts a mapping
+option, echoes it back on `GET _mapping`, and then does not honour it.
+
+```sh
+curl -s -X PUT localhost:9200/dvtest -H 'Content-Type: application/json' -d '{
+ "mappings":{"properties":{
+ "kw_dv_off":{"type":"keyword","doc_values":false},
+ "kw_dv_on":{"type":"keyword"},
+ "body":{"type":"text"}}}}'
+```
+
+`GET /dvtest/_mapping` returns the field with `"doc_values": false` intact — so the
+option survives round-trip. Then, with two documents indexed:
+
+```sh
+# Aggregate on the doc_values:false field
+curl -s -X POST localhost:9200/dvtest/_search -H 'Content-Type: application/json' \
+ -d '{"size":0,"aggs":{"a":{"terms":{"field":"kw_dv_off"}}}}'
+```
+
+**Result: it succeeds**, returning `buckets: [{alpha,1},{beta,1}]`. Sorting on the
+same field also succeeds and returns a populated `sort` key.
+
+Elasticsearch fails both with an explicit error, because doc-values are the only
+structure that can serve an aggregation or a sort on a keyword field. XERJ
+succeeding proves the doc-values were built regardless of the mapping.
+
+**Why this matters beyond bytes.** A user who sets `"doc_values": false` is making
+an explicit storage decision. Today they get neither the saving nor an error, and
+nothing in the response tells them the option did nothing. Whatever we decide about
+the footprint lever, the honest options are to *honour* the flag or to *reject* it —
+silently ignoring an accepted option is the one behaviour that cannot be defended.
+
+---
+
+## Experiment 2 — a `text`-only index still builds a full `.dv` column
+
+The sharper test: an index whose mapping contains **exactly one field, of type
+`text`**. There is no keyword, no numeric, no date. Lucene builds **zero**
+doc-values for such an index, because you cannot sort or aggregate on an analyzed
+body and the bytes would be unusable.
+
+```sh
+curl -s -X PUT localhost:9200/dvtext -H 'Content-Type: application/json' \
+ -d '{"mappings":{"properties":{"body":{"type":"text"}}}}'
+# 2,000 docs, each a 60-token body drawn from a 5,000-term vocabulary
+# ... bulk index ... then:
+curl -s -X POST "localhost:9200/dvtext/_forcemerge?max_num_segments=1"
+curl -s -X POST "localhost:9200/dvtext/_flush"
+```
+
+Segment artifacts after force-merge to a single segment:
+
+| artifact | bytes | share |
+|---|---:|---:|
+| `.seg` stored `_source` | 39,656 | 43.3% |
+| **`.dv` doc-values** | **36,333** | **39.7%** |
+| `.post` postings | 9,142 | 10.0% |
+| `.fst` term dictionary | 187 | 0.2% |
+| **total** | **85,318** | |
+
+**The doc-values column is four times the size of the postings** on a pure-text
+index, and it is storing the full value of every analyzed body — the one thing
+that can never be sorted or aggregated on.
+
+This is consistent with the RFC's measurement on the WooCommerce corpus (`.dv` =
+22.92 MiB of 68.14 MiB = 34%). The isolated experiment shows the effect is not an
+artifact of that corpus's keyword fields: it is inherent to how XERJ treats `text`.
+
+---
+
+## What this does and does not establish
+
+**Established:** doc-values are built for analyzed `text` fields; the
+`"doc_values": false` mapping option is accepted and ignored; on a pure-text index
+the resulting column is ~40% of the segment.
+
+**Not established here:** the net saving on a realistic mixed corpus, and — the
+crux — what currently *reads* `.dv` for a text field. XERJ's read path grew
+doc-values prefilters for `term`/`terms`/`range`/`bool`, and the `size:0` count
+shortcut reads from doc-values too. If any of those fire on `text` fields, naively
+dropping the column moves queries onto a slower path instead of saving anything,
+and the disk win would be paid for in latency. That interaction must be settled
+before this lever is scheduled — it is the difference between a real win and a
+trade.
diff --git a/demo/playbooks/rc12/measure_rc12.sh b/demo/playbooks/rc12/measure_rc12.sh
new file mode 100755
index 000000000..bf6f401f8
--- /dev/null
+++ b/demo/playbooks/rc12/measure_rc12.sh
@@ -0,0 +1,277 @@
+#!/usr/bin/env bash
+# rc.12 gate harness — measures BOTH axes the release must defend:
+# (1) on-disk bytes per artifact type for the same logical corpus
+# (2) query latency across the shapes rc.12 claims to improve
+#
+# Methodology follows demo/playbooks/DISK_SIZE_2026-07-09.md so the numbers are
+# comparable to that baseline: deterministic corpus, single index, force-merged
+# to one segment so we measure STEADY STATE rather than a transient many-segment
+# ingest state (that conflation is what produced the bogus "XERJ ~2x ES disk"
+# claim historically).
+#
+# Usage:
+# measure_rc12.sh <label> [docs]
+# Writes results to demo/playbooks/rc12/results-<label>.json and prints a table.
+#
+# Run it once on the base commit and once after a change; compare the JSONs.
+set -euo pipefail
+
+LABEL="${1:?usage: measure_rc12.sh <label> [docs]}"
+DOCS="${2:-100000}"
+PORT="${XERJ_BENCH_PORT:-9400}"
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
+BIN="${XERJ_BIN:-$ROOT/engine/target/release/xerj}"
+DATA="${XERJ_BENCH_DATA:-/tmp/claude-1001/xerj-rc12-bench/$LABEL}"
+OUT="$ROOT/demo/playbooks/rc12/results-$LABEL.json"
+IDX="rc12bench"
+
+[ -x "$BIN" ] || { echo "no xerj binary at $BIN — build it first" >&2; exit 2; }
+mkdir -p "$(dirname "$OUT")"
+
+echo "==> rc.12 harness: label=$LABEL docs=$DOCS port=$PORT"
+echo " binary: $BIN"
+echo " data: $DATA"
+
+# ── fresh instance ───────────────────────────────────────────────────────────
+# There is no --port flag; the listen ports come from a TOML config.
+pkill -f "[x]erj -c $DATA/xerj.toml" 2>/dev/null || true
+sleep 1
+rm -rf "$DATA"; mkdir -p "$DATA"
+cat > "$DATA/xerj.toml" <<EOF
+[server]
+bind_address = "127.0.0.1"
+es_compat_port = $PORT
+rest_port = $((PORT + 1))
+grpc_port = $((PORT + 2))
+EOF
+nohup "$BIN" -c "$DATA/xerj.toml" --data-dir "$DATA" --insecure \
+ > "$DATA/server.log" 2>&1 &
+disown
+
+URL="http://localhost:$PORT"
+for _ in $(seq 1 120); do
+ curl -fsS -m 3 "$URL/_cluster/health" 2>/dev/null | grep -q '"status":"\(green\|yellow\)"' && break
+ sleep 1
+done
+curl -fsS -m 5 "$URL/_cluster/health" >/dev/null || { echo "server never came up" >&2; exit 1; }
+echo "==> server up"
+
+# ── corpus: deterministic, mixed-type (keyword + text + numeric + date + bool) ─
+# Mirrors the 2026-07-09 baseline shape so artifact shares are comparable.
+CORPUS="$DATA/corpus.ndjson"
+python3 - "$DOCS" "$IDX" "$CORPUS" <<'PY'
+import json, random, sys
+n, idx, out = int(sys.argv[1]), sys.argv[2], sys.argv[3]
+
+# The `body` text must have REALISTIC term diversity or the measurement is
+# worthless: a small vocabulary makes postings compress absurdly well and makes
+# every text query trivially fast, which flatters both axes. Real text is
+# Zipfian over a large vocabulary, so we synthesise that explicitly.
+# Seeded, so the corpus is byte-identical between runs and A/B is valid.
+rng = random.Random(20261207)
+VOCAB_N = 60000
+STEMS = ("index segment posting token merge flush shard query filter score rank "
+ "cache buffer stream codec vector cluster commit replica offset column "
+ "record field parse encode decode compress bitmap ordinal dictionary").split()
+VOCAB = [f"{STEMS[i % len(STEMS)]}{i//len(STEMS):04d}" for i in range(VOCAB_N)]
+# Zipf weights: a few very common terms, a long tail of rare ones.
+# Draw ONE large pool up front rather than sampling per token: per-call weighted
+# sampling over a 60k vocabulary rebuilds the cumulative table every time and is
+# far too slow at corpus scale. A rolling window over the pool preserves the
+# distribution while making generation linear.
+WEIGHTS = [1.0 / (i + 1) ** 1.07 for i in range(VOCAB_N)]
+POOL_N = 2_000_003 # prime, so the rolling window does not resonate
+POOL = rng.choices(VOCAB, weights=WEIGHTS, k=POOL_N)
+
+MODELS = ["fable-5", "opus-5", "sonnet-5", "haiku-4-5"]
+STATUS = ["ok", "error", "timeout", "throttled"]
+REGION = ["us-east-1", "eu-west-2", "ap-south-1"]
+with open(out, "w") as f:
+ for i in range(n):
+ # ~40 tokens drawn Zipfian — drives realistic positional postings and
+ # gives phrase/match queries a real term-frequency distribution.
+ off = (i * 41) % (POOL_N - 40)
+ body = " ".join(POOL[off:off + 40])
+ doc = {
+ "doc_id": f"doc-{i:08d}", # high-cardinality keyword
+ "model": MODELS[i % len(MODELS)], # low-cardinality keyword
+ "status": STATUS[i % len(STATUS)],
+ "region": REGION[i % len(REGION)],
+ "session": f"sess-{i % 5000:05d}",
+ "tenant": f"tenant-{i % 97:03d}",
+ "body": body, # analyzed text
+ "latency_ms": (i * 37) % 5000, # integer-structured
+ "context_tokens": 1000 + (i % 100000), # monotonic-ish
+ "cost_usd": round((i % 997) / 1000.0, 4),# float
+ "retries": i % 5, # tiny range
+ "score": round(((i * 13) % 1000) / 100.0, 2),
+ "ts": 1700000000000 + i * 1000, # epoch-ms, monotonic
+ "cached": (i % 3 == 0), # boolean
+ }
+ f.write(json.dumps({"index": {"_index": idx, "_id": doc["doc_id"]}}) + "\n")
+ f.write(json.dumps(doc) + "\n")
+print(f"corpus: {n} docs -> {out}", file=sys.stderr)
+PY
+
+RAW_BYTES=$(python3 - "$CORPUS" <<'PY'
+import json,sys
+# raw = the source documents only, excluding bulk action lines
+tot=0
+with open(sys.argv[1]) as f:
+ for i,l in enumerate(f):
+ if i % 2 == 1: tot += len(l.encode())
+print(tot)
+PY
+)
+echo "==> corpus raw source bytes: $RAW_BYTES"
+
+# ── mapping: explicit, so the measurement is not at the mercy of inference ────
+curl -fsS -X PUT "$URL/$IDX" -H 'Content-Type: application/json' -d '{
+ "mappings": {"properties": {
+ "doc_id":{"type":"keyword"}, "model":{"type":"keyword"},
+ "status":{"type":"keyword"}, "region":{"type":"keyword"},
+ "session":{"type":"keyword"}, "tenant":{"type":"keyword"},
+ "body":{"type":"text"},
+ "latency_ms":{"type":"long"}, "context_tokens":{"type":"long"},
+ "cost_usd":{"type":"double"}, "retries":{"type":"long"},
+ "score":{"type":"double"}, "ts":{"type":"date"}, "cached":{"type":"boolean"}
+ }}}' > /dev/null
+echo "==> mapping created"
+
+# ── ingest (timed) ───────────────────────────────────────────────────────────
+INGEST_START=$(date +%s.%N)
+split -l 20000 "$CORPUS" "$DATA/chunk-"
+for c in "$DATA"/chunk-*; do
+ curl -fsS -X POST "$URL/_bulk" -H 'Content-Type: application/x-ndjson' \
+ --data-binary "@$c" > /dev/null
+done
+curl -fsS -X POST "$URL/$IDX/_refresh" > /dev/null
+INGEST_END=$(date +%s.%N)
+INGEST_S=$(python3 -c "print(round($INGEST_END-$INGEST_START,2))")
+echo "==> ingest: ${INGEST_S}s"
+
+# ── force-merge to steady state ──────────────────────────────────────────────
+MERGE_START=$(date +%s.%N)
+curl -fsS -X POST "$URL/$IDX/_forcemerge?max_num_segments=1" > /dev/null || true
+curl -fsS -X POST "$URL/$IDX/_flush" > /dev/null || true
+sleep 10
+MERGE_END=$(date +%s.%N)
+MERGE_S=$(python3 -c "print(round($MERGE_END-$MERGE_START,2))")
+echo "==> force-merge: ${MERGE_S}s"
+
+COUNT=$(curl -fsS "$URL/$IDX/_count" | sed 's/.*"count":\([0-9]*\).*/\1/')
+echo "==> indexed docs: $COUNT"
+
+# ── disk by artifact type ────────────────────────────────────────────────────
+python3 - "$DATA" "$IDX" "$RAW_BYTES" "$COUNT" "$INGEST_S" "$MERGE_S" "$LABEL" "$OUT" <<'PY'
+import json, os, sys, collections
+data, idx, raw, count, ing, mrg, label, out = sys.argv[1:9]
+by_ext = collections.Counter()
+total = 0
+for dirpath, _, files in os.walk(data):
+ if "corpus.ndjson" in files or dirpath.endswith(data): pass
+ for fn in files:
+ if fn in ("corpus.ndjson","server.log") or fn.startswith("chunk-"): continue
+ p = os.path.join(dirpath, fn)
+ try: sz = os.path.getsize(p)
+ except OSError: continue
+ ext = fn.rsplit(".", 1)[-1] if "." in fn else "(noext)"
+ by_ext[ext] += sz; total += sz
+res = {
+ "label": label, "docs": int(count), "raw_source_bytes": int(raw),
+ "index_total_bytes": total,
+ "index_over_raw": round(total / int(raw), 4) if int(raw) else None,
+ "ingest_seconds": float(ing), "forcemerge_seconds": float(mrg),
+ "by_artifact": {k: v for k, v in by_ext.most_common()},
+}
+json.dump(res, open(out, "w"), indent=2)
+print(f"\n{'artifact':<12}{'bytes':>14}{'share':>9}")
+for k, v in by_ext.most_common(12):
+ print(f"{k:<12}{v:>14,}{v/total*100:>8.1f}%")
+print(f"{'TOTAL':<12}{total:>14,}")
+print(f"\nraw source : {int(raw):,} B")
+print(f"index/raw : {res['index_over_raw']}x")
+PY
+
+# ── query latency suite ──────────────────────────────────────────────────────
+echo ""
+echo "==> query latency (p50/p99 ms over 40 runs each)"
+python3 - "$URL" "$IDX" "$OUT" <<'PY'
+import json, sys, time, urllib.request
+url, idx, out = sys.argv[1], sys.argv[2], sys.argv[3]
+# Cheap point-lookup shapes were all sub-0.5ms at 100k docs — at that level HTTP
+# and JSON overhead dominate and no engine change is measurable. So the suite
+# deliberately includes EXPENSIVE shapes (large result sets, deep pages, high
+# cardinality, phrase over diverse text) where engine work actually shows up.
+QUERIES = {
+ # --- cheap shapes: regression guards, not improvement targets ---
+ "term_keyword_low_card": {"query":{"term":{"model":"opus-5"}}},
+ "term_keyword_high_card":{"query":{"term":{"doc_id":"doc-00050000"}}},
+ "range_date_narrow": {"query":{"range":{"ts":{"gte":1700000000000,"lt":1700000050000}}}},
+ # --- expensive shapes: where a 2x would have to come from ---
+ "range_long_wide": {"query":{"range":{"latency_ms":{"gte":0,"lt":4900}}}},
+ "match_text_common": {"query":{"match":{"body":"index0000"}}},
+ "match_text_multi": {"query":{"match":{"body":"index0000 segment0000 token0000"}}},
+ "match_phrase": {"query":{"match_phrase":{"body":"index0000 segment0000"}}},
+ "bool_must_filter": {"query":{"bool":{"must":[{"match":{"body":"index0000"}}],
+ "filter":[{"term":{"status":"ok"}},
+ {"range":{"latency_ms":{"lt":2500}}}]}}},
+ "deep_page": {"from":9000,"size":100,
+ "query":{"range":{"latency_ms":{"gte":0,"lt":4900}}}},
+ "large_page": {"size":500,"query":{"match":{"body":"index0000"}}},
+ "sort_by_numeric": {"size":100,"sort":[{"latency_ms":"desc"}],
+ "query":{"match_all":{}}},
+ # --- aggregations ---
+ "agg_terms_low_card": {"size":0,"aggs":{"m":{"terms":{"field":"model"}}}},
+ "agg_terms_high_card": {"size":0,"aggs":{"s":{"terms":{"field":"session","size":100}}}},
+ "agg_stats_numeric": {"size":0,"aggs":{"s":{"stats":{"field":"latency_ms"}}}},
+ "agg_date_histogram": {"size":0,"aggs":{"h":{"date_histogram":{"field":"ts",
+ "fixed_interval":"1h"}}}},
+ # numeric histogram is still the brute-force O(N) path (task #35) — a named target
+ "agg_histogram_numeric": {"size":0,"aggs":{"h":{"histogram":{"field":"latency_ms",
+ "interval":250}}}},
+ "agg_nested_terms_stats":{"size":0,"aggs":{"m":{"terms":{"field":"model"},
+ "aggs":{"s":{"stats":{"field":"cost_usd"}}}}}},
+}
+def run(body):
+ d = json.dumps(body).encode()
+ r = urllib.request.Request(f"{url}/{idx}/_search", data=d,
+ headers={"Content-Type":"application/json"})
+ t0 = time.perf_counter()
+ with urllib.request.urlopen(r) as resp: resp.read()
+ return (time.perf_counter()-t0)*1000
+# Report the engine's own `took` alongside wall-clock: when wall-clock is
+# sub-millisecond, transport dominates and `took` is the honest engine signal.
+def took(body):
+ d = json.dumps(body).encode()
+ r = urllib.request.Request(f"{url}/{idx}/_search", data=d,
+ headers={"Content-Type":"application/json"})
+ with urllib.request.urlopen(r) as resp:
+ return json.loads(resp.read()).get("took", -1)
+results = {}
+print(f"{'query':<26}{'p50':>9}{'p99':>9}{'took':>8}{'hits':>12}")
+for name, body in QUERIES.items():
+ try:
+ for _ in range(5): run(body) # warm
+ ts = sorted(run(body) for _ in range(40))
+ p50, p99 = ts[len(ts)//2], ts[int(len(ts)*0.99)-1]
+ tk = took(body)
+ d = json.dumps(body).encode()
+ r = urllib.request.Request(f"{url}/{idx}/_search", data=d,
+ headers={"Content-Type":"application/json"})
+ with urllib.request.urlopen(r) as resp:
+ tot = json.loads(resp.read()).get("hits",{}).get("total",{}).get("value",-1)
+ results[name] = {"p50_ms": round(p50,3), "p99_ms": round(p99,3),
+ "took_ms": tk, "hits": tot}
+ print(f"{name:<26}{p50:>9.2f}{p99:>9.2f}{tk:>8}{tot:>12,}")
+ except Exception as e:
+ results[name] = {"error": str(e)[:200]}
+ print(f"{name:<26}{'ERROR':>9} {str(e)[:60]}")
+res = json.load(open(out)); res["queries"] = results
+json.dump(res, open(out,"w"), indent=2)
+print(f"\nwrote {out}")
+PY
+
+echo ""
+echo "==> done. server still running on $PORT (data: $DATA)"
+echo " stop it with: pkill -f 'xerj --data-dir $DATA'"
diff --git a/demo/playbooks/rc12/results-baseline-main.json b/demo/playbooks/rc12/results-baseline-main.json
new file mode 100644
index 000000000..fb17f5619
--- /dev/null
+++ b/demo/playbooks/rc12/results-baseline-main.json
@@ -0,0 +1,128 @@
+{
+ "label": "baseline-main",
+ "docs": 500000,
+ "raw_source_bytes": 356593195,
+ "index_total_bytes": 154413168,
+ "index_over_raw": 0.433,
+ "ingest_seconds": 33.0,
+ "forcemerge_seconds": 116.52,
+ "by_artifact": {
+ "dv": 57357593,
+ "seg": 51449817,
+ "post": 40951589,
+ "ids": 3057388,
+ "meta": 1464327,
+ "fst": 64485,
+ "json": 35338,
+ "norms": 28305,
+ "wal": 4119,
+ "toml": 104,
+ "xerj_master_key": 64,
+ "sidx": 32,
+ "lock": 7
+ },
+ "queries": {
+ "term_keyword_low_card": {
+ "p50_ms": 0.297,
+ "p99_ms": 0.488,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "term_keyword_high_card": {
+ "p50_ms": 0.208,
+ "p99_ms": 0.559,
+ "took_ms": 0,
+ "hits": 1
+ },
+ "range_date_narrow": {
+ "p50_ms": 0.327,
+ "p99_ms": 0.584,
+ "took_ms": 0,
+ "hits": 50
+ },
+ "range_long_wide": {
+ "p50_ms": 0.302,
+ "p99_ms": 0.455,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "match_text_common": {
+ "p50_ms": 0.287,
+ "p99_ms": 0.361,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "match_text_multi": {
+ "p50_ms": 0.313,
+ "p99_ms": 0.406,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "match_phrase": {
+ "p50_ms": 0.282,
+ "p99_ms": 0.544,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "bool_must_filter": {
+ "p50_ms": 0.368,
+ "p99_ms": 0.743,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "deep_page": {
+ "p50_ms": 1.043,
+ "p99_ms": 1.161,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "large_page": {
+ "p50_ms": 4.718,
+ "p99_ms": 5.937,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "sort_by_numeric": {
+ "p50_ms": 1.171,
+ "p99_ms": 1.38,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_terms_low_card": {
+ "p50_ms": 0.208,
+ "p99_ms": 0.296,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_terms_high_card": {
+ "p50_ms": 0.239,
+ "p99_ms": 0.348,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_stats_numeric": {
+ "p50_ms": 0.194,
+ "p99_ms": 0.308,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_date_histogram": {
+ "p50_ms": 0.329,
+ "p99_ms": 0.456,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_histogram_numeric": {
+ "p50_ms": 0.229,
+ "p99_ms": 0.379,
+ "took_ms": 0,
+ "hits": 10000
+ },
+ "agg_nested_terms_stats": {
+ "p50_ms": 0.213,
+ "p99_ms": 0.61,
+ "took_ms": 0,
+ "hits": 10000
+ }
+ }
+}
\ No newline at end of file