Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

feat(intelligence): BEIR matrix + bootstrap CIs + 2nd dataset (ADR-086, 3.10.26)

ruv1490278176da
diff --git a/docs/benchmarks/BEIR-MATRIX.md b/docs/benchmarks/BEIR-MATRIX.md
new file mode 100644
index 0000000000..f6d979af6b
--- /dev/null
+++ b/docs/benchmarks/BEIR-MATRIX.md
@@ -0,0 +1,133 @@
+# BEIR Public-Benchmark Results β€” ruflo
+
+This page tracks ruflo's measured retrieval performance on BEIR datasets.
+Every cell is reproducible from the commands in the rightmost column;
+every cell has a run JSON in `docs/benchmarks/runs/`. Published baseline
+numbers come from Thakur et al. 2021 (BEIR paper) and the BAAI BGE paper.
+
+## Result Matrix
+
+> Numbers from `docs/benchmarks/runs/beir-{dataset}-bge-latest.json`. ADR-085
+> (harness + BGE swap) + ADR-086 (significance testing).
+
+| Dataset | Corpus | Test Q | Pipeline | Model | nDCG@10 | 95% CI | vs BM25 (CI verdict) | vs Best Listed Baseline | Rank | Latency | Run JSON |
+|---|---:|---:|---|---|---:|---|---|---:|---:|---:|---|
+| **NFCorpus** | 3,633 | 323 | direct dense (no rerank) | BGE-base-en-v1.5 (110M) | **0.352** | [0.317, 0.387] | +0.027 ↑ (n.s.) | -0.028 ↓ BGE-large 0.380 (n.s.) | **2/11** | 388 ms | `beir-nfcorpus-bge-latest.json` |
+| **NFCorpus** | 3,633 | 323 | pure BM25 (silent hash-fallback path) | _no real dense_ | 0.289 | _n/a_ | -0.036 ↓ | -0.091 ↓ | 11/11 | 950 ms | `beir-nfcorpus-2026-05-30T19-16-23-024Z.json` |
+| **SciFact** | 5,183 | 300 | direct dense (no rerank) | BGE-base-en-v1.5 (110M) | 0.626 | [0.577, 0.672] | **-0.053 ↓ (p<0.05)** | -0.096 ↓ BGE-large 0.722 (p<0.05) | 10/11 | 410 ms | `beir-scifact-bge-latest.json` |
+
+### Bootstrap CI summary (per ADR-086, 10k resamples, seed=42)
+
+The 95% confidence intervals tell the rigorous story. On NFCorpus, we beat BM25 by 0.027 *point estimate* but the CI overlaps the baseline (n.s. at p<0.05) β€” the "rank-2" headline is a single-realisation outcome, not a statistically distinguishable win. On SciFact, we lose to BM25 by 0.053 and the CI **excludes** the baseline (significant at p<0.05) β€” that loss is real, not noise.
+
+### Two-dataset mean (rough generalisation gauge)
+
+| System | NFCorpus | SciFact | Mean |
+|---|---:|---:|---:|
+| BGE-large-v1.5 (published) | 0.380 | 0.722 | 0.551 |
+| **ruflo + BGE-base-en-v1.5** | **0.352** | **0.626** | **0.489** |
+| SPLADE++ | 0.347 | 0.704 | 0.526 |
+| BM25 (Lucene) | 0.325 | 0.679 | 0.502 |
+
+**We're below BM25 on the 2-dataset mean** (0.489 vs 0.502). The BEIR-average story requires more datasets *and* domain-specific tuning. The NFCorpus rank-2 is real but not representative.
+
+> **What pipeline is reported here:** the NFCorpus 0.352 row is the **direct
+> BGE dense path** β€” no fine-tuning, no hybrid BM25+dense fusion, no
+> cross-encoder reranker. The hybrid pipeline (cosine + multi-field BM25 +
+> MMR + opt-in rerank, ADRs 078-083) is what ruflo uses internally for
+> small-corpus retrieval; the BEIR runner deliberately isolates the dense
+> path for clean comparison to dense baselines. Hybrid + rerank variants on
+> BEIR are tracked for a future ADR.
+
+## Published Baselines (for reference)
+
+### NFCorpus nDCG@10 (medical IR, n=323 test queries)
+
+| Method | Params | nDCG@10 | Source |
+|---|---:|---:|---|
+| BGE-large-v1.5 | 335M | 0.380 | BAAI BGE paper |
+| **ruflo + BGE-base-en-v1.5** | **110M** | **0.352** | **this repo** |
+| SPLADE++ | 110M | 0.347 | Formal et al. 2022 |
+| GTR-XL | 1.2B | 0.343 | Ni et al. 2022 |
+| DocT5query | 60M | 0.328 | Nogueira & Lin 2019 |
+| Contriever | 110M | 0.328 | Izacard et al. 2022 |
+| BM25 (Lucene) | β€” | 0.325 | Thakur et al. 2021 |
+| TAS-B | 66M | 0.319 | HofstΓ€tter et al. 2021 |
+| GenQ | 110M | 0.319 | Thakur et al. 2021 |
+| ColBERT | 110M | 0.305 | Khattab & Zaharia 2020 |
+| SBERT (msmarco) | 110M | 0.272 | Reimers & Gurevych 2019 |
+
+### SciFact nDCG@10 (scientific IR, n=300 test queries)
+
+| Method | nDCG@10 | Source |
+|---|---:|---|
+| BGE-large-v1.5 | 0.722 | BAAI BGE paper |
+| SPLADE++ | 0.704 | Formal et al. 2022 |
+| BM25 (Lucene) | 0.679 | Thakur et al. 2021 |
+| Contriever | 0.677 | Izacard et al. 2022 |
+| DocT5query | 0.675 | Nogueira & Lin 2019 |
+| ColBERT | 0.671 | Khattab & Zaharia 2020 |
+| GTR-XL | 0.662 | Ni et al. 2022 |
+| GenQ | 0.644 | Thakur et al. 2021 |
+| TAS-B | 0.643 | HofstΓ€tter et al. 2021 |
+| SBERT (msmarco) | 0.555 | Reimers & Gurevych 2019 |
+
+## How to reproduce
+
+```bash
+git clone https://github.com/ruvnet/ruflo && cd ruflo
+npm install && ( cd v3/@claude-flow/cli && npx tsc )
+
+# NFCorpus
+mkdir -p /tmp/beir-nfcorpus && cd /tmp/beir-nfcorpus
+curl -sL -o nfcorpus.zip 'https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/nfcorpus.zip' && unzip -q nfcorpus.zip
+node /path/to/ruflo/v3/@claude-flow/cli/scripts/run-beir-bge.mjs
+# β†’ nDCG@10 0.352, rank 2 of 11
+
+# SciFact
+mkdir -p /tmp/beir-scifact && cd /tmp/beir-scifact
+curl -sL -o scifact.zip 'https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip' && unzip -q scifact.zip
+BEIR_DATA_DIR=/tmp/beir-scifact/scifact node /path/to/ruflo/v3/@claude-flow/cli/scripts/run-beir-bge.mjs
+
+# Paired bootstrap significance test (ADR-086)
+node /path/to/ruflo/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs \
+ /path/to/ruflo/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
+```
+
+## Model size / speed / quality trade-offs
+
+| Model | Params | Embed dim | Cache size (NFCorpus) | Ingest (3,633 docs) | Query latency |
+|---|---:|---:|---:|---:|---:|
+| `Xenova/bge-small-en-v1.5` | 33M | 384 | ~5.5 MB | ~15 min | ~250 ms |
+| `Xenova/bge-base-en-v1.5` | 110M | 768 | ~11 MB | ~25 min | ~330 ms |
+| `Xenova/bge-large-en-v1.5` | 335M | 1024 | ~15 MB | ~60 min (est.) | ~700 ms (est.) |
+
+Per-row latency is on Apple Silicon CPU through `@xenova/transformers`
+int8-quantised ONNX. GPU would be ~10-50Γ— faster.
+
+## Methodology notes
+
+- **No fine-tuning.** All numbers are zero-shot β€” we use BAAI's released
+ BGE models as-is. NFCorpus has a 110K-pair train split that fine-tuning
+ could exploit for an additional ~0.02-0.05 nDCG lift; not done here.
+- **`@xenova/transformers` direct API** (not `pipeline()`) used to bypass
+ the `sharp`/`libvips` transitive dependency that breaks on
+ darwin-arm64 (ADR-085 Β§"sharp-on-darwin-arm64 bug").
+- **CLS-token pooling + L2 normalisation** per BAAI's BGE spec; cosine
+ becomes dot product on normalised vectors.
+- **Graded relevance for nDCG** β€” qrels use 0/1/2 grades; we use
+ `(2^rel - 1) / log2(i+1)` per BEIR convention.
+- **Reproducibility**: `BOOTSTRAP_SEED=42` for the significance test
+ (mulberry32 PRNG). Run JSONs include full per-query metrics so
+ external bootstrap-CI checks reproduce exactly.
+
+## Limits & next steps
+
+- **Two-dataset coverage isn't BEIR-average.** BEIR ships 18 datasets;
+ the published "BEIR average" is the standard generalisation gauge.
+ Tracking: TREC-COVID, FiQA-2018, ArguAna, HotpotQA, NQ next.
+- **Single-annotator labelled retrieval** for internal ruflo bench
+ (ADR-081); not relevant to BEIR's externally-curated qrels.
+- **The 0.005 gap to SPLADE++** (0.352 vs 0.347) is on the edge of noise
+ at N=323. The paired bootstrap test (ADR-086) gives a confidence
+ interval; report both point estimate AND CI.
diff --git a/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.json b/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.json
new file mode 100644
index 0000000000..a0f342b224
--- /dev/null
+++ b/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.json
@@ -0,0 +1,2292 @@
+{
+ "runAt": "2026-05-30T20:29:42.702Z",
+ "benchmark": "beir-nfcorpus-bge",
+ "dataset": "nfcorpus",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 323,
+ "corpusSize": 3633,
+ "metrics": {
+ "ndcg10": 0.35171471732484655,
+ "mrr10": 0.5456029780333185,
+ "recall10": 0.16652743860112934,
+ "recall100": 0.30466713166229387,
+ "avgQueryLatencyMs": 900.598096879257
+ },
+ "perQuery": [
+ {
+ "qid": "PLAIN-2",
+ "ndcg10": 0.7412469912547754,
+ "mrr10": 1,
+ "recall10": 0.2916666666666667,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-12",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.23333333333333334
+ },
+ {
+ "qid": "PLAIN-23",
+ "ndcg10": 0.4951328496468998,
+ "mrr10": 1,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.15555555555555556
+ },
+ {
+ "qid": "PLAIN-33",
+ "ndcg10": 0.46667017129647237,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-44",
+ "ndcg10": 0.2940620318598474,
+ "mrr10": 1,
+ "recall10": 0.05172413793103448,
+ "recall100": 0.1724137931034483
+ },
+ {
+ "qid": "PLAIN-56",
+ "ndcg10": 0.5097750261375381,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.26
+ },
+ {
+ "qid": "PLAIN-68",
+ "ndcg10": 0.8588798377757242,
+ "mrr10": 1,
+ "recall10": 0.12727272727272726,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-78",
+ "ndcg10": 0.08966284300766811,
+ "mrr10": 1,
+ "recall10": 0.016129032258064516,
+ "recall100": 0.12903225806451613
+ },
+ {
+ "qid": "PLAIN-91",
+ "ndcg10": 0.6081351105039879,
+ "mrr10": 1,
+ "recall10": 0.140625,
+ "recall100": 0.328125
+ },
+ {
+ "qid": "PLAIN-102",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-112",
+ "ndcg10": 0.6858781710646398,
+ "mrr10": 1,
+ "recall10": 0.08928571428571429,
+ "recall100": 0.17857142857142858
+ },
+ {
+ "qid": "PLAIN-123",
+ "ndcg10": 0.2848186035513736,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.09803921568627451,
+ "recall100": 0.1568627450980392
+ },
+ {
+ "qid": "PLAIN-133",
+ "ndcg10": 0.08649891544052932,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.027777777777777776,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-143",
+ "ndcg10": 0.37061294442664394,
+ "mrr10": 1,
+ "recall10": 0.0392156862745098,
+ "recall100": 0.2549019607843137
+ },
+ {
+ "qid": "PLAIN-153",
+ "ndcg10": 0.8948763559161851,
+ "mrr10": 1,
+ "recall10": 0.1702127659574468,
+ "recall100": 0.2978723404255319
+ },
+ {
+ "qid": "PLAIN-165",
+ "ndcg10": 0.3937404317428725,
+ "mrr10": 0.5,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-175",
+ "ndcg10": 0.29608176037159845,
+ "mrr10": 1,
+ "recall10": 0.10810810810810811,
+ "recall100": 0.13513513513513514
+ },
+ {
+ "qid": "PLAIN-186",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-196",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.043478260869565216
+ },
+ {
+ "qid": "PLAIN-207",
+ "ndcg10": 0.6578986223737446,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.2545454545454545
+ },
+ {
+ "qid": "PLAIN-217",
+ "ndcg10": 0.12987501165334076,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-227",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1
+ },
+ {
+ "qid": "PLAIN-238",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.037037037037037035
+ },
+ {
+ "qid": "PLAIN-248",
+ "ndcg10": 0.4666234168001789,
+ "mrr10": 0.5,
+ "recall10": 0.16129032258064516,
+ "recall100": 0.25806451612903225
+ },
+ {
+ "qid": "PLAIN-259",
+ "ndcg10": 0.2984900352767555,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-270",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.019230769230769232,
+ "recall100": 0.21153846153846154
+ },
+ {
+ "qid": "PLAIN-280",
+ "ndcg10": 0.3436966913921454,
+ "mrr10": 0.5,
+ "recall10": 0.15,
+ "recall100": 0.55
+ },
+ {
+ "qid": "PLAIN-291",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.5384615384615384
+ },
+ {
+ "qid": "PLAIN-307",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-320",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2619047619047619
+ },
+ {
+ "qid": "PLAIN-332",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-344",
+ "ndcg10": 0.6166392118848165,
+ "mrr10": 0.5,
+ "recall10": 0.11475409836065574,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-358",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-371",
+ "ndcg10": 0.27541155237618664,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-383",
+ "ndcg10": 0.14606834984270645,
+ "mrr10": 0.25,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-395",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-407",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-418",
+ "ndcg10": 0.8344475896326595,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-430",
+ "ndcg10": 0.4967105728753997,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-441",
+ "ndcg10": 0.727463892412172,
+ "mrr10": 1,
+ "recall10": 0.8,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-457",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-468",
+ "ndcg10": 0.521256019952062,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-478",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-488",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.7333333333333333
+ },
+ {
+ "qid": "PLAIN-499",
+ "ndcg10": 0.2532954041105318,
+ "mrr10": 0.25,
+ "recall10": 0.08823529411764706,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-510",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-531",
+ "ndcg10": 0.2906246349515899,
+ "mrr10": 0.5,
+ "recall10": 0.00974025974025974,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-541",
+ "ndcg10": 0.10778915452451093,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-551",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-561",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.12
+ },
+ {
+ "qid": "PLAIN-571",
+ "ndcg10": 0.2162957183053307,
+ "mrr10": 0.25,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-583",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.022727272727272728,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-593",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-603",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-613",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-623",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.024390243902439025,
+ "recall100": 0.12195121951219512
+ },
+ {
+ "qid": "PLAIN-634",
+ "ndcg10": 0.424926013816671,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-645",
+ "ndcg10": 0.07336392209936005,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-660",
+ "ndcg10": 0.9363792118010483,
+ "mrr10": 1,
+ "recall10": 0.018947368421052633,
+ "recall100": 0.0863157894736842
+ },
+ {
+ "qid": "PLAIN-671",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-681",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-691",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-701",
+ "ndcg10": 0.1662433493697512,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.25,
+ "recall100": 0.375
+ },
+ {
+ "qid": "PLAIN-711",
+ "ndcg10": 0.5579780239523999,
+ "mrr10": 1,
+ "recall10": 0.08771929824561403,
+ "recall100": 0.22807017543859648
+ },
+ {
+ "qid": "PLAIN-721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.64
+ },
+ {
+ "qid": "PLAIN-731",
+ "ndcg10": 0.7241528458281337,
+ "mrr10": 1,
+ "recall10": 0.12962962962962962,
+ "recall100": 0.37037037037037035
+ },
+ {
+ "qid": "PLAIN-741",
+ "ndcg10": 0.800693766409882,
+ "mrr10": 1,
+ "recall10": 0.3684210526315789,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-751",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-761",
+ "ndcg10": 0.4103915680233244,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-771",
+ "ndcg10": 0.9134015924715543,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-782",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-792",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-806",
+ "ndcg10": 0.933745776545611,
+ "mrr10": 1,
+ "recall10": 0.09375,
+ "recall100": 0.22916666666666666
+ },
+ {
+ "qid": "PLAIN-817",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-827",
+ "ndcg10": 0.84860265890399,
+ "mrr10": 1,
+ "recall10": 0.03278688524590164,
+ "recall100": 0.21721311475409835
+ },
+ {
+ "qid": "PLAIN-838",
+ "ndcg10": 0.6930221935231562,
+ "mrr10": 1,
+ "recall10": 0.5555555555555556,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-850",
+ "ndcg10": 0.5541432109622958,
+ "mrr10": 1,
+ "recall10": 0.07017543859649122,
+ "recall100": 0.17543859649122806
+ },
+ {
+ "qid": "PLAIN-872",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-882",
+ "ndcg10": 0.8701249883466593,
+ "mrr10": 1,
+ "recall10": 0.5333333333333333,
+ "recall100": 0.8
+ },
+ {
+ "qid": "PLAIN-892",
+ "ndcg10": 0.6332196796270029,
+ "mrr10": 1,
+ "recall10": 0.05434782608695652,
+ "recall100": 0.21739130434782608
+ },
+ {
+ "qid": "PLAIN-902",
+ "ndcg10": 0.3903800499921017,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-913",
+ "ndcg10": 0.8603818544462509,
+ "mrr10": 1,
+ "recall10": 0.23529411764705882,
+ "recall100": 0.35294117647058826
+ },
+ {
+ "qid": "PLAIN-924",
+ "ndcg10": 0.31516255047698366,
+ "mrr10": 0.5,
+ "recall10": 0.09375,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-934",
+ "ndcg10": 0.7917267193679839,
+ "mrr10": 1,
+ "recall10": 0.06930693069306931,
+ "recall100": 0.31683168316831684
+ },
+ {
+ "qid": "PLAIN-946",
+ "ndcg10": 0.13568544539116628,
+ "mrr10": 0.125,
+ "recall10": 0.06451612903225806,
+ "recall100": 0.12903225806451613
+ },
+ {
+ "qid": "PLAIN-956",
+ "ndcg10": 0.6594803070902429,
+ "mrr10": 1,
+ "recall10": 0.02912621359223301,
+ "recall100": 0.17475728155339806
+ },
+ {
+ "qid": "PLAIN-966",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-977",
+ "ndcg10": 0.2960819109658652,
+ "mrr10": 0.5,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-987",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-997",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1008",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1018",
+ "ndcg10": 0.4521472607752954,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1028",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1039",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1050",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.06382978723404255
+ },
+ {
+ "qid": "PLAIN-1066",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1088",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1098",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1109",
+ "ndcg10": 0.684383072597662,
+ "mrr10": 1,
+ "recall10": 0.061224489795918366,
+ "recall100": 0.35714285714285715
+ },
+ {
+ "qid": "PLAIN-1119",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1141",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1151",
+ "ndcg10": 0.7065443116025598,
+ "mrr10": 0.5,
+ "recall10": 0.05228758169934641,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-1161",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1172",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.15789473684210525
+ },
+ {
+ "qid": "PLAIN-1183",
+ "ndcg10": 0.23365080824310533,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.08
+ },
+ {
+ "qid": "PLAIN-1193",
+ "ndcg10": 0.20483424751859086,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1203",
+ "ndcg10": 0.5174612499126158,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1214",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1225",
+ "ndcg10": 0.6830838076905166,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1236",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1249",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1262",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1275",
+ "ndcg10": 0.43735247915031,
+ "mrr10": 1,
+ "recall10": 0.05454545454545454,
+ "recall100": 0.2545454545454545
+ },
+ {
+ "qid": "PLAIN-1288",
+ "ndcg10": 0.5933607200949346,
+ "mrr10": 1,
+ "recall10": 0.025510204081632654,
+ "recall100": 0.12755102040816327
+ },
+ {
+ "qid": "PLAIN-1299",
+ "ndcg10": 0.11004588314904008,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.01639344262295082,
+ "recall100": 0.06557377049180328
+ },
+ {
+ "qid": "PLAIN-1309",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1320",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1331",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1342",
+ "ndcg10": 0.4440973278132557,
+ "mrr10": 1,
+ "recall10": 0.1875,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1353",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1363",
+ "ndcg10": 0.24655055544758614,
+ "mrr10": 0.25,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.7272727272727273
+ },
+ {
+ "qid": "PLAIN-1374",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.19230769230769232
+ },
+ {
+ "qid": "PLAIN-1387",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1398",
+ "ndcg10": 0.9216017310213247,
+ "mrr10": 1,
+ "recall10": 0.08737864077669903,
+ "recall100": 0.22330097087378642
+ },
+ {
+ "qid": "PLAIN-1409",
+ "ndcg10": 0.24936270394794902,
+ "mrr10": 0.25,
+ "recall10": 0.011857707509881422,
+ "recall100": 0.1541501976284585
+ },
+ {
+ "qid": "PLAIN-1419",
+ "ndcg10": 0.4885468020156227,
+ "mrr10": 1,
+ "recall10": 0.06153846153846154,
+ "recall100": 0.1076923076923077
+ },
+ {
+ "qid": "PLAIN-1429",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1441",
+ "ndcg10": 0.5599332595218056,
+ "mrr10": 1,
+ "recall10": 0.026881720430107527,
+ "recall100": 0.15053763440860216
+ },
+ {
+ "qid": "PLAIN-1453",
+ "ndcg10": 0.5103515512676448,
+ "mrr10": 1,
+ "recall10": 0.0163265306122449,
+ "recall100": 0.06938775510204082
+ },
+ {
+ "qid": "PLAIN-1463",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.2916666666666667
+ },
+ {
+ "qid": "PLAIN-1473",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1485",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1496",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1506",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.13333333333333333
+ },
+ {
+ "qid": "PLAIN-1516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1527",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.04455445544554455,
+ "recall100": 0.18811881188118812
+ },
+ {
+ "qid": "PLAIN-1537",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.022222222222222223,
+ "recall100": 0.15555555555555556
+ },
+ {
+ "qid": "PLAIN-1547",
+ "ndcg10": 0.13032376591037062,
+ "mrr10": 0.25,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1557",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.11538461538461539
+ },
+ {
+ "qid": "PLAIN-1568",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1579",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1590",
+ "ndcg10": 0.06625422345438903,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-1601",
+ "ndcg10": 0.6457545768494583,
+ "mrr10": 1,
+ "recall10": 0.08108108108108109,
+ "recall100": 0.24324324324324326
+ },
+ {
+ "qid": "PLAIN-1611",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1621",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1635",
+ "ndcg10": 0.9266360779006401,
+ "mrr10": 1,
+ "recall10": 0.02088167053364269,
+ "recall100": 0.14153132250580047
+ },
+ {
+ "qid": "PLAIN-1645",
+ "ndcg10": 0.15130120674940672,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1656",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07142857142857142
+ },
+ {
+ "qid": "PLAIN-1667",
+ "ndcg10": 0.7877736340270854,
+ "mrr10": 1,
+ "recall10": 0.050314465408805034,
+ "recall100": 0.1509433962264151
+ },
+ {
+ "qid": "PLAIN-1679",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1690",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1700",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1710",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.625,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1721",
+ "ndcg10": 0.40995091484653445,
+ "mrr10": 0.5,
+ "recall10": 0.12903225806451613,
+ "recall100": 0.16129032258064516
+ },
+ {
+ "qid": "PLAIN-1731",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1741",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.11666666666666667
+ },
+ {
+ "qid": "PLAIN-1752",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1762",
+ "ndcg10": 0.5784505554216093,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1772",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1784",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1794",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1805",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.07758620689655173,
+ "recall100": 0.2672413793103448
+ },
+ {
+ "qid": "PLAIN-1817",
+ "ndcg10": 0.42838543210841196,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2962962962962963
+ },
+ {
+ "qid": "PLAIN-1827",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.32142857142857145
+ },
+ {
+ "qid": "PLAIN-1847",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.09090909090909091,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1857",
+ "ndcg10": 0.5957616317188126,
+ "mrr10": 1,
+ "recall10": 0.15151515151515152,
+ "recall100": 0.42424242424242425
+ },
+ {
+ "qid": "PLAIN-1867",
+ "ndcg10": 0.07945707943276742,
+ "mrr10": 0.1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1877",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-1887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1897",
+ "ndcg10": 0.17723928678404774,
+ "mrr10": 0.1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1909",
+ "ndcg10": 0.8263333286520083,
+ "mrr10": 1,
+ "recall10": 0.017278617710583154,
+ "recall100": 0.1101511879049676
+ },
+ {
+ "qid": "PLAIN-1919",
+ "ndcg10": 0.7967610662472993,
+ "mrr10": 1,
+ "recall10": 0.25925925925925924,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1929",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1940",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-1950",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-1962",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1972",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-1983",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.3125,
+ "recall100": 0.6875
+ },
+ {
+ "qid": "PLAIN-1995",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2009",
+ "ndcg10": 0.3700049343491939,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-2019",
+ "ndcg10": 0.13565197343244778,
+ "mrr10": 0.1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2030",
+ "ndcg10": 0.19092086617893467,
+ "mrr10": 0.5,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-2040",
+ "ndcg10": 0.4950940324881554,
+ "mrr10": 0.5,
+ "recall10": 0.03787878787878788,
+ "recall100": 0.13636363636363635
+ },
+ {
+ "qid": "PLAIN-2051",
+ "ndcg10": 0.5423640154200349,
+ "mrr10": 1,
+ "recall10": 0.011267605633802818,
+ "recall100": 0.09859154929577464
+ },
+ {
+ "qid": "PLAIN-2061",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.024390243902439025,
+ "recall100": 0.13902439024390245
+ },
+ {
+ "qid": "PLAIN-2071",
+ "ndcg10": 0.5455017237786198,
+ "mrr10": 0.5,
+ "recall10": 0.13333333333333333,
+ "recall100": 0.37777777777777777
+ },
+ {
+ "qid": "PLAIN-2081",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2092",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2102",
+ "ndcg10": 0.7600791051976192,
+ "mrr10": 1,
+ "recall10": 0.0166270783847981,
+ "recall100": 0.1163895486935867
+ },
+ {
+ "qid": "PLAIN-2113",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2124",
+ "ndcg10": 0.5455206287241073,
+ "mrr10": 1,
+ "recall10": 0.29411764705882354,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-2134",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.08333333333333333
+ },
+ {
+ "qid": "PLAIN-2145",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2156",
+ "ndcg10": 0.7751482523375255,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.8333333333333334
+ },
+ {
+ "qid": "PLAIN-2167",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2177",
+ "ndcg10": 0.5072380780124418,
+ "mrr10": 0.5,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2187",
+ "ndcg10": 0.23504554941448536,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2197",
+ "ndcg10": 0.6496872770120411,
+ "mrr10": 1,
+ "recall10": 0.10344827586206896,
+ "recall100": 0.3103448275862069
+ },
+ {
+ "qid": "PLAIN-2209",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2220",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2230",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2240",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-2250",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2261",
+ "ndcg10": 0.5637884576902256,
+ "mrr10": 1,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.15
+ },
+ {
+ "qid": "PLAIN-2271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2291",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2301",
+ "ndcg10": 0.13961814555374907,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.3076923076923077
+ },
+ {
+ "qid": "PLAIN-2311",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2321",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2332",
+ "ndcg10": 0.7663491917568946,
+ "mrr10": 1,
+ "recall10": 0.08247422680412371,
+ "recall100": 0.4020618556701031
+ },
+ {
+ "qid": "PLAIN-2343",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2354",
+ "ndcg10": 0.5087403079104241,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2364",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2375",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2386",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-2396",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2408",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2430",
+ "ndcg10": 0.5740106608557803,
+ "mrr10": 1,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.41304347826086957
+ },
+ {
+ "qid": "PLAIN-2440",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.047619047619047616
+ },
+ {
+ "qid": "PLAIN-2450",
+ "ndcg10": 0.2336508082431053,
+ "mrr10": 0.5,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.13157894736842105
+ },
+ {
+ "qid": "PLAIN-2460",
+ "ndcg10": 0.43174458805633853,
+ "mrr10": 1,
+ "recall10": 0.12903225806451613,
+ "recall100": 0.4838709677419355
+ },
+ {
+ "qid": "PLAIN-2470",
+ "ndcg10": 0.7024216173807135,
+ "mrr10": 1,
+ "recall10": 0.0821917808219178,
+ "recall100": 0.3287671232876712
+ },
+ {
+ "qid": "PLAIN-2480",
+ "ndcg10": 0.4407905902246705,
+ "mrr10": 1,
+ "recall10": 0.2777777777777778,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2490",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.34782608695652173
+ },
+ {
+ "qid": "PLAIN-2500",
+ "ndcg10": 0.15487585161383474,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-2510",
+ "ndcg10": 0.589664883987166,
+ "mrr10": 0.5,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.5869565217391305
+ },
+ {
+ "qid": "PLAIN-2520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1095890410958904
+ },
+ {
+ "qid": "PLAIN-2530",
+ "ndcg10": 0.955830517697074,
+ "mrr10": 1,
+ "recall10": 0.1388888888888889,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-2540",
+ "ndcg10": 0.4941492558420748,
+ "mrr10": 0.5,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2550",
+ "ndcg10": 0.24746798875357182,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2560",
+ "ndcg10": 0.9363792118010483,
+ "mrr10": 1,
+ "recall10": 0.42857142857142855,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2570",
+ "ndcg10": 0.25523504782501816,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.075,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2580",
+ "ndcg10": 0.1135664319411187,
+ "mrr10": 1,
+ "recall10": 0.043478260869565216,
+ "recall100": 0.13043478260869565
+ },
+ {
+ "qid": "PLAIN-2590",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07936507936507936
+ },
+ {
+ "qid": "PLAIN-2600",
+ "ndcg10": 0.6613158235505792,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-2610",
+ "ndcg10": 0.4973811325345484,
+ "mrr10": 1,
+ "recall10": 0.06557377049180328,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-2620",
+ "ndcg10": 0.6770586454665828,
+ "mrr10": 1,
+ "recall10": 0.16363636363636364,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-2630",
+ "ndcg10": 0.8188458073987622,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3958333333333333
+ },
+ {
+ "qid": "PLAIN-2640",
+ "ndcg10": 0.39923129859010653,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5416666666666666
+ },
+ {
+ "qid": "PLAIN-2650",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.1724137931034483,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2660",
+ "ndcg10": 0.40030566811860463,
+ "mrr10": 0.5,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.48148148148148145
+ },
+ {
+ "qid": "PLAIN-2670",
+ "ndcg10": 0.3406992958233561,
+ "mrr10": 1,
+ "recall10": 0.02857142857142857,
+ "recall100": 0.05714285714285714
+ },
+ {
+ "qid": "PLAIN-2680",
+ "ndcg10": 0.454740507204534,
+ "mrr10": 0.5,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2690",
+ "ndcg10": 0.6852219536434856,
+ "mrr10": 1,
+ "recall10": 0.20930232558139536,
+ "recall100": 0.4418604651162791
+ },
+ {
+ "qid": "PLAIN-2700",
+ "ndcg10": 0.5438770046596358,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.7
+ },
+ {
+ "qid": "PLAIN-2710",
+ "ndcg10": 0.6906119546391791,
+ "mrr10": 1,
+ "recall10": 0.23809523809523808,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2720",
+ "ndcg10": 0.24639745374629807,
+ "mrr10": 0.2,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2777777777777778
+ },
+ {
+ "qid": "PLAIN-2730",
+ "ndcg10": 0.5235258308273208,
+ "mrr10": 1,
+ "recall10": 0.13793103448275862,
+ "recall100": 0.41379310344827586
+ },
+ {
+ "qid": "PLAIN-2740",
+ "ndcg10": 0.6782347310356166,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.16
+ },
+ {
+ "qid": "PLAIN-2750",
+ "ndcg10": 0.532003134830924,
+ "mrr10": 1,
+ "recall10": 0.1016949152542373,
+ "recall100": 0.23728813559322035
+ },
+ {
+ "qid": "PLAIN-2760",
+ "ndcg10": 0.4702199443548266,
+ "mrr10": 0.5,
+ "recall10": 0.25,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2770",
+ "ndcg10": 0.325096961030005,
+ "mrr10": 0.5,
+ "recall10": 0.12195121951219512,
+ "recall100": 0.4878048780487805
+ },
+ {
+ "qid": "PLAIN-2780",
+ "ndcg10": 0.4033852369868885,
+ "mrr10": 1,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.4090909090909091
+ },
+ {
+ "qid": "PLAIN-2790",
+ "ndcg10": 0.652358405855227,
+ "mrr10": 1,
+ "recall10": 0.1016949152542373,
+ "recall100": 0.2033898305084746
+ },
+ {
+ "qid": "PLAIN-2800",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.041666666666666664
+ },
+ {
+ "qid": "PLAIN-2810",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-2820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2830",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2840",
+ "ndcg10": 0.09478836436955078,
+ "mrr10": 0.25,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-2850",
+ "ndcg10": 0.3026286548261086,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.4444444444444444
+ },
+ {
+ "qid": "PLAIN-2860",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2870",
+ "ndcg10": 0.0781764177375172,
+ "mrr10": 0.2,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2880",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-2890",
+ "ndcg10": 0.1974505933948842,
+ "mrr10": 0.25,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-2900",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2910",
+ "ndcg10": 0.5330346140547638,
+ "mrr10": 1,
+ "recall10": 0.13333333333333333,
+ "recall100": 0.26666666666666666
+ },
+ {
+ "qid": "PLAIN-2920",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-2930",
+ "ndcg10": 0.5175857101991203,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-2940",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.045454545454545456,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2950",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2960",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2970",
+ "ndcg10": 0.6338262955454459,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-2981",
+ "ndcg10": 0.5321036168710066,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-2991",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3001",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3014",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3026",
+ "ndcg10": 0.6309297535714574,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3037",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3053",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3063",
+ "ndcg10": 0.19519002499605084,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.25,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-3074",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3085",
+ "ndcg10": 0.38162168437253385,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3097",
+ "ndcg10": 0.515157140670655,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3116",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3131",
+ "ndcg10": 0.6516433935264055,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3141",
+ "ndcg10": 0.3170808479749099,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.20588235294117646
+ },
+ {
+ "qid": "PLAIN-3151",
+ "ndcg10": 0.7456217791401517,
+ "mrr10": 1,
+ "recall10": 0.3888888888888889,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-3161",
+ "ndcg10": 0.524282944620724,
+ "mrr10": 1,
+ "recall10": 0.046511627906976744,
+ "recall100": 0.27906976744186046
+ },
+ {
+ "qid": "PLAIN-3171",
+ "ndcg10": 0.044175472610956126,
+ "mrr10": 0.1,
+ "recall10": 0.029411764705882353,
+ "recall100": 0.058823529411764705
+ },
+ {
+ "qid": "PLAIN-3181",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.037037037037037035
+ },
+ {
+ "qid": "PLAIN-3191",
+ "ndcg10": 0.4300543345163763,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3201",
+ "ndcg10": 0.2037495755301314,
+ "mrr10": 0.2,
+ "recall10": 0.0625,
+ "recall100": 0.09375
+ },
+ {
+ "qid": "PLAIN-3211",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-3221",
+ "ndcg10": 0.06581686446496139,
+ "mrr10": 0.25,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-3231",
+ "ndcg10": 0.2501157675209561,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3241",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.06896551724137931
+ },
+ {
+ "qid": "PLAIN-3251",
+ "ndcg10": 0.16330891279672524,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.047619047619047616,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-3261",
+ "ndcg10": 0.17034964791167806,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.07692307692307693
+ },
+ {
+ "qid": "PLAIN-3271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10204081632653061
+ },
+ {
+ "qid": "PLAIN-3281",
+ "ndcg10": 0.24249682153453903,
+ "mrr10": 0.5,
+ "recall10": 0.038461538461538464,
+ "recall100": 0.07692307692307693
+ },
+ {
+ "qid": "PLAIN-3292",
+ "ndcg10": 0.34839716434274914,
+ "mrr10": 1,
+ "recall10": 0.42857142857142855,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-3302",
+ "ndcg10": 0.13084731762486884,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.05405405405405406,
+ "recall100": 0.3783783783783784
+ },
+ {
+ "qid": "PLAIN-3312",
+ "ndcg10": 0.30114238162847884,
+ "mrr10": 0.5,
+ "recall10": 0.3076923076923077,
+ "recall100": 0.6153846153846154
+ },
+ {
+ "qid": "PLAIN-3322",
+ "ndcg10": 0.28165758790969186,
+ "mrr10": 0.5,
+ "recall10": 0.0379746835443038,
+ "recall100": 0.13924050632911392
+ },
+ {
+ "qid": "PLAIN-3332",
+ "ndcg10": 0.7386266690451878,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3342",
+ "ndcg10": 0.1671604549662023,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-3352",
+ "ndcg10": 0.3843483686763149,
+ "mrr10": 1,
+ "recall10": 0.02564102564102564,
+ "recall100": 0.1282051282051282
+ },
+ {
+ "qid": "PLAIN-3362",
+ "ndcg10": 0.6268451902108538,
+ "mrr10": 1,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-3372",
+ "ndcg10": 0.27744302190680586,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3382",
+ "ndcg10": 0.4594761912515673,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-3392",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.13043478260869565
+ },
+ {
+ "qid": "PLAIN-3402",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3412",
+ "ndcg10": 0.18329273373298458,
+ "mrr10": 1,
+ "recall10": 0.04081632653061224,
+ "recall100": 0.24489795918367346
+ },
+ {
+ "qid": "PLAIN-3422",
+ "ndcg10": 0.02321714761064369,
+ "mrr10": 0.1,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.45161290322580644
+ },
+ {
+ "qid": "PLAIN-3432",
+ "ndcg10": 0.06362078819895171,
+ "mrr10": 0.1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-3442",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-3452",
+ "ndcg10": 0.4247645055987381,
+ "mrr10": 1,
+ "recall10": 0.11764705882352941,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-3462",
+ "ndcg10": 0.6378891765242927,
+ "mrr10": 0.5,
+ "recall10": 0.109375,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3472",
+ "ndcg10": 0.2873431773791806,
+ "mrr10": 1,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.4117647058823529
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.325,
+ "DocT5query": 0.328,
+ "TAS-B": 0.319,
+ "GenQ": 0.319,
+ "ColBERT": 0.305,
+ "Contriever": 0.328,
+ "GTR-XL": 0.343,
+ "SPLADE++": 0.347,
+ "BGE-large-v1.5 (pub)": 0.38,
+ "SBERT msmarco": 0.272
+ },
+ "ourRank": 2,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json b/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
index 4798a06c10..a0f342b224 100644
--- a/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
+++ b/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
@@ -1,5 +1,5 @@
{
- "runAt": "2026-05-30T20:06:41.238Z",
+ "runAt": "2026-05-30T20:29:42.702Z",
"benchmark": "beir-nfcorpus-bge",
"dataset": "nfcorpus",
"model": "Xenova/bge-base-en-v1.5",
@@ -10,8 +10,2271 @@
"mrr10": 0.5456029780333185,
"recall10": 0.16652743860112934,
"recall100": 0.30466713166229387,
- "avgQueryLatencyMs": 388.4598924148607
+ "avgQueryLatencyMs": 900.598096879257
},
+ "perQuery": [
+ {
+ "qid": "PLAIN-2",
+ "ndcg10": 0.7412469912547754,
+ "mrr10": 1,
+ "recall10": 0.2916666666666667,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-12",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.23333333333333334
+ },
+ {
+ "qid": "PLAIN-23",
+ "ndcg10": 0.4951328496468998,
+ "mrr10": 1,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.15555555555555556
+ },
+ {
+ "qid": "PLAIN-33",
+ "ndcg10": 0.46667017129647237,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-44",
+ "ndcg10": 0.2940620318598474,
+ "mrr10": 1,
+ "recall10": 0.05172413793103448,
+ "recall100": 0.1724137931034483
+ },
+ {
+ "qid": "PLAIN-56",
+ "ndcg10": 0.5097750261375381,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.26
+ },
+ {
+ "qid": "PLAIN-68",
+ "ndcg10": 0.8588798377757242,
+ "mrr10": 1,
+ "recall10": 0.12727272727272726,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-78",
+ "ndcg10": 0.08966284300766811,
+ "mrr10": 1,
+ "recall10": 0.016129032258064516,
+ "recall100": 0.12903225806451613
+ },
+ {
+ "qid": "PLAIN-91",
+ "ndcg10": 0.6081351105039879,
+ "mrr10": 1,
+ "recall10": 0.140625,
+ "recall100": 0.328125
+ },
+ {
+ "qid": "PLAIN-102",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-112",
+ "ndcg10": 0.6858781710646398,
+ "mrr10": 1,
+ "recall10": 0.08928571428571429,
+ "recall100": 0.17857142857142858
+ },
+ {
+ "qid": "PLAIN-123",
+ "ndcg10": 0.2848186035513736,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.09803921568627451,
+ "recall100": 0.1568627450980392
+ },
+ {
+ "qid": "PLAIN-133",
+ "ndcg10": 0.08649891544052932,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.027777777777777776,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-143",
+ "ndcg10": 0.37061294442664394,
+ "mrr10": 1,
+ "recall10": 0.0392156862745098,
+ "recall100": 0.2549019607843137
+ },
+ {
+ "qid": "PLAIN-153",
+ "ndcg10": 0.8948763559161851,
+ "mrr10": 1,
+ "recall10": 0.1702127659574468,
+ "recall100": 0.2978723404255319
+ },
+ {
+ "qid": "PLAIN-165",
+ "ndcg10": 0.3937404317428725,
+ "mrr10": 0.5,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-175",
+ "ndcg10": 0.29608176037159845,
+ "mrr10": 1,
+ "recall10": 0.10810810810810811,
+ "recall100": 0.13513513513513514
+ },
+ {
+ "qid": "PLAIN-186",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-196",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.043478260869565216
+ },
+ {
+ "qid": "PLAIN-207",
+ "ndcg10": 0.6578986223737446,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.2545454545454545
+ },
+ {
+ "qid": "PLAIN-217",
+ "ndcg10": 0.12987501165334076,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-227",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1
+ },
+ {
+ "qid": "PLAIN-238",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.037037037037037035
+ },
+ {
+ "qid": "PLAIN-248",
+ "ndcg10": 0.4666234168001789,
+ "mrr10": 0.5,
+ "recall10": 0.16129032258064516,
+ "recall100": 0.25806451612903225
+ },
+ {
+ "qid": "PLAIN-259",
+ "ndcg10": 0.2984900352767555,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-270",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.019230769230769232,
+ "recall100": 0.21153846153846154
+ },
+ {
+ "qid": "PLAIN-280",
+ "ndcg10": 0.3436966913921454,
+ "mrr10": 0.5,
+ "recall10": 0.15,
+ "recall100": 0.55
+ },
+ {
+ "qid": "PLAIN-291",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.5384615384615384
+ },
+ {
+ "qid": "PLAIN-307",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-320",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2619047619047619
+ },
+ {
+ "qid": "PLAIN-332",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-344",
+ "ndcg10": 0.6166392118848165,
+ "mrr10": 0.5,
+ "recall10": 0.11475409836065574,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-358",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-371",
+ "ndcg10": 0.27541155237618664,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-383",
+ "ndcg10": 0.14606834984270645,
+ "mrr10": 0.25,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-395",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-407",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-418",
+ "ndcg10": 0.8344475896326595,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-430",
+ "ndcg10": 0.4967105728753997,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-441",
+ "ndcg10": 0.727463892412172,
+ "mrr10": 1,
+ "recall10": 0.8,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-457",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-468",
+ "ndcg10": 0.521256019952062,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-478",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-488",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.7333333333333333
+ },
+ {
+ "qid": "PLAIN-499",
+ "ndcg10": 0.2532954041105318,
+ "mrr10": 0.25,
+ "recall10": 0.08823529411764706,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-510",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-531",
+ "ndcg10": 0.2906246349515899,
+ "mrr10": 0.5,
+ "recall10": 0.00974025974025974,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-541",
+ "ndcg10": 0.10778915452451093,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-551",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-561",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.12
+ },
+ {
+ "qid": "PLAIN-571",
+ "ndcg10": 0.2162957183053307,
+ "mrr10": 0.25,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-583",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.022727272727272728,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-593",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-603",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-613",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-623",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.024390243902439025,
+ "recall100": 0.12195121951219512
+ },
+ {
+ "qid": "PLAIN-634",
+ "ndcg10": 0.424926013816671,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-645",
+ "ndcg10": 0.07336392209936005,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-660",
+ "ndcg10": 0.9363792118010483,
+ "mrr10": 1,
+ "recall10": 0.018947368421052633,
+ "recall100": 0.0863157894736842
+ },
+ {
+ "qid": "PLAIN-671",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-681",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-691",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-701",
+ "ndcg10": 0.1662433493697512,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.25,
+ "recall100": 0.375
+ },
+ {
+ "qid": "PLAIN-711",
+ "ndcg10": 0.5579780239523999,
+ "mrr10": 1,
+ "recall10": 0.08771929824561403,
+ "recall100": 0.22807017543859648
+ },
+ {
+ "qid": "PLAIN-721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.64
+ },
+ {
+ "qid": "PLAIN-731",
+ "ndcg10": 0.7241528458281337,
+ "mrr10": 1,
+ "recall10": 0.12962962962962962,
+ "recall100": 0.37037037037037035
+ },
+ {
+ "qid": "PLAIN-741",
+ "ndcg10": 0.800693766409882,
+ "mrr10": 1,
+ "recall10": 0.3684210526315789,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-751",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-761",
+ "ndcg10": 0.4103915680233244,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-771",
+ "ndcg10": 0.9134015924715543,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-782",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-792",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-806",
+ "ndcg10": 0.933745776545611,
+ "mrr10": 1,
+ "recall10": 0.09375,
+ "recall100": 0.22916666666666666
+ },
+ {
+ "qid": "PLAIN-817",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-827",
+ "ndcg10": 0.84860265890399,
+ "mrr10": 1,
+ "recall10": 0.03278688524590164,
+ "recall100": 0.21721311475409835
+ },
+ {
+ "qid": "PLAIN-838",
+ "ndcg10": 0.6930221935231562,
+ "mrr10": 1,
+ "recall10": 0.5555555555555556,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-850",
+ "ndcg10": 0.5541432109622958,
+ "mrr10": 1,
+ "recall10": 0.07017543859649122,
+ "recall100": 0.17543859649122806
+ },
+ {
+ "qid": "PLAIN-872",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-882",
+ "ndcg10": 0.8701249883466593,
+ "mrr10": 1,
+ "recall10": 0.5333333333333333,
+ "recall100": 0.8
+ },
+ {
+ "qid": "PLAIN-892",
+ "ndcg10": 0.6332196796270029,
+ "mrr10": 1,
+ "recall10": 0.05434782608695652,
+ "recall100": 0.21739130434782608
+ },
+ {
+ "qid": "PLAIN-902",
+ "ndcg10": 0.3903800499921017,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-913",
+ "ndcg10": 0.8603818544462509,
+ "mrr10": 1,
+ "recall10": 0.23529411764705882,
+ "recall100": 0.35294117647058826
+ },
+ {
+ "qid": "PLAIN-924",
+ "ndcg10": 0.31516255047698366,
+ "mrr10": 0.5,
+ "recall10": 0.09375,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-934",
+ "ndcg10": 0.7917267193679839,
+ "mrr10": 1,
+ "recall10": 0.06930693069306931,
+ "recall100": 0.31683168316831684
+ },
+ {
+ "qid": "PLAIN-946",
+ "ndcg10": 0.13568544539116628,
+ "mrr10": 0.125,
+ "recall10": 0.06451612903225806,
+ "recall100": 0.12903225806451613
+ },
+ {
+ "qid": "PLAIN-956",
+ "ndcg10": 0.6594803070902429,
+ "mrr10": 1,
+ "recall10": 0.02912621359223301,
+ "recall100": 0.17475728155339806
+ },
+ {
+ "qid": "PLAIN-966",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-977",
+ "ndcg10": 0.2960819109658652,
+ "mrr10": 0.5,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-987",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-997",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1008",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1018",
+ "ndcg10": 0.4521472607752954,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1028",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1039",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1050",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.06382978723404255
+ },
+ {
+ "qid": "PLAIN-1066",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1088",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1098",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1109",
+ "ndcg10": 0.684383072597662,
+ "mrr10": 1,
+ "recall10": 0.061224489795918366,
+ "recall100": 0.35714285714285715
+ },
+ {
+ "qid": "PLAIN-1119",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1141",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1151",
+ "ndcg10": 0.7065443116025598,
+ "mrr10": 0.5,
+ "recall10": 0.05228758169934641,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-1161",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1172",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.15789473684210525
+ },
+ {
+ "qid": "PLAIN-1183",
+ "ndcg10": 0.23365080824310533,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.08
+ },
+ {
+ "qid": "PLAIN-1193",
+ "ndcg10": 0.20483424751859086,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1203",
+ "ndcg10": 0.5174612499126158,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1214",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1225",
+ "ndcg10": 0.6830838076905166,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1236",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1249",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1262",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1275",
+ "ndcg10": 0.43735247915031,
+ "mrr10": 1,
+ "recall10": 0.05454545454545454,
+ "recall100": 0.2545454545454545
+ },
+ {
+ "qid": "PLAIN-1288",
+ "ndcg10": 0.5933607200949346,
+ "mrr10": 1,
+ "recall10": 0.025510204081632654,
+ "recall100": 0.12755102040816327
+ },
+ {
+ "qid": "PLAIN-1299",
+ "ndcg10": 0.11004588314904008,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.01639344262295082,
+ "recall100": 0.06557377049180328
+ },
+ {
+ "qid": "PLAIN-1309",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1320",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1331",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1342",
+ "ndcg10": 0.4440973278132557,
+ "mrr10": 1,
+ "recall10": 0.1875,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1353",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1363",
+ "ndcg10": 0.24655055544758614,
+ "mrr10": 0.25,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.7272727272727273
+ },
+ {
+ "qid": "PLAIN-1374",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.19230769230769232
+ },
+ {
+ "qid": "PLAIN-1387",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1398",
+ "ndcg10": 0.9216017310213247,
+ "mrr10": 1,
+ "recall10": 0.08737864077669903,
+ "recall100": 0.22330097087378642
+ },
+ {
+ "qid": "PLAIN-1409",
+ "ndcg10": 0.24936270394794902,
+ "mrr10": 0.25,
+ "recall10": 0.011857707509881422,
+ "recall100": 0.1541501976284585
+ },
+ {
+ "qid": "PLAIN-1419",
+ "ndcg10": 0.4885468020156227,
+ "mrr10": 1,
+ "recall10": 0.06153846153846154,
+ "recall100": 0.1076923076923077
+ },
+ {
+ "qid": "PLAIN-1429",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1441",
+ "ndcg10": 0.5599332595218056,
+ "mrr10": 1,
+ "recall10": 0.026881720430107527,
+ "recall100": 0.15053763440860216
+ },
+ {
+ "qid": "PLAIN-1453",
+ "ndcg10": 0.5103515512676448,
+ "mrr10": 1,
+ "recall10": 0.0163265306122449,
+ "recall100": 0.06938775510204082
+ },
+ {
+ "qid": "PLAIN-1463",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.2916666666666667
+ },
+ {
+ "qid": "PLAIN-1473",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1485",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1496",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1506",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.13333333333333333
+ },
+ {
+ "qid": "PLAIN-1516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1527",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.04455445544554455,
+ "recall100": 0.18811881188118812
+ },
+ {
+ "qid": "PLAIN-1537",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.022222222222222223,
+ "recall100": 0.15555555555555556
+ },
+ {
+ "qid": "PLAIN-1547",
+ "ndcg10": 0.13032376591037062,
+ "mrr10": 0.25,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1557",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.11538461538461539
+ },
+ {
+ "qid": "PLAIN-1568",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1579",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1590",
+ "ndcg10": 0.06625422345438903,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-1601",
+ "ndcg10": 0.6457545768494583,
+ "mrr10": 1,
+ "recall10": 0.08108108108108109,
+ "recall100": 0.24324324324324326
+ },
+ {
+ "qid": "PLAIN-1611",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1621",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1635",
+ "ndcg10": 0.9266360779006401,
+ "mrr10": 1,
+ "recall10": 0.02088167053364269,
+ "recall100": 0.14153132250580047
+ },
+ {
+ "qid": "PLAIN-1645",
+ "ndcg10": 0.15130120674940672,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1656",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07142857142857142
+ },
+ {
+ "qid": "PLAIN-1667",
+ "ndcg10": 0.7877736340270854,
+ "mrr10": 1,
+ "recall10": 0.050314465408805034,
+ "recall100": 0.1509433962264151
+ },
+ {
+ "qid": "PLAIN-1679",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1690",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1700",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1710",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.625,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1721",
+ "ndcg10": 0.40995091484653445,
+ "mrr10": 0.5,
+ "recall10": 0.12903225806451613,
+ "recall100": 0.16129032258064516
+ },
+ {
+ "qid": "PLAIN-1731",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1741",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.11666666666666667
+ },
+ {
+ "qid": "PLAIN-1752",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1762",
+ "ndcg10": 0.5784505554216093,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1772",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1784",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1794",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1805",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.07758620689655173,
+ "recall100": 0.2672413793103448
+ },
+ {
+ "qid": "PLAIN-1817",
+ "ndcg10": 0.42838543210841196,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2962962962962963
+ },
+ {
+ "qid": "PLAIN-1827",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.32142857142857145
+ },
+ {
+ "qid": "PLAIN-1847",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.09090909090909091,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1857",
+ "ndcg10": 0.5957616317188126,
+ "mrr10": 1,
+ "recall10": 0.15151515151515152,
+ "recall100": 0.42424242424242425
+ },
+ {
+ "qid": "PLAIN-1867",
+ "ndcg10": 0.07945707943276742,
+ "mrr10": 0.1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1877",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-1887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1897",
+ "ndcg10": 0.17723928678404774,
+ "mrr10": 0.1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1909",
+ "ndcg10": 0.8263333286520083,
+ "mrr10": 1,
+ "recall10": 0.017278617710583154,
+ "recall100": 0.1101511879049676
+ },
+ {
+ "qid": "PLAIN-1919",
+ "ndcg10": 0.7967610662472993,
+ "mrr10": 1,
+ "recall10": 0.25925925925925924,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1929",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1940",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-1950",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-1962",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1972",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-1983",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.3125,
+ "recall100": 0.6875
+ },
+ {
+ "qid": "PLAIN-1995",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2009",
+ "ndcg10": 0.3700049343491939,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-2019",
+ "ndcg10": 0.13565197343244778,
+ "mrr10": 0.1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2030",
+ "ndcg10": 0.19092086617893467,
+ "mrr10": 0.5,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-2040",
+ "ndcg10": 0.4950940324881554,
+ "mrr10": 0.5,
+ "recall10": 0.03787878787878788,
+ "recall100": 0.13636363636363635
+ },
+ {
+ "qid": "PLAIN-2051",
+ "ndcg10": 0.5423640154200349,
+ "mrr10": 1,
+ "recall10": 0.011267605633802818,
+ "recall100": 0.09859154929577464
+ },
+ {
+ "qid": "PLAIN-2061",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.024390243902439025,
+ "recall100": 0.13902439024390245
+ },
+ {
+ "qid": "PLAIN-2071",
+ "ndcg10": 0.5455017237786198,
+ "mrr10": 0.5,
+ "recall10": 0.13333333333333333,
+ "recall100": 0.37777777777777777
+ },
+ {
+ "qid": "PLAIN-2081",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2092",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2102",
+ "ndcg10": 0.7600791051976192,
+ "mrr10": 1,
+ "recall10": 0.0166270783847981,
+ "recall100": 0.1163895486935867
+ },
+ {
+ "qid": "PLAIN-2113",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2124",
+ "ndcg10": 0.5455206287241073,
+ "mrr10": 1,
+ "recall10": 0.29411764705882354,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-2134",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.08333333333333333
+ },
+ {
+ "qid": "PLAIN-2145",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2156",
+ "ndcg10": 0.7751482523375255,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.8333333333333334
+ },
+ {
+ "qid": "PLAIN-2167",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2177",
+ "ndcg10": 0.5072380780124418,
+ "mrr10": 0.5,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2187",
+ "ndcg10": 0.23504554941448536,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2197",
+ "ndcg10": 0.6496872770120411,
+ "mrr10": 1,
+ "recall10": 0.10344827586206896,
+ "recall100": 0.3103448275862069
+ },
+ {
+ "qid": "PLAIN-2209",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2220",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2230",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2240",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-2250",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2261",
+ "ndcg10": 0.5637884576902256,
+ "mrr10": 1,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.15
+ },
+ {
+ "qid": "PLAIN-2271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2291",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2301",
+ "ndcg10": 0.13961814555374907,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.3076923076923077
+ },
+ {
+ "qid": "PLAIN-2311",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2321",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2332",
+ "ndcg10": 0.7663491917568946,
+ "mrr10": 1,
+ "recall10": 0.08247422680412371,
+ "recall100": 0.4020618556701031
+ },
+ {
+ "qid": "PLAIN-2343",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2354",
+ "ndcg10": 0.5087403079104241,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2364",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2375",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2386",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-2396",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2408",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2430",
+ "ndcg10": 0.5740106608557803,
+ "mrr10": 1,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.41304347826086957
+ },
+ {
+ "qid": "PLAIN-2440",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.047619047619047616
+ },
+ {
+ "qid": "PLAIN-2450",
+ "ndcg10": 0.2336508082431053,
+ "mrr10": 0.5,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.13157894736842105
+ },
+ {
+ "qid": "PLAIN-2460",
+ "ndcg10": 0.43174458805633853,
+ "mrr10": 1,
+ "recall10": 0.12903225806451613,
+ "recall100": 0.4838709677419355
+ },
+ {
+ "qid": "PLAIN-2470",
+ "ndcg10": 0.7024216173807135,
+ "mrr10": 1,
+ "recall10": 0.0821917808219178,
+ "recall100": 0.3287671232876712
+ },
+ {
+ "qid": "PLAIN-2480",
+ "ndcg10": 0.4407905902246705,
+ "mrr10": 1,
+ "recall10": 0.2777777777777778,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2490",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.34782608695652173
+ },
+ {
+ "qid": "PLAIN-2500",
+ "ndcg10": 0.15487585161383474,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-2510",
+ "ndcg10": 0.589664883987166,
+ "mrr10": 0.5,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.5869565217391305
+ },
+ {
+ "qid": "PLAIN-2520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1095890410958904
+ },
+ {
+ "qid": "PLAIN-2530",
+ "ndcg10": 0.955830517697074,
+ "mrr10": 1,
+ "recall10": 0.1388888888888889,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-2540",
+ "ndcg10": 0.4941492558420748,
+ "mrr10": 0.5,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2550",
+ "ndcg10": 0.24746798875357182,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2560",
+ "ndcg10": 0.9363792118010483,
+ "mrr10": 1,
+ "recall10": 0.42857142857142855,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2570",
+ "ndcg10": 0.25523504782501816,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.075,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2580",
+ "ndcg10": 0.1135664319411187,
+ "mrr10": 1,
+ "recall10": 0.043478260869565216,
+ "recall100": 0.13043478260869565
+ },
+ {
+ "qid": "PLAIN-2590",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07936507936507936
+ },
+ {
+ "qid": "PLAIN-2600",
+ "ndcg10": 0.6613158235505792,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-2610",
+ "ndcg10": 0.4973811325345484,
+ "mrr10": 1,
+ "recall10": 0.06557377049180328,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-2620",
+ "ndcg10": 0.6770586454665828,
+ "mrr10": 1,
+ "recall10": 0.16363636363636364,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-2630",
+ "ndcg10": 0.8188458073987622,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3958333333333333
+ },
+ {
+ "qid": "PLAIN-2640",
+ "ndcg10": 0.39923129859010653,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5416666666666666
+ },
+ {
+ "qid": "PLAIN-2650",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.1724137931034483,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2660",
+ "ndcg10": 0.40030566811860463,
+ "mrr10": 0.5,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.48148148148148145
+ },
+ {
+ "qid": "PLAIN-2670",
+ "ndcg10": 0.3406992958233561,
+ "mrr10": 1,
+ "recall10": 0.02857142857142857,
+ "recall100": 0.05714285714285714
+ },
+ {
+ "qid": "PLAIN-2680",
+ "ndcg10": 0.454740507204534,
+ "mrr10": 0.5,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2690",
+ "ndcg10": 0.6852219536434856,
+ "mrr10": 1,
+ "recall10": 0.20930232558139536,
+ "recall100": 0.4418604651162791
+ },
+ {
+ "qid": "PLAIN-2700",
+ "ndcg10": 0.5438770046596358,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.7
+ },
+ {
+ "qid": "PLAIN-2710",
+ "ndcg10": 0.6906119546391791,
+ "mrr10": 1,
+ "recall10": 0.23809523809523808,
+ "recall100": 0.6190476190476191
+ },
+ {
+ "qid": "PLAIN-2720",
+ "ndcg10": 0.24639745374629807,
+ "mrr10": 0.2,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.2777777777777778
+ },
+ {
+ "qid": "PLAIN-2730",
+ "ndcg10": 0.5235258308273208,
+ "mrr10": 1,
+ "recall10": 0.13793103448275862,
+ "recall100": 0.41379310344827586
+ },
+ {
+ "qid": "PLAIN-2740",
+ "ndcg10": 0.6782347310356166,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.16
+ },
+ {
+ "qid": "PLAIN-2750",
+ "ndcg10": 0.532003134830924,
+ "mrr10": 1,
+ "recall10": 0.1016949152542373,
+ "recall100": 0.23728813559322035
+ },
+ {
+ "qid": "PLAIN-2760",
+ "ndcg10": 0.4702199443548266,
+ "mrr10": 0.5,
+ "recall10": 0.25,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2770",
+ "ndcg10": 0.325096961030005,
+ "mrr10": 0.5,
+ "recall10": 0.12195121951219512,
+ "recall100": 0.4878048780487805
+ },
+ {
+ "qid": "PLAIN-2780",
+ "ndcg10": 0.4033852369868885,
+ "mrr10": 1,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.4090909090909091
+ },
+ {
+ "qid": "PLAIN-2790",
+ "ndcg10": 0.652358405855227,
+ "mrr10": 1,
+ "recall10": 0.1016949152542373,
+ "recall100": 0.2033898305084746
+ },
+ {
+ "qid": "PLAIN-2800",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.041666666666666664
+ },
+ {
+ "qid": "PLAIN-2810",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-2820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2830",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2840",
+ "ndcg10": 0.09478836436955078,
+ "mrr10": 0.25,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-2850",
+ "ndcg10": 0.3026286548261086,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.4444444444444444
+ },
+ {
+ "qid": "PLAIN-2860",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2870",
+ "ndcg10": 0.0781764177375172,
+ "mrr10": 0.2,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2880",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-2890",
+ "ndcg10": 0.1974505933948842,
+ "mrr10": 0.25,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-2900",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2910",
+ "ndcg10": 0.5330346140547638,
+ "mrr10": 1,
+ "recall10": 0.13333333333333333,
+ "recall100": 0.26666666666666666
+ },
+ {
+ "qid": "PLAIN-2920",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-2930",
+ "ndcg10": 0.5175857101991203,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-2940",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.045454545454545456,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2950",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2960",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2970",
+ "ndcg10": 0.6338262955454459,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-2981",
+ "ndcg10": 0.5321036168710066,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-2991",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3001",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3014",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3026",
+ "ndcg10": 0.6309297535714574,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3037",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3053",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3063",
+ "ndcg10": 0.19519002499605084,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.25,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-3074",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3085",
+ "ndcg10": 0.38162168437253385,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3097",
+ "ndcg10": 0.515157140670655,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3116",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3131",
+ "ndcg10": 0.6516433935264055,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3141",
+ "ndcg10": 0.3170808479749099,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.20588235294117646
+ },
+ {
+ "qid": "PLAIN-3151",
+ "ndcg10": 0.7456217791401517,
+ "mrr10": 1,
+ "recall10": 0.3888888888888889,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-3161",
+ "ndcg10": 0.524282944620724,
+ "mrr10": 1,
+ "recall10": 0.046511627906976744,
+ "recall100": 0.27906976744186046
+ },
+ {
+ "qid": "PLAIN-3171",
+ "ndcg10": 0.044175472610956126,
+ "mrr10": 0.1,
+ "recall10": 0.029411764705882353,
+ "recall100": 0.058823529411764705
+ },
+ {
+ "qid": "PLAIN-3181",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.037037037037037035
+ },
+ {
+ "qid": "PLAIN-3191",
+ "ndcg10": 0.4300543345163763,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3201",
+ "ndcg10": 0.2037495755301314,
+ "mrr10": 0.2,
+ "recall10": 0.0625,
+ "recall100": 0.09375
+ },
+ {
+ "qid": "PLAIN-3211",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-3221",
+ "ndcg10": 0.06581686446496139,
+ "mrr10": 0.25,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-3231",
+ "ndcg10": 0.2501157675209561,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3241",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.06896551724137931
+ },
+ {
+ "qid": "PLAIN-3251",
+ "ndcg10": 0.16330891279672524,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.047619047619047616,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-3261",
+ "ndcg10": 0.17034964791167806,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.07692307692307693
+ },
+ {
+ "qid": "PLAIN-3271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10204081632653061
+ },
+ {
+ "qid": "PLAIN-3281",
+ "ndcg10": 0.24249682153453903,
+ "mrr10": 0.5,
+ "recall10": 0.038461538461538464,
+ "recall100": 0.07692307692307693
+ },
+ {
+ "qid": "PLAIN-3292",
+ "ndcg10": 0.34839716434274914,
+ "mrr10": 1,
+ "recall10": 0.42857142857142855,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-3302",
+ "ndcg10": 0.13084731762486884,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.05405405405405406,
+ "recall100": 0.3783783783783784
+ },
+ {
+ "qid": "PLAIN-3312",
+ "ndcg10": 0.30114238162847884,
+ "mrr10": 0.5,
+ "recall10": 0.3076923076923077,
+ "recall100": 0.6153846153846154
+ },
+ {
+ "qid": "PLAIN-3322",
+ "ndcg10": 0.28165758790969186,
+ "mrr10": 0.5,
+ "recall10": 0.0379746835443038,
+ "recall100": 0.13924050632911392
+ },
+ {
+ "qid": "PLAIN-3332",
+ "ndcg10": 0.7386266690451878,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3342",
+ "ndcg10": 0.1671604549662023,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-3352",
+ "ndcg10": 0.3843483686763149,
+ "mrr10": 1,
+ "recall10": 0.02564102564102564,
+ "recall100": 0.1282051282051282
+ },
+ {
+ "qid": "PLAIN-3362",
+ "ndcg10": 0.6268451902108538,
+ "mrr10": 1,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-3372",
+ "ndcg10": 0.27744302190680586,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3382",
+ "ndcg10": 0.4594761912515673,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-3392",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.13043478260869565
+ },
+ {
+ "qid": "PLAIN-3402",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3412",
+ "ndcg10": 0.18329273373298458,
+ "mrr10": 1,
+ "recall10": 0.04081632653061224,
+ "recall100": 0.24489795918367346
+ },
+ {
+ "qid": "PLAIN-3422",
+ "ndcg10": 0.02321714761064369,
+ "mrr10": 0.1,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.45161290322580644
+ },
+ {
+ "qid": "PLAIN-3432",
+ "ndcg10": 0.06362078819895171,
+ "mrr10": 0.1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-3442",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-3452",
+ "ndcg10": 0.4247645055987381,
+ "mrr10": 1,
+ "recall10": 0.11764705882352941,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-3462",
+ "ndcg10": 0.6378891765242927,
+ "mrr10": 0.5,
+ "recall10": 0.109375,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3472",
+ "ndcg10": 0.2873431773791806,
+ "mrr10": 1,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.4117647058823529
+ }
+ ],
"baselines": {
"BM25 (Lucene)": 0.325,
"DocT5query": 0.328,
diff --git a/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.json b/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.json
new file mode 100644
index 0000000000..e516f870e8
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.json
@@ -0,0 +1,2131 @@
+{
+ "runAt": "2026-05-30T20:46:06.540Z",
+ "benchmark": "beir-scifact-bge",
+ "dataset": "scifact",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 300,
+ "corpusSize": 5183,
+ "metrics": {
+ "ndcg10": 0.6255647791860727,
+ "mrr10": 0.5931878306878305,
+ "recall10": 0.7412222222222221,
+ "recall100": 0.8283333333333334,
+ "avgQueryLatencyMs": 355.40267458333335
+ },
+ "perQuery": [
+ {
+ "qid": "1",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "3",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "5",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "13",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "36",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "42",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "48",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "49",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "50",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "51",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "53",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "54",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "56",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "57",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "70",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "72",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "75",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "94",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "99",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "100",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "113",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "115",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "118",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "124",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "127",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "128",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "129",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "130",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "132",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "133",
+ "ndcg10": 0.6164336326286644,
+ "mrr10": 1,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "141",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "142",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "143",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "146",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "148",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "171",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "179",
+ "ndcg10": 0.8048099750039491,
+ "mrr10": 1,
+ "recall10": 0.75,
+ "recall100": 0.75
+ },
+ {
+ "qid": "180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "183",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "185",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "198",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "208",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "212",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "217",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "218",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "219",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "230",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "233",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "236",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "237",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "238",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "239",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "248",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "249",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "261",
+ "ndcg10": 0.2640681225725909,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "268",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "269",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "274",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "275",
+ "ndcg10": 0.7039180890341347,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "294",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "295",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "298",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "300",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "303",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "312",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "314",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "324",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "327",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "338",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "343",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "350",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "354",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "362",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "380",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "384",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "385",
+ "ndcg10": 0.6934264036172708,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "386",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "388",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "399",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "410",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "411",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "415",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "421",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "431",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "436",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "437",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "439",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "440",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "443",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "452",
+ "ndcg10": 0.2640681225725909,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "475",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "478",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "491",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "501",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "502",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "507",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "508",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "513",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "514",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "517",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "521",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "525",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "527",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "528",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "532",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "533",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "535",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "536",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "539",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "540",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "544",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "549",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "551",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "552",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "554",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "560",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "569",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "575",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "577",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "578",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "587",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "589",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "593",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "597",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "598",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "613",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "619",
+ "ndcg10": 0.17723928678404774,
+ "mrr10": 0.1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "623",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "628",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "636",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "637",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "641",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "644",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "649",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "659",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "660",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "674",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "684",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "690",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "691",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "692",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "693",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "700",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "702",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "715",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "716",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "718",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "723",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "727",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "728",
+ "ndcg10": 0.8772153153380493,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "729",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "742",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "743",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "744",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "756",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "759",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "768",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "770",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "775",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "781",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "783",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "784",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "785",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "793",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "800",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "805",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "808",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "811",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "814",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "821",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "823",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "830",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "831",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "832",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "834",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "839",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "845",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "847",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "852",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "859",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "870",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "873",
+ "ndcg10": 0.5043776700708298,
+ "mrr10": 0.5,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "879",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "880",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "882",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "903",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "904",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "907",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "911",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "913",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "914",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "921",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "922",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "936",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "956",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "957",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "960",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "967",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "971",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "975",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "982",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "985",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "993",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1012",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1014",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1019",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1020",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1021",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1024",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1029",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1041",
+ "ndcg10": 0.39564672360221187,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1049",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1062",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1086",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1088",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1089",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1099",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1100",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1104",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1107",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1110",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1121",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1132",
+ "ndcg10": 0.39780880120575696,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1140",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1144",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1150",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1175",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1179",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1185",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1187",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1191",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1194",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1196",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1197",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1199",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1200",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1202",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1204",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1207",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1221",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1225",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1226",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1241",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1245",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1259",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1262",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1266",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1270",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1272",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1273",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1274",
+ "ndcg10": 0.9674679834891693,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1278",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1280",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1281",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1282",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1290",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1292",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1298",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1303",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1316",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1319",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1320",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1332",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1335",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1336",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1337",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1339",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1344",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1352",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1359",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1362",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1363",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1368",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1370",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1379",
+ "ndcg10": 0.8318724637288826,
+ "mrr10": 1,
+ "recall10": 0.75,
+ "recall100": 0.75
+ },
+ {
+ "qid": "1382",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1385",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1389",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1395",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.679,
+ "DocT5query": 0.675,
+ "TAS-B": 0.643,
+ "GenQ": 0.644,
+ "ColBERT": 0.671,
+ "Contriever": 0.677,
+ "GTR-XL": 0.662,
+ "SPLADE++": 0.704,
+ "BGE-large-v1.5 (pub)": 0.722,
+ "SBERT msmarco": 0.555
+ },
+ "ourRank": 10,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-bge-latest.json b/docs/benchmarks/runs/beir-scifact-bge-latest.json
new file mode 100644
index 0000000000..e516f870e8
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-bge-latest.json
@@ -0,0 +1,2131 @@
+{
+ "runAt": "2026-05-30T20:46:06.540Z",
+ "benchmark": "beir-scifact-bge",
+ "dataset": "scifact",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 300,
+ "corpusSize": 5183,
+ "metrics": {
+ "ndcg10": 0.6255647791860727,
+ "mrr10": 0.5931878306878305,
+ "recall10": 0.7412222222222221,
+ "recall100": 0.8283333333333334,
+ "avgQueryLatencyMs": 355.40267458333335
+ },
+ "perQuery": [
+ {
+ "qid": "1",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "3",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "5",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "13",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "36",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "42",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "48",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "49",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "50",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "51",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "53",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "54",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "56",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "57",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "70",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "72",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "75",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "94",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "99",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "100",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "113",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "115",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "118",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "124",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "127",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "128",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "129",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "130",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "132",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "133",
+ "ndcg10": 0.6164336326286644,
+ "mrr10": 1,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "141",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "142",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "143",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "146",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "148",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "171",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "179",
+ "ndcg10": 0.8048099750039491,
+ "mrr10": 1,
+ "recall10": 0.75,
+ "recall100": 0.75
+ },
+ {
+ "qid": "180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "183",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "185",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "198",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "208",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "212",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "217",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "218",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "219",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "230",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "233",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "236",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "237",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "238",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "239",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "248",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "249",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "261",
+ "ndcg10": 0.2640681225725909,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "268",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "269",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "274",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "275",
+ "ndcg10": 0.7039180890341347,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "294",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "295",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "298",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "300",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "303",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "312",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "314",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "324",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "327",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "338",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "343",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "350",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "354",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "362",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "380",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "384",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "385",
+ "ndcg10": 0.6934264036172708,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "386",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "388",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "399",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "410",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "411",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "415",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "421",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "431",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "436",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "437",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "439",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "440",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "443",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "452",
+ "ndcg10": 0.2640681225725909,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "475",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "478",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "491",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "501",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "502",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "507",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "508",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "513",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "514",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "517",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "521",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "525",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "527",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "528",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "532",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "533",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "535",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "536",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "539",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "540",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "544",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "549",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "551",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "552",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "554",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "560",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "569",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "575",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "577",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "578",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "587",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "589",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "593",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "597",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "598",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "613",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "619",
+ "ndcg10": 0.17723928678404774,
+ "mrr10": 0.1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "623",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "628",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "636",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "637",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "641",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "644",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "649",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "659",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "660",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "674",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "684",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "690",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "691",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "692",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "693",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "700",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "702",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "715",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "716",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "718",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "723",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "727",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "728",
+ "ndcg10": 0.8772153153380493,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "729",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "742",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "743",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "744",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "756",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "759",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "768",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "770",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "775",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "781",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "783",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "784",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "785",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "793",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "800",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "805",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "808",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "811",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "814",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "821",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "823",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "830",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "831",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "832",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "834",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "839",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "845",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "847",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "852",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "859",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "870",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "873",
+ "ndcg10": 0.5043776700708298,
+ "mrr10": 0.5,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "879",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "880",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "882",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "903",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "904",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "907",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "911",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "913",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "914",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "921",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "922",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "936",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "956",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "957",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "960",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "967",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "971",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "975",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "982",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "985",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "993",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1012",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1014",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1019",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1020",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1021",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1024",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1029",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1041",
+ "ndcg10": 0.39564672360221187,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1049",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1062",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1086",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1088",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1089",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1099",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1100",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1104",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1107",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1110",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1121",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1132",
+ "ndcg10": 0.39780880120575696,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1140",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1144",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1150",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1175",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1179",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1185",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1187",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1191",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1194",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1196",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1197",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1199",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1200",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1202",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1204",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1207",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1221",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1225",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1226",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1241",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1245",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1259",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1262",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1266",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1270",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1272",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1273",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1274",
+ "ndcg10": 0.9674679834891693,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1278",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1280",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1281",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1282",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1290",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1292",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1298",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1303",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1316",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1319",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1320",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1332",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1335",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1336",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1337",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1339",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1344",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1352",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1359",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1362",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1363",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1368",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1370",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1379",
+ "ndcg10": 0.8318724637288826,
+ "mrr10": 1,
+ "recall10": 0.75,
+ "recall100": 0.75
+ },
+ {
+ "qid": "1382",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1385",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1389",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1395",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.679,
+ "DocT5query": 0.675,
+ "TAS-B": 0.643,
+ "GenQ": 0.644,
+ "ColBERT": 0.671,
+ "Contriever": 0.677,
+ "GTR-XL": 0.662,
+ "SPLADE++": 0.704,
+ "BGE-large-v1.5 (pub)": 0.722,
+ "SBERT msmarco": 0.555
+ },
+ "ourRank": 10,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/package-lock.json b/package-lock.json
index 4142f377f8..7907a48308 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "claude-flow",
- "version": "3.10.25",
+ "version": "3.10.26",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "claude-flow",
- "version": "3.10.25",
+ "version": "3.10.26",
"license": "MIT",
"dependencies": {
"@claude-flow/cli-core": "^3.7.0-alpha.5",
diff --git a/package.json b/package.json
index e02c3f40f1..35f9081356 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "claude-flow",
- "version": "3.10.25",
+ "version": "3.10.26",
"description": "Ruflo - Enterprise AI agent orchestration for Claude Code. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration",
"main": "dist/index.js",
"type": "module",
diff --git a/ruflo/package.json b/ruflo/package.json
index 7fa1d22397..1ecef8447f 100644
--- a/ruflo/package.json
+++ b/ruflo/package.json
@@ -1,6 +1,6 @@
{
"name": "ruflo",
- "version": "3.10.25",
+ "version": "3.10.26",
"description": "Ruflo - Enterprise AI agent orchestration platform. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration",
"main": "bin/ruflo.js",
"type": "module",
diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.json
index 9a94a55e28..4ff66a857d 100644
--- a/v3/@claude-flow/cli/package.json
+++ b/v3/@claude-flow/cli/package.json
@@ -1,6 +1,6 @@
{
"name": "@claude-flow/cli",
- "version": "3.10.25",
+ "version": "3.10.26",
"type": "module",
"description": "Ruflo CLI - Enterprise AI agent orchestration with 60+ specialized agents, swarm coordination, MCP server, self-learning hooks, and vector memory for Claude Code",
"main": "dist/src/index.js",
diff --git a/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs b/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs
new file mode 100644
index 0000000000..495cb9563f
--- /dev/null
+++ b/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs
@@ -0,0 +1,162 @@
+#!/usr/bin/env node
+// beir-bootstrap-significance.mjs β€” paired bootstrap significance test for
+// BEIR retrieval results (ADR-086).
+//
+// Given a run JSON with per-query nDCG@10 and a baseline mean nDCG@10,
+// estimates whether our result is significantly above the baseline.
+//
+// Method: paired bootstrap β€” resample N=10000 with replacement from our
+// per-query nDCG scores, compute mean each time, take 2.5%-97.5% percentile
+// for 95% CI. Compare baseline to our CI: if baseline < CI lower bound,
+// the difference is significant at p < 0.05.
+//
+// For a paired test against ANOTHER per-query run (e.g. our pure-BM25 vs
+// our BGE-base), resample paired DIFFERENCES instead. Pass --paired
+// path/to/baseline-run.json.
+//
+// Usage:
+// node scripts/beir-bootstrap-significance.mjs <run-json-path>
+// node scripts/beir-bootstrap-significance.mjs <ours> --paired <baseline>
+//
+// Example:
+// node scripts/beir-bootstrap-significance.mjs \
+// docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
+
+import { readFileSync } from 'node:fs';
+import { performance } from 'node:perf_hooks';
+
+const args = process.argv.slice(2);
+if (args.length === 0) {
+ console.error('Usage: node beir-bootstrap-significance.mjs <run-json> [--paired <baseline-run-json>]');
+ process.exit(1);
+}
+const RUN_PATH = args[0];
+const PAIRED_IDX = args.indexOf('--paired');
+const PAIRED_PATH = PAIRED_IDX > 0 ? args[PAIRED_IDX + 1] : null;
+const ITERATIONS = Number(process.env.BOOTSTRAP_ITERATIONS) || 10000;
+
+// Deterministic PRNG (Mulberry32) β€” so re-runs are reproducible.
+const SEED = Number(process.env.BOOTSTRAP_SEED) || 42;
+function mulberry32(a) {
+ return function () {
+ a |= 0; a = (a + 0x6D2B79F5) | 0;
+ let t = a;
+ t = Math.imul(t ^ (t >>> 15), t | 1);
+ t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+ };
+}
+
+function mean(arr) { let s = 0; for (const v of arr) s += v; return s / arr.length; }
+
+function percentile(sortedArr, p) {
+ const idx = Math.max(0, Math.min(sortedArr.length - 1, Math.floor(p * sortedArr.length)));
+ return sortedArr[idx];
+}
+
+function pointEstimate(scores) {
+ return {
+ mean: mean(scores),
+ n: scores.length,
+ };
+}
+
+function bootstrapCI(scores, iterations, rng, percentiles = [0.025, 0.975]) {
+ const means = new Float64Array(iterations);
+ const n = scores.length;
+ for (let i = 0; i < iterations; i++) {
+ let s = 0;
+ for (let j = 0; j < n; j++) s += scores[Math.floor(rng() * n)];
+ means[i] = s / n;
+ }
+ const sorted = Array.from(means).sort((a, b) => a - b);
+ return {
+ point: mean(scores),
+ lower: percentile(sorted, percentiles[0]),
+ upper: percentile(sorted, percentiles[1]),
+ p_value_vs_baseline_zero: null,
+ };
+}
+
+// Paired bootstrap: compare our per-query scores to a baseline's per-query
+// scores. Resample (idx) and average paired differences. The 95% CI on the
+// mean difference tells us if it's significantly non-zero.
+function pairedBootstrap(oursByQid, baselineByQid, iterations, rng) {
+ const sharedIds = [...oursByQid.keys()].filter((id) => baselineByQid.has(id));
+ const diffs = sharedIds.map((id) => oursByQid.get(id) - baselineByQid.get(id));
+ const n = diffs.length;
+ const means = new Float64Array(iterations);
+ for (let i = 0; i < iterations; i++) {
+ let s = 0;
+ for (let j = 0; j < n; j++) s += diffs[Math.floor(rng() * n)];
+ means[i] = s / n;
+ }
+ const sorted = Array.from(means).sort((a, b) => a - b);
+ // One-sided p β‰ˆ fraction of resamples where mean diff ≀ 0
+ let negCount = 0;
+ for (const m of means) if (m <= 0) negCount++;
+ return {
+ n_paired: n,
+ point_diff: mean(diffs),
+ lower_95: percentile(sorted, 0.025),
+ upper_95: percentile(sorted, 0.975),
+ p_one_sided: negCount / iterations,
+ };
+}
+
+function loadRun(path) {
+ const data = JSON.parse(readFileSync(path, 'utf-8'));
+ if (!data.perQuery || !Array.isArray(data.perQuery)) {
+ console.error(`Run JSON at ${path} has no perQuery array β€” re-run the bench with the updated harness.`);
+ process.exit(2);
+ }
+ return data;
+}
+
+const ours = loadRun(RUN_PATH);
+console.log(`# BEIR significance test (ADR-086)`);
+console.log(`Run: ${RUN_PATH}`);
+console.log(`Dataset: ${ours.dataset ?? '?'} Model: ${ours.model ?? '?'} Queries: ${ours.perQuery.length}`);
+console.log(`Bootstrap: ${ITERATIONS} iterations, seed=${SEED}`);
+
+const rng = mulberry32(SEED);
+const ourScores = ours.perQuery.map((q) => q.ndcg10);
+
+const t0 = performance.now();
+const oursCI = bootstrapCI(ourScores, ITERATIONS, rng);
+console.log(`\n=== Our nDCG@10 (1-sample bootstrap CI) ===`);
+console.log(` point: ${oursCI.point.toFixed(4)}`);
+console.log(` 95% CI: [${oursCI.lower.toFixed(4)}, ${oursCI.upper.toFixed(4)}]`);
+
+if (ours.baselines) {
+ console.log(`\n=== vs each published baseline (CI overlap) ===`);
+ for (const [name, score] of Object.entries(ours.baselines)) {
+ const ciDiff = oursCI.point - score;
+ const lowerDiff = oursCI.lower - score;
+ const upperDiff = oursCI.upper - score;
+ const direction = ciDiff > 0 ? '↑ above' : '↓ below';
+ const sig = (lowerDiff > 0 && upperDiff > 0) ? 'p<0.05 (95% CI excludes baseline)'
+ : (lowerDiff < 0 && upperDiff < 0) ? 'p<0.05 (95% CI below baseline)'
+ : 'n.s. (95% CI overlaps baseline)';
+ console.log(` ${score.toFixed(3)} ${name.padEnd(28)} Ξ”=${ciDiff >= 0 ? '+' : ''}${ciDiff.toFixed(4)} ${direction} [${sig}]`);
+ }
+}
+
+if (PAIRED_PATH) {
+ const baseline = loadRun(PAIRED_PATH);
+ console.log(`\n=== Paired bootstrap vs ${PAIRED_PATH} ===`);
+ const oursByQid = new Map(ours.perQuery.map((q) => [q.qid, q.ndcg10]));
+ const baseByQid = new Map(baseline.perQuery.map((q) => [q.qid, q.ndcg10]));
+ const rng2 = mulberry32(SEED + 1);
+ const paired = pairedBootstrap(oursByQid, baseByQid, ITERATIONS, rng2);
+ console.log(` paired queries: ${paired.n_paired}`);
+ console.log(` Ξ” (ours - baseline): ${paired.point_diff >= 0 ? '+' : ''}${paired.point_diff.toFixed(4)}`);
+ console.log(` 95% CI: [${paired.lower_95.toFixed(4)}, ${paired.upper_95.toFixed(4)}]`);
+ console.log(` one-sided p (Ξ” ≀ 0): ${paired.p_one_sided.toFixed(4)}`);
+ const sig = paired.lower_95 > 0 ? 'SIGNIFICANT improvement (p<0.025 one-sided)' :
+ paired.upper_95 < 0 ? 'SIGNIFICANT regression (p<0.025 one-sided)' :
+ 'NOT SIGNIFICANT β€” CI overlaps zero';
+ console.log(` verdict: ${sig}`);
+}
+
+console.log(`\nBootstrap took ${((performance.now() - t0) / 1000).toFixed(1)}s.`);
diff --git a/v3/@claude-flow/cli/scripts/run-beir-bge.mjs b/v3/@claude-flow/cli/scripts/run-beir-bge.mjs
index cea54b3fe9..4865c12a0f 100644
--- a/v3/@claude-flow/cli/scripts/run-beir-bge.mjs
+++ b/v3/@claude-flow/cli/scripts/run-beir-bge.mjs
@@ -218,25 +218,26 @@ async function main() {
console.log(`\nRunning ${evalQueryIds.length} queries...`);
let nSum = 0, mSum = 0, r10Sum = 0, r100Sum = 0, n = 0;
+ // ADR-086 β€” save per-query metrics for paired bootstrap significance testing.
+ const perQuery = [];
const tQ = performance.now();
for (const qid of evalQueryIds) {
const qtext = queriesById.get(qid);
if (!qtext) continue;
- // BGE recommends prepending "Represent this sentence for searching relevant passages:" to queries
- // but only for v1.5 *en-icl* variants. v1.5 (non-icl) works without it.
+ // BGE v1.5 non-icl works without the query prefix.
const qEmb = await emb.embed(qtext);
- // Score every doc, sort.
const scores = new Array(docEmbeds.length);
for (let i = 0; i < docEmbeds.length; i++) scores[i] = { id: docIds[i], score: cosine(qEmb, docEmbeds[i]) };
scores.sort((a, b) => b.score - a.score);
const top100 = scores.slice(0, 100).map((s) => s.id);
const qmap = qrels.get(qid);
- nSum += ndcg(top100, qmap, 10);
- mSum += mrr(top100, qmap, 10);
- r10Sum += recall(top100, qmap, 10);
- r100Sum += recall(top100, qmap, 100);
- n++;
+ const qNdcg = ndcg(top100, qmap, 10);
+ const qMrr = mrr(top100, qmap, 10);
+ const qR10 = recall(top100, qmap, 10);
+ const qR100 = recall(top100, qmap, 100);
+ nSum += qNdcg; mSum += qMrr; r10Sum += qR10; r100Sum += qR100; n++;
+ perQuery.push({ qid, ndcg10: qNdcg, mrr10: qMrr, recall10: qR10, recall100: qR100 });
if (n % 50 === 0) {
const elapsed = (performance.now() - tQ) / 1000;
console.log(` ${n}/${evalQueryIds.length} in ${elapsed.toFixed(0)}s running nDCG@10=${(nSum / n).toFixed(4)}`);
@@ -276,6 +277,7 @@ async function main() {
queries: n,
corpusSize: corpus.length,
metrics: { ndcg10, mrr10, recall10, recall100, avgQueryLatencyMs: queryMs / n },
+ perQuery, // ADR-086: per-query metrics for paired bootstrap significance testing
baselines: BASELINES_NDCG10,
ourRank,
leaderboardLength: ranking.length,
diff --git a/v3/docs/adr/ADR-085-beir-public-benchmark.md b/v3/docs/adr/ADR-085-beir-public-benchmark.md
index 1d1d80e190..9026909af0 100644
--- a/v3/docs/adr/ADR-085-beir-public-benchmark.md
+++ b/v3/docs/adr/ADR-085-beir-public-benchmark.md
@@ -48,7 +48,9 @@ Standalone runner that bypasses the neural-tools BM25/hybrid pipeline entirely
Why we're slightly below: our multi-field BM25 (subjectWeight=2.0) was tuned for ruflo's commit-style corpus where the "subject" field carries different IDF distribution than NFCorpus's title field. With `subjectWeight=0` (pure body BM25) we get **the same 0.289** β€” meaning the doc `_id` we passed as the "subject" carries no signal. A tighter BEIR-specific tokenizer + stopword list (Lucene-style) would close the remaining 0.04 gap.
-### BGE-base-en-v1.5 (dense retrieval) β€” **TOP-2 RESULT**
+### BGE-base-en-v1.5 (dense retrieval) β€” **TOP-2 on BEIR NFCorpus** (1 dataset, not the BEIR average)
+
+**Pipeline reported:** direct BGE dense path (no fine-tuning, no hybrid BM25+dense fusion, no cross-encoder reranker). Our internal hybrid pipeline (ADRs 078-083) is deliberately isolated from this BEIR comparison so the dense-vs-dense numbers stay honest.
Measured N=323 test queries, full corpus 3,633 docs:
@@ -76,7 +78,12 @@ Measured N=323 test queries, full corpus 3,633 docs:
| 10 | ColBERT | 0.305 |
| 11 | SBERT msmarco | 0.272 |
-We beat SPLADE++ (the published "best" before BGE-large landed), GTR-XL, and every other listed dense retriever using a 3Γ— smaller base model (110M vs 335M). The only thing beating us is BGE-large at 0.380, and that's a model-size advantage we could close by passing `BGE_MODEL=Xenova/bge-large-en-v1.5` to the same harness.
+We beat SPLADE++ (the published "best" before BGE-large landed), GTR-XL, and every other listed dense retriever using a 3Γ— smaller base model (110M vs 335M).
+
+**Caveats on the framing:**
+- The 0.005 gap to SPLADE++ (0.352 vs 0.347) is small. A paired bootstrap significance test (ADR-086) is needed to claim a statistically significant win β€” pending re-run with per-query metrics.
+- **The "swap to BGE-large to close the BGE-large gap" claim is directional, not guaranteed.** Larger model = different dim (1024 vs 768), ~3Γ— larger cache, ~2Γ— query latency. Real number requires the run; we have not yet measured BGE-large on this stack.
+- **One BEIR dataset is not BEIR-SOTA.** BEIR is explicitly an 18-dataset heterogeneous benchmark; rank-2 on NFCorpus is a strong signal, not a generalisation claim. SciFact run (2nd dataset) is queued; broader coverage is tracked.
## Cumulative SOTA context
diff --git a/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.md b/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.md
new file mode 100644
index 0000000000..4d3a8a627e
--- /dev/null
+++ b/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.md
@@ -0,0 +1,202 @@
+# ADR-086 β€” Silent-Fallback Bug Story + Bootstrap Significance Testing
+
+**Status**: Accepted β€” Implemented in ruflo 3.10.26
+**Date**: 2026-05-30
+**Tracking**: continuation of self-learning hardening cluster (ADR-077 β†’ ... β†’ 085)
+**Related**: ADR-085 (BEIR harness)
+
+## Context
+
+Two threads from the BEIR work that deserve their own ADRs:
+
+1. **The silent-fallback embedder bug.** While building ADR-085's BEIR harness on darwin-arm64, every retrieval result looked plausible but cosine similarity scores were essentially random. The neural store reported `_realEmbedding: true`. The hash-fallback was carrying the entire dense signal β€” and we didn't know it. This is the kind of bug that *steals years of debugging* from people who don't notice.
+
+2. **Statistical significance on the 0.005 SPLADE++ gap.** Our NFCorpus result (0.352) is 0.005 above SPLADE++ (0.347). The natural pushback is "noise" β€” and with N=323 queries it's a fair pushback. We need a paired bootstrap CI to either confirm the gap is real or admit it's not.
+
+Both are about honesty in reporting, not new features.
+
+## Decision
+
+### 1. Bug story: silent hash-fallback path
+
+**Root cause chain:**
+
+```
+neural_patterns store action
+ β†’ realEmbeddings.embed(text)
+ β†’ agentic-flow/reasoningbank.computeEmbedding(text)
+ β†’ @xenova/transformers pipeline('feature-extraction', 'all-MiniLM-L6-v2')
+ β†’ @xenova/transformers loads tokenizer + sharp (for image models)
+ β†’ require('sharp')
+ β†’ require('../build/Release/sharp-darwin-arm64v8.node')
+ β†’ MODULE_NOT_FOUND
+ ↓ THROWS
+ ↓ THROWS
+ ↓ THROWS
+ ↓ but the THROW happens per-CALL, not at import time
+ ↓ neural-tools wraps each call in try/catch and silently hash-falls-back
+ ↓ store returns successfully
+↓ neural_patterns reports _realEmbedding: true (because realEmbeddings is non-null at module-load)
+```
+
+**Where the lie lives:** `src/mcp-tools/neural-tools.ts` initialises
+`realEmbeddings = { embed: ... }` at module load by importing
+`agentic-flow/reasoningbank`. That import succeeds even when the
+downstream `transformers.js β†’ sharp` chain breaks β€” because the import is
+just type metadata, not an actual model load. The lazy model load happens
+on the first `computeEmbedding()` call, which throws. The outer try/catch
+swallows it and silently falls back to hash. But `_realEmbedding` was set
+to `true` based on the import success.
+
+**What we did about it:**
+
+- **Bypassed the sharp dependency** in ADR-085 by loading BGE directly
+ via `@xenova/transformers` `AutoTokenizer` + `AutoModel`. Text bi-encoders
+ don't need image preprocessing β€” `sharp` is a transitive dep of the
+ full pipeline that's never needed for retrieval.
+- **Discovered it via the BEIR bench.** A 0.262 nDCG@10 result (vs BM25
+ published 0.325) was suspicious enough to investigate. Without an
+ external benchmark, the bug could have persisted indefinitely because
+ every internal test passed (since hash-based embeds are deterministic
+ and our internal labels are short tokens that BM25 dominates anyway).
+- **Documented in `BEIR-MATRIX.md`** and ADR-085 Β§"sharp-on-darwin-arm64
+ bug" so users on other platforms know to check.
+
+**What we did NOT do** (deferred to a separate fix):
+- Fix the underlying `_realEmbedding: true` lie when per-call embeds
+ throw. The honest fix is a probe-embed at module load that updates
+ the flag if it fails. That's a real change in `neural-tools.ts` and
+ warrants its own dedicated tracking. Leaving as known issue with the
+ bypass.
+
+**The lesson:** every "is the embedder real?" check needs to verify an
+actual embed succeeded, not just that the import didn't throw. Type-load
+success β‰  runtime correctness. External benchmarks expose the gap because
+they don't share the bias of internal labels.
+
+### 2. Paired bootstrap significance test
+
+`scripts/beir-bootstrap-significance.mjs` β€” given a run JSON with
+`perQuery: Array<{qid, ndcg10, ...}>`, computes:
+
+- **1-sample bootstrap CI** on the mean nDCG@10 (10K resamples, deterministic
+ mulberry32 seed=42). 95% CI for the point estimate.
+- **CI overlap test vs each published baseline** β€” if our 95% CI lower
+ bound exceeds the baseline, the difference is significant at p<0.05.
+- **Paired bootstrap** (via `--paired <other-run.json>`) β€” resamples the
+ per-query *differences* between two runs. The one-sided p-value tells us
+ if our run is significantly above (or below) the baseline. The 95% CI on
+ the difference gives both magnitude and uncertainty.
+
+Why paired matters: the same hard queries are hard for everyone. A
+1-sample test treats query difficulty as noise; the paired test conditions
+on it. This is what BEIR papers report.
+
+**Reusable as `BEIR_DATA_DIR=... node beir-bootstrap-significance.mjs`** on
+any run JSON our bench emits.
+
+### 3. Per-query metrics in run JSON
+
+Extended `scripts/run-beir-bge.mjs` to save `perQuery: [{qid, ndcg10,
+mrr10, recall10, recall100}, ...]` in every run JSON. Without this,
+external bootstrap CI testing isn't possible.
+
+### 4. BEIR matrix page
+
+`docs/benchmarks/BEIR-MATRIX.md` β€” dataset Γ— model Γ— metric grid. Every
+cell links to its run JSON and reproduction command. The honest-reporting
+counterpart to the leaderboard table in ADR-085.
+
+## Measured proof β€” bootstrap CIs on both datasets
+
+### NFCorpus (N=323 queries, bootstrap 10k seed=42)
+
+```
+=== Our nDCG@10 (1-sample bootstrap CI) ===
+ point: 0.3517
+ 95% CI: [0.3171, 0.3873]
+
+=== vs each published baseline (CI overlap) ===
+ 0.272 SBERT msmarco Ξ”=+0.0797 ↑ above [p<0.05 β€” significant win]
+ 0.305 ColBERT Ξ”=+0.0467 ↑ above [p<0.05 β€” significant win]
+ 0.319 TAS-B Ξ”=+0.0327 ↑ above n.s.
+ 0.319 GenQ Ξ”=+0.0327 ↑ above n.s.
+ 0.325 BM25 (Lucene) Ξ”=+0.0267 ↑ above n.s.
+ 0.328 DocT5query Ξ”=+0.0237 ↑ above n.s.
+ 0.328 Contriever Ξ”=+0.0237 ↑ above n.s.
+ 0.343 GTR-XL Ξ”=+0.0087 ↑ above n.s.
+ 0.347 SPLADE++ Ξ”=+0.0047 ↑ above n.s.
+ 0.380 BGE-large-v1.5 Ξ”=-0.0283 ↓ below n.s.
+```
+
+**Two significant wins** (SBERT, ColBERT). Seven point-estimate wins are within sampling noise (n.s.) β€” the "rank-2" headline is a single-realisation outcome, not a statistically distinguishable lead over SPLADE++/GTR-XL/BM25 etc. **The 0.005 SPLADE++ gap is noise**, as expected.
+
+### SciFact (N=300 queries, bootstrap 10k seed=42)
+
+```
+=== Our nDCG@10 (1-sample bootstrap CI) ===
+ point: 0.6256
+ 95% CI: [0.5772, 0.6723]
+
+=== vs each published baseline (CI overlap) ===
+ 0.555 SBERT msmarco Ξ”=+0.0706 ↑ above [p<0.05 β€” significant win]
+ 0.643 TAS-B Ξ”=-0.0174 ↓ below n.s.
+ 0.644 GenQ Ξ”=-0.0184 ↓ below n.s.
+ 0.662 GTR-XL Ξ”=-0.0364 ↓ below n.s.
+ 0.671 ColBERT Ξ”=-0.0454 ↓ below n.s.
+ 0.675 DocT5query Ξ”=-0.0494 ↓ below [p<0.05 β€” significant loss]
+ 0.677 Contriever Ξ”=-0.0514 ↓ below [p<0.05 β€” significant loss]
+ 0.679 BM25 (Lucene) Ξ”=-0.0534 ↓ below [p<0.05 β€” significant loss]
+ 0.704 SPLADE++ Ξ”=-0.0784 ↓ below [p<0.05 β€” significant loss]
+ 0.722 BGE-large-v1.5 Ξ”=-0.0964 ↓ below [p<0.05 β€” significant loss]
+```
+
+**One significant win** (SBERT). **Five significant losses** including to BM25. SciFact is a fact-verification benchmark where exact scientific terms favor lexical retrieval; zero-shot BGE-base doesn't have the in-domain training that BGE-large + SPLADE++ have.
+
+### The two-dataset truth
+
+| System | NFCorpus | SciFact | Mean |
+|---|---:|---:|---:|
+| BGE-large-v1.5 (335M, published) | 0.380 | 0.722 | 0.551 |
+| **ruflo + BGE-base-en-v1.5 (110M)** | **0.352** | **0.626** | **0.489** |
+| SPLADE++ | 0.347 | 0.704 | 0.526 |
+| BM25 (Lucene) | 0.325 | 0.679 | 0.502 |
+
+**On the two-dataset mean, we lose to BM25** (0.489 vs 0.502). The NFCorpus rank-2 is real but not representative. The acceptance test "beats BM25 on both datasets" fails β€” and reporting that honestly is the point of this ADR.
+
+## Honest limits
+
+- **N=323 is the dataset's full test split** β€” we can't add more queries
+ without leaving NFCorpus.
+- **Bootstrap assumes per-query independence** β€” true for IID query sets,
+ approximately true for BEIR.
+- **The `_realEmbedding: true` lie is not yet fixed** in `neural-tools.ts`;
+ the BGE bypass works around it but other call paths through
+ `neural_patterns` may still report the wrong flag value. Tracked.
+
+## Deliberately NOT in this round
+
+- **Fix the `_realEmbedding: true` lie** at source β€” needs a probe-embed
+ at module init. Tracked separately.
+- **Lucene-style BM25 improvements** to close our 0.289 β†’ 0.325 BM25 gap.
+ Stemming + bigger stopword list would help; not in scope here.
+
+## Verification
+
+```bash
+git clone https://github.com/ruvnet/ruflo && cd ruflo
+npm install && ( cd v3/@claude-flow/cli && npx tsc )
+
+# Re-run the NFCorpus bench with updated harness (writes perQuery to JSON)
+cd /tmp/beir-nfcorpus
+SKIP_INGEST=1 node /path/to/scripts/run-beir-bge.mjs
+
+# Bootstrap significance test (10K resamples, ~1s)
+node /path/to/scripts/beir-bootstrap-significance.mjs \
+ /path/to/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json
+
+# Paired test vs our pure-BM25 baseline
+node /path/to/scripts/beir-bootstrap-significance.mjs \
+ /path/to/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json \
+ --paired /path/to/docs/benchmarks/runs/beir-nfcorpus-2026-05-30T19-16-23-024Z.json
+```