π The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
feat(intelligence): BEIR matrix + bootstrap CIs + 2nd dataset (ADR-086, 3.10.26)
ruv1490278176da
diff --git a/docs/benchmarks/BEIR-MATRIX.md b/docs/benchmarks/BEIR-MATRIX.mdnew file mode 100644index 0000000000..f6d979af6b--- /dev/null+++ b/docs/benchmarks/BEIR-MATRIX.md@@ -0,0 +1,133 @@+# BEIR Public-Benchmark Results β ruflo++This page tracks ruflo's measured retrieval performance on BEIR datasets.+Every cell is reproducible from the commands in the rightmost column;+every cell has a run JSON in `docs/benchmarks/runs/`. Published baseline+numbers come from Thakur et al. 2021 (BEIR paper) and the BAAI BGE paper.++## Result Matrix++> Numbers from `docs/benchmarks/runs/beir-{dataset}-bge-latest.json`. ADR-085+> (harness + BGE swap) + ADR-086 (significance testing).++| Dataset | Corpus | Test Q | Pipeline | Model | nDCG@10 | 95% CI | vs BM25 (CI verdict) | vs Best Listed Baseline | Rank | Latency | Run JSON |+|---|---:|---:|---|---|---:|---|---|---:|---:|---:|---|+| **NFCorpus** | 3,633 | 323 | direct dense (no rerank) | BGE-base-en-v1.5 (110M) | **0.352** | [0.317, 0.387] | +0.027 β (n.s.) | -0.028 β BGE-large 0.380 (n.s.) | **2/11** | 388 ms | `beir-nfcorpus-bge-latest.json` |+| **NFCorpus** | 3,633 | 323 | pure BM25 (silent hash-fallback path) | _no real dense_ | 0.289 | _n/a_ | -0.036 β | -0.091 β | 11/11 | 950 ms | `beir-nfcorpus-2026-05-30T19-16-23-024Z.json` |+| **SciFact** | 5,183 | 300 | direct dense (no rerank) | BGE-base-en-v1.5 (110M) | 0.626 | [0.577, 0.672] | **-0.053 β (p<0.05)** | -0.096 β BGE-large 0.722 (p<0.05) | 10/11 | 410 ms | `beir-scifact-bge-latest.json` |++### Bootstrap CI summary (per ADR-086, 10k resamples, seed=42)++The 95% confidence intervals tell the rigorous story. On NFCorpus, we beat BM25 by 0.027 *point estimate* but the CI overlaps the baseline (n.s. at p<0.05) β the "rank-2" headline is a single-realisation outcome, not a statistically distinguishable win. On SciFact, we lose to BM25 by 0.053 and the CI **excludes** the baseline (significant at p<0.05) β that loss is real, not noise.++### Two-dataset mean (rough generalisation gauge)++| System | NFCorpus | SciFact | Mean |+|---|---:|---:|---:|+| BGE-large-v1.5 (published) | 0.380 | 0.722 | 0.551 |+| **ruflo + BGE-base-en-v1.5** | **0.352** | **0.626** | **0.489** |+| SPLADE++ | 0.347 | 0.704 | 0.526 |+| BM25 (Lucene) | 0.325 | 0.679 | 0.502 |++**We're below BM25 on the 2-dataset mean** (0.489 vs 0.502). The BEIR-average story requires more datasets *and* domain-specific tuning. The NFCorpus rank-2 is real but not representative.++> **What pipeline is reported here:** the NFCorpus 0.352 row is the **direct+> BGE dense path** β no fine-tuning, no hybrid BM25+dense fusion, no+> cross-encoder reranker. The hybrid pipeline (cosine + multi-field BM25 ++> MMR + opt-in rerank, ADRs 078-083) is what ruflo uses internally for+> small-corpus retrieval; the BEIR runner deliberately isolates the dense+> path for clean comparison to dense baselines. Hybrid + rerank variants on+> BEIR are tracked for a future ADR.++## Published Baselines (for reference)++### NFCorpus nDCG@10 (medical IR, n=323 test queries)++| Method | Params | nDCG@10 | Source |+|---|---:|---:|---|+| BGE-large-v1.5 | 335M | 0.380 | BAAI BGE paper |+| **ruflo + BGE-base-en-v1.5** | **110M** | **0.352** | **this repo** |+| SPLADE++ | 110M | 0.347 | Formal et al. 2022 |+| GTR-XL | 1.2B | 0.343 | Ni et al. 2022 |+| DocT5query | 60M | 0.328 | Nogueira & Lin 2019 |+| Contriever | 110M | 0.328 | Izacard et al. 2022 |+| BM25 (Lucene) | β | 0.325 | Thakur et al. 2021 |+| TAS-B | 66M | 0.319 | HofstΓ€tter et al. 2021 |+| GenQ | 110M | 0.319 | Thakur et al. 2021 |+| ColBERT | 110M | 0.305 | Khattab & Zaharia 2020 |+| SBERT (msmarco) | 110M | 0.272 | Reimers & Gurevych 2019 |++### SciFact nDCG@10 (scientific IR, n=300 test queries)++| Method | nDCG@10 | Source |+|---|---:|---|+| BGE-large-v1.5 | 0.722 | BAAI BGE paper |+| SPLADE++ | 0.704 | Formal et al. 2022 |+| BM25 (Lucene) | 0.679 | Thakur et al. 2021 |+| Contriever | 0.677 | Izacard et al. 2022 |+| DocT5query | 0.675 | Nogueira & Lin 2019 |+| ColBERT | 0.671 | Khattab & Zaharia 2020 |+| GTR-XL | 0.662 | Ni et al. 2022 |+| GenQ | 0.644 | Thakur et al. 2021 |+| TAS-B | 0.643 | HofstΓ€tter et al. 2021 |+| SBERT (msmarco) | 0.555 | Reimers & Gurevych 2019 |++## How to reproduce++```bash+git clone https://github.com/ruvnet/ruflo && cd ruflo+npm install && ( cd v3/@claude-flow/cli && npx tsc )++# NFCorpus+mkdir -p /tmp/beir-nfcorpus && cd /tmp/beir-nfcorpus+curl -sL -o nfcorpus.zip 'https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/nfcorpus.zip' && unzip -q nfcorpus.zip+node /path/to/ruflo/v3/@claude-flow/cli/scripts/run-beir-bge.mjs+# β nDCG@10 0.352, rank 2 of 11++# SciFact+mkdir -p /tmp/beir-scifact && cd /tmp/beir-scifact+curl -sL -o scifact.zip 'https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip' && unzip -q scifact.zip+BEIR_DATA_DIR=/tmp/beir-scifact/scifact node /path/to/ruflo/v3/@claude-flow/cli/scripts/run-beir-bge.mjs++# Paired bootstrap significance test (ADR-086)+node /path/to/ruflo/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs \+ /path/to/ruflo/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json+```++## Model size / speed / quality trade-offs++| Model | Params | Embed dim | Cache size (NFCorpus) | Ingest (3,633 docs) | Query latency |+|---|---:|---:|---:|---:|---:|+| `Xenova/bge-small-en-v1.5` | 33M | 384 | ~5.5 MB | ~15 min | ~250 ms |+| `Xenova/bge-base-en-v1.5` | 110M | 768 | ~11 MB | ~25 min | ~330 ms |+| `Xenova/bge-large-en-v1.5` | 335M | 1024 | ~15 MB | ~60 min (est.) | ~700 ms (est.) |++Per-row latency is on Apple Silicon CPU through `@xenova/transformers`+int8-quantised ONNX. GPU would be ~10-50Γ faster.++## Methodology notes++- **No fine-tuning.** All numbers are zero-shot β we use BAAI's released+ BGE models as-is. NFCorpus has a 110K-pair train split that fine-tuning+ could exploit for an additional ~0.02-0.05 nDCG lift; not done here.+- **`@xenova/transformers` direct API** (not `pipeline()`) used to bypass+ the `sharp`/`libvips` transitive dependency that breaks on+ darwin-arm64 (ADR-085 Β§"sharp-on-darwin-arm64 bug").+- **CLS-token pooling + L2 normalisation** per BAAI's BGE spec; cosine+ becomes dot product on normalised vectors.+- **Graded relevance for nDCG** β qrels use 0/1/2 grades; we use+ `(2^rel - 1) / log2(i+1)` per BEIR convention.+- **Reproducibility**: `BOOTSTRAP_SEED=42` for the significance test+ (mulberry32 PRNG). Run JSONs include full per-query metrics so+ external bootstrap-CI checks reproduce exactly.++## Limits & next steps++- **Two-dataset coverage isn't BEIR-average.** BEIR ships 18 datasets;+ the published "BEIR average" is the standard generalisation gauge.+ Tracking: TREC-COVID, FiQA-2018, ArguAna, HotpotQA, NQ next.+- **Single-annotator labelled retrieval** for internal ruflo bench+ (ADR-081); not relevant to BEIR's externally-curated qrels.+- **The 0.005 gap to SPLADE++** (0.352 vs 0.347) is on the edge of noise+ at N=323. The paired bootstrap test (ADR-086) gives a confidence+ interval; report both point estimate AND CI.diff --git a/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.json b/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.jsonnew file mode 100644index 0000000000..a0f342b224--- /dev/null+++ b/docs/benchmarks/runs/beir-nfcorpus-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-29-42-702Z.json@@ -0,0 +1,2292 @@+{+ "runAt": "2026-05-30T20:29:42.702Z",+ "benchmark": "beir-nfcorpus-bge",+ "dataset": "nfcorpus",+ "model": "Xenova/bge-base-en-v1.5",+ "queries": 323,+ "corpusSize": 3633,+ "metrics": {+ "ndcg10": 0.35171471732484655,+ "mrr10": 0.5456029780333185,+ "recall10": 0.16652743860112934,+ "recall100": 0.30466713166229387,+ "avgQueryLatencyMs": 900.598096879257+ },+ "perQuery": [+ {+ "qid": "PLAIN-2",+ "ndcg10": 0.7412469912547754,+ "mrr10": 1,+ "recall10": 0.2916666666666667,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-12",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.23333333333333334+ },+ {+ "qid": "PLAIN-23",+ "ndcg10": 0.4951328496468998,+ "mrr10": 1,+ "recall10": 0.05555555555555555,+ "recall100": 0.15555555555555556+ },+ {+ "qid": "PLAIN-33",+ "ndcg10": 0.46667017129647237,+ "mrr10": 1,+ "recall10": 0.125,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-44",+ "ndcg10": 0.2940620318598474,+ "mrr10": 1,+ "recall10": 0.05172413793103448,+ "recall100": 0.1724137931034483+ },+ {+ "qid": "PLAIN-56",+ "ndcg10": 0.5097750261375381,+ "mrr10": 1,+ "recall10": 0.1,+ "recall100": 0.26+ },+ {+ "qid": "PLAIN-68",+ "ndcg10": 0.8588798377757242,+ "mrr10": 1,+ "recall10": 0.12727272727272726,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-78",+ "ndcg10": 0.08966284300766811,+ "mrr10": 1,+ "recall10": 0.016129032258064516,+ "recall100": 0.12903225806451613+ },+ {+ "qid": "PLAIN-91",+ "ndcg10": 0.6081351105039879,+ "mrr10": 1,+ "recall10": 0.140625,+ "recall100": 0.328125+ },+ {+ "qid": "PLAIN-102",+ "ndcg10": 0.12811612289210494,+ "mrr10": 1,+ "recall10": 0.041666666666666664,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-112",+ "ndcg10": 0.6858781710646398,+ "mrr10": 1,+ "recall10": 0.08928571428571429,+ "recall100": 0.17857142857142858+ },+ {+ "qid": "PLAIN-123",+ "ndcg10": 0.2848186035513736,+ "mrr10": 0.3333333333333333,+ "recall10": 0.09803921568627451,+ "recall100": 0.1568627450980392+ },+ {+ "qid": "PLAIN-133",+ "ndcg10": 0.08649891544052932,+ "mrr10": 0.1111111111111111,+ "recall10": 0.027777777777777776,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-143",+ "ndcg10": 0.37061294442664394,+ "mrr10": 1,+ "recall10": 0.0392156862745098,+ "recall100": 0.2549019607843137+ },+ {+ "qid": "PLAIN-153",+ "ndcg10": 0.8948763559161851,+ "mrr10": 1,+ "recall10": 0.1702127659574468,+ "recall100": 0.2978723404255319+ },+ {+ "qid": "PLAIN-165",+ "ndcg10": 0.3937404317428725,+ "mrr10": 0.5,+ "recall10": 0.16666666666666666,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-175",+ "ndcg10": 0.29608176037159845,+ "mrr10": 1,+ "recall10": 0.10810810810810811,+ "recall100": 0.13513513513513514+ },+ {+ "qid": "PLAIN-186",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.041666666666666664,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-196",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.043478260869565216+ },+ {+ "qid": "PLAIN-207",+ "ndcg10": 0.6578986223737446,+ "mrr10": 1,+ "recall10": 0.10909090909090909,+ "recall100": 0.2545454545454545+ },+ {+ "qid": "PLAIN-217",+ "ndcg10": 0.12987501165334076,+ "mrr10": 0.1111111111111111,+ "recall10": 0.05555555555555555,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-227",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1+ },+ {+ "qid": "PLAIN-238",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.037037037037037035+ },+ {+ "qid": "PLAIN-248",+ "ndcg10": 0.4666234168001789,+ "mrr10": 0.5,+ "recall10": 0.16129032258064516,+ "recall100": 0.25806451612903225+ },+ {+ "qid": "PLAIN-259",+ "ndcg10": 0.2984900352767555,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-270",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.019230769230769232,+ "recall100": 0.21153846153846154+ },+ {+ "qid": "PLAIN-280",+ "ndcg10": 0.3436966913921454,+ "mrr10": 0.5,+ "recall10": 0.15,+ "recall100": 0.55+ },+ {+ "qid": "PLAIN-291",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.15384615384615385,+ "recall100": 0.5384615384615384+ },+ {+ "qid": "PLAIN-307",+ "ndcg10": 0.7653606369886218,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "PLAIN-320",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2619047619047619+ },+ {+ "qid": "PLAIN-332",+ "ndcg10": 0.6131471927654584,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-344",+ "ndcg10": 0.6166392118848165,+ "mrr10": 0.5,+ "recall10": 0.11475409836065574,+ "recall100": 0.2459016393442623+ },+ {+ "qid": "PLAIN-358",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-371",+ "ndcg10": 0.27541155237618664,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "PLAIN-383",+ "ndcg10": 0.14606834984270645,+ "mrr10": 0.25,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-395",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-407",+ "ndcg10": 0.7262287617954056,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-418",+ "ndcg10": 0.8344475896326595,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-430",+ "ndcg10": 0.4967105728753997,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-441",+ "ndcg10": 0.727463892412172,+ "mrr10": 1,+ "recall10": 0.8,+ "recall100": 1+ },+ {+ "qid": "PLAIN-457",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-468",+ "ndcg10": 0.521256019952062,+ "mrr10": 1,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-478",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-488",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.7333333333333333+ },+ {+ "qid": "PLAIN-499",+ "ndcg10": 0.2532954041105318,+ "mrr10": 0.25,+ "recall10": 0.08823529411764706,+ "recall100": 0.14705882352941177+ },+ {+ "qid": "PLAIN-510",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-520",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-531",+ "ndcg10": 0.2906246349515899,+ "mrr10": 0.5,+ "recall10": 0.00974025974025974,+ "recall100": 0.09090909090909091+ },+ {+ "qid": "PLAIN-541",+ "ndcg10": 0.10778915452451093,+ "mrr10": 0.16666666666666666,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-551",+ "ndcg10": 0.6131471927654584,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-561",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.08,+ "recall100": 0.12+ },+ {+ "qid": "PLAIN-571",+ "ndcg10": 0.2162957183053307,+ "mrr10": 0.25,+ "recall10": 0.2857142857142857,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-583",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.022727272727272728,+ "recall100": 0.09090909090909091+ },+ {+ "qid": "PLAIN-593",+ "ndcg10": 0.46927872602275644,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-603",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-613",+ "ndcg10": 0.13886244387355454,+ "mrr10": 0.5,+ "recall10": 0.1,+ "recall100": 0.3+ },+ {+ "qid": "PLAIN-623",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.024390243902439025,+ "recall100": 0.12195121951219512+ },+ {+ "qid": "PLAIN-634",+ "ndcg10": 0.424926013816671,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-645",+ "ndcg10": 0.07336392209936005,+ "mrr10": 0.14285714285714285,+ "recall10": 0.034482758620689655,+ "recall100": 0.10344827586206896+ },+ {+ "qid": "PLAIN-660",+ "ndcg10": 0.9363792118010483,+ "mrr10": 1,+ "recall10": 0.018947368421052633,+ "recall100": 0.0863157894736842+ },+ {+ "qid": "PLAIN-671",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-681",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-691",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.09523809523809523+ },+ {+ "qid": "PLAIN-701",+ "ndcg10": 0.1662433493697512,+ "mrr10": 0.16666666666666666,+ "recall10": 0.25,+ "recall100": 0.375+ },+ {+ "qid": "PLAIN-711",+ "ndcg10": 0.5579780239523999,+ "mrr10": 1,+ "recall10": 0.08771929824561403,+ "recall100": 0.22807017543859648+ },+ {+ "qid": "PLAIN-721",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.64+ },+ {+ "qid": "PLAIN-731",+ "ndcg10": 0.7241528458281337,+ "mrr10": 1,+ "recall10": 0.12962962962962962,+ "recall100": 0.37037037037037035+ },+ {+ "qid": "PLAIN-741",+ "ndcg10": 0.800693766409882,+ "mrr10": 1,+ "recall10": 0.3684210526315789,+ "recall100": 0.3684210526315789+ },+ {+ "qid": "PLAIN-751",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-761",+ "ndcg10": 0.4103915680233244,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-771",+ "ndcg10": 0.9134015924715543,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-782",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-792",+ "ndcg10": 0.30260241349881345,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-806",+ "ndcg10": 0.933745776545611,+ "mrr10": 1,+ "recall10": 0.09375,+ "recall100": 0.22916666666666666+ },+ {+ "qid": "PLAIN-817",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "PLAIN-827",+ "ndcg10": 0.84860265890399,+ "mrr10": 1,+ "recall10": 0.03278688524590164,+ "recall100": 0.21721311475409835+ },+ {+ "qid": "PLAIN-838",+ "ndcg10": 0.6930221935231562,+ "mrr10": 1,+ "recall10": 0.5555555555555556,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-850",+ "ndcg10": 0.5541432109622958,+ "mrr10": 1,+ "recall10": 0.07017543859649122,+ "recall100": 0.17543859649122806+ },+ {+ "qid": "PLAIN-872",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-882",+ "ndcg10": 0.8701249883466593,+ "mrr10": 1,+ "recall10": 0.5333333333333333,+ "recall100": 0.8+ },+ {+ "qid": "PLAIN-892",+ "ndcg10": 0.6332196796270029,+ "mrr10": 1,+ "recall10": 0.05434782608695652,+ "recall100": 0.21739130434782608+ },+ {+ "qid": "PLAIN-902",+ "ndcg10": 0.3903800499921017,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-913",+ "ndcg10": 0.8603818544462509,+ "mrr10": 1,+ "recall10": 0.23529411764705882,+ "recall100": 0.35294117647058826+ },+ {+ "qid": "PLAIN-924",+ "ndcg10": 0.31516255047698366,+ "mrr10": 0.5,+ "recall10": 0.09375,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-934",+ "ndcg10": 0.7917267193679839,+ "mrr10": 1,+ "recall10": 0.06930693069306931,+ "recall100": 0.31683168316831684+ },+ {+ "qid": "PLAIN-946",+ "ndcg10": 0.13568544539116628,+ "mrr10": 0.125,+ "recall10": 0.06451612903225806,+ "recall100": 0.12903225806451613+ },+ {+ "qid": "PLAIN-956",+ "ndcg10": 0.6594803070902429,+ "mrr10": 1,+ "recall10": 0.02912621359223301,+ "recall100": 0.17475728155339806+ },+ {+ "qid": "PLAIN-966",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-977",+ "ndcg10": 0.2960819109658652,+ "mrr10": 0.5,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-987",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-997",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1008",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1018",+ "ndcg10": 0.4521472607752954,+ "mrr10": 0.5,+ "recall10": 0.08333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1028",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1039",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1050",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.06382978723404255+ },+ {+ "qid": "PLAIN-1066",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1088",+ "ndcg10": 0.5531464700081437,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-1098",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1109",+ "ndcg10": 0.684383072597662,+ "mrr10": 1,+ "recall10": 0.061224489795918366,+ "recall100": 0.35714285714285715+ },+ {+ "qid": "PLAIN-1119",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1130",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1141",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.0625,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-1151",+ "ndcg10": 0.7065443116025598,+ "mrr10": 0.5,+ "recall10": 0.05228758169934641,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-1161",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1172",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.05263157894736842,+ "recall100": 0.15789473684210525+ },+ {+ "qid": "PLAIN-1183",+ "ndcg10": 0.23365080824310533,+ "mrr10": 0.5,+ "recall10": 0.08,+ "recall100": 0.08+ },+ {+ "qid": "PLAIN-1193",+ "ndcg10": 0.20483424751859086,+ "mrr10": 0.3333333333333333,+ "recall10": 0.1111111111111111,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-1203",+ "ndcg10": 0.5174612499126158,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1214",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1225",+ "ndcg10": 0.6830838076905166,+ "mrr10": 1,+ "recall10": 0.2222222222222222,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-1236",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1249",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1262",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1275",+ "ndcg10": 0.43735247915031,+ "mrr10": 1,+ "recall10": 0.05454545454545454,+ "recall100": 0.2545454545454545+ },+ {+ "qid": "PLAIN-1288",+ "ndcg10": 0.5933607200949346,+ "mrr10": 1,+ "recall10": 0.025510204081632654,+ "recall100": 0.12755102040816327+ },+ {+ "qid": "PLAIN-1299",+ "ndcg10": 0.11004588314904008,+ "mrr10": 0.3333333333333333,+ "recall10": 0.01639344262295082,+ "recall100": 0.06557377049180328+ },+ {+ "qid": "PLAIN-1309",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1320",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.16666666666666666+ },+ {+ "qid": "PLAIN-1331",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1342",+ "ndcg10": 0.4440973278132557,+ "mrr10": 1,+ "recall10": 0.1875,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-1353",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1363",+ "ndcg10": 0.24655055544758614,+ "mrr10": 0.25,+ "recall10": 0.2727272727272727,+ "recall100": 0.7272727272727273+ },+ {+ "qid": "PLAIN-1374",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.19230769230769232+ },+ {+ "qid": "PLAIN-1387",+ "ndcg10": 0.7183627972686238,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-1398",+ "ndcg10": 0.9216017310213247,+ "mrr10": 1,+ "recall10": 0.08737864077669903,+ "recall100": 0.22330097087378642+ },+ {+ "qid": "PLAIN-1409",+ "ndcg10": 0.24936270394794902,+ "mrr10": 0.25,+ "recall10": 0.011857707509881422,+ "recall100": 0.1541501976284585+ },+ {+ "qid": "PLAIN-1419",+ "ndcg10": 0.4885468020156227,+ "mrr10": 1,+ "recall10": 0.06153846153846154,+ "recall100": 0.1076923076923077+ },+ {+ "qid": "PLAIN-1429",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1441",+ "ndcg10": 0.5599332595218056,+ "mrr10": 1,+ "recall10": 0.026881720430107527,+ "recall100": 0.15053763440860216+ },+ {+ "qid": "PLAIN-1453",+ "ndcg10": 0.5103515512676448,+ "mrr10": 1,+ "recall10": 0.0163265306122449,+ "recall100": 0.06938775510204082+ },+ {+ "qid": "PLAIN-1463",+ "ndcg10": 0.2082936658103318,+ "mrr10": 0.5,+ "recall10": 0.08333333333333333,+ "recall100": 0.2916666666666667+ },+ {+ "qid": "PLAIN-1473",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1485",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1496",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1506",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.13333333333333333+ },+ {+ "qid": "PLAIN-1516",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1527",+ "ndcg10": 0.9052116356304493,+ "mrr10": 1,+ "recall10": 0.04455445544554455,+ "recall100": 0.18811881188118812+ },+ {+ "qid": "PLAIN-1537",+ "ndcg10": 0.13886244387355454,+ "mrr10": 0.5,+ "recall10": 0.022222222222222223,+ "recall100": 0.15555555555555556+ },+ {+ "qid": "PLAIN-1547",+ "ndcg10": 0.13032376591037062,+ "mrr10": 0.25,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1557",+ "ndcg10": 0.2489083270225946,+ "mrr10": 0.5,+ "recall10": 0.07692307692307693,+ "recall100": 0.11538461538461539+ },+ {+ "qid": "PLAIN-1568",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1579",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1590",+ "ndcg10": 0.06625422345438903,+ "mrr10": 0.1111111111111111,+ "recall10": 0.05555555555555555,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-1601",+ "ndcg10": 0.6457545768494583,+ "mrr10": 1,+ "recall10": 0.08108108108108109,+ "recall100": 0.24324324324324326+ },+ {+ "qid": "PLAIN-1611",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1621",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1635",+ "ndcg10": 0.9266360779006401,+ "mrr10": 1,+ "recall10": 0.02088167053364269,+ "recall100": 0.14153132250580047+ },+ {+ "qid": "PLAIN-1645",+ "ndcg10": 0.15130120674940672,+ "mrr10": 0.3333333333333333,+ "recall10": 0.16666666666666666,+ "recall100": 0.16666666666666666+ },+ {+ "qid": "PLAIN-1656",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.07142857142857142+ },+ {+ "qid": "PLAIN-1667",+ "ndcg10": 0.7877736340270854,+ "mrr10": 1,+ "recall10": 0.050314465408805034,+ "recall100": 0.1509433962264151+ },+ {+ "qid": "PLAIN-1679",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1690",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1700",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-1710",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.625,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-1721",+ "ndcg10": 0.40995091484653445,+ "mrr10": 0.5,+ "recall10": 0.12903225806451613,+ "recall100": 0.16129032258064516+ },+ {+ "qid": "PLAIN-1731",+ "ndcg10": 0.7653606369886217,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1741",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.023809523809523808,+ "recall100": 0.11666666666666667+ },+ {+ "qid": "PLAIN-1752",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1762",+ "ndcg10": 0.5784505554216093,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1772",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1784",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1794",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1805",+ "ndcg10": 0.9052116356304493,+ "mrr10": 1,+ "recall10": 0.07758620689655173,+ "recall100": 0.2672413793103448+ },+ {+ "qid": "PLAIN-1817",+ "ndcg10": 0.42838543210841196,+ "mrr10": 1,+ "recall10": 0.1111111111111111,+ "recall100": 0.2962962962962963+ },+ {+ "qid": "PLAIN-1827",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1837",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.05102040816326531,+ "recall100": 0.32142857142857145+ },+ {+ "qid": "PLAIN-1847",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.09090909090909091,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-1857",+ "ndcg10": 0.5957616317188126,+ "mrr10": 1,+ "recall10": 0.15151515151515152,+ "recall100": 0.42424242424242425+ },+ {+ "qid": "PLAIN-1867",+ "ndcg10": 0.07945707943276742,+ "mrr10": 0.1,+ "recall10": 0.14285714285714285,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-1877",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.05555555555555555+ },+ {+ "qid": "PLAIN-1887",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-1897",+ "ndcg10": 0.17723928678404774,+ "mrr10": 0.1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1909",+ "ndcg10": 0.8263333286520083,+ "mrr10": 1,+ "recall10": 0.017278617710583154,+ "recall100": 0.1101511879049676+ },+ {+ "qid": "PLAIN-1919",+ "ndcg10": 0.7967610662472993,+ "mrr10": 1,+ "recall10": 0.25925925925925924,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1929",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-1940",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.5833333333333334+ },+ {+ "qid": "PLAIN-1950",+ "ndcg10": 0.2489083270225946,+ "mrr10": 0.5,+ "recall10": 0.15384615384615385,+ "recall100": 0.23076923076923078+ },+ {+ "qid": "PLAIN-1962",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1972",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.36363636363636365+ },+ {+ "qid": "PLAIN-1983",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.3125,+ "recall100": 0.6875+ },+ {+ "qid": "PLAIN-1995",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.18181818181818182+ },+ {+ "qid": "PLAIN-2009",+ "ndcg10": 0.3700049343491939,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 0.625+ },+ {+ "qid": "PLAIN-2019",+ "ndcg10": 0.13565197343244778,+ "mrr10": 0.1,+ "recall10": 0.3333333333333333,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2030",+ "ndcg10": 0.19092086617893467,+ "mrr10": 0.5,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-2040",+ "ndcg10": 0.4950940324881554,+ "mrr10": 0.5,+ "recall10": 0.03787878787878788,+ "recall100": 0.13636363636363635+ },+ {+ "qid": "PLAIN-2051",+ "ndcg10": 0.5423640154200349,+ "mrr10": 1,+ "recall10": 0.011267605633802818,+ "recall100": 0.09859154929577464+ },+ {+ "qid": "PLAIN-2061",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.024390243902439025,+ "recall100": 0.13902439024390245+ },+ {+ "qid": "PLAIN-2071",+ "ndcg10": 0.5455017237786198,+ "mrr10": 0.5,+ "recall10": 0.13333333333333333,+ "recall100": 0.37777777777777777+ },+ {+ "qid": "PLAIN-2081",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2092",+ "ndcg10": 0.6489315753318465,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2102",+ "ndcg10": 0.7600791051976192,+ "mrr10": 1,+ "recall10": 0.0166270783847981,+ "recall100": 0.1163895486935867+ },+ {+ "qid": "PLAIN-2113",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2124",+ "ndcg10": 0.5455206287241073,+ "mrr10": 1,+ "recall10": 0.29411764705882354,+ "recall100": 0.5294117647058824+ },+ {+ "qid": "PLAIN-2134",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.08333333333333333+ },+ {+ "qid": "PLAIN-2145",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-2156",+ "ndcg10": 0.7751482523375255,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.8333333333333334+ },+ {+ "qid": "PLAIN-2167",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2177",+ "ndcg10": 0.5072380780124418,+ "mrr10": 0.5,+ "recall10": 0.20833333333333334,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2187",+ "ndcg10": 0.23504554941448536,+ "mrr10": 1,+ "recall10": 0.1111111111111111,+ "recall100": 0.2222222222222222+ },+ {+ "qid": "PLAIN-2197",+ "ndcg10": 0.6496872770120411,+ "mrr10": 1,+ "recall10": 0.10344827586206896,+ "recall100": 0.3103448275862069+ },+ {+ "qid": "PLAIN-2209",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2220",+ "ndcg10": 0.38334277998463445,+ "mrr10": 1,+ "recall10": 0.2222222222222222,+ "recall100": 0.2222222222222222+ },+ {+ "qid": "PLAIN-2230",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2240",+ "ndcg10": 0.2082936658103318,+ "mrr10": 0.5,+ "recall10": 0.18181818181818182,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-2250",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2261",+ "ndcg10": 0.5637884576902256,+ "mrr10": 1,+ "recall10": 0.06666666666666667,+ "recall100": 0.15+ },+ {+ "qid": "PLAIN-2271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2281",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2291",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-2301",+ "ndcg10": 0.13961814555374907,+ "mrr10": 0.14285714285714285,+ "recall10": 0.15384615384615385,+ "recall100": 0.3076923076923077+ },+ {+ "qid": "PLAIN-2311",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2321",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2332",+ "ndcg10": 0.7663491917568946,+ "mrr10": 1,+ "recall10": 0.08247422680412371,+ "recall100": 0.4020618556701031+ },+ {+ "qid": "PLAIN-2343",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2354",+ "ndcg10": 0.5087403079104241,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2364",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2375",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2386",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3684210526315789+ },+ {+ "qid": "PLAIN-2396",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-2408",+ "ndcg10": 0.5531464700081437,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2430",+ "ndcg10": 0.5740106608557803,+ "mrr10": 1,+ "recall10": 0.15217391304347827,+ "recall100": 0.41304347826086957+ },+ {+ "qid": "PLAIN-2440",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.047619047619047616+ },+ {+ "qid": "PLAIN-2450",+ "ndcg10": 0.2336508082431053,+ "mrr10": 0.5,+ "recall10": 0.05263157894736842,+ "recall100": 0.13157894736842105+ },+ {+ "qid": "PLAIN-2460",+ "ndcg10": 0.43174458805633853,+ "mrr10": 1,+ "recall10": 0.12903225806451613,+ "recall100": 0.4838709677419355+ },+ {+ "qid": "PLAIN-2470",+ "ndcg10": 0.7024216173807135,+ "mrr10": 1,+ "recall10": 0.0821917808219178,+ "recall100": 0.3287671232876712+ },+ {+ "qid": "PLAIN-2480",+ "ndcg10": 0.4407905902246705,+ "mrr10": 1,+ "recall10": 0.2777777777777778,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2490",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.34782608695652173+ },+ {+ "qid": "PLAIN-2500",+ "ndcg10": 0.15487585161383474,+ "mrr10": 0.16666666666666666,+ "recall10": 0.14285714285714285,+ "recall100": 0.7142857142857143+ },+ {+ "qid": "PLAIN-2510",+ "ndcg10": 0.589664883987166,+ "mrr10": 0.5,+ "recall10": 0.15217391304347827,+ "recall100": 0.5869565217391305+ },+ {+ "qid": "PLAIN-2520",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1095890410958904+ },+ {+ "qid": "PLAIN-2530",+ "ndcg10": 0.955830517697074,+ "mrr10": 1,+ "recall10": 0.1388888888888889,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-2540",+ "ndcg10": 0.4941492558420748,+ "mrr10": 0.5,+ "recall10": 0.2857142857142857,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2550",+ "ndcg10": 0.24746798875357182,+ "mrr10": 0.5,+ "recall10": 0.08,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-2560",+ "ndcg10": 0.9363792118010483,+ "mrr10": 1,+ "recall10": 0.42857142857142855,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2570",+ "ndcg10": 0.25523504782501816,+ "mrr10": 0.3333333333333333,+ "recall10": 0.075,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-2580",+ "ndcg10": 0.1135664319411187,+ "mrr10": 1,+ "recall10": 0.043478260869565216,+ "recall100": 0.13043478260869565+ },+ {+ "qid": "PLAIN-2590",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.07936507936507936+ },+ {+ "qid": "PLAIN-2600",+ "ndcg10": 0.6613158235505792,+ "mrr10": 1,+ "recall10": 0.3,+ "recall100": 0.3+ },+ {+ "qid": "PLAIN-2610",+ "ndcg10": 0.4973811325345484,+ "mrr10": 1,+ "recall10": 0.06557377049180328,+ "recall100": 0.2459016393442623+ },+ {+ "qid": "PLAIN-2620",+ "ndcg10": 0.6770586454665828,+ "mrr10": 1,+ "recall10": 0.16363636363636364,+ "recall100": 0.36363636363636365+ },+ {+ "qid": "PLAIN-2630",+ "ndcg10": 0.8188458073987622,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.3958333333333333+ },+ {+ "qid": "PLAIN-2640",+ "ndcg10": 0.39923129859010653,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.5416666666666666+ },+ {+ "qid": "PLAIN-2650",+ "ndcg10": 0.6489315753318465,+ "mrr10": 1,+ "recall10": 0.1724137931034483,+ "recall100": 0.3448275862068966+ },+ {+ "qid": "PLAIN-2660",+ "ndcg10": 0.40030566811860463,+ "mrr10": 0.5,+ "recall10": 0.14814814814814814,+ "recall100": 0.48148148148148145+ },+ {+ "qid": "PLAIN-2670",+ "ndcg10": 0.3406992958233561,+ "mrr10": 1,+ "recall10": 0.02857142857142857,+ "recall100": 0.05714285714285714+ },+ {+ "qid": "PLAIN-2680",+ "ndcg10": 0.454740507204534,+ "mrr10": 0.5,+ "recall10": 0.2857142857142857,+ "recall100": 0.42857142857142855+ },+ {+ "qid": "PLAIN-2690",+ "ndcg10": 0.6852219536434856,+ "mrr10": 1,+ "recall10": 0.20930232558139536,+ "recall100": 0.4418604651162791+ },+ {+ "qid": "PLAIN-2700",+ "ndcg10": 0.5438770046596358,+ "mrr10": 1,+ "recall10": 0.3,+ "recall100": 0.7+ },+ {+ "qid": "PLAIN-2710",+ "ndcg10": 0.6906119546391791,+ "mrr10": 1,+ "recall10": 0.23809523809523808,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2720",+ "ndcg10": 0.24639745374629807,+ "mrr10": 0.2,+ "recall10": 0.1111111111111111,+ "recall100": 0.2777777777777778+ },+ {+ "qid": "PLAIN-2730",+ "ndcg10": 0.5235258308273208,+ "mrr10": 1,+ "recall10": 0.13793103448275862,+ "recall100": 0.41379310344827586+ },+ {+ "qid": "PLAIN-2740",+ "ndcg10": 0.6782347310356166,+ "mrr10": 1,+ "recall10": 0.1,+ "recall100": 0.16+ },+ {+ "qid": "PLAIN-2750",+ "ndcg10": 0.532003134830924,+ "mrr10": 1,+ "recall10": 0.1016949152542373,+ "recall100": 0.23728813559322035+ },+ {+ "qid": "PLAIN-2760",+ "ndcg10": 0.4702199443548266,+ "mrr10": 0.5,+ "recall10": 0.25,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2770",+ "ndcg10": 0.325096961030005,+ "mrr10": 0.5,+ "recall10": 0.12195121951219512,+ "recall100": 0.4878048780487805+ },+ {+ "qid": "PLAIN-2780",+ "ndcg10": 0.4033852369868885,+ "mrr10": 1,+ "recall10": 0.2727272727272727,+ "recall100": 0.4090909090909091+ },+ {+ "qid": "PLAIN-2790",+ "ndcg10": 0.652358405855227,+ "mrr10": 1,+ "recall10": 0.1016949152542373,+ "recall100": 0.2033898305084746+ },+ {+ "qid": "PLAIN-2800",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.041666666666666664+ },+ {+ "qid": "PLAIN-2810",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-2820",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2830",+ "ndcg10": 0.7262287617954056,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2840",+ "ndcg10": 0.09478836436955078,+ "mrr10": 0.25,+ "recall10": 0.058823529411764705,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-2850",+ "ndcg10": 0.3026286548261086,+ "mrr10": 0.5,+ "recall10": 0.1111111111111111,+ "recall100": 0.4444444444444444+ },+ {+ "qid": "PLAIN-2860",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2870",+ "ndcg10": 0.0781764177375172,+ "mrr10": 0.2,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-2880",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-2890",+ "ndcg10": 0.1974505933948842,+ "mrr10": 0.25,+ "recall10": 0.034482758620689655,+ "recall100": 0.034482758620689655+ },+ {+ "qid": "PLAIN-2900",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2910",+ "ndcg10": 0.5330346140547638,+ "mrr10": 1,+ "recall10": 0.13333333333333333,+ "recall100": 0.26666666666666666+ },+ {+ "qid": "PLAIN-2920",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-2930",+ "ndcg10": 0.5175857101991203,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-2940",+ "ndcg10": 0.12811612289210494,+ "mrr10": 0.14285714285714285,+ "recall10": 0.045454545454545456,+ "recall100": 0.18181818181818182+ },+ {+ "qid": "PLAIN-2950",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2960",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-2970",+ "ndcg10": 0.6338262955454459,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-2981",+ "ndcg10": 0.5321036168710066,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-2991",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3001",+ "ndcg10": 0.3065735963827292,+ "mrr10": 0.3333333333333333,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3014",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3026",+ "ndcg10": 0.6309297535714574,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3037",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3053",+ "ndcg10": 0.7653606369886218,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-3063",+ "ndcg10": 0.19519002499605084,+ "mrr10": 0.3333333333333333,+ "recall10": 0.25,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-3074",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3085",+ "ndcg10": 0.38162168437253385,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3097",+ "ndcg10": 0.515157140670655,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-3116",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3131",+ "ndcg10": 0.6516433935264055,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3141",+ "ndcg10": 0.3170808479749099,+ "mrr10": 0.3333333333333333,+ "recall10": 0.058823529411764705,+ "recall100": 0.20588235294117646+ },+ {+ "qid": "PLAIN-3151",+ "ndcg10": 0.7456217791401517,+ "mrr10": 1,+ "recall10": 0.3888888888888889,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-3161",+ "ndcg10": 0.524282944620724,+ "mrr10": 1,+ "recall10": 0.046511627906976744,+ "recall100": 0.27906976744186046+ },+ {+ "qid": "PLAIN-3171",+ "ndcg10": 0.044175472610956126,+ "mrr10": 0.1,+ "recall10": 0.029411764705882353,+ "recall100": 0.058823529411764705+ },+ {+ "qid": "PLAIN-3181",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.037037037037037035+ },+ {+ "qid": "PLAIN-3191",+ "ndcg10": 0.4300543345163763,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3201",+ "ndcg10": 0.2037495755301314,+ "mrr10": 0.2,+ "recall10": 0.0625,+ "recall100": 0.09375+ },+ {+ "qid": "PLAIN-3211",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.05555555555555555+ },+ {+ "qid": "PLAIN-3221",+ "ndcg10": 0.06581686446496139,+ "mrr10": 0.25,+ "recall10": 0.041666666666666664,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-3231",+ "ndcg10": 0.2501157675209561,+ "mrr10": 0.5,+ "recall10": 0.5,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-3241",+ "ndcg10": 0.45846608015576257,+ "mrr10": 1,+ "recall10": 0.034482758620689655,+ "recall100": 0.06896551724137931+ },+ {+ "qid": "PLAIN-3251",+ "ndcg10": 0.16330891279672524,+ "mrr10": 0.16666666666666666,+ "recall10": 0.047619047619047616,+ "recall100": 0.09523809523809523+ },+ {+ "qid": "PLAIN-3261",+ "ndcg10": 0.17034964791167806,+ "mrr10": 0.3333333333333333,+ "recall10": 0.07692307692307693,+ "recall100": 0.07692307692307693+ },+ {+ "qid": "PLAIN-3271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.10204081632653061+ },+ {+ "qid": "PLAIN-3281",+ "ndcg10": 0.24249682153453903,+ "mrr10": 0.5,+ "recall10": 0.038461538461538464,+ "recall100": 0.07692307692307693+ },+ {+ "qid": "PLAIN-3292",+ "ndcg10": 0.34839716434274914,+ "mrr10": 1,+ "recall10": 0.42857142857142855,+ "recall100": 0.5714285714285714+ },+ {+ "qid": "PLAIN-3302",+ "ndcg10": 0.13084731762486884,+ "mrr10": 0.3333333333333333,+ "recall10": 0.05405405405405406,+ "recall100": 0.3783783783783784+ },+ {+ "qid": "PLAIN-3312",+ "ndcg10": 0.30114238162847884,+ "mrr10": 0.5,+ "recall10": 0.3076923076923077,+ "recall100": 0.6153846153846154+ },+ {+ "qid": "PLAIN-3322",+ "ndcg10": 0.28165758790969186,+ "mrr10": 0.5,+ "recall10": 0.0379746835443038,+ "recall100": 0.13924050632911392+ },+ {+ "qid": "PLAIN-3332",+ "ndcg10": 0.7386266690451878,+ "mrr10": 1,+ "recall10": 0.35714285714285715,+ "recall100": 0.8571428571428571+ },+ {+ "qid": "PLAIN-3342",+ "ndcg10": 0.1671604549662023,+ "mrr10": 0.16666666666666666,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-3352",+ "ndcg10": 0.3843483686763149,+ "mrr10": 1,+ "recall10": 0.02564102564102564,+ "recall100": 0.1282051282051282+ },+ {+ "qid": "PLAIN-3362",+ "ndcg10": 0.6268451902108538,+ "mrr10": 1,+ "recall10": 0.18181818181818182,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-3372",+ "ndcg10": 0.27744302190680586,+ "mrr10": 0.3333333333333333,+ "recall10": 0.14285714285714285,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-3382",+ "ndcg10": 0.4594761912515673,+ "mrr10": 1,+ "recall10": 0.35714285714285715,+ "recall100": 0.7142857142857143+ },+ {+ "qid": "PLAIN-3392",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.13043478260869565+ },+ {+ "qid": "PLAIN-3402",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-3412",+ "ndcg10": 0.18329273373298458,+ "mrr10": 1,+ "recall10": 0.04081632653061224,+ "recall100": 0.24489795918367346+ },+ {+ "qid": "PLAIN-3422",+ "ndcg10": 0.02321714761064369,+ "mrr10": 0.1,+ "recall10": 0.03225806451612903,+ "recall100": 0.45161290322580644+ },+ {+ "qid": "PLAIN-3432",+ "ndcg10": 0.06362078819895171,+ "mrr10": 0.1,+ "recall10": 0.023809523809523808,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-3442",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.034482758620689655+ },+ {+ "qid": "PLAIN-3452",+ "ndcg10": 0.4247645055987381,+ "mrr10": 1,+ "recall10": 0.11764705882352941,+ "recall100": 0.5294117647058824+ },+ {+ "qid": "PLAIN-3462",+ "ndcg10": 0.6378891765242927,+ "mrr10": 0.5,+ "recall10": 0.109375,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3472",+ "ndcg10": 0.2873431773791806,+ "mrr10": 1,+ "recall10": 0.058823529411764705,+ "recall100": 0.4117647058823529+ }+ ],+ "baselines": {+ "BM25 (Lucene)": 0.325,+ "DocT5query": 0.328,+ "TAS-B": 0.319,+ "GenQ": 0.319,+ "ColBERT": 0.305,+ "Contriever": 0.328,+ "GTR-XL": 0.343,+ "SPLADE++": 0.347,+ "BGE-large-v1.5 (pub)": 0.38,+ "SBERT msmarco": 0.272+ },+ "ourRank": 2,+ "leaderboardLength": 11+}\ No newline at end of filediff --git a/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json b/docs/benchmarks/runs/beir-nfcorpus-bge-latest.jsonindex 4798a06c10..a0f342b224 100644--- a/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json+++ b/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json@@ -1,5 +1,5 @@{- "runAt": "2026-05-30T20:06:41.238Z",+ "runAt": "2026-05-30T20:29:42.702Z","benchmark": "beir-nfcorpus-bge","dataset": "nfcorpus","model": "Xenova/bge-base-en-v1.5",@@ -10,8 +10,2271 @@"mrr10": 0.5456029780333185,"recall10": 0.16652743860112934,"recall100": 0.30466713166229387,- "avgQueryLatencyMs": 388.4598924148607+ "avgQueryLatencyMs": 900.598096879257},+ "perQuery": [+ {+ "qid": "PLAIN-2",+ "ndcg10": 0.7412469912547754,+ "mrr10": 1,+ "recall10": 0.2916666666666667,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-12",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.23333333333333334+ },+ {+ "qid": "PLAIN-23",+ "ndcg10": 0.4951328496468998,+ "mrr10": 1,+ "recall10": 0.05555555555555555,+ "recall100": 0.15555555555555556+ },+ {+ "qid": "PLAIN-33",+ "ndcg10": 0.46667017129647237,+ "mrr10": 1,+ "recall10": 0.125,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-44",+ "ndcg10": 0.2940620318598474,+ "mrr10": 1,+ "recall10": 0.05172413793103448,+ "recall100": 0.1724137931034483+ },+ {+ "qid": "PLAIN-56",+ "ndcg10": 0.5097750261375381,+ "mrr10": 1,+ "recall10": 0.1,+ "recall100": 0.26+ },+ {+ "qid": "PLAIN-68",+ "ndcg10": 0.8588798377757242,+ "mrr10": 1,+ "recall10": 0.12727272727272726,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-78",+ "ndcg10": 0.08966284300766811,+ "mrr10": 1,+ "recall10": 0.016129032258064516,+ "recall100": 0.12903225806451613+ },+ {+ "qid": "PLAIN-91",+ "ndcg10": 0.6081351105039879,+ "mrr10": 1,+ "recall10": 0.140625,+ "recall100": 0.328125+ },+ {+ "qid": "PLAIN-102",+ "ndcg10": 0.12811612289210494,+ "mrr10": 1,+ "recall10": 0.041666666666666664,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-112",+ "ndcg10": 0.6858781710646398,+ "mrr10": 1,+ "recall10": 0.08928571428571429,+ "recall100": 0.17857142857142858+ },+ {+ "qid": "PLAIN-123",+ "ndcg10": 0.2848186035513736,+ "mrr10": 0.3333333333333333,+ "recall10": 0.09803921568627451,+ "recall100": 0.1568627450980392+ },+ {+ "qid": "PLAIN-133",+ "ndcg10": 0.08649891544052932,+ "mrr10": 0.1111111111111111,+ "recall10": 0.027777777777777776,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-143",+ "ndcg10": 0.37061294442664394,+ "mrr10": 1,+ "recall10": 0.0392156862745098,+ "recall100": 0.2549019607843137+ },+ {+ "qid": "PLAIN-153",+ "ndcg10": 0.8948763559161851,+ "mrr10": 1,+ "recall10": 0.1702127659574468,+ "recall100": 0.2978723404255319+ },+ {+ "qid": "PLAIN-165",+ "ndcg10": 0.3937404317428725,+ "mrr10": 0.5,+ "recall10": 0.16666666666666666,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-175",+ "ndcg10": 0.29608176037159845,+ "mrr10": 1,+ "recall10": 0.10810810810810811,+ "recall100": 0.13513513513513514+ },+ {+ "qid": "PLAIN-186",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.041666666666666664,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-196",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.043478260869565216+ },+ {+ "qid": "PLAIN-207",+ "ndcg10": 0.6578986223737446,+ "mrr10": 1,+ "recall10": 0.10909090909090909,+ "recall100": 0.2545454545454545+ },+ {+ "qid": "PLAIN-217",+ "ndcg10": 0.12987501165334076,+ "mrr10": 0.1111111111111111,+ "recall10": 0.05555555555555555,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-227",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1+ },+ {+ "qid": "PLAIN-238",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.037037037037037035+ },+ {+ "qid": "PLAIN-248",+ "ndcg10": 0.4666234168001789,+ "mrr10": 0.5,+ "recall10": 0.16129032258064516,+ "recall100": 0.25806451612903225+ },+ {+ "qid": "PLAIN-259",+ "ndcg10": 0.2984900352767555,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-270",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.019230769230769232,+ "recall100": 0.21153846153846154+ },+ {+ "qid": "PLAIN-280",+ "ndcg10": 0.3436966913921454,+ "mrr10": 0.5,+ "recall10": 0.15,+ "recall100": 0.55+ },+ {+ "qid": "PLAIN-291",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.15384615384615385,+ "recall100": 0.5384615384615384+ },+ {+ "qid": "PLAIN-307",+ "ndcg10": 0.7653606369886218,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "PLAIN-320",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2619047619047619+ },+ {+ "qid": "PLAIN-332",+ "ndcg10": 0.6131471927654584,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-344",+ "ndcg10": 0.6166392118848165,+ "mrr10": 0.5,+ "recall10": 0.11475409836065574,+ "recall100": 0.2459016393442623+ },+ {+ "qid": "PLAIN-358",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-371",+ "ndcg10": 0.27541155237618664,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "PLAIN-383",+ "ndcg10": 0.14606834984270645,+ "mrr10": 0.25,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-395",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-407",+ "ndcg10": 0.7262287617954056,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-418",+ "ndcg10": 0.8344475896326595,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-430",+ "ndcg10": 0.4967105728753997,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-441",+ "ndcg10": 0.727463892412172,+ "mrr10": 1,+ "recall10": 0.8,+ "recall100": 1+ },+ {+ "qid": "PLAIN-457",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-468",+ "ndcg10": 0.521256019952062,+ "mrr10": 1,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-478",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-488",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.7333333333333333+ },+ {+ "qid": "PLAIN-499",+ "ndcg10": 0.2532954041105318,+ "mrr10": 0.25,+ "recall10": 0.08823529411764706,+ "recall100": 0.14705882352941177+ },+ {+ "qid": "PLAIN-510",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-520",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-531",+ "ndcg10": 0.2906246349515899,+ "mrr10": 0.5,+ "recall10": 0.00974025974025974,+ "recall100": 0.09090909090909091+ },+ {+ "qid": "PLAIN-541",+ "ndcg10": 0.10778915452451093,+ "mrr10": 0.16666666666666666,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-551",+ "ndcg10": 0.6131471927654584,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-561",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.08,+ "recall100": 0.12+ },+ {+ "qid": "PLAIN-571",+ "ndcg10": 0.2162957183053307,+ "mrr10": 0.25,+ "recall10": 0.2857142857142857,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-583",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.022727272727272728,+ "recall100": 0.09090909090909091+ },+ {+ "qid": "PLAIN-593",+ "ndcg10": 0.46927872602275644,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-603",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-613",+ "ndcg10": 0.13886244387355454,+ "mrr10": 0.5,+ "recall10": 0.1,+ "recall100": 0.3+ },+ {+ "qid": "PLAIN-623",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.024390243902439025,+ "recall100": 0.12195121951219512+ },+ {+ "qid": "PLAIN-634",+ "ndcg10": 0.424926013816671,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-645",+ "ndcg10": 0.07336392209936005,+ "mrr10": 0.14285714285714285,+ "recall10": 0.034482758620689655,+ "recall100": 0.10344827586206896+ },+ {+ "qid": "PLAIN-660",+ "ndcg10": 0.9363792118010483,+ "mrr10": 1,+ "recall10": 0.018947368421052633,+ "recall100": 0.0863157894736842+ },+ {+ "qid": "PLAIN-671",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-681",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-691",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.09523809523809523+ },+ {+ "qid": "PLAIN-701",+ "ndcg10": 0.1662433493697512,+ "mrr10": 0.16666666666666666,+ "recall10": 0.25,+ "recall100": 0.375+ },+ {+ "qid": "PLAIN-711",+ "ndcg10": 0.5579780239523999,+ "mrr10": 1,+ "recall10": 0.08771929824561403,+ "recall100": 0.22807017543859648+ },+ {+ "qid": "PLAIN-721",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.64+ },+ {+ "qid": "PLAIN-731",+ "ndcg10": 0.7241528458281337,+ "mrr10": 1,+ "recall10": 0.12962962962962962,+ "recall100": 0.37037037037037035+ },+ {+ "qid": "PLAIN-741",+ "ndcg10": 0.800693766409882,+ "mrr10": 1,+ "recall10": 0.3684210526315789,+ "recall100": 0.3684210526315789+ },+ {+ "qid": "PLAIN-751",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-761",+ "ndcg10": 0.4103915680233244,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-771",+ "ndcg10": 0.9134015924715543,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-782",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-792",+ "ndcg10": 0.30260241349881345,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-806",+ "ndcg10": 0.933745776545611,+ "mrr10": 1,+ "recall10": 0.09375,+ "recall100": 0.22916666666666666+ },+ {+ "qid": "PLAIN-817",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "PLAIN-827",+ "ndcg10": 0.84860265890399,+ "mrr10": 1,+ "recall10": 0.03278688524590164,+ "recall100": 0.21721311475409835+ },+ {+ "qid": "PLAIN-838",+ "ndcg10": 0.6930221935231562,+ "mrr10": 1,+ "recall10": 0.5555555555555556,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-850",+ "ndcg10": 0.5541432109622958,+ "mrr10": 1,+ "recall10": 0.07017543859649122,+ "recall100": 0.17543859649122806+ },+ {+ "qid": "PLAIN-872",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-882",+ "ndcg10": 0.8701249883466593,+ "mrr10": 1,+ "recall10": 0.5333333333333333,+ "recall100": 0.8+ },+ {+ "qid": "PLAIN-892",+ "ndcg10": 0.6332196796270029,+ "mrr10": 1,+ "recall10": 0.05434782608695652,+ "recall100": 0.21739130434782608+ },+ {+ "qid": "PLAIN-902",+ "ndcg10": 0.3903800499921017,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-913",+ "ndcg10": 0.8603818544462509,+ "mrr10": 1,+ "recall10": 0.23529411764705882,+ "recall100": 0.35294117647058826+ },+ {+ "qid": "PLAIN-924",+ "ndcg10": 0.31516255047698366,+ "mrr10": 0.5,+ "recall10": 0.09375,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-934",+ "ndcg10": 0.7917267193679839,+ "mrr10": 1,+ "recall10": 0.06930693069306931,+ "recall100": 0.31683168316831684+ },+ {+ "qid": "PLAIN-946",+ "ndcg10": 0.13568544539116628,+ "mrr10": 0.125,+ "recall10": 0.06451612903225806,+ "recall100": 0.12903225806451613+ },+ {+ "qid": "PLAIN-956",+ "ndcg10": 0.6594803070902429,+ "mrr10": 1,+ "recall10": 0.02912621359223301,+ "recall100": 0.17475728155339806+ },+ {+ "qid": "PLAIN-966",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-977",+ "ndcg10": 0.2960819109658652,+ "mrr10": 0.5,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-987",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-997",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1008",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1018",+ "ndcg10": 0.4521472607752954,+ "mrr10": 0.5,+ "recall10": 0.08333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1028",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1039",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1050",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.06382978723404255+ },+ {+ "qid": "PLAIN-1066",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1088",+ "ndcg10": 0.5531464700081437,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-1098",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1109",+ "ndcg10": 0.684383072597662,+ "mrr10": 1,+ "recall10": 0.061224489795918366,+ "recall100": 0.35714285714285715+ },+ {+ "qid": "PLAIN-1119",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1130",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1141",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.0625,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-1151",+ "ndcg10": 0.7065443116025598,+ "mrr10": 0.5,+ "recall10": 0.05228758169934641,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-1161",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1172",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.05263157894736842,+ "recall100": 0.15789473684210525+ },+ {+ "qid": "PLAIN-1183",+ "ndcg10": 0.23365080824310533,+ "mrr10": 0.5,+ "recall10": 0.08,+ "recall100": 0.08+ },+ {+ "qid": "PLAIN-1193",+ "ndcg10": 0.20483424751859086,+ "mrr10": 0.3333333333333333,+ "recall10": 0.1111111111111111,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-1203",+ "ndcg10": 0.5174612499126158,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1214",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1225",+ "ndcg10": 0.6830838076905166,+ "mrr10": 1,+ "recall10": 0.2222222222222222,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-1236",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1249",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1262",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1275",+ "ndcg10": 0.43735247915031,+ "mrr10": 1,+ "recall10": 0.05454545454545454,+ "recall100": 0.2545454545454545+ },+ {+ "qid": "PLAIN-1288",+ "ndcg10": 0.5933607200949346,+ "mrr10": 1,+ "recall10": 0.025510204081632654,+ "recall100": 0.12755102040816327+ },+ {+ "qid": "PLAIN-1299",+ "ndcg10": 0.11004588314904008,+ "mrr10": 0.3333333333333333,+ "recall10": 0.01639344262295082,+ "recall100": 0.06557377049180328+ },+ {+ "qid": "PLAIN-1309",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1320",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.16666666666666666+ },+ {+ "qid": "PLAIN-1331",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1342",+ "ndcg10": 0.4440973278132557,+ "mrr10": 1,+ "recall10": 0.1875,+ "recall100": 0.3125+ },+ {+ "qid": "PLAIN-1353",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1363",+ "ndcg10": 0.24655055544758614,+ "mrr10": 0.25,+ "recall10": 0.2727272727272727,+ "recall100": 0.7272727272727273+ },+ {+ "qid": "PLAIN-1374",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.19230769230769232+ },+ {+ "qid": "PLAIN-1387",+ "ndcg10": 0.7183627972686238,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-1398",+ "ndcg10": 0.9216017310213247,+ "mrr10": 1,+ "recall10": 0.08737864077669903,+ "recall100": 0.22330097087378642+ },+ {+ "qid": "PLAIN-1409",+ "ndcg10": 0.24936270394794902,+ "mrr10": 0.25,+ "recall10": 0.011857707509881422,+ "recall100": 0.1541501976284585+ },+ {+ "qid": "PLAIN-1419",+ "ndcg10": 0.4885468020156227,+ "mrr10": 1,+ "recall10": 0.06153846153846154,+ "recall100": 0.1076923076923077+ },+ {+ "qid": "PLAIN-1429",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1441",+ "ndcg10": 0.5599332595218056,+ "mrr10": 1,+ "recall10": 0.026881720430107527,+ "recall100": 0.15053763440860216+ },+ {+ "qid": "PLAIN-1453",+ "ndcg10": 0.5103515512676448,+ "mrr10": 1,+ "recall10": 0.0163265306122449,+ "recall100": 0.06938775510204082+ },+ {+ "qid": "PLAIN-1463",+ "ndcg10": 0.2082936658103318,+ "mrr10": 0.5,+ "recall10": 0.08333333333333333,+ "recall100": 0.2916666666666667+ },+ {+ "qid": "PLAIN-1473",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1485",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1496",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1506",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.13333333333333333+ },+ {+ "qid": "PLAIN-1516",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1527",+ "ndcg10": 0.9052116356304493,+ "mrr10": 1,+ "recall10": 0.04455445544554455,+ "recall100": 0.18811881188118812+ },+ {+ "qid": "PLAIN-1537",+ "ndcg10": 0.13886244387355454,+ "mrr10": 0.5,+ "recall10": 0.022222222222222223,+ "recall100": 0.15555555555555556+ },+ {+ "qid": "PLAIN-1547",+ "ndcg10": 0.13032376591037062,+ "mrr10": 0.25,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1557",+ "ndcg10": 0.2489083270225946,+ "mrr10": 0.5,+ "recall10": 0.07692307692307693,+ "recall100": 0.11538461538461539+ },+ {+ "qid": "PLAIN-1568",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1579",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1590",+ "ndcg10": 0.06625422345438903,+ "mrr10": 0.1111111111111111,+ "recall10": 0.05555555555555555,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-1601",+ "ndcg10": 0.6457545768494583,+ "mrr10": 1,+ "recall10": 0.08108108108108109,+ "recall100": 0.24324324324324326+ },+ {+ "qid": "PLAIN-1611",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1621",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1635",+ "ndcg10": 0.9266360779006401,+ "mrr10": 1,+ "recall10": 0.02088167053364269,+ "recall100": 0.14153132250580047+ },+ {+ "qid": "PLAIN-1645",+ "ndcg10": 0.15130120674940672,+ "mrr10": 0.3333333333333333,+ "recall10": 0.16666666666666666,+ "recall100": 0.16666666666666666+ },+ {+ "qid": "PLAIN-1656",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.07142857142857142+ },+ {+ "qid": "PLAIN-1667",+ "ndcg10": 0.7877736340270854,+ "mrr10": 1,+ "recall10": 0.050314465408805034,+ "recall100": 0.1509433962264151+ },+ {+ "qid": "PLAIN-1679",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1690",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1700",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-1710",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.625,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-1721",+ "ndcg10": 0.40995091484653445,+ "mrr10": 0.5,+ "recall10": 0.12903225806451613,+ "recall100": 0.16129032258064516+ },+ {+ "qid": "PLAIN-1731",+ "ndcg10": 0.7653606369886217,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1741",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.023809523809523808,+ "recall100": 0.11666666666666667+ },+ {+ "qid": "PLAIN-1752",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1762",+ "ndcg10": 0.5784505554216093,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1772",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1784",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1794",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-1805",+ "ndcg10": 0.9052116356304493,+ "mrr10": 1,+ "recall10": 0.07758620689655173,+ "recall100": 0.2672413793103448+ },+ {+ "qid": "PLAIN-1817",+ "ndcg10": 0.42838543210841196,+ "mrr10": 1,+ "recall10": 0.1111111111111111,+ "recall100": 0.2962962962962963+ },+ {+ "qid": "PLAIN-1827",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1837",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.05102040816326531,+ "recall100": 0.32142857142857145+ },+ {+ "qid": "PLAIN-1847",+ "ndcg10": 0.22009176629808017,+ "mrr10": 1,+ "recall10": 0.09090909090909091,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-1857",+ "ndcg10": 0.5957616317188126,+ "mrr10": 1,+ "recall10": 0.15151515151515152,+ "recall100": 0.42424242424242425+ },+ {+ "qid": "PLAIN-1867",+ "ndcg10": 0.07945707943276742,+ "mrr10": 0.1,+ "recall10": 0.14285714285714285,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-1877",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.05555555555555555+ },+ {+ "qid": "PLAIN-1887",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-1897",+ "ndcg10": 0.17723928678404774,+ "mrr10": 0.1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-1909",+ "ndcg10": 0.8263333286520083,+ "mrr10": 1,+ "recall10": 0.017278617710583154,+ "recall100": 0.1101511879049676+ },+ {+ "qid": "PLAIN-1919",+ "ndcg10": 0.7967610662472993,+ "mrr10": 1,+ "recall10": 0.25925925925925924,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-1929",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-1940",+ "ndcg10": 0.3589542101716347,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.5833333333333334+ },+ {+ "qid": "PLAIN-1950",+ "ndcg10": 0.2489083270225946,+ "mrr10": 0.5,+ "recall10": 0.15384615384615385,+ "recall100": 0.23076923076923078+ },+ {+ "qid": "PLAIN-1962",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-1972",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.36363636363636365+ },+ {+ "qid": "PLAIN-1983",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.3125,+ "recall100": 0.6875+ },+ {+ "qid": "PLAIN-1995",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.18181818181818182+ },+ {+ "qid": "PLAIN-2009",+ "ndcg10": 0.3700049343491939,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 0.625+ },+ {+ "qid": "PLAIN-2019",+ "ndcg10": 0.13565197343244778,+ "mrr10": 0.1,+ "recall10": 0.3333333333333333,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2030",+ "ndcg10": 0.19092086617893467,+ "mrr10": 0.5,+ "recall10": 0.16666666666666666,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-2040",+ "ndcg10": 0.4950940324881554,+ "mrr10": 0.5,+ "recall10": 0.03787878787878788,+ "recall100": 0.13636363636363635+ },+ {+ "qid": "PLAIN-2051",+ "ndcg10": 0.5423640154200349,+ "mrr10": 1,+ "recall10": 0.011267605633802818,+ "recall100": 0.09859154929577464+ },+ {+ "qid": "PLAIN-2061",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 0.024390243902439025,+ "recall100": 0.13902439024390245+ },+ {+ "qid": "PLAIN-2071",+ "ndcg10": 0.5455017237786198,+ "mrr10": 0.5,+ "recall10": 0.13333333333333333,+ "recall100": 0.37777777777777777+ },+ {+ "qid": "PLAIN-2081",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2092",+ "ndcg10": 0.6489315753318465,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2102",+ "ndcg10": 0.7600791051976192,+ "mrr10": 1,+ "recall10": 0.0166270783847981,+ "recall100": 0.1163895486935867+ },+ {+ "qid": "PLAIN-2113",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2124",+ "ndcg10": 0.5455206287241073,+ "mrr10": 1,+ "recall10": 0.29411764705882354,+ "recall100": 0.5294117647058824+ },+ {+ "qid": "PLAIN-2134",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.08333333333333333+ },+ {+ "qid": "PLAIN-2145",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-2156",+ "ndcg10": 0.7751482523375255,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.8333333333333334+ },+ {+ "qid": "PLAIN-2167",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2177",+ "ndcg10": 0.5072380780124418,+ "mrr10": 0.5,+ "recall10": 0.20833333333333334,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2187",+ "ndcg10": 0.23504554941448536,+ "mrr10": 1,+ "recall10": 0.1111111111111111,+ "recall100": 0.2222222222222222+ },+ {+ "qid": "PLAIN-2197",+ "ndcg10": 0.6496872770120411,+ "mrr10": 1,+ "recall10": 0.10344827586206896,+ "recall100": 0.3103448275862069+ },+ {+ "qid": "PLAIN-2209",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2220",+ "ndcg10": 0.38334277998463445,+ "mrr10": 1,+ "recall10": 0.2222222222222222,+ "recall100": 0.2222222222222222+ },+ {+ "qid": "PLAIN-2230",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2240",+ "ndcg10": 0.2082936658103318,+ "mrr10": 0.5,+ "recall10": 0.18181818181818182,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-2250",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2261",+ "ndcg10": 0.5637884576902256,+ "mrr10": 1,+ "recall10": 0.06666666666666667,+ "recall100": 0.15+ },+ {+ "qid": "PLAIN-2271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2281",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2291",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-2301",+ "ndcg10": 0.13961814555374907,+ "mrr10": 0.14285714285714285,+ "recall10": 0.15384615384615385,+ "recall100": 0.3076923076923077+ },+ {+ "qid": "PLAIN-2311",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2321",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2332",+ "ndcg10": 0.7663491917568946,+ "mrr10": 1,+ "recall10": 0.08247422680412371,+ "recall100": 0.4020618556701031+ },+ {+ "qid": "PLAIN-2343",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2354",+ "ndcg10": 0.5087403079104241,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2364",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2375",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2386",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.3684210526315789+ },+ {+ "qid": "PLAIN-2396",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1111111111111111+ },+ {+ "qid": "PLAIN-2408",+ "ndcg10": 0.5531464700081437,+ "mrr10": 1,+ "recall10": 0.4,+ "recall100": 0.6+ },+ {+ "qid": "PLAIN-2430",+ "ndcg10": 0.5740106608557803,+ "mrr10": 1,+ "recall10": 0.15217391304347827,+ "recall100": 0.41304347826086957+ },+ {+ "qid": "PLAIN-2440",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.047619047619047616+ },+ {+ "qid": "PLAIN-2450",+ "ndcg10": 0.2336508082431053,+ "mrr10": 0.5,+ "recall10": 0.05263157894736842,+ "recall100": 0.13157894736842105+ },+ {+ "qid": "PLAIN-2460",+ "ndcg10": 0.43174458805633853,+ "mrr10": 1,+ "recall10": 0.12903225806451613,+ "recall100": 0.4838709677419355+ },+ {+ "qid": "PLAIN-2470",+ "ndcg10": 0.7024216173807135,+ "mrr10": 1,+ "recall10": 0.0821917808219178,+ "recall100": 0.3287671232876712+ },+ {+ "qid": "PLAIN-2480",+ "ndcg10": 0.4407905902246705,+ "mrr10": 1,+ "recall10": 0.2777777777777778,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2490",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.34782608695652173+ },+ {+ "qid": "PLAIN-2500",+ "ndcg10": 0.15487585161383474,+ "mrr10": 0.16666666666666666,+ "recall10": 0.14285714285714285,+ "recall100": 0.7142857142857143+ },+ {+ "qid": "PLAIN-2510",+ "ndcg10": 0.589664883987166,+ "mrr10": 0.5,+ "recall10": 0.15217391304347827,+ "recall100": 0.5869565217391305+ },+ {+ "qid": "PLAIN-2520",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.1095890410958904+ },+ {+ "qid": "PLAIN-2530",+ "ndcg10": 0.955830517697074,+ "mrr10": 1,+ "recall10": 0.1388888888888889,+ "recall100": 0.3888888888888889+ },+ {+ "qid": "PLAIN-2540",+ "ndcg10": 0.4941492558420748,+ "mrr10": 0.5,+ "recall10": 0.2857142857142857,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2550",+ "ndcg10": 0.24746798875357182,+ "mrr10": 0.5,+ "recall10": 0.08,+ "recall100": 0.4+ },+ {+ "qid": "PLAIN-2560",+ "ndcg10": 0.9363792118010483,+ "mrr10": 1,+ "recall10": 0.42857142857142855,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2570",+ "ndcg10": 0.25523504782501816,+ "mrr10": 0.3333333333333333,+ "recall10": 0.075,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-2580",+ "ndcg10": 0.1135664319411187,+ "mrr10": 1,+ "recall10": 0.043478260869565216,+ "recall100": 0.13043478260869565+ },+ {+ "qid": "PLAIN-2590",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.07936507936507936+ },+ {+ "qid": "PLAIN-2600",+ "ndcg10": 0.6613158235505792,+ "mrr10": 1,+ "recall10": 0.3,+ "recall100": 0.3+ },+ {+ "qid": "PLAIN-2610",+ "ndcg10": 0.4973811325345484,+ "mrr10": 1,+ "recall10": 0.06557377049180328,+ "recall100": 0.2459016393442623+ },+ {+ "qid": "PLAIN-2620",+ "ndcg10": 0.6770586454665828,+ "mrr10": 1,+ "recall10": 0.16363636363636364,+ "recall100": 0.36363636363636365+ },+ {+ "qid": "PLAIN-2630",+ "ndcg10": 0.8188458073987622,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.3958333333333333+ },+ {+ "qid": "PLAIN-2640",+ "ndcg10": 0.39923129859010653,+ "mrr10": 1,+ "recall10": 0.16666666666666666,+ "recall100": 0.5416666666666666+ },+ {+ "qid": "PLAIN-2650",+ "ndcg10": 0.6489315753318465,+ "mrr10": 1,+ "recall10": 0.1724137931034483,+ "recall100": 0.3448275862068966+ },+ {+ "qid": "PLAIN-2660",+ "ndcg10": 0.40030566811860463,+ "mrr10": 0.5,+ "recall10": 0.14814814814814814,+ "recall100": 0.48148148148148145+ },+ {+ "qid": "PLAIN-2670",+ "ndcg10": 0.3406992958233561,+ "mrr10": 1,+ "recall10": 0.02857142857142857,+ "recall100": 0.05714285714285714+ },+ {+ "qid": "PLAIN-2680",+ "ndcg10": 0.454740507204534,+ "mrr10": 0.5,+ "recall10": 0.2857142857142857,+ "recall100": 0.42857142857142855+ },+ {+ "qid": "PLAIN-2690",+ "ndcg10": 0.6852219536434856,+ "mrr10": 1,+ "recall10": 0.20930232558139536,+ "recall100": 0.4418604651162791+ },+ {+ "qid": "PLAIN-2700",+ "ndcg10": 0.5438770046596358,+ "mrr10": 1,+ "recall10": 0.3,+ "recall100": 0.7+ },+ {+ "qid": "PLAIN-2710",+ "ndcg10": 0.6906119546391791,+ "mrr10": 1,+ "recall10": 0.23809523809523808,+ "recall100": 0.6190476190476191+ },+ {+ "qid": "PLAIN-2720",+ "ndcg10": 0.24639745374629807,+ "mrr10": 0.2,+ "recall10": 0.1111111111111111,+ "recall100": 0.2777777777777778+ },+ {+ "qid": "PLAIN-2730",+ "ndcg10": 0.5235258308273208,+ "mrr10": 1,+ "recall10": 0.13793103448275862,+ "recall100": 0.41379310344827586+ },+ {+ "qid": "PLAIN-2740",+ "ndcg10": 0.6782347310356166,+ "mrr10": 1,+ "recall10": 0.1,+ "recall100": 0.16+ },+ {+ "qid": "PLAIN-2750",+ "ndcg10": 0.532003134830924,+ "mrr10": 1,+ "recall10": 0.1016949152542373,+ "recall100": 0.23728813559322035+ },+ {+ "qid": "PLAIN-2760",+ "ndcg10": 0.4702199443548266,+ "mrr10": 0.5,+ "recall10": 0.25,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2770",+ "ndcg10": 0.325096961030005,+ "mrr10": 0.5,+ "recall10": 0.12195121951219512,+ "recall100": 0.4878048780487805+ },+ {+ "qid": "PLAIN-2780",+ "ndcg10": 0.4033852369868885,+ "mrr10": 1,+ "recall10": 0.2727272727272727,+ "recall100": 0.4090909090909091+ },+ {+ "qid": "PLAIN-2790",+ "ndcg10": 0.652358405855227,+ "mrr10": 1,+ "recall10": 0.1016949152542373,+ "recall100": 0.2033898305084746+ },+ {+ "qid": "PLAIN-2800",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.041666666666666664+ },+ {+ "qid": "PLAIN-2810",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-2820",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2830",+ "ndcg10": 0.7262287617954056,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-2840",+ "ndcg10": 0.09478836436955078,+ "mrr10": 0.25,+ "recall10": 0.058823529411764705,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-2850",+ "ndcg10": 0.3026286548261086,+ "mrr10": 0.5,+ "recall10": 0.1111111111111111,+ "recall100": 0.4444444444444444+ },+ {+ "qid": "PLAIN-2860",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-2870",+ "ndcg10": 0.0781764177375172,+ "mrr10": 0.2,+ "recall10": 0.2,+ "recall100": 0.2+ },+ {+ "qid": "PLAIN-2880",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.17647058823529413+ },+ {+ "qid": "PLAIN-2890",+ "ndcg10": 0.1974505933948842,+ "mrr10": 0.25,+ "recall10": 0.034482758620689655,+ "recall100": 0.034482758620689655+ },+ {+ "qid": "PLAIN-2900",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-2910",+ "ndcg10": 0.5330346140547638,+ "mrr10": 1,+ "recall10": 0.13333333333333333,+ "recall100": 0.26666666666666666+ },+ {+ "qid": "PLAIN-2920",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-2930",+ "ndcg10": 0.5175857101991203,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-2940",+ "ndcg10": 0.12811612289210494,+ "mrr10": 0.14285714285714285,+ "recall10": 0.045454545454545456,+ "recall100": 0.18181818181818182+ },+ {+ "qid": "PLAIN-2950",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-2960",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-2970",+ "ndcg10": 0.6338262955454459,+ "mrr10": 1,+ "recall10": 0.3333333333333333,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-2981",+ "ndcg10": 0.5321036168710066,+ "mrr10": 1,+ "recall10": 0.14285714285714285,+ "recall100": 0.2857142857142857+ },+ {+ "qid": "PLAIN-2991",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3001",+ "ndcg10": 0.3065735963827292,+ "mrr10": 0.3333333333333333,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3014",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3026",+ "ndcg10": 0.6309297535714574,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3037",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3053",+ "ndcg10": 0.7653606369886218,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 0.6666666666666666+ },+ {+ "qid": "PLAIN-3063",+ "ndcg10": 0.19519002499605084,+ "mrr10": 0.3333333333333333,+ "recall10": 0.25,+ "recall100": 0.25+ },+ {+ "qid": "PLAIN-3074",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3085",+ "ndcg10": 0.38162168437253385,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "PLAIN-3097",+ "ndcg10": 0.515157140670655,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-3116",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "PLAIN-3131",+ "ndcg10": 0.6516433935264055,+ "mrr10": 1,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3141",+ "ndcg10": 0.3170808479749099,+ "mrr10": 0.3333333333333333,+ "recall10": 0.058823529411764705,+ "recall100": 0.20588235294117646+ },+ {+ "qid": "PLAIN-3151",+ "ndcg10": 0.7456217791401517,+ "mrr10": 1,+ "recall10": 0.3888888888888889,+ "recall100": 0.5555555555555556+ },+ {+ "qid": "PLAIN-3161",+ "ndcg10": 0.524282944620724,+ "mrr10": 1,+ "recall10": 0.046511627906976744,+ "recall100": 0.27906976744186046+ },+ {+ "qid": "PLAIN-3171",+ "ndcg10": 0.044175472610956126,+ "mrr10": 0.1,+ "recall10": 0.029411764705882353,+ "recall100": 0.058823529411764705+ },+ {+ "qid": "PLAIN-3181",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.037037037037037035+ },+ {+ "qid": "PLAIN-3191",+ "ndcg10": 0.4300543345163763,+ "mrr10": 1,+ "recall10": 0.25,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3201",+ "ndcg10": 0.2037495755301314,+ "mrr10": 0.2,+ "recall10": 0.0625,+ "recall100": 0.09375+ },+ {+ "qid": "PLAIN-3211",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.05555555555555555+ },+ {+ "qid": "PLAIN-3221",+ "ndcg10": 0.06581686446496139,+ "mrr10": 0.25,+ "recall10": 0.041666666666666664,+ "recall100": 0.125+ },+ {+ "qid": "PLAIN-3231",+ "ndcg10": 0.2501157675209561,+ "mrr10": 0.5,+ "recall10": 0.5,+ "recall100": 0.75+ },+ {+ "qid": "PLAIN-3241",+ "ndcg10": 0.45846608015576257,+ "mrr10": 1,+ "recall10": 0.034482758620689655,+ "recall100": 0.06896551724137931+ },+ {+ "qid": "PLAIN-3251",+ "ndcg10": 0.16330891279672524,+ "mrr10": 0.16666666666666666,+ "recall10": 0.047619047619047616,+ "recall100": 0.09523809523809523+ },+ {+ "qid": "PLAIN-3261",+ "ndcg10": 0.17034964791167806,+ "mrr10": 0.3333333333333333,+ "recall10": 0.07692307692307693,+ "recall100": 0.07692307692307693+ },+ {+ "qid": "PLAIN-3271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.10204081632653061+ },+ {+ "qid": "PLAIN-3281",+ "ndcg10": 0.24249682153453903,+ "mrr10": 0.5,+ "recall10": 0.038461538461538464,+ "recall100": 0.07692307692307693+ },+ {+ "qid": "PLAIN-3292",+ "ndcg10": 0.34839716434274914,+ "mrr10": 1,+ "recall10": 0.42857142857142855,+ "recall100": 0.5714285714285714+ },+ {+ "qid": "PLAIN-3302",+ "ndcg10": 0.13084731762486884,+ "mrr10": 0.3333333333333333,+ "recall10": 0.05405405405405406,+ "recall100": 0.3783783783783784+ },+ {+ "qid": "PLAIN-3312",+ "ndcg10": 0.30114238162847884,+ "mrr10": 0.5,+ "recall10": 0.3076923076923077,+ "recall100": 0.6153846153846154+ },+ {+ "qid": "PLAIN-3322",+ "ndcg10": 0.28165758790969186,+ "mrr10": 0.5,+ "recall10": 0.0379746835443038,+ "recall100": 0.13924050632911392+ },+ {+ "qid": "PLAIN-3332",+ "ndcg10": 0.7386266690451878,+ "mrr10": 1,+ "recall10": 0.35714285714285715,+ "recall100": 0.8571428571428571+ },+ {+ "qid": "PLAIN-3342",+ "ndcg10": 0.1671604549662023,+ "mrr10": 0.16666666666666666,+ "recall10": 0.3333333333333333,+ "recall100": 0.3333333333333333+ },+ {+ "qid": "PLAIN-3352",+ "ndcg10": 0.3843483686763149,+ "mrr10": 1,+ "recall10": 0.02564102564102564,+ "recall100": 0.1282051282051282+ },+ {+ "qid": "PLAIN-3362",+ "ndcg10": 0.6268451902108538,+ "mrr10": 1,+ "recall10": 0.18181818181818182,+ "recall100": 0.2727272727272727+ },+ {+ "qid": "PLAIN-3372",+ "ndcg10": 0.27744302190680586,+ "mrr10": 0.3333333333333333,+ "recall10": 0.14285714285714285,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-3382",+ "ndcg10": 0.4594761912515673,+ "mrr10": 1,+ "recall10": 0.35714285714285715,+ "recall100": 0.7142857142857143+ },+ {+ "qid": "PLAIN-3392",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.13043478260869565+ },+ {+ "qid": "PLAIN-3402",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.21428571428571427+ },+ {+ "qid": "PLAIN-3412",+ "ndcg10": 0.18329273373298458,+ "mrr10": 1,+ "recall10": 0.04081632653061224,+ "recall100": 0.24489795918367346+ },+ {+ "qid": "PLAIN-3422",+ "ndcg10": 0.02321714761064369,+ "mrr10": 0.1,+ "recall10": 0.03225806451612903,+ "recall100": 0.45161290322580644+ },+ {+ "qid": "PLAIN-3432",+ "ndcg10": 0.06362078819895171,+ "mrr10": 0.1,+ "recall10": 0.023809523809523808,+ "recall100": 0.14285714285714285+ },+ {+ "qid": "PLAIN-3442",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0.034482758620689655+ },+ {+ "qid": "PLAIN-3452",+ "ndcg10": 0.4247645055987381,+ "mrr10": 1,+ "recall10": 0.11764705882352941,+ "recall100": 0.5294117647058824+ },+ {+ "qid": "PLAIN-3462",+ "ndcg10": 0.6378891765242927,+ "mrr10": 0.5,+ "recall10": 0.109375,+ "recall100": 0.5+ },+ {+ "qid": "PLAIN-3472",+ "ndcg10": 0.2873431773791806,+ "mrr10": 1,+ "recall10": 0.058823529411764705,+ "recall100": 0.4117647058823529+ }+ ],"baselines": {"BM25 (Lucene)": 0.325,"DocT5query": 0.328,diff --git a/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.json b/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.jsonnew file mode 100644index 0000000000..e516f870e8--- /dev/null+++ b/docs/benchmarks/runs/beir-scifact-bge-Xenova_bge-base-en-v1.5-2026-05-30T20-46-06-540Z.json@@ -0,0 +1,2131 @@+{+ "runAt": "2026-05-30T20:46:06.540Z",+ "benchmark": "beir-scifact-bge",+ "dataset": "scifact",+ "model": "Xenova/bge-base-en-v1.5",+ "queries": 300,+ "corpusSize": 5183,+ "metrics": {+ "ndcg10": 0.6255647791860727,+ "mrr10": 0.5931878306878305,+ "recall10": 0.7412222222222221,+ "recall100": 0.8283333333333334,+ "avgQueryLatencyMs": 355.40267458333335+ },+ "perQuery": [+ {+ "qid": "1",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "3",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "5",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "13",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "36",+ "ndcg10": 0.23719771276929622,+ "mrr10": 0.2,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "42",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "48",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "49",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "50",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "51",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "53",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "54",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "56",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "57",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "70",+ "ndcg10": 0.9197207891481876,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "72",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "75",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "94",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "99",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "100",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "113",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "115",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "118",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "124",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "127",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "128",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "129",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "130",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "132",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "133",+ "ndcg10": 0.6164336326286644,+ "mrr10": 1,+ "recall10": 0.6,+ "recall100": 1+ },+ {+ "qid": "137",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "141",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "142",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "143",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "146",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "148",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "163",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "171",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "179",+ "ndcg10": 0.8048099750039491,+ "mrr10": 1,+ "recall10": 0.75,+ "recall100": 0.75+ },+ {+ "qid": "180",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "183",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "185",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "198",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "208",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "212",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "213",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "216",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "217",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "218",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "219",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "230",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "232",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "233",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "236",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "237",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "238",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "239",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "248",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "249",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "261",+ "ndcg10": 0.2640681225725909,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "268",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "269",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "274",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "275",+ "ndcg10": 0.7039180890341347,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "279",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "294",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "295",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "298",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "300",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "303",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "312",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "314",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "324",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "327",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "338",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "343",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "350",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "354",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "362",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "380",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "384",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "385",+ "ndcg10": 0.6934264036172708,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "386",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "388",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "399",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "410",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "411",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "415",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "421",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "431",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "436",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "437",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "439",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "440",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "443",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "452",+ "ndcg10": 0.2640681225725909,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "475",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "478",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "491",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "501",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "502",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "507",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "508",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "513",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "514",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "516",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "517",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "521",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "525",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "527",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "528",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "532",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "533",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "535",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "536",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "539",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "540",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.5,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "544",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "549",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "551",+ "ndcg10": 0.3333333333333333,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "552",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "554",+ "ndcg10": 0.3010299956639812,+ "mrr10": 0.1111111111111111,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "560",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "569",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "575",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "577",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "578",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "587",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "589",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "593",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "597",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "598",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "613",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "619",+ "ndcg10": 0.17723928678404774,+ "mrr10": 0.1,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "623",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "628",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "636",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "637",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "641",+ "ndcg10": 0.23719771276929622,+ "mrr10": 0.2,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "644",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "649",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "659",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "660",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "674",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "684",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "690",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "691",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "692",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "693",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "700",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "702",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "715",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "716",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "718",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "721",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "723",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "727",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "728",+ "ndcg10": 0.8772153153380493,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "729",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "742",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "743",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "744",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "756",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "759",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "768",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "770",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "775",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "781",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "783",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "784",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "785",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "793",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "800",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "805",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "808",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "811",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "814",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "820",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "821",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "823",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "830",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "831",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "832",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "834",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "837",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "839",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "845",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "847",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "852",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "859",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "870",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "873",+ "ndcg10": 0.5043776700708298,+ "mrr10": 0.5,+ "recall10": 0.6,+ "recall100": 1+ },+ {+ "qid": "879",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "880",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "882",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "887",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "903",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "904",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "907",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "911",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "913",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "914",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "921",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "922",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "936",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "956",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "957",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "960",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "967",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "971",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "975",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "982",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "985",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "993",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1012",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1014",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1019",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1020",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1021",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1024",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1029",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1041",+ "ndcg10": 0.39564672360221187,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1049",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1062",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1086",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1088",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1089",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1099",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1100",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1104",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1107",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1110",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1121",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1130",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1132",+ "ndcg10": 0.39780880120575696,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1137",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1140",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1144",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1146",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1150",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1163",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1175",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1179",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1180",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1185",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1187",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1191",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1194",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1196",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1197",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1199",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1200",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1202",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1204",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1207",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1213",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1216",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1221",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1225",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1226",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1232",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1241",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1245",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1259",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1262",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1266",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1270",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1272",+ "ndcg10": 0.3333333333333333,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1273",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1274",+ "ndcg10": 0.9674679834891693,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1278",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1279",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1280",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1281",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1282",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1290",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1292",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1298",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1303",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1316",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1319",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1320",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1332",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1335",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1336",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1337",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1339",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1344",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1352",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1359",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1362",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1363",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1368",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1370",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1379",+ "ndcg10": 0.8318724637288826,+ "mrr10": 1,+ "recall10": 0.75,+ "recall100": 0.75+ },+ {+ "qid": "1382",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1385",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1389",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1395",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ }+ ],+ "baselines": {+ "BM25 (Lucene)": 0.679,+ "DocT5query": 0.675,+ "TAS-B": 0.643,+ "GenQ": 0.644,+ "ColBERT": 0.671,+ "Contriever": 0.677,+ "GTR-XL": 0.662,+ "SPLADE++": 0.704,+ "BGE-large-v1.5 (pub)": 0.722,+ "SBERT msmarco": 0.555+ },+ "ourRank": 10,+ "leaderboardLength": 11+}\ No newline at end of filediff --git a/docs/benchmarks/runs/beir-scifact-bge-latest.json b/docs/benchmarks/runs/beir-scifact-bge-latest.jsonnew file mode 100644index 0000000000..e516f870e8--- /dev/null+++ b/docs/benchmarks/runs/beir-scifact-bge-latest.json@@ -0,0 +1,2131 @@+{+ "runAt": "2026-05-30T20:46:06.540Z",+ "benchmark": "beir-scifact-bge",+ "dataset": "scifact",+ "model": "Xenova/bge-base-en-v1.5",+ "queries": 300,+ "corpusSize": 5183,+ "metrics": {+ "ndcg10": 0.6255647791860727,+ "mrr10": 0.5931878306878305,+ "recall10": 0.7412222222222221,+ "recall100": 0.8283333333333334,+ "avgQueryLatencyMs": 355.40267458333335+ },+ "perQuery": [+ {+ "qid": "1",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "3",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "5",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "13",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "36",+ "ndcg10": 0.23719771276929622,+ "mrr10": 0.2,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "42",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "48",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "49",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "50",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "51",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "53",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "54",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "56",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "57",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "70",+ "ndcg10": 0.9197207891481876,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "72",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "75",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "94",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "99",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "100",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "113",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "115",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "118",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "124",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "127",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "128",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "129",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "130",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "132",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "133",+ "ndcg10": 0.6164336326286644,+ "mrr10": 1,+ "recall10": 0.6,+ "recall100": 1+ },+ {+ "qid": "137",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "141",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "142",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "143",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "146",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "148",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "163",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "171",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "179",+ "ndcg10": 0.8048099750039491,+ "mrr10": 1,+ "recall10": 0.75,+ "recall100": 0.75+ },+ {+ "qid": "180",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "183",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "185",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "198",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "208",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "212",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "213",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "216",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "217",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "218",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "219",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "230",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "232",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "233",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "236",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "237",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "238",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "239",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "248",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "249",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "261",+ "ndcg10": 0.2640681225725909,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "268",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "269",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "274",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "275",+ "ndcg10": 0.7039180890341347,+ "mrr10": 1,+ "recall10": 0.6666666666666666,+ "recall100": 1+ },+ {+ "qid": "279",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "294",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "295",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "298",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "300",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "303",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "312",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "314",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "324",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "327",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "338",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "343",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "350",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "354",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "362",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "380",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "384",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "385",+ "ndcg10": 0.6934264036172708,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "386",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "388",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "399",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "410",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "411",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "415",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "421",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "431",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "436",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "437",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "439",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "440",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "443",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "452",+ "ndcg10": 0.2640681225725909,+ "mrr10": 0.25,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "475",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "478",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "491",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "501",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "502",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "507",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "508",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "513",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "514",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "516",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "517",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "521",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "525",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "527",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "528",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "532",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "533",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "535",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "536",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "539",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "540",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.5,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "544",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "549",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "551",+ "ndcg10": 0.3333333333333333,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "552",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "554",+ "ndcg10": 0.3010299956639812,+ "mrr10": 0.1111111111111111,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "560",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "569",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "575",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "577",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "578",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "587",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "589",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "593",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "597",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "598",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "613",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "619",+ "ndcg10": 0.17723928678404774,+ "mrr10": 0.1,+ "recall10": 0.5,+ "recall100": 1+ },+ {+ "qid": "623",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "628",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "636",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "637",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "641",+ "ndcg10": 0.23719771276929622,+ "mrr10": 0.2,+ "recall10": 0.5,+ "recall100": 0.5+ },+ {+ "qid": "644",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "649",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "659",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "660",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "674",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "684",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "690",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "691",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "692",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "693",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "700",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "702",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "715",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "716",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "718",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "721",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "723",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "727",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "728",+ "ndcg10": 0.8772153153380493,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "729",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "742",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "743",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "744",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "756",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "759",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "768",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "770",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "775",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "781",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "783",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "784",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "785",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "793",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "800",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "805",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "808",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "811",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "814",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "820",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "821",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "823",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "830",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "831",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "832",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "834",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "837",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "839",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "845",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "847",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "852",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "859",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "870",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "873",+ "ndcg10": 0.5043776700708298,+ "mrr10": 0.5,+ "recall10": 0.6,+ "recall100": 1+ },+ {+ "qid": "879",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "880",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "882",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "887",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "903",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "904",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "907",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "911",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "913",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "914",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "921",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "922",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "936",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "956",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "957",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "960",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "967",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "971",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "975",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "982",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "985",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "993",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1012",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1014",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1019",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1020",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1021",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1024",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1029",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1041",+ "ndcg10": 0.39564672360221187,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1049",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1062",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1086",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1088",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1089",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1099",+ "ndcg10": 0.3562071871080222,+ "mrr10": 0.16666666666666666,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1100",+ "ndcg10": 0.2890648263178879,+ "mrr10": 0.1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1104",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1107",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1110",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1121",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1130",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1132",+ "ndcg10": 0.39780880120575696,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1137",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1140",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1144",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1146",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1150",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1163",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1175",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1179",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1180",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1185",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1187",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1191",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1194",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1196",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1197",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1199",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1200",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1202",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1204",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1207",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1213",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1216",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1221",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1225",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1226",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1232",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1241",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1245",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1259",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1262",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1266",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1270",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1271",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1272",+ "ndcg10": 0.3333333333333333,+ "mrr10": 0.14285714285714285,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1273",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1274",+ "ndcg10": 0.9674679834891693,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1278",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1279",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1280",+ "ndcg10": 0.43067655807339306,+ "mrr10": 0.25,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1281",+ "ndcg10": 0.38685280723454163,+ "mrr10": 0.2,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1282",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1290",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1292",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1298",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1303",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1316",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1319",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1320",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1332",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 1+ },+ {+ "qid": "1335",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1336",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1337",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1339",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1344",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1352",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1359",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1362",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1363",+ "ndcg10": 0,+ "mrr10": 0,+ "recall10": 0,+ "recall100": 0+ },+ {+ "qid": "1368",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1370",+ "ndcg10": 0.5,+ "mrr10": 0.3333333333333333,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1379",+ "ndcg10": 0.8318724637288826,+ "mrr10": 1,+ "recall10": 0.75,+ "recall100": 0.75+ },+ {+ "qid": "1382",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1385",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1389",+ "ndcg10": 1,+ "mrr10": 1,+ "recall10": 1,+ "recall100": 1+ },+ {+ "qid": "1395",+ "ndcg10": 0.6309297535714575,+ "mrr10": 0.5,+ "recall10": 1,+ "recall100": 1+ }+ ],+ "baselines": {+ "BM25 (Lucene)": 0.679,+ "DocT5query": 0.675,+ "TAS-B": 0.643,+ "GenQ": 0.644,+ "ColBERT": 0.671,+ "Contriever": 0.677,+ "GTR-XL": 0.662,+ "SPLADE++": 0.704,+ "BGE-large-v1.5 (pub)": 0.722,+ "SBERT msmarco": 0.555+ },+ "ourRank": 10,+ "leaderboardLength": 11+}\ No newline at end of filediff --git a/package-lock.json b/package-lock.jsonindex 4142f377f8..7907a48308 100644--- a/package-lock.json+++ b/package-lock.json@@ -1,12 +1,12 @@{"name": "claude-flow",- "version": "3.10.25",+ "version": "3.10.26","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "claude-flow",- "version": "3.10.25",+ "version": "3.10.26","license": "MIT","dependencies": {"@claude-flow/cli-core": "^3.7.0-alpha.5",diff --git a/package.json b/package.jsonindex e02c3f40f1..35f9081356 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "claude-flow",- "version": "3.10.25",+ "version": "3.10.26","description": "Ruflo - Enterprise AI agent orchestration for Claude Code. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "dist/index.js","type": "module",diff --git a/ruflo/package.json b/ruflo/package.jsonindex 7fa1d22397..1ecef8447f 100644--- a/ruflo/package.json+++ b/ruflo/package.json@@ -1,6 +1,6 @@{"name": "ruflo",- "version": "3.10.25",+ "version": "3.10.26","description": "Ruflo - Enterprise AI agent orchestration platform. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "bin/ruflo.js","type": "module",diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.jsonindex 9a94a55e28..4ff66a857d 100644--- a/v3/@claude-flow/cli/package.json+++ b/v3/@claude-flow/cli/package.json@@ -1,6 +1,6 @@{"name": "@claude-flow/cli",- "version": "3.10.25",+ "version": "3.10.26","type": "module","description": "Ruflo CLI - Enterprise AI agent orchestration with 60+ specialized agents, swarm coordination, MCP server, self-learning hooks, and vector memory for Claude Code","main": "dist/src/index.js",diff --git a/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs b/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjsnew file mode 100644index 0000000000..495cb9563f--- /dev/null+++ b/v3/@claude-flow/cli/scripts/beir-bootstrap-significance.mjs@@ -0,0 +1,162 @@+#!/usr/bin/env node+// beir-bootstrap-significance.mjs β paired bootstrap significance test for+// BEIR retrieval results (ADR-086).+//+// Given a run JSON with per-query nDCG@10 and a baseline mean nDCG@10,+// estimates whether our result is significantly above the baseline.+//+// Method: paired bootstrap β resample N=10000 with replacement from our+// per-query nDCG scores, compute mean each time, take 2.5%-97.5% percentile+// for 95% CI. Compare baseline to our CI: if baseline < CI lower bound,+// the difference is significant at p < 0.05.+//+// For a paired test against ANOTHER per-query run (e.g. our pure-BM25 vs+// our BGE-base), resample paired DIFFERENCES instead. Pass --paired+// path/to/baseline-run.json.+//+// Usage:+// node scripts/beir-bootstrap-significance.mjs <run-json-path>+// node scripts/beir-bootstrap-significance.mjs <ours> --paired <baseline>+//+// Example:+// node scripts/beir-bootstrap-significance.mjs \+// docs/benchmarks/runs/beir-nfcorpus-bge-latest.json++import { readFileSync } from 'node:fs';+import { performance } from 'node:perf_hooks';++const args = process.argv.slice(2);+if (args.length === 0) {+ console.error('Usage: node beir-bootstrap-significance.mjs <run-json> [--paired <baseline-run-json>]');+ process.exit(1);+}+const RUN_PATH = args[0];+const PAIRED_IDX = args.indexOf('--paired');+const PAIRED_PATH = PAIRED_IDX > 0 ? args[PAIRED_IDX + 1] : null;+const ITERATIONS = Number(process.env.BOOTSTRAP_ITERATIONS) || 10000;++// Deterministic PRNG (Mulberry32) β so re-runs are reproducible.+const SEED = Number(process.env.BOOTSTRAP_SEED) || 42;+function mulberry32(a) {+ return function () {+ a |= 0; a = (a + 0x6D2B79F5) | 0;+ let t = a;+ t = Math.imul(t ^ (t >>> 15), t | 1);+ t ^= t + Math.imul(t ^ (t >>> 7), t | 61);+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;+ };+}++function mean(arr) { let s = 0; for (const v of arr) s += v; return s / arr.length; }++function percentile(sortedArr, p) {+ const idx = Math.max(0, Math.min(sortedArr.length - 1, Math.floor(p * sortedArr.length)));+ return sortedArr[idx];+}++function pointEstimate(scores) {+ return {+ mean: mean(scores),+ n: scores.length,+ };+}++function bootstrapCI(scores, iterations, rng, percentiles = [0.025, 0.975]) {+ const means = new Float64Array(iterations);+ const n = scores.length;+ for (let i = 0; i < iterations; i++) {+ let s = 0;+ for (let j = 0; j < n; j++) s += scores[Math.floor(rng() * n)];+ means[i] = s / n;+ }+ const sorted = Array.from(means).sort((a, b) => a - b);+ return {+ point: mean(scores),+ lower: percentile(sorted, percentiles[0]),+ upper: percentile(sorted, percentiles[1]),+ p_value_vs_baseline_zero: null,+ };+}++// Paired bootstrap: compare our per-query scores to a baseline's per-query+// scores. Resample (idx) and average paired differences. The 95% CI on the+// mean difference tells us if it's significantly non-zero.+function pairedBootstrap(oursByQid, baselineByQid, iterations, rng) {+ const sharedIds = [...oursByQid.keys()].filter((id) => baselineByQid.has(id));+ const diffs = sharedIds.map((id) => oursByQid.get(id) - baselineByQid.get(id));+ const n = diffs.length;+ const means = new Float64Array(iterations);+ for (let i = 0; i < iterations; i++) {+ let s = 0;+ for (let j = 0; j < n; j++) s += diffs[Math.floor(rng() * n)];+ means[i] = s / n;+ }+ const sorted = Array.from(means).sort((a, b) => a - b);+ // One-sided p β fraction of resamples where mean diff β€ 0+ let negCount = 0;+ for (const m of means) if (m <= 0) negCount++;+ return {+ n_paired: n,+ point_diff: mean(diffs),+ lower_95: percentile(sorted, 0.025),+ upper_95: percentile(sorted, 0.975),+ p_one_sided: negCount / iterations,+ };+}++function loadRun(path) {+ const data = JSON.parse(readFileSync(path, 'utf-8'));+ if (!data.perQuery || !Array.isArray(data.perQuery)) {+ console.error(`Run JSON at ${path} has no perQuery array β re-run the bench with the updated harness.`);+ process.exit(2);+ }+ return data;+}++const ours = loadRun(RUN_PATH);+console.log(`# BEIR significance test (ADR-086)`);+console.log(`Run: ${RUN_PATH}`);+console.log(`Dataset: ${ours.dataset ?? '?'} Model: ${ours.model ?? '?'} Queries: ${ours.perQuery.length}`);+console.log(`Bootstrap: ${ITERATIONS} iterations, seed=${SEED}`);++const rng = mulberry32(SEED);+const ourScores = ours.perQuery.map((q) => q.ndcg10);++const t0 = performance.now();+const oursCI = bootstrapCI(ourScores, ITERATIONS, rng);+console.log(`\n=== Our nDCG@10 (1-sample bootstrap CI) ===`);+console.log(` point: ${oursCI.point.toFixed(4)}`);+console.log(` 95% CI: [${oursCI.lower.toFixed(4)}, ${oursCI.upper.toFixed(4)}]`);++if (ours.baselines) {+ console.log(`\n=== vs each published baseline (CI overlap) ===`);+ for (const [name, score] of Object.entries(ours.baselines)) {+ const ciDiff = oursCI.point - score;+ const lowerDiff = oursCI.lower - score;+ const upperDiff = oursCI.upper - score;+ const direction = ciDiff > 0 ? 'β above' : 'β below';+ const sig = (lowerDiff > 0 && upperDiff > 0) ? 'p<0.05 (95% CI excludes baseline)'+ : (lowerDiff < 0 && upperDiff < 0) ? 'p<0.05 (95% CI below baseline)'+ : 'n.s. (95% CI overlaps baseline)';+ console.log(` ${score.toFixed(3)} ${name.padEnd(28)} Ξ=${ciDiff >= 0 ? '+' : ''}${ciDiff.toFixed(4)} ${direction} [${sig}]`);+ }+}++if (PAIRED_PATH) {+ const baseline = loadRun(PAIRED_PATH);+ console.log(`\n=== Paired bootstrap vs ${PAIRED_PATH} ===`);+ const oursByQid = new Map(ours.perQuery.map((q) => [q.qid, q.ndcg10]));+ const baseByQid = new Map(baseline.perQuery.map((q) => [q.qid, q.ndcg10]));+ const rng2 = mulberry32(SEED + 1);+ const paired = pairedBootstrap(oursByQid, baseByQid, ITERATIONS, rng2);+ console.log(` paired queries: ${paired.n_paired}`);+ console.log(` Ξ (ours - baseline): ${paired.point_diff >= 0 ? '+' : ''}${paired.point_diff.toFixed(4)}`);+ console.log(` 95% CI: [${paired.lower_95.toFixed(4)}, ${paired.upper_95.toFixed(4)}]`);+ console.log(` one-sided p (Ξ β€ 0): ${paired.p_one_sided.toFixed(4)}`);+ const sig = paired.lower_95 > 0 ? 'SIGNIFICANT improvement (p<0.025 one-sided)' :+ paired.upper_95 < 0 ? 'SIGNIFICANT regression (p<0.025 one-sided)' :+ 'NOT SIGNIFICANT β CI overlaps zero';+ console.log(` verdict: ${sig}`);+}++console.log(`\nBootstrap took ${((performance.now() - t0) / 1000).toFixed(1)}s.`);diff --git a/v3/@claude-flow/cli/scripts/run-beir-bge.mjs b/v3/@claude-flow/cli/scripts/run-beir-bge.mjsindex cea54b3fe9..4865c12a0f 100644--- a/v3/@claude-flow/cli/scripts/run-beir-bge.mjs+++ b/v3/@claude-flow/cli/scripts/run-beir-bge.mjs@@ -218,25 +218,26 @@ async function main() {console.log(`\nRunning ${evalQueryIds.length} queries...`);let nSum = 0, mSum = 0, r10Sum = 0, r100Sum = 0, n = 0;+ // ADR-086 β save per-query metrics for paired bootstrap significance testing.+ const perQuery = [];const tQ = performance.now();for (const qid of evalQueryIds) {const qtext = queriesById.get(qid);if (!qtext) continue;- // BGE recommends prepending "Represent this sentence for searching relevant passages:" to queries- // but only for v1.5 *en-icl* variants. v1.5 (non-icl) works without it.+ // BGE v1.5 non-icl works without the query prefix.const qEmb = await emb.embed(qtext);- // Score every doc, sort.const scores = new Array(docEmbeds.length);for (let i = 0; i < docEmbeds.length; i++) scores[i] = { id: docIds[i], score: cosine(qEmb, docEmbeds[i]) };scores.sort((a, b) => b.score - a.score);const top100 = scores.slice(0, 100).map((s) => s.id);const qmap = qrels.get(qid);- nSum += ndcg(top100, qmap, 10);- mSum += mrr(top100, qmap, 10);- r10Sum += recall(top100, qmap, 10);- r100Sum += recall(top100, qmap, 100);- n++;+ const qNdcg = ndcg(top100, qmap, 10);+ const qMrr = mrr(top100, qmap, 10);+ const qR10 = recall(top100, qmap, 10);+ const qR100 = recall(top100, qmap, 100);+ nSum += qNdcg; mSum += qMrr; r10Sum += qR10; r100Sum += qR100; n++;+ perQuery.push({ qid, ndcg10: qNdcg, mrr10: qMrr, recall10: qR10, recall100: qR100 });if (n % 50 === 0) {const elapsed = (performance.now() - tQ) / 1000;console.log(` ${n}/${evalQueryIds.length} in ${elapsed.toFixed(0)}s running nDCG@10=${(nSum / n).toFixed(4)}`);@@ -276,6 +277,7 @@ async function main() {queries: n,corpusSize: corpus.length,metrics: { ndcg10, mrr10, recall10, recall100, avgQueryLatencyMs: queryMs / n },+ perQuery, // ADR-086: per-query metrics for paired bootstrap significance testingbaselines: BASELINES_NDCG10,ourRank,leaderboardLength: ranking.length,diff --git a/v3/docs/adr/ADR-085-beir-public-benchmark.md b/v3/docs/adr/ADR-085-beir-public-benchmark.mdindex 1d1d80e190..9026909af0 100644--- a/v3/docs/adr/ADR-085-beir-public-benchmark.md+++ b/v3/docs/adr/ADR-085-beir-public-benchmark.md@@ -48,7 +48,9 @@ Standalone runner that bypasses the neural-tools BM25/hybrid pipeline entirelyWhy we're slightly below: our multi-field BM25 (subjectWeight=2.0) was tuned for ruflo's commit-style corpus where the "subject" field carries different IDF distribution than NFCorpus's title field. With `subjectWeight=0` (pure body BM25) we get **the same 0.289** β meaning the doc `_id` we passed as the "subject" carries no signal. A tighter BEIR-specific tokenizer + stopword list (Lucene-style) would close the remaining 0.04 gap.-### BGE-base-en-v1.5 (dense retrieval) β **TOP-2 RESULT**+### BGE-base-en-v1.5 (dense retrieval) β **TOP-2 on BEIR NFCorpus** (1 dataset, not the BEIR average)++**Pipeline reported:** direct BGE dense path (no fine-tuning, no hybrid BM25+dense fusion, no cross-encoder reranker). Our internal hybrid pipeline (ADRs 078-083) is deliberately isolated from this BEIR comparison so the dense-vs-dense numbers stay honest.Measured N=323 test queries, full corpus 3,633 docs:@@ -76,7 +78,12 @@ Measured N=323 test queries, full corpus 3,633 docs:| 10 | ColBERT | 0.305 || 11 | SBERT msmarco | 0.272 |-We beat SPLADE++ (the published "best" before BGE-large landed), GTR-XL, and every other listed dense retriever using a 3Γ smaller base model (110M vs 335M). The only thing beating us is BGE-large at 0.380, and that's a model-size advantage we could close by passing `BGE_MODEL=Xenova/bge-large-en-v1.5` to the same harness.+We beat SPLADE++ (the published "best" before BGE-large landed), GTR-XL, and every other listed dense retriever using a 3Γ smaller base model (110M vs 335M).++**Caveats on the framing:**+- The 0.005 gap to SPLADE++ (0.352 vs 0.347) is small. A paired bootstrap significance test (ADR-086) is needed to claim a statistically significant win β pending re-run with per-query metrics.+- **The "swap to BGE-large to close the BGE-large gap" claim is directional, not guaranteed.** Larger model = different dim (1024 vs 768), ~3Γ larger cache, ~2Γ query latency. Real number requires the run; we have not yet measured BGE-large on this stack.+- **One BEIR dataset is not BEIR-SOTA.** BEIR is explicitly an 18-dataset heterogeneous benchmark; rank-2 on NFCorpus is a strong signal, not a generalisation claim. SciFact run (2nd dataset) is queued; broader coverage is tracked.## Cumulative SOTA contextdiff --git a/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.md b/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.mdnew file mode 100644index 0000000000..4d3a8a627e--- /dev/null+++ b/v3/docs/adr/ADR-086-silent-fallback-and-bootstrap.md@@ -0,0 +1,202 @@+# ADR-086 β Silent-Fallback Bug Story + Bootstrap Significance Testing++**Status**: Accepted β Implemented in ruflo 3.10.26+**Date**: 2026-05-30+**Tracking**: continuation of self-learning hardening cluster (ADR-077 β ... β 085)+**Related**: ADR-085 (BEIR harness)++## Context++Two threads from the BEIR work that deserve their own ADRs:++1. **The silent-fallback embedder bug.** While building ADR-085's BEIR harness on darwin-arm64, every retrieval result looked plausible but cosine similarity scores were essentially random. The neural store reported `_realEmbedding: true`. The hash-fallback was carrying the entire dense signal β and we didn't know it. This is the kind of bug that *steals years of debugging* from people who don't notice.++2. **Statistical significance on the 0.005 SPLADE++ gap.** Our NFCorpus result (0.352) is 0.005 above SPLADE++ (0.347). The natural pushback is "noise" β and with N=323 queries it's a fair pushback. We need a paired bootstrap CI to either confirm the gap is real or admit it's not.++Both are about honesty in reporting, not new features.++## Decision++### 1. Bug story: silent hash-fallback path++**Root cause chain:**++```+neural_patterns store action+ β realEmbeddings.embed(text)+ β agentic-flow/reasoningbank.computeEmbedding(text)+ β @xenova/transformers pipeline('feature-extraction', 'all-MiniLM-L6-v2')+ β @xenova/transformers loads tokenizer + sharp (for image models)+ β require('sharp')+ β require('../build/Release/sharp-darwin-arm64v8.node')+ β MODULE_NOT_FOUND+ β THROWS+ β THROWS+ β THROWS+ β but the THROW happens per-CALL, not at import time+ β neural-tools wraps each call in try/catch and silently hash-falls-back+ β store returns successfully+β neural_patterns reports _realEmbedding: true (because realEmbeddings is non-null at module-load)+```++**Where the lie lives:** `src/mcp-tools/neural-tools.ts` initialises+`realEmbeddings = { embed: ... }` at module load by importing+`agentic-flow/reasoningbank`. That import succeeds even when the+downstream `transformers.js β sharp` chain breaks β because the import is+just type metadata, not an actual model load. The lazy model load happens+on the first `computeEmbedding()` call, which throws. The outer try/catch+swallows it and silently falls back to hash. But `_realEmbedding` was set+to `true` based on the import success.++**What we did about it:**++- **Bypassed the sharp dependency** in ADR-085 by loading BGE directly+ via `@xenova/transformers` `AutoTokenizer` + `AutoModel`. Text bi-encoders+ don't need image preprocessing β `sharp` is a transitive dep of the+ full pipeline that's never needed for retrieval.+- **Discovered it via the BEIR bench.** A 0.262 nDCG@10 result (vs BM25+ published 0.325) was suspicious enough to investigate. Without an+ external benchmark, the bug could have persisted indefinitely because+ every internal test passed (since hash-based embeds are deterministic+ and our internal labels are short tokens that BM25 dominates anyway).+- **Documented in `BEIR-MATRIX.md`** and ADR-085 Β§"sharp-on-darwin-arm64+ bug" so users on other platforms know to check.++**What we did NOT do** (deferred to a separate fix):+- Fix the underlying `_realEmbedding: true` lie when per-call embeds+ throw. The honest fix is a probe-embed at module load that updates+ the flag if it fails. That's a real change in `neural-tools.ts` and+ warrants its own dedicated tracking. Leaving as known issue with the+ bypass.++**The lesson:** every "is the embedder real?" check needs to verify an+actual embed succeeded, not just that the import didn't throw. Type-load+success β runtime correctness. External benchmarks expose the gap because+they don't share the bias of internal labels.++### 2. Paired bootstrap significance test++`scripts/beir-bootstrap-significance.mjs` β given a run JSON with+`perQuery: Array<{qid, ndcg10, ...}>`, computes:++- **1-sample bootstrap CI** on the mean nDCG@10 (10K resamples, deterministic+ mulberry32 seed=42). 95% CI for the point estimate.+- **CI overlap test vs each published baseline** β if our 95% CI lower+ bound exceeds the baseline, the difference is significant at p<0.05.+- **Paired bootstrap** (via `--paired <other-run.json>`) β resamples the+ per-query *differences* between two runs. The one-sided p-value tells us+ if our run is significantly above (or below) the baseline. The 95% CI on+ the difference gives both magnitude and uncertainty.++Why paired matters: the same hard queries are hard for everyone. A+1-sample test treats query difficulty as noise; the paired test conditions+on it. This is what BEIR papers report.++**Reusable as `BEIR_DATA_DIR=... node beir-bootstrap-significance.mjs`** on+any run JSON our bench emits.++### 3. Per-query metrics in run JSON++Extended `scripts/run-beir-bge.mjs` to save `perQuery: [{qid, ndcg10,+mrr10, recall10, recall100}, ...]` in every run JSON. Without this,+external bootstrap CI testing isn't possible.++### 4. BEIR matrix page++`docs/benchmarks/BEIR-MATRIX.md` β dataset Γ model Γ metric grid. Every+cell links to its run JSON and reproduction command. The honest-reporting+counterpart to the leaderboard table in ADR-085.++## Measured proof β bootstrap CIs on both datasets++### NFCorpus (N=323 queries, bootstrap 10k seed=42)++```+=== Our nDCG@10 (1-sample bootstrap CI) ===+ point: 0.3517+ 95% CI: [0.3171, 0.3873]++=== vs each published baseline (CI overlap) ===+ 0.272 SBERT msmarco Ξ=+0.0797 β above [p<0.05 β significant win]+ 0.305 ColBERT Ξ=+0.0467 β above [p<0.05 β significant win]+ 0.319 TAS-B Ξ=+0.0327 β above n.s.+ 0.319 GenQ Ξ=+0.0327 β above n.s.+ 0.325 BM25 (Lucene) Ξ=+0.0267 β above n.s.+ 0.328 DocT5query Ξ=+0.0237 β above n.s.+ 0.328 Contriever Ξ=+0.0237 β above n.s.+ 0.343 GTR-XL Ξ=+0.0087 β above n.s.+ 0.347 SPLADE++ Ξ=+0.0047 β above n.s.+ 0.380 BGE-large-v1.5 Ξ=-0.0283 β below n.s.+```++**Two significant wins** (SBERT, ColBERT). Seven point-estimate wins are within sampling noise (n.s.) β the "rank-2" headline is a single-realisation outcome, not a statistically distinguishable lead over SPLADE++/GTR-XL/BM25 etc. **The 0.005 SPLADE++ gap is noise**, as expected.++### SciFact (N=300 queries, bootstrap 10k seed=42)++```+=== Our nDCG@10 (1-sample bootstrap CI) ===+ point: 0.6256+ 95% CI: [0.5772, 0.6723]++=== vs each published baseline (CI overlap) ===+ 0.555 SBERT msmarco Ξ=+0.0706 β above [p<0.05 β significant win]+ 0.643 TAS-B Ξ=-0.0174 β below n.s.+ 0.644 GenQ Ξ=-0.0184 β below n.s.+ 0.662 GTR-XL Ξ=-0.0364 β below n.s.+ 0.671 ColBERT Ξ=-0.0454 β below n.s.+ 0.675 DocT5query Ξ=-0.0494 β below [p<0.05 β significant loss]+ 0.677 Contriever Ξ=-0.0514 β below [p<0.05 β significant loss]+ 0.679 BM25 (Lucene) Ξ=-0.0534 β below [p<0.05 β significant loss]+ 0.704 SPLADE++ Ξ=-0.0784 β below [p<0.05 β significant loss]+ 0.722 BGE-large-v1.5 Ξ=-0.0964 β below [p<0.05 β significant loss]+```++**One significant win** (SBERT). **Five significant losses** including to BM25. SciFact is a fact-verification benchmark where exact scientific terms favor lexical retrieval; zero-shot BGE-base doesn't have the in-domain training that BGE-large + SPLADE++ have.++### The two-dataset truth++| System | NFCorpus | SciFact | Mean |+|---|---:|---:|---:|+| BGE-large-v1.5 (335M, published) | 0.380 | 0.722 | 0.551 |+| **ruflo + BGE-base-en-v1.5 (110M)** | **0.352** | **0.626** | **0.489** |+| SPLADE++ | 0.347 | 0.704 | 0.526 |+| BM25 (Lucene) | 0.325 | 0.679 | 0.502 |++**On the two-dataset mean, we lose to BM25** (0.489 vs 0.502). The NFCorpus rank-2 is real but not representative. The acceptance test "beats BM25 on both datasets" fails β and reporting that honestly is the point of this ADR.++## Honest limits++- **N=323 is the dataset's full test split** β we can't add more queries+ without leaving NFCorpus.+- **Bootstrap assumes per-query independence** β true for IID query sets,+ approximately true for BEIR.+- **The `_realEmbedding: true` lie is not yet fixed** in `neural-tools.ts`;+ the BGE bypass works around it but other call paths through+ `neural_patterns` may still report the wrong flag value. Tracked.++## Deliberately NOT in this round++- **Fix the `_realEmbedding: true` lie** at source β needs a probe-embed+ at module init. Tracked separately.+- **Lucene-style BM25 improvements** to close our 0.289 β 0.325 BM25 gap.+ Stemming + bigger stopword list would help; not in scope here.++## Verification++```bash+git clone https://github.com/ruvnet/ruflo && cd ruflo+npm install && ( cd v3/@claude-flow/cli && npx tsc )++# Re-run the NFCorpus bench with updated harness (writes perQuery to JSON)+cd /tmp/beir-nfcorpus+SKIP_INGEST=1 node /path/to/scripts/run-beir-bge.mjs++# Bootstrap significance test (10K resamples, ~1s)+node /path/to/scripts/beir-bootstrap-significance.mjs \+ /path/to/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json++# Paired test vs our pure-BM25 baseline+node /path/to/scripts/beir-bootstrap-significance.mjs \+ /path/to/docs/benchmarks/runs/beir-nfcorpus-bge-latest.json \+ --paired /path/to/docs/benchmarks/runs/beir-nfcorpus-2026-05-30T19-16-23-024Z.json+```