Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

feat(beir): Lucene BM25 + RRF + CE rerank β€” passes the acceptance test (ADR-088, 3.10.28)

ruvaa810856ffbe
diff --git a/docs/benchmarks/BEIR-MATRIX.md b/docs/benchmarks/BEIR-MATRIX.md
index f326c3639f..feb564f5a8 100644
--- a/docs/benchmarks/BEIR-MATRIX.md
+++ b/docs/benchmarks/BEIR-MATRIX.md
@@ -48,6 +48,40 @@ Recall@100 **does** improve (RRF surfaces more candidates) β€” which makes RRF a
The default BEIR runner stays at dense-only. RRF is opt-in.
+### ADR-088 β€” Lucene BM25 + cross-encoder rerank (3.10.28) β€” the pipeline that works
+
+Fixing the BM25 (Porter stemmer + Lucene stopwords + length norm, single-field over title+text) closes the asymmetric-strength problem and makes RRF + cross-encoder rerank produce real wins.
+
+| Configuration | NFCorpus | SciFact | Mean | Notes |
+|---|---:|---:|---:|---|
+| dense alone (BGE-base) | 0.352 | 0.626 | 0.489 | baseline |
+| **Lucene BM25 alone** | 0.328 | 0.681 | **0.505** | matches published baseline (0.325 / 0.679) |
+| Lucene RRF k=30 (no CE) | **0.363** | 0.639 | 0.501 | RRF works once BM25 is strong |
+| Multi-field RRF + CE rerank | 0.355 | 0.685 | 0.520 | rerank rescues weak BM25 |
+| **Lucene RRF + CE rerank (best)** | **0.358** | **0.683** | **0.521** | rank 2 NFCorpus, rank 3 SciFact |
+
+**Acceptance test PASSES** with Lucene RRF + CE rerank: beats published BM25 on both datasets (+0.033 NFCorpus, +0.004 SciFact). Mean 0.521 beats every listed BEIR baseline except SPLADE++ (0.526) and BGE-large (0.551).
+
+### Final two-dataset means leaderboard
+
+| System | Params | Mean nDCG@10 | NFCorpus | SciFact |
+|---|---:|---:|---:|---:|
+| BGE-large-v1.5 (published) | 335M | **0.551** | 0.380 | 0.722 |
+| SPLADE++ (published) | 110M | **0.526** | 0.347 | 0.704 |
+| **ruflo Lucene RRF + CE rerank (3.10.28)** | **110M** | **0.521** | **0.358** | **0.683** |
+| ruflo multi-field RRF + CE rerank | 110M | 0.520 | 0.355 | 0.685 |
+| ruflo Lucene BM25 alone | β€” | 0.505 | 0.328 | 0.681 |
+| BM25 (published Lucene) | β€” | 0.502 | 0.325 | 0.679 |
+| Contriever (published) | 110M | 0.502 | 0.328 | 0.677 |
+| DocT5query (published) | 60M | 0.501 | 0.328 | 0.675 |
+| ColBERT (published) | 110M | 0.488 | 0.305 | 0.671 |
+| GTR-XL (published) | 1.2B | 0.502 | 0.343 | 0.662 |
+| ruflo dense alone (BGE-base) | 110M | 0.489 | 0.352 | 0.626 |
+| TAS-B (published) | 66M | 0.481 | 0.319 | 0.643 |
+| SBERT msmarco (published) | 110M | 0.414 | 0.272 | 0.555 |
+
+We rank **3rd of 13 entries on the 2-dataset mean**. Using a 110M-param base model (vs BGE-large's 335M and GTR-XL's 1.2B).
+
> **What pipeline is reported here:** the NFCorpus 0.352 row is the **direct
> BGE dense path** β€” no fine-tuning, no hybrid BM25+dense fusion, no
> cross-encoder reranker. The hybrid pipeline (cosine + multi-field BM25 +
diff --git a/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-2026-05-30T22-55-50-389Z.json b/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-2026-05-30T22-55-50-389Z.json
new file mode 100644
index 0000000000..0f42cdec4e
--- /dev/null
+++ b/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-2026-05-30T22-55-50-389Z.json
@@ -0,0 +1,2295 @@
+{
+ "runAt": "2026-05-30T22:55:50.389Z",
+ "benchmark": "beir-nfcorpus-hybrid-rerank",
+ "dataset": "nfcorpus",
+ "pipeline": "BM25+BGE RRF+CE-rerank",
+ "bgeModel": "Xenova/bge-base-en-v1.5",
+ "rrfK": 60,
+ "rerank": true,
+ "queries": 323,
+ "corpusSize": 3633,
+ "metrics": {
+ "ndcg10": 0.3579401956512521,
+ "mrr10": 0.5858002850262912,
+ "recall10": 0.16552038911035225,
+ "recall100": 0.3269348953720285,
+ "avgQueryLatencyMs": 4629.8292253622285
+ },
+ "perQuery": [
+ {
+ "qid": "PLAIN-2",
+ "ndcg10": 0.7567216677934727,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-12",
+ "ndcg10": 0.09818965828900982,
+ "mrr10": 0.2,
+ "recall10": 0.03333333333333333,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-23",
+ "ndcg10": 0.41597401548426627,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-33",
+ "ndcg10": 0.517063508277242,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-44",
+ "ndcg10": 0.10764273675942825,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.22413793103448276
+ },
+ {
+ "qid": "PLAIN-56",
+ "ndcg10": 0.6644368413213309,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-68",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.01818181818181818,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-78",
+ "ndcg10": 0.25937570933770104,
+ "mrr10": 1,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.11290322580645161
+ },
+ {
+ "qid": "PLAIN-91",
+ "ndcg10": 0.32622064218245284,
+ "mrr10": 1,
+ "recall10": 0.046875,
+ "recall100": 0.34375
+ },
+ {
+ "qid": "PLAIN-102",
+ "ndcg10": 0.3843483686763149,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-112",
+ "ndcg10": 0.570530415158296,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.16071428571428573
+ },
+ {
+ "qid": "PLAIN-123",
+ "ndcg10": 0.22550350195645952,
+ "mrr10": 0.5,
+ "recall10": 0.0784313725490196,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-133",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-143",
+ "ndcg10": 0.6820218010517335,
+ "mrr10": 1,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.23529411764705882
+ },
+ {
+ "qid": "PLAIN-153",
+ "ndcg10": 0.9558245273890439,
+ "mrr10": 1,
+ "recall10": 0.19148936170212766,
+ "recall100": 0.425531914893617
+ },
+ {
+ "qid": "PLAIN-165",
+ "ndcg10": 0.4763960541386417,
+ "mrr10": 1,
+ "recall10": 0.19444444444444445,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-175",
+ "ndcg10": 0.3568919684926741,
+ "mrr10": 1,
+ "recall10": 0.10810810810810811,
+ "recall100": 0.10810810810810811
+ },
+ {
+ "qid": "PLAIN-186",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-196",
+ "ndcg10": 0.7116179486056782,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.10144927536231885
+ },
+ {
+ "qid": "PLAIN-207",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.32727272727272727
+ },
+ {
+ "qid": "PLAIN-217",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.19444444444444445
+ },
+ {
+ "qid": "PLAIN-227",
+ "ndcg10": 0.08514311764162098,
+ "mrr10": 0.2,
+ "recall10": 0.025,
+ "recall100": 0.05
+ },
+ {
+ "qid": "PLAIN-238",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.018518518518518517,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-248",
+ "ndcg10": 0.2835149363066189,
+ "mrr10": 0.5,
+ "recall10": 0.0967741935483871,
+ "recall100": 0.2903225806451613
+ },
+ {
+ "qid": "PLAIN-259",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.35714285714285715
+ },
+ {
+ "qid": "PLAIN-270",
+ "ndcg10": 0.2984900352767555,
+ "mrr10": 1,
+ "recall10": 0.038461538461538464,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-280",
+ "ndcg10": 0.3963918729015093,
+ "mrr10": 1,
+ "recall10": 0.15,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-291",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.46153846153846156
+ },
+ {
+ "qid": "PLAIN-307",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-320",
+ "ndcg10": 0.07336392209936005,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-332",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-344",
+ "ndcg10": 0.2193331461698658,
+ "mrr10": 1,
+ "recall10": 0.04918032786885246,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-358",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-371",
+ "ndcg10": 0.27541155237618664,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-383",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-395",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-407",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-418",
+ "ndcg10": 0.9072836011519267,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-430",
+ "ndcg10": 0.6278760013165626,
+ "mrr10": 1,
+ "recall10": 0.375,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-441",
+ "ndcg10": 0.7744138298590112,
+ "mrr10": 1,
+ "recall10": 0.8,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-457",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-468",
+ "ndcg10": 0.35819720929062837,
+ "mrr10": 0.5,
+ "recall10": 0.2,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-478",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-488",
+ "ndcg10": 0.9305687780632227,
+ "mrr10": 1,
+ "recall10": 0.6,
+ "recall100": 0.7333333333333333
+ },
+ {
+ "qid": "PLAIN-499",
+ "ndcg10": 0.4480300279758384,
+ "mrr10": 1,
+ "recall10": 0.11764705882352941,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-510",
+ "ndcg10": 0.15130120674940672,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-531",
+ "ndcg10": 0.17993148201117176,
+ "mrr10": 0.25,
+ "recall10": 0.006493506493506494,
+ "recall100": 0.09740259740259741
+ },
+ {
+ "qid": "PLAIN-541",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-551",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-561",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.12
+ },
+ {
+ "qid": "PLAIN-571",
+ "ndcg10": 0.24377484748737527,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-583",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.022727272727272728,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-593",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-603",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-613",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-623",
+ "ndcg10": 0.3436966913921454,
+ "mrr10": 0.5,
+ "recall10": 0.07317073170731707,
+ "recall100": 0.14634146341463414
+ },
+ {
+ "qid": "PLAIN-634",
+ "ndcg10": 0.424926013816671,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-645",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-660",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.021052631578947368,
+ "recall100": 0.09052631578947369
+ },
+ {
+ "qid": "PLAIN-671",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-681",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-691",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.047619047619047616
+ },
+ {
+ "qid": "PLAIN-701",
+ "ndcg10": 0.647939623894138,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-711",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.03508771929824561,
+ "recall100": 0.19298245614035087
+ },
+ {
+ "qid": "PLAIN-721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.72
+ },
+ {
+ "qid": "PLAIN-731",
+ "ndcg10": 0.8521705090845474,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.42592592592592593
+ },
+ {
+ "qid": "PLAIN-741",
+ "ndcg10": 0.857980942822373,
+ "mrr10": 1,
+ "recall10": 0.42105263157894735,
+ "recall100": 0.42105263157894735
+ },
+ {
+ "qid": "PLAIN-751",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-761",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-771",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-782",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-792",
+ "ndcg10": 0.4935232796777481,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-806",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.10416666666666667,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-817",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-827",
+ "ndcg10": 0.6848374495230163,
+ "mrr10": 1,
+ "recall10": 0.028688524590163935,
+ "recall100": 0.22540983606557377
+ },
+ {
+ "qid": "PLAIN-838",
+ "ndcg10": 0.7767471075223497,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.8888888888888888
+ },
+ {
+ "qid": "PLAIN-850",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.08771929824561403,
+ "recall100": 0.19298245614035087
+ },
+ {
+ "qid": "PLAIN-872",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-882",
+ "ndcg10": 0.5036067026046991,
+ "mrr10": 1,
+ "recall10": 0.26666666666666666,
+ "recall100": 0.8
+ },
+ {
+ "qid": "PLAIN-892",
+ "ndcg10": 0.8701249883466593,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.2391304347826087
+ },
+ {
+ "qid": "PLAIN-902",
+ "ndcg10": 0.6366824387328317,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-913",
+ "ndcg10": 0.800693766409882,
+ "mrr10": 1,
+ "recall10": 0.20588235294117646,
+ "recall100": 0.35294117647058826
+ },
+ {
+ "qid": "PLAIN-924",
+ "ndcg10": 0.33013764944712026,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-934",
+ "ndcg10": 0.47010174003740723,
+ "mrr10": 0.5,
+ "recall10": 0.04950495049504951,
+ "recall100": 0.36633663366336633
+ },
+ {
+ "qid": "PLAIN-946",
+ "ndcg10": 0.1635413866202963,
+ "mrr10": 0.2,
+ "recall10": 0.06451612903225806,
+ "recall100": 0.1935483870967742
+ },
+ {
+ "qid": "PLAIN-956",
+ "ndcg10": 0.6726934039987301,
+ "mrr10": 1,
+ "recall10": 0.03398058252427184,
+ "recall100": 0.19902912621359223
+ },
+ {
+ "qid": "PLAIN-966",
+ "ndcg10": 0.18154179253735267,
+ "mrr10": 0.2,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-977",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-987",
+ "ndcg10": 0.23463936301137822,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-997",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1008",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1018",
+ "ndcg10": 0.5111707781750245,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.1,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1028",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1039",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1050",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.0425531914893617
+ },
+ {
+ "qid": "PLAIN-1066",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1088",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1098",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1109",
+ "ndcg10": 0.6203974344166848,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.3163265306122449
+ },
+ {
+ "qid": "PLAIN-1119",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1141",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1151",
+ "ndcg10": 0.3012042966046398,
+ "mrr10": 0.25,
+ "recall10": 0.026143790849673203,
+ "recall100": 0.20915032679738563
+ },
+ {
+ "qid": "PLAIN-1161",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1172",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.15789473684210525
+ },
+ {
+ "qid": "PLAIN-1183",
+ "ndcg10": 0.23365080824310533,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.08
+ },
+ {
+ "qid": "PLAIN-1193",
+ "ndcg10": 0.09478836436955078,
+ "mrr10": 0.25,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1203",
+ "ndcg10": 0.48882922182497546,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.4166666666666667
+ },
+ {
+ "qid": "PLAIN-1214",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1225",
+ "ndcg10": 0.6578986223737446,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.6296296296296297
+ },
+ {
+ "qid": "PLAIN-1236",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1249",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1262",
+ "ndcg10": 0.41528076708874123,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-1275",
+ "ndcg10": 0.6569815082938864,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1288",
+ "ndcg10": 0.7133080652109929,
+ "mrr10": 1,
+ "recall10": 0.03571428571428571,
+ "recall100": 0.12244897959183673
+ },
+ {
+ "qid": "PLAIN-1299",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.01639344262295082,
+ "recall100": 0.09836065573770492
+ },
+ {
+ "qid": "PLAIN-1309",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1320",
+ "ndcg10": 0.561692774772731,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1331",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1342",
+ "ndcg10": 0.4885468020156227,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1353",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1363",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.09090909090909091,
+ "recall100": 0.7272727272727273
+ },
+ {
+ "qid": "PLAIN-1374",
+ "ndcg10": 0.17630010660342912,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.2692307692307692
+ },
+ {
+ "qid": "PLAIN-1387",
+ "ndcg10": 0.693761356516045,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1398",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.0970873786407767,
+ "recall100": 0.2621359223300971
+ },
+ {
+ "qid": "PLAIN-1409",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.007905138339920948,
+ "recall100": 0.11857707509881422
+ },
+ {
+ "qid": "PLAIN-1419",
+ "ndcg10": 0.42520843362602373,
+ "mrr10": 1,
+ "recall10": 0.046153846153846156,
+ "recall100": 0.12307692307692308
+ },
+ {
+ "qid": "PLAIN-1429",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1441",
+ "ndcg10": 0.5582604437617528,
+ "mrr10": 1,
+ "recall10": 0.026881720430107527,
+ "recall100": 0.1774193548387097
+ },
+ {
+ "qid": "PLAIN-1453",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.00816326530612245,
+ "recall100": 0.12244897959183673
+ },
+ {
+ "qid": "PLAIN-1463",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1473",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1485",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1496",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1506",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1527",
+ "ndcg10": 0.7471589728148227,
+ "mrr10": 1,
+ "recall10": 0.034653465346534656,
+ "recall100": 0.18811881188118812
+ },
+ {
+ "qid": "PLAIN-1537",
+ "ndcg10": 0.485813635742251,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.17777777777777778
+ },
+ {
+ "qid": "PLAIN-1547",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1557",
+ "ndcg10": 0.3222722491219547,
+ "mrr10": 0.5,
+ "recall10": 0.11538461538461539,
+ "recall100": 0.19230769230769232
+ },
+ {
+ "qid": "PLAIN-1568",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1579",
+ "ndcg10": 0.14606834984270645,
+ "mrr10": 0.25,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1590",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1601",
+ "ndcg10": 0.642186726668901,
+ "mrr10": 1,
+ "recall10": 0.06756756756756757,
+ "recall100": 0.3108108108108108
+ },
+ {
+ "qid": "PLAIN-1611",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1621",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1635",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.02320185614849188,
+ "recall100": 0.14153132250580047
+ },
+ {
+ "qid": "PLAIN-1645",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1656",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10714285714285714
+ },
+ {
+ "qid": "PLAIN-1667",
+ "ndcg10": 0.804810999209339,
+ "mrr10": 1,
+ "recall10": 0.050314465408805034,
+ "recall100": 0.20754716981132076
+ },
+ {
+ "qid": "PLAIN-1679",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1690",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1700",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1710",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.625,
+ "recall100": 0.9375
+ },
+ {
+ "qid": "PLAIN-1721",
+ "ndcg10": 0.5617131059245698,
+ "mrr10": 0.5,
+ "recall10": 0.1935483870967742,
+ "recall100": 0.22580645161290322
+ },
+ {
+ "qid": "PLAIN-1731",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1741",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1752",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1762",
+ "ndcg10": 0.41253177989255346,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-1772",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1784",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1794",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1805",
+ "ndcg10": 0.9305687780632227,
+ "mrr10": 1,
+ "recall10": 0.07758620689655173,
+ "recall100": 0.31896551724137934
+ },
+ {
+ "qid": "PLAIN-1817",
+ "ndcg10": 0.5326208815196265,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1827",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.3520408163265306
+ },
+ {
+ "qid": "PLAIN-1847",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1857",
+ "ndcg10": 0.6371523797895856,
+ "mrr10": 1,
+ "recall10": 0.15151515151515152,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1867",
+ "ndcg10": 0.27487633291429087,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1877",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-1887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1897",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1909",
+ "ndcg10": 0.841492960259019,
+ "mrr10": 1,
+ "recall10": 0.017278617710583154,
+ "recall100": 0.11879049676025918
+ },
+ {
+ "qid": "PLAIN-1919",
+ "ndcg10": 0.727329844310522,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.7037037037037037
+ },
+ {
+ "qid": "PLAIN-1929",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-1940",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-1950",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-1962",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1972",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-1983",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.3125,
+ "recall100": 0.71875
+ },
+ {
+ "qid": "PLAIN-1995",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2009",
+ "ndcg10": 0.10893649261775588,
+ "mrr10": 0.25,
+ "recall10": 0.125,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-2019",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2030",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-2040",
+ "ndcg10": 0.5423640154200349,
+ "mrr10": 1,
+ "recall10": 0.030303030303030304,
+ "recall100": 0.15151515151515152
+ },
+ {
+ "qid": "PLAIN-2051",
+ "ndcg10": 0.7503359712972109,
+ "mrr10": 1,
+ "recall10": 0.01971830985915493,
+ "recall100": 0.11267605633802817
+ },
+ {
+ "qid": "PLAIN-2061",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.02195121951219512,
+ "recall100": 0.15365853658536585
+ },
+ {
+ "qid": "PLAIN-2071",
+ "ndcg10": 0.5077181160122074,
+ "mrr10": 1,
+ "recall10": 0.08888888888888889,
+ "recall100": 0.4222222222222222
+ },
+ {
+ "qid": "PLAIN-2081",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2092",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2102",
+ "ndcg10": 0.8236998933965709,
+ "mrr10": 1,
+ "recall10": 0.019002375296912115,
+ "recall100": 0.12826603325415678
+ },
+ {
+ "qid": "PLAIN-2113",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2124",
+ "ndcg10": 0.7100226348397882,
+ "mrr10": 1,
+ "recall10": 0.4117647058823529,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-2134",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.08333333333333333
+ },
+ {
+ "qid": "PLAIN-2145",
+ "ndcg10": 0.1482972305701491,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2156",
+ "ndcg10": 0.644824486427155,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.8333333333333334
+ },
+ {
+ "qid": "PLAIN-2167",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2177",
+ "ndcg10": 0.4883491838252097,
+ "mrr10": 0.5,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2187",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2197",
+ "ndcg10": 0.9216017310213247,
+ "mrr10": 1,
+ "recall10": 0.15517241379310345,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2209",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2220",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2230",
+ "ndcg10": 0.13743816645714543,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-2240",
+ "ndcg10": 0.642186726668901,
+ "mrr10": 1,
+ "recall10": 0.45454545454545453,
+ "recall100": 0.5454545454545454
+ },
+ {
+ "qid": "PLAIN-2250",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2261",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.18333333333333332
+ },
+ {
+ "qid": "PLAIN-2271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2291",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2301",
+ "ndcg10": 0.06943122193677727,
+ "mrr10": 0.125,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-2311",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2321",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2332",
+ "ndcg10": 0.5116508161747901,
+ "mrr10": 1,
+ "recall10": 0.05154639175257732,
+ "recall100": 0.3402061855670103
+ },
+ {
+ "qid": "PLAIN-2343",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2354",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2364",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2375",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2386",
+ "ndcg10": 0.11004588314904008,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-2396",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2408",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2430",
+ "ndcg10": 0.45374257454118544,
+ "mrr10": 1,
+ "recall10": 0.06521739130434782,
+ "recall100": 0.391304347826087
+ },
+ {
+ "qid": "PLAIN-2440",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2450",
+ "ndcg10": 0.29990503169749433,
+ "mrr10": 0.5,
+ "recall10": 0.07894736842105263,
+ "recall100": 0.18421052631578946
+ },
+ {
+ "qid": "PLAIN-2460",
+ "ndcg10": 0.08649891544052932,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.3225806451612903
+ },
+ {
+ "qid": "PLAIN-2470",
+ "ndcg10": 0.6464260278126966,
+ "mrr10": 1,
+ "recall10": 0.0547945205479452,
+ "recall100": 0.3287671232876712
+ },
+ {
+ "qid": "PLAIN-2480",
+ "ndcg10": 0.3534433112980961,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.6111111111111112
+ },
+ {
+ "qid": "PLAIN-2490",
+ "ndcg10": 0.31248003685671516,
+ "mrr10": 0.5,
+ "recall10": 0.13043478260869565,
+ "recall100": 0.34782608695652173
+ },
+ {
+ "qid": "PLAIN-2500",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-2510",
+ "ndcg10": 0.7241528458281337,
+ "mrr10": 1,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.5869565217391305
+ },
+ {
+ "qid": "PLAIN-2520",
+ "ndcg10": 0.03668196104968002,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.0136986301369863,
+ "recall100": 0.1232876712328767
+ },
+ {
+ "qid": "PLAIN-2530",
+ "ndcg10": 0.9069212362975007,
+ "mrr10": 1,
+ "recall10": 0.1388888888888889,
+ "recall100": 0.3611111111111111
+ },
+ {
+ "qid": "PLAIN-2540",
+ "ndcg10": 0.31579804414741475,
+ "mrr10": 0.5,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-2550",
+ "ndcg10": 0.10405896756643919,
+ "mrr10": 0.2,
+ "recall10": 0.04,
+ "recall100": 0.24
+ },
+ {
+ "qid": "PLAIN-2560",
+ "ndcg10": 0.8701249883466592,
+ "mrr10": 1,
+ "recall10": 0.38095238095238093,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2570",
+ "ndcg10": 0.16014043519656038,
+ "mrr10": 0.5,
+ "recall10": 0.025,
+ "recall100": 0.475
+ },
+ {
+ "qid": "PLAIN-2580",
+ "ndcg10": 0.41235173674195585,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.17391304347826086
+ },
+ {
+ "qid": "PLAIN-2590",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-2600",
+ "ndcg10": 0.5991689570583907,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2610",
+ "ndcg10": 0.3746661058764784,
+ "mrr10": 1,
+ "recall10": 0.04918032786885246,
+ "recall100": 0.19672131147540983
+ },
+ {
+ "qid": "PLAIN-2620",
+ "ndcg10": 0.24327815416627566,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.07272727272727272,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2630",
+ "ndcg10": 0.6772733739526907,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.4791666666666667
+ },
+ {
+ "qid": "PLAIN-2640",
+ "ndcg10": 0.5107640114919196,
+ "mrr10": 1,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-2650",
+ "ndcg10": 0.46318965958284924,
+ "mrr10": 1,
+ "recall10": 0.13793103448275862,
+ "recall100": 0.3793103448275862
+ },
+ {
+ "qid": "PLAIN-2660",
+ "ndcg10": 0.4518648409659427,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.5185185185185185
+ },
+ {
+ "qid": "PLAIN-2670",
+ "ndcg10": 0.3406992958233561,
+ "mrr10": 1,
+ "recall10": 0.02857142857142857,
+ "recall100": 0.05714285714285714
+ },
+ {
+ "qid": "PLAIN-2680",
+ "ndcg10": 0.5075417611009456,
+ "mrr10": 1,
+ "recall10": 0.21428571428571427,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2690",
+ "ndcg10": 0.3764251307383051,
+ "mrr10": 1,
+ "recall10": 0.06976744186046512,
+ "recall100": 0.37209302325581395
+ },
+ {
+ "qid": "PLAIN-2700",
+ "ndcg10": 0.4391834785971612,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2710",
+ "ndcg10": 0.6812457182571504,
+ "mrr10": 1,
+ "recall10": 0.23809523809523808,
+ "recall100": 0.47619047619047616
+ },
+ {
+ "qid": "PLAIN-2720",
+ "ndcg10": 0.3995688713838975,
+ "mrr10": 1,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3611111111111111
+ },
+ {
+ "qid": "PLAIN-2730",
+ "ndcg10": 0.6077697019194394,
+ "mrr10": 0.5,
+ "recall10": 0.20689655172413793,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2740",
+ "ndcg10": 0.6155267282106165,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.24
+ },
+ {
+ "qid": "PLAIN-2750",
+ "ndcg10": 0.5987514293054288,
+ "mrr10": 1,
+ "recall10": 0.13559322033898305,
+ "recall100": 0.3389830508474576
+ },
+ {
+ "qid": "PLAIN-2760",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2770",
+ "ndcg10": 0.330622400869393,
+ "mrr10": 1,
+ "recall10": 0.07317073170731707,
+ "recall100": 0.5121951219512195
+ },
+ {
+ "qid": "PLAIN-2780",
+ "ndcg10": 0.3663797094518679,
+ "mrr10": 1,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.3181818181818182
+ },
+ {
+ "qid": "PLAIN-2790",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.0847457627118644,
+ "recall100": 0.2542372881355932
+ },
+ {
+ "qid": "PLAIN-2800",
+ "ndcg10": 0.14673120006323176,
+ "mrr10": 0.25,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-2810",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2830",
+ "ndcg10": 0.2809436351950179,
+ "mrr10": 0.2,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2840",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.058823529411764705
+ },
+ {
+ "qid": "PLAIN-2850",
+ "ndcg10": 0.04621713645665386,
+ "mrr10": 0.1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.4444444444444444
+ },
+ {
+ "qid": "PLAIN-2860",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2870",
+ "ndcg10": 0.10104155414609708,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2880",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.11764705882352941
+ },
+ {
+ "qid": "PLAIN-2890",
+ "ndcg10": 0.13252641783286837,
+ "mrr10": 0.1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-2900",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-2910",
+ "ndcg10": 0.24249682153453903,
+ "mrr10": 0.5,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.26666666666666666
+ },
+ {
+ "qid": "PLAIN-2920",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2930",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2940",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.22727272727272727
+ },
+ {
+ "qid": "PLAIN-2950",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2960",
+ "ndcg10": 0.35583989829307827,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2970",
+ "ndcg10": 0.47965506954306864,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-2981",
+ "ndcg10": 0.5321036168710066,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2991",
+ "ndcg10": 0.3868528072345416,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3001",
+ "ndcg10": 0.3868528072345416,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3014",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3026",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3037",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3053",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3063",
+ "ndcg10": 0.15101961822780524,
+ "mrr10": 0.2,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3074",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3085",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3097",
+ "ndcg10": 0.515157140670655,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3116",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3131",
+ "ndcg10": 0.6516433935264055,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3141",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.20588235294117646
+ },
+ {
+ "qid": "PLAIN-3151",
+ "ndcg10": 0.519838015658837,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.3888888888888889,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3161",
+ "ndcg10": 0.5486455668071759,
+ "mrr10": 1,
+ "recall10": 0.06976744186046512,
+ "recall100": 0.32558139534883723
+ },
+ {
+ "qid": "PLAIN-3171",
+ "ndcg10": 0.04820998182892457,
+ "mrr10": 0.125,
+ "recall10": 0.029411764705882353,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-3181",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07407407407407407
+ },
+ {
+ "qid": "PLAIN-3191",
+ "ndcg10": 0.2204386078501931,
+ "mrr10": 0.5,
+ "recall10": 0.125,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3201",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.03125,
+ "recall100": 0.09375
+ },
+ {
+ "qid": "PLAIN-3211",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3221",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-3231",
+ "ndcg10": 0.14810444518925672,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.125,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3241",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-3251",
+ "ndcg10": 0.24162606600584033,
+ "mrr10": 0.25,
+ "recall10": 0.09523809523809523,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-3261",
+ "ndcg10": 0.17034964791167806,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.15384615384615385
+ },
+ {
+ "qid": "PLAIN-3271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10204081632653061
+ },
+ {
+ "qid": "PLAIN-3281",
+ "ndcg10": 0.4954499631132973,
+ "mrr10": 1,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.11538461538461539
+ },
+ {
+ "qid": "PLAIN-3292",
+ "ndcg10": 0.5146983300029805,
+ "mrr10": 0.5,
+ "recall10": 0.7142857142857143,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3302",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.35135135135135137
+ },
+ {
+ "qid": "PLAIN-3312",
+ "ndcg10": 0.23741542281870187,
+ "mrr10": 0.5,
+ "recall10": 0.38461538461538464,
+ "recall100": 0.6923076923076923
+ },
+ {
+ "qid": "PLAIN-3322",
+ "ndcg10": 0.06943122193677727,
+ "mrr10": 0.125,
+ "recall10": 0.012658227848101266,
+ "recall100": 0.13924050632911392
+ },
+ {
+ "qid": "PLAIN-3332",
+ "ndcg10": 0.6034115249626371,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3342",
+ "ndcg10": 0.8072467605676947,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3352",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.15384615384615385
+ },
+ {
+ "qid": "PLAIN-3362",
+ "ndcg10": 0.6573548268593187,
+ "mrr10": 1,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-3372",
+ "ndcg10": 0.1446299454867737,
+ "mrr10": 0.125,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3382",
+ "ndcg10": 0.4575997404793817,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-3392",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.043478260869565216
+ },
+ {
+ "qid": "PLAIN-3402",
+ "ndcg10": 0.06362078819895171,
+ "mrr10": 0.1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.07142857142857142
+ },
+ {
+ "qid": "PLAIN-3412",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 1,
+ "recall10": 0.02040816326530612,
+ "recall100": 0.24489795918367346
+ },
+ {
+ "qid": "PLAIN-3422",
+ "ndcg10": 0.03459113773405908,
+ "mrr10": 0.25,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.5161290322580645
+ },
+ {
+ "qid": "PLAIN-3432",
+ "ndcg10": 0.3932783996463063,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3442",
+ "ndcg10": 0.051723574529084305,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.06896551724137931
+ },
+ {
+ "qid": "PLAIN-3452",
+ "ndcg10": 0.5844153148080933,
+ "mrr10": 1,
+ "recall10": 0.23529411764705882,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-3462",
+ "ndcg10": 0.6975961695061491,
+ "mrr10": 1,
+ "recall10": 0.109375,
+ "recall100": 0.515625
+ },
+ {
+ "qid": "PLAIN-3472",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.35294117647058826
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.325,
+ "DocT5query": 0.328,
+ "TAS-B": 0.319,
+ "GenQ": 0.319,
+ "ColBERT": 0.305,
+ "Contriever": 0.328,
+ "GTR-XL": 0.343,
+ "SPLADE++": 0.347,
+ "BGE-large-v1.5 (pub)": 0.38,
+ "SBERT msmarco": 0.272
+ },
+ "ourRank": 2,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-latest.json b/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-latest.json
new file mode 100644
index 0000000000..0f42cdec4e
--- /dev/null
+++ b/docs/benchmarks/runs/beir-nfcorpus-hybrid-rerank-latest.json
@@ -0,0 +1,2295 @@
+{
+ "runAt": "2026-05-30T22:55:50.389Z",
+ "benchmark": "beir-nfcorpus-hybrid-rerank",
+ "dataset": "nfcorpus",
+ "pipeline": "BM25+BGE RRF+CE-rerank",
+ "bgeModel": "Xenova/bge-base-en-v1.5",
+ "rrfK": 60,
+ "rerank": true,
+ "queries": 323,
+ "corpusSize": 3633,
+ "metrics": {
+ "ndcg10": 0.3579401956512521,
+ "mrr10": 0.5858002850262912,
+ "recall10": 0.16552038911035225,
+ "recall100": 0.3269348953720285,
+ "avgQueryLatencyMs": 4629.8292253622285
+ },
+ "perQuery": [
+ {
+ "qid": "PLAIN-2",
+ "ndcg10": 0.7567216677934727,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-12",
+ "ndcg10": 0.09818965828900982,
+ "mrr10": 0.2,
+ "recall10": 0.03333333333333333,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-23",
+ "ndcg10": 0.41597401548426627,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-33",
+ "ndcg10": 0.517063508277242,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-44",
+ "ndcg10": 0.10764273675942825,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.22413793103448276
+ },
+ {
+ "qid": "PLAIN-56",
+ "ndcg10": 0.6644368413213309,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-68",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.01818181818181818,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-78",
+ "ndcg10": 0.25937570933770104,
+ "mrr10": 1,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.11290322580645161
+ },
+ {
+ "qid": "PLAIN-91",
+ "ndcg10": 0.32622064218245284,
+ "mrr10": 1,
+ "recall10": 0.046875,
+ "recall100": 0.34375
+ },
+ {
+ "qid": "PLAIN-102",
+ "ndcg10": 0.3843483686763149,
+ "mrr10": 1,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-112",
+ "ndcg10": 0.570530415158296,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.16071428571428573
+ },
+ {
+ "qid": "PLAIN-123",
+ "ndcg10": 0.22550350195645952,
+ "mrr10": 0.5,
+ "recall10": 0.0784313725490196,
+ "recall100": 0.17647058823529413
+ },
+ {
+ "qid": "PLAIN-133",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-143",
+ "ndcg10": 0.6820218010517335,
+ "mrr10": 1,
+ "recall10": 0.058823529411764705,
+ "recall100": 0.23529411764705882
+ },
+ {
+ "qid": "PLAIN-153",
+ "ndcg10": 0.9558245273890439,
+ "mrr10": 1,
+ "recall10": 0.19148936170212766,
+ "recall100": 0.425531914893617
+ },
+ {
+ "qid": "PLAIN-165",
+ "ndcg10": 0.4763960541386417,
+ "mrr10": 1,
+ "recall10": 0.19444444444444445,
+ "recall100": 0.3888888888888889
+ },
+ {
+ "qid": "PLAIN-175",
+ "ndcg10": 0.3568919684926741,
+ "mrr10": 1,
+ "recall10": 0.10810810810810811,
+ "recall100": 0.10810810810810811
+ },
+ {
+ "qid": "PLAIN-186",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.125
+ },
+ {
+ "qid": "PLAIN-196",
+ "ndcg10": 0.7116179486056782,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.10144927536231885
+ },
+ {
+ "qid": "PLAIN-207",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.32727272727272727
+ },
+ {
+ "qid": "PLAIN-217",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.19444444444444445
+ },
+ {
+ "qid": "PLAIN-227",
+ "ndcg10": 0.08514311764162098,
+ "mrr10": 0.2,
+ "recall10": 0.025,
+ "recall100": 0.05
+ },
+ {
+ "qid": "PLAIN-238",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.018518518518518517,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-248",
+ "ndcg10": 0.2835149363066189,
+ "mrr10": 0.5,
+ "recall10": 0.0967741935483871,
+ "recall100": 0.2903225806451613
+ },
+ {
+ "qid": "PLAIN-259",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.35714285714285715
+ },
+ {
+ "qid": "PLAIN-270",
+ "ndcg10": 0.2984900352767555,
+ "mrr10": 1,
+ "recall10": 0.038461538461538464,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-280",
+ "ndcg10": 0.3963918729015093,
+ "mrr10": 1,
+ "recall10": 0.15,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-291",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.46153846153846156
+ },
+ {
+ "qid": "PLAIN-307",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-320",
+ "ndcg10": 0.07336392209936005,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-332",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-344",
+ "ndcg10": 0.2193331461698658,
+ "mrr10": 1,
+ "recall10": 0.04918032786885246,
+ "recall100": 0.2459016393442623
+ },
+ {
+ "qid": "PLAIN-358",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-371",
+ "ndcg10": 0.27541155237618664,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-383",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-395",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-407",
+ "ndcg10": 0.7262287617954056,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-418",
+ "ndcg10": 0.9072836011519267,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-430",
+ "ndcg10": 0.6278760013165626,
+ "mrr10": 1,
+ "recall10": 0.375,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-441",
+ "ndcg10": 0.7744138298590112,
+ "mrr10": 1,
+ "recall10": 0.8,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-457",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-468",
+ "ndcg10": 0.35819720929062837,
+ "mrr10": 0.5,
+ "recall10": 0.2,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-478",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-488",
+ "ndcg10": 0.9305687780632227,
+ "mrr10": 1,
+ "recall10": 0.6,
+ "recall100": 0.7333333333333333
+ },
+ {
+ "qid": "PLAIN-499",
+ "ndcg10": 0.4480300279758384,
+ "mrr10": 1,
+ "recall10": 0.11764705882352941,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-510",
+ "ndcg10": 0.15130120674940672,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-520",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-531",
+ "ndcg10": 0.17993148201117176,
+ "mrr10": 0.25,
+ "recall10": 0.006493506493506494,
+ "recall100": 0.09740259740259741
+ },
+ {
+ "qid": "PLAIN-541",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-551",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-561",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.08,
+ "recall100": 0.12
+ },
+ {
+ "qid": "PLAIN-571",
+ "ndcg10": 0.24377484748737527,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.2857142857142857,
+ "recall100": 0.2857142857142857
+ },
+ {
+ "qid": "PLAIN-583",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.022727272727272728,
+ "recall100": 0.09090909090909091
+ },
+ {
+ "qid": "PLAIN-593",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-603",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-613",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-623",
+ "ndcg10": 0.3436966913921454,
+ "mrr10": 0.5,
+ "recall10": 0.07317073170731707,
+ "recall100": 0.14634146341463414
+ },
+ {
+ "qid": "PLAIN-634",
+ "ndcg10": 0.424926013816671,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-645",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-660",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.021052631578947368,
+ "recall100": 0.09052631578947369
+ },
+ {
+ "qid": "PLAIN-671",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-681",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-691",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.047619047619047616
+ },
+ {
+ "qid": "PLAIN-701",
+ "ndcg10": 0.647939623894138,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-711",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.03508771929824561,
+ "recall100": 0.19298245614035087
+ },
+ {
+ "qid": "PLAIN-721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.72
+ },
+ {
+ "qid": "PLAIN-731",
+ "ndcg10": 0.8521705090845474,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.42592592592592593
+ },
+ {
+ "qid": "PLAIN-741",
+ "ndcg10": 0.857980942822373,
+ "mrr10": 1,
+ "recall10": 0.42105263157894735,
+ "recall100": 0.42105263157894735
+ },
+ {
+ "qid": "PLAIN-751",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-761",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-771",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-782",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-792",
+ "ndcg10": 0.4935232796777481,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-806",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.10416666666666667,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-817",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-827",
+ "ndcg10": 0.6848374495230163,
+ "mrr10": 1,
+ "recall10": 0.028688524590163935,
+ "recall100": 0.22540983606557377
+ },
+ {
+ "qid": "PLAIN-838",
+ "ndcg10": 0.7767471075223497,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.8888888888888888
+ },
+ {
+ "qid": "PLAIN-850",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.08771929824561403,
+ "recall100": 0.19298245614035087
+ },
+ {
+ "qid": "PLAIN-872",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-882",
+ "ndcg10": 0.5036067026046991,
+ "mrr10": 1,
+ "recall10": 0.26666666666666666,
+ "recall100": 0.8
+ },
+ {
+ "qid": "PLAIN-892",
+ "ndcg10": 0.8701249883466593,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.2391304347826087
+ },
+ {
+ "qid": "PLAIN-902",
+ "ndcg10": 0.6366824387328317,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-913",
+ "ndcg10": 0.800693766409882,
+ "mrr10": 1,
+ "recall10": 0.20588235294117646,
+ "recall100": 0.35294117647058826
+ },
+ {
+ "qid": "PLAIN-924",
+ "ndcg10": 0.33013764944712026,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-934",
+ "ndcg10": 0.47010174003740723,
+ "mrr10": 0.5,
+ "recall10": 0.04950495049504951,
+ "recall100": 0.36633663366336633
+ },
+ {
+ "qid": "PLAIN-946",
+ "ndcg10": 0.1635413866202963,
+ "mrr10": 0.2,
+ "recall10": 0.06451612903225806,
+ "recall100": 0.1935483870967742
+ },
+ {
+ "qid": "PLAIN-956",
+ "ndcg10": 0.6726934039987301,
+ "mrr10": 1,
+ "recall10": 0.03398058252427184,
+ "recall100": 0.19902912621359223
+ },
+ {
+ "qid": "PLAIN-966",
+ "ndcg10": 0.18154179253735267,
+ "mrr10": 0.2,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-977",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-987",
+ "ndcg10": 0.23463936301137822,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-997",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1008",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1018",
+ "ndcg10": 0.5111707781750245,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.1,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1028",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1039",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1050",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.0425531914893617
+ },
+ {
+ "qid": "PLAIN-1066",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1088",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-1098",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1109",
+ "ndcg10": 0.6203974344166848,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.3163265306122449
+ },
+ {
+ "qid": "PLAIN-1119",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1141",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.0625,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1151",
+ "ndcg10": 0.3012042966046398,
+ "mrr10": 0.25,
+ "recall10": 0.026143790849673203,
+ "recall100": 0.20915032679738563
+ },
+ {
+ "qid": "PLAIN-1161",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1172",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.15789473684210525
+ },
+ {
+ "qid": "PLAIN-1183",
+ "ndcg10": 0.23365080824310533,
+ "mrr10": 0.5,
+ "recall10": 0.08,
+ "recall100": 0.08
+ },
+ {
+ "qid": "PLAIN-1193",
+ "ndcg10": 0.09478836436955078,
+ "mrr10": 0.25,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-1203",
+ "ndcg10": 0.48882922182497546,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.4166666666666667
+ },
+ {
+ "qid": "PLAIN-1214",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1225",
+ "ndcg10": 0.6578986223737446,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.6296296296296297
+ },
+ {
+ "qid": "PLAIN-1236",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1249",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1262",
+ "ndcg10": 0.41528076708874123,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-1275",
+ "ndcg10": 0.6569815082938864,
+ "mrr10": 1,
+ "recall10": 0.10909090909090909,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1288",
+ "ndcg10": 0.7133080652109929,
+ "mrr10": 1,
+ "recall10": 0.03571428571428571,
+ "recall100": 0.12244897959183673
+ },
+ {
+ "qid": "PLAIN-1299",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.01639344262295082,
+ "recall100": 0.09836065573770492
+ },
+ {
+ "qid": "PLAIN-1309",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1320",
+ "ndcg10": 0.561692774772731,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1331",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1342",
+ "ndcg10": 0.4885468020156227,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.3125
+ },
+ {
+ "qid": "PLAIN-1353",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1363",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.09090909090909091,
+ "recall100": 0.7272727272727273
+ },
+ {
+ "qid": "PLAIN-1374",
+ "ndcg10": 0.17630010660342912,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.2692307692307692
+ },
+ {
+ "qid": "PLAIN-1387",
+ "ndcg10": 0.693761356516045,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-1398",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.0970873786407767,
+ "recall100": 0.2621359223300971
+ },
+ {
+ "qid": "PLAIN-1409",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.007905138339920948,
+ "recall100": 0.11857707509881422
+ },
+ {
+ "qid": "PLAIN-1419",
+ "ndcg10": 0.42520843362602373,
+ "mrr10": 1,
+ "recall10": 0.046153846153846156,
+ "recall100": 0.12307692307692308
+ },
+ {
+ "qid": "PLAIN-1429",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1441",
+ "ndcg10": 0.5582604437617528,
+ "mrr10": 1,
+ "recall10": 0.026881720430107527,
+ "recall100": 0.1774193548387097
+ },
+ {
+ "qid": "PLAIN-1453",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.00816326530612245,
+ "recall100": 0.12244897959183673
+ },
+ {
+ "qid": "PLAIN-1463",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1473",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1485",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1496",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1506",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1516",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1527",
+ "ndcg10": 0.7471589728148227,
+ "mrr10": 1,
+ "recall10": 0.034653465346534656,
+ "recall100": 0.18811881188118812
+ },
+ {
+ "qid": "PLAIN-1537",
+ "ndcg10": 0.485813635742251,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.17777777777777778
+ },
+ {
+ "qid": "PLAIN-1547",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1557",
+ "ndcg10": 0.3222722491219547,
+ "mrr10": 0.5,
+ "recall10": 0.11538461538461539,
+ "recall100": 0.19230769230769232
+ },
+ {
+ "qid": "PLAIN-1568",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1579",
+ "ndcg10": 0.14606834984270645,
+ "mrr10": 0.25,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1590",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1601",
+ "ndcg10": 0.642186726668901,
+ "mrr10": 1,
+ "recall10": 0.06756756756756757,
+ "recall100": 0.3108108108108108
+ },
+ {
+ "qid": "PLAIN-1611",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1621",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1635",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.02320185614849188,
+ "recall100": 0.14153132250580047
+ },
+ {
+ "qid": "PLAIN-1645",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-1656",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10714285714285714
+ },
+ {
+ "qid": "PLAIN-1667",
+ "ndcg10": 0.804810999209339,
+ "mrr10": 1,
+ "recall10": 0.050314465408805034,
+ "recall100": 0.20754716981132076
+ },
+ {
+ "qid": "PLAIN-1679",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1690",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1700",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-1710",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.625,
+ "recall100": 0.9375
+ },
+ {
+ "qid": "PLAIN-1721",
+ "ndcg10": 0.5617131059245698,
+ "mrr10": 0.5,
+ "recall10": 0.1935483870967742,
+ "recall100": 0.22580645161290322
+ },
+ {
+ "qid": "PLAIN-1731",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1741",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.023809523809523808,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1752",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1762",
+ "ndcg10": 0.41253177989255346,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-1772",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1784",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1794",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-1805",
+ "ndcg10": 0.9305687780632227,
+ "mrr10": 1,
+ "recall10": 0.07758620689655173,
+ "recall100": 0.31896551724137934
+ },
+ {
+ "qid": "PLAIN-1817",
+ "ndcg10": 0.5326208815196265,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1827",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1837",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.05102040816326531,
+ "recall100": 0.3520408163265306
+ },
+ {
+ "qid": "PLAIN-1847",
+ "ndcg10": 0.2082936658103318,
+ "mrr10": 0.5,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-1857",
+ "ndcg10": 0.6371523797895856,
+ "mrr10": 1,
+ "recall10": 0.15151515151515152,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-1867",
+ "ndcg10": 0.27487633291429087,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-1877",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.05555555555555555
+ },
+ {
+ "qid": "PLAIN-1887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-1897",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-1909",
+ "ndcg10": 0.841492960259019,
+ "mrr10": 1,
+ "recall10": 0.017278617710583154,
+ "recall100": 0.11879049676025918
+ },
+ {
+ "qid": "PLAIN-1919",
+ "ndcg10": 0.727329844310522,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.7037037037037037
+ },
+ {
+ "qid": "PLAIN-1929",
+ "ndcg10": 0.22009176629808017,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.3
+ },
+ {
+ "qid": "PLAIN-1940",
+ "ndcg10": 0.3589542101716347,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-1950",
+ "ndcg10": 0.2489083270225946,
+ "mrr10": 0.5,
+ "recall10": 0.15384615384615385,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-1962",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-1972",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.36363636363636365
+ },
+ {
+ "qid": "PLAIN-1983",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 0.3125,
+ "recall100": 0.71875
+ },
+ {
+ "qid": "PLAIN-1995",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.18181818181818182
+ },
+ {
+ "qid": "PLAIN-2009",
+ "ndcg10": 0.10893649261775588,
+ "mrr10": 0.25,
+ "recall10": 0.125,
+ "recall100": 0.625
+ },
+ {
+ "qid": "PLAIN-2019",
+ "ndcg10": 0.46927872602275644,
+ "mrr10": 1,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2030",
+ "ndcg10": 0.30260241349881345,
+ "mrr10": 1,
+ "recall10": 0.16666666666666666,
+ "recall100": 0.3333333333333333
+ },
+ {
+ "qid": "PLAIN-2040",
+ "ndcg10": 0.5423640154200349,
+ "mrr10": 1,
+ "recall10": 0.030303030303030304,
+ "recall100": 0.15151515151515152
+ },
+ {
+ "qid": "PLAIN-2051",
+ "ndcg10": 0.7503359712972109,
+ "mrr10": 1,
+ "recall10": 0.01971830985915493,
+ "recall100": 0.11267605633802817
+ },
+ {
+ "qid": "PLAIN-2061",
+ "ndcg10": 0.9052116356304493,
+ "mrr10": 1,
+ "recall10": 0.02195121951219512,
+ "recall100": 0.15365853658536585
+ },
+ {
+ "qid": "PLAIN-2071",
+ "ndcg10": 0.5077181160122074,
+ "mrr10": 1,
+ "recall10": 0.08888888888888889,
+ "recall100": 0.4222222222222222
+ },
+ {
+ "qid": "PLAIN-2081",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2092",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2102",
+ "ndcg10": 0.8236998933965709,
+ "mrr10": 1,
+ "recall10": 0.019002375296912115,
+ "recall100": 0.12826603325415678
+ },
+ {
+ "qid": "PLAIN-2113",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2124",
+ "ndcg10": 0.7100226348397882,
+ "mrr10": 1,
+ "recall10": 0.4117647058823529,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-2134",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.08333333333333333
+ },
+ {
+ "qid": "PLAIN-2145",
+ "ndcg10": 0.1482972305701491,
+ "mrr10": 0.5,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2156",
+ "ndcg10": 0.644824486427155,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.8333333333333334
+ },
+ {
+ "qid": "PLAIN-2167",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2177",
+ "ndcg10": 0.4883491838252097,
+ "mrr10": 0.5,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2187",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2197",
+ "ndcg10": 0.9216017310213247,
+ "mrr10": 1,
+ "recall10": 0.15517241379310345,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2209",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2220",
+ "ndcg10": 0.38334277998463445,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.2222222222222222
+ },
+ {
+ "qid": "PLAIN-2230",
+ "ndcg10": 0.13743816645714543,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.14285714285714285
+ },
+ {
+ "qid": "PLAIN-2240",
+ "ndcg10": 0.642186726668901,
+ "mrr10": 1,
+ "recall10": 0.45454545454545453,
+ "recall100": 0.5454545454545454
+ },
+ {
+ "qid": "PLAIN-2250",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2261",
+ "ndcg10": 0.7183627972686238,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.18333333333333332
+ },
+ {
+ "qid": "PLAIN-2271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2291",
+ "ndcg10": 0.3391602052736161,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2301",
+ "ndcg10": 0.06943122193677727,
+ "mrr10": 0.125,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.23076923076923078
+ },
+ {
+ "qid": "PLAIN-2311",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2321",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2332",
+ "ndcg10": 0.5116508161747901,
+ "mrr10": 1,
+ "recall10": 0.05154639175257732,
+ "recall100": 0.3402061855670103
+ },
+ {
+ "qid": "PLAIN-2343",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2354",
+ "ndcg10": 0.5531464700081437,
+ "mrr10": 1,
+ "recall10": 0.4,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2364",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2375",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2386",
+ "ndcg10": 0.11004588314904008,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.05263157894736842,
+ "recall100": 0.3684210526315789
+ },
+ {
+ "qid": "PLAIN-2396",
+ "ndcg10": 0.13886244387355454,
+ "mrr10": 0.5,
+ "recall10": 0.05555555555555555,
+ "recall100": 0.1111111111111111
+ },
+ {
+ "qid": "PLAIN-2408",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2430",
+ "ndcg10": 0.45374257454118544,
+ "mrr10": 1,
+ "recall10": 0.06521739130434782,
+ "recall100": 0.391304347826087
+ },
+ {
+ "qid": "PLAIN-2440",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2450",
+ "ndcg10": 0.29990503169749433,
+ "mrr10": 0.5,
+ "recall10": 0.07894736842105263,
+ "recall100": 0.18421052631578946
+ },
+ {
+ "qid": "PLAIN-2460",
+ "ndcg10": 0.08649891544052932,
+ "mrr10": 0.1111111111111111,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.3225806451612903
+ },
+ {
+ "qid": "PLAIN-2470",
+ "ndcg10": 0.6464260278126966,
+ "mrr10": 1,
+ "recall10": 0.0547945205479452,
+ "recall100": 0.3287671232876712
+ },
+ {
+ "qid": "PLAIN-2480",
+ "ndcg10": 0.3534433112980961,
+ "mrr10": 1,
+ "recall10": 0.2222222222222222,
+ "recall100": 0.6111111111111112
+ },
+ {
+ "qid": "PLAIN-2490",
+ "ndcg10": 0.31248003685671516,
+ "mrr10": 0.5,
+ "recall10": 0.13043478260869565,
+ "recall100": 0.34782608695652173
+ },
+ {
+ "qid": "PLAIN-2500",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-2510",
+ "ndcg10": 0.7241528458281337,
+ "mrr10": 1,
+ "recall10": 0.15217391304347827,
+ "recall100": 0.5869565217391305
+ },
+ {
+ "qid": "PLAIN-2520",
+ "ndcg10": 0.03668196104968002,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.0136986301369863,
+ "recall100": 0.1232876712328767
+ },
+ {
+ "qid": "PLAIN-2530",
+ "ndcg10": 0.9069212362975007,
+ "mrr10": 1,
+ "recall10": 0.1388888888888889,
+ "recall100": 0.3611111111111111
+ },
+ {
+ "qid": "PLAIN-2540",
+ "ndcg10": 0.31579804414741475,
+ "mrr10": 0.5,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.5714285714285714
+ },
+ {
+ "qid": "PLAIN-2550",
+ "ndcg10": 0.10405896756643919,
+ "mrr10": 0.2,
+ "recall10": 0.04,
+ "recall100": 0.24
+ },
+ {
+ "qid": "PLAIN-2560",
+ "ndcg10": 0.8701249883466592,
+ "mrr10": 1,
+ "recall10": 0.38095238095238093,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2570",
+ "ndcg10": 0.16014043519656038,
+ "mrr10": 0.5,
+ "recall10": 0.025,
+ "recall100": 0.475
+ },
+ {
+ "qid": "PLAIN-2580",
+ "ndcg10": 0.41235173674195585,
+ "mrr10": 1,
+ "recall10": 0.08695652173913043,
+ "recall100": 0.17391304347826086
+ },
+ {
+ "qid": "PLAIN-2590",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-2600",
+ "ndcg10": 0.5991689570583907,
+ "mrr10": 1,
+ "recall10": 0.3,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2610",
+ "ndcg10": 0.3746661058764784,
+ "mrr10": 1,
+ "recall10": 0.04918032786885246,
+ "recall100": 0.19672131147540983
+ },
+ {
+ "qid": "PLAIN-2620",
+ "ndcg10": 0.24327815416627566,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.07272727272727272,
+ "recall100": 0.4
+ },
+ {
+ "qid": "PLAIN-2630",
+ "ndcg10": 0.6772733739526907,
+ "mrr10": 1,
+ "recall10": 0.125,
+ "recall100": 0.4791666666666667
+ },
+ {
+ "qid": "PLAIN-2640",
+ "ndcg10": 0.5107640114919196,
+ "mrr10": 1,
+ "recall10": 0.20833333333333334,
+ "recall100": 0.5833333333333334
+ },
+ {
+ "qid": "PLAIN-2650",
+ "ndcg10": 0.46318965958284924,
+ "mrr10": 1,
+ "recall10": 0.13793103448275862,
+ "recall100": 0.3793103448275862
+ },
+ {
+ "qid": "PLAIN-2660",
+ "ndcg10": 0.4518648409659427,
+ "mrr10": 1,
+ "recall10": 0.14814814814814814,
+ "recall100": 0.5185185185185185
+ },
+ {
+ "qid": "PLAIN-2670",
+ "ndcg10": 0.3406992958233561,
+ "mrr10": 1,
+ "recall10": 0.02857142857142857,
+ "recall100": 0.05714285714285714
+ },
+ {
+ "qid": "PLAIN-2680",
+ "ndcg10": 0.5075417611009456,
+ "mrr10": 1,
+ "recall10": 0.21428571428571427,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2690",
+ "ndcg10": 0.3764251307383051,
+ "mrr10": 1,
+ "recall10": 0.06976744186046512,
+ "recall100": 0.37209302325581395
+ },
+ {
+ "qid": "PLAIN-2700",
+ "ndcg10": 0.4391834785971612,
+ "mrr10": 1,
+ "recall10": 0.2,
+ "recall100": 0.6
+ },
+ {
+ "qid": "PLAIN-2710",
+ "ndcg10": 0.6812457182571504,
+ "mrr10": 1,
+ "recall10": 0.23809523809523808,
+ "recall100": 0.47619047619047616
+ },
+ {
+ "qid": "PLAIN-2720",
+ "ndcg10": 0.3995688713838975,
+ "mrr10": 1,
+ "recall10": 0.08333333333333333,
+ "recall100": 0.3611111111111111
+ },
+ {
+ "qid": "PLAIN-2730",
+ "ndcg10": 0.6077697019194394,
+ "mrr10": 0.5,
+ "recall10": 0.20689655172413793,
+ "recall100": 0.3448275862068966
+ },
+ {
+ "qid": "PLAIN-2740",
+ "ndcg10": 0.6155267282106165,
+ "mrr10": 1,
+ "recall10": 0.1,
+ "recall100": 0.24
+ },
+ {
+ "qid": "PLAIN-2750",
+ "ndcg10": 0.5987514293054288,
+ "mrr10": 1,
+ "recall10": 0.13559322033898305,
+ "recall100": 0.3389830508474576
+ },
+ {
+ "qid": "PLAIN-2760",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2770",
+ "ndcg10": 0.330622400869393,
+ "mrr10": 1,
+ "recall10": 0.07317073170731707,
+ "recall100": 0.5121951219512195
+ },
+ {
+ "qid": "PLAIN-2780",
+ "ndcg10": 0.3663797094518679,
+ "mrr10": 1,
+ "recall10": 0.18181818181818182,
+ "recall100": 0.3181818181818182
+ },
+ {
+ "qid": "PLAIN-2790",
+ "ndcg10": 0.6489315753318465,
+ "mrr10": 1,
+ "recall10": 0.0847457627118644,
+ "recall100": 0.2542372881355932
+ },
+ {
+ "qid": "PLAIN-2800",
+ "ndcg10": 0.14673120006323176,
+ "mrr10": 0.25,
+ "recall10": 0.041666666666666664,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-2810",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.25
+ },
+ {
+ "qid": "PLAIN-2830",
+ "ndcg10": 0.2809436351950179,
+ "mrr10": 0.2,
+ "recall10": 0.3333333333333333,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-2840",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.058823529411764705
+ },
+ {
+ "qid": "PLAIN-2850",
+ "ndcg10": 0.04621713645665386,
+ "mrr10": 0.1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.4444444444444444
+ },
+ {
+ "qid": "PLAIN-2860",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-2870",
+ "ndcg10": 0.10104155414609708,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.2,
+ "recall100": 0.2
+ },
+ {
+ "qid": "PLAIN-2880",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.11764705882352941
+ },
+ {
+ "qid": "PLAIN-2890",
+ "ndcg10": 0.13252641783286837,
+ "mrr10": 0.1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.034482758620689655
+ },
+ {
+ "qid": "PLAIN-2900",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-2910",
+ "ndcg10": 0.24249682153453903,
+ "mrr10": 0.5,
+ "recall10": 0.06666666666666667,
+ "recall100": 0.26666666666666666
+ },
+ {
+ "qid": "PLAIN-2920",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2930",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2940",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.22727272727272727
+ },
+ {
+ "qid": "PLAIN-2950",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-2960",
+ "ndcg10": 0.35583989829307827,
+ "mrr10": 0.25,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-2970",
+ "ndcg10": 0.47965506954306864,
+ "mrr10": 1,
+ "recall10": 0.1111111111111111,
+ "recall100": 0.5555555555555556
+ },
+ {
+ "qid": "PLAIN-2981",
+ "ndcg10": 0.5321036168710066,
+ "mrr10": 1,
+ "recall10": 0.14285714285714285,
+ "recall100": 0.42857142857142855
+ },
+ {
+ "qid": "PLAIN-2991",
+ "ndcg10": 0.3868528072345416,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3001",
+ "ndcg10": 0.3868528072345416,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3014",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3026",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3037",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3053",
+ "ndcg10": 0.7653606369886218,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3063",
+ "ndcg10": 0.15101961822780524,
+ "mrr10": 0.2,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3074",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3085",
+ "ndcg10": 0.23719771276929622,
+ "mrr10": 0.2,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3097",
+ "ndcg10": 0.515157140670655,
+ "mrr10": 1,
+ "recall10": 0.25,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3116",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3131",
+ "ndcg10": 0.6516433935264055,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3141",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.20588235294117646
+ },
+ {
+ "qid": "PLAIN-3151",
+ "ndcg10": 0.519838015658837,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.3888888888888889,
+ "recall100": 0.6666666666666666
+ },
+ {
+ "qid": "PLAIN-3161",
+ "ndcg10": 0.5486455668071759,
+ "mrr10": 1,
+ "recall10": 0.06976744186046512,
+ "recall100": 0.32558139534883723
+ },
+ {
+ "qid": "PLAIN-3171",
+ "ndcg10": 0.04820998182892457,
+ "mrr10": 0.125,
+ "recall10": 0.029411764705882353,
+ "recall100": 0.14705882352941177
+ },
+ {
+ "qid": "PLAIN-3181",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.07407407407407407
+ },
+ {
+ "qid": "PLAIN-3191",
+ "ndcg10": 0.2204386078501931,
+ "mrr10": 0.5,
+ "recall10": 0.125,
+ "recall100": 0.5
+ },
+ {
+ "qid": "PLAIN-3201",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.03125,
+ "recall100": 0.09375
+ },
+ {
+ "qid": "PLAIN-3211",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "PLAIN-3221",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.16666666666666666
+ },
+ {
+ "qid": "PLAIN-3231",
+ "ndcg10": 0.14810444518925672,
+ "mrr10": 0.16666666666666666,
+ "recall10": 0.125,
+ "recall100": 0.75
+ },
+ {
+ "qid": "PLAIN-3241",
+ "ndcg10": 0.45846608015576257,
+ "mrr10": 1,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.10344827586206896
+ },
+ {
+ "qid": "PLAIN-3251",
+ "ndcg10": 0.24162606600584033,
+ "mrr10": 0.25,
+ "recall10": 0.09523809523809523,
+ "recall100": 0.09523809523809523
+ },
+ {
+ "qid": "PLAIN-3261",
+ "ndcg10": 0.17034964791167806,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.15384615384615385
+ },
+ {
+ "qid": "PLAIN-3271",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.10204081632653061
+ },
+ {
+ "qid": "PLAIN-3281",
+ "ndcg10": 0.4954499631132973,
+ "mrr10": 1,
+ "recall10": 0.07692307692307693,
+ "recall100": 0.11538461538461539
+ },
+ {
+ "qid": "PLAIN-3292",
+ "ndcg10": 0.5146983300029805,
+ "mrr10": 0.5,
+ "recall10": 0.7142857142857143,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3302",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.35135135135135137
+ },
+ {
+ "qid": "PLAIN-3312",
+ "ndcg10": 0.23741542281870187,
+ "mrr10": 0.5,
+ "recall10": 0.38461538461538464,
+ "recall100": 0.6923076923076923
+ },
+ {
+ "qid": "PLAIN-3322",
+ "ndcg10": 0.06943122193677727,
+ "mrr10": 0.125,
+ "recall10": 0.012658227848101266,
+ "recall100": 0.13924050632911392
+ },
+ {
+ "qid": "PLAIN-3332",
+ "ndcg10": 0.6034115249626371,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.8571428571428571
+ },
+ {
+ "qid": "PLAIN-3342",
+ "ndcg10": 0.8072467605676947,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "PLAIN-3352",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.15384615384615385
+ },
+ {
+ "qid": "PLAIN-3362",
+ "ndcg10": 0.6573548268593187,
+ "mrr10": 1,
+ "recall10": 0.2727272727272727,
+ "recall100": 0.2727272727272727
+ },
+ {
+ "qid": "PLAIN-3372",
+ "ndcg10": 0.1446299454867737,
+ "mrr10": 0.125,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3382",
+ "ndcg10": 0.4575997404793817,
+ "mrr10": 1,
+ "recall10": 0.35714285714285715,
+ "recall100": 0.7142857142857143
+ },
+ {
+ "qid": "PLAIN-3392",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.043478260869565216
+ },
+ {
+ "qid": "PLAIN-3402",
+ "ndcg10": 0.06362078819895171,
+ "mrr10": 0.1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.07142857142857142
+ },
+ {
+ "qid": "PLAIN-3412",
+ "ndcg10": 0.12811612289210494,
+ "mrr10": 1,
+ "recall10": 0.02040816326530612,
+ "recall100": 0.24489795918367346
+ },
+ {
+ "qid": "PLAIN-3422",
+ "ndcg10": 0.03459113773405908,
+ "mrr10": 0.25,
+ "recall10": 0.03225806451612903,
+ "recall100": 0.5161290322580645
+ },
+ {
+ "qid": "PLAIN-3432",
+ "ndcg10": 0.3932783996463063,
+ "mrr10": 1,
+ "recall10": 0.07142857142857142,
+ "recall100": 0.21428571428571427
+ },
+ {
+ "qid": "PLAIN-3442",
+ "ndcg10": 0.051723574529084305,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.034482758620689655,
+ "recall100": 0.06896551724137931
+ },
+ {
+ "qid": "PLAIN-3452",
+ "ndcg10": 0.5844153148080933,
+ "mrr10": 1,
+ "recall10": 0.23529411764705882,
+ "recall100": 0.5294117647058824
+ },
+ {
+ "qid": "PLAIN-3462",
+ "ndcg10": 0.6975961695061491,
+ "mrr10": 1,
+ "recall10": 0.109375,
+ "recall100": 0.515625
+ },
+ {
+ "qid": "PLAIN-3472",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0.35294117647058826
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.325,
+ "DocT5query": 0.328,
+ "TAS-B": 0.319,
+ "GenQ": 0.319,
+ "ColBERT": 0.305,
+ "Contriever": 0.328,
+ "GTR-XL": 0.343,
+ "SPLADE++": 0.347,
+ "BGE-large-v1.5 (pub)": 0.38,
+ "SBERT msmarco": 0.272
+ },
+ "ourRank": 2,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-2026-05-30T22-11-24-625Z.json b/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-2026-05-30T22-11-24-625Z.json
new file mode 100644
index 0000000000..c13700ec01
--- /dev/null
+++ b/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-2026-05-30T22-11-24-625Z.json
@@ -0,0 +1,34 @@
+{
+ "runAt": "2026-05-30T22:11:24.625Z",
+ "benchmark": "beir-nfcorpus-lucene-bm25",
+ "dataset": "nfcorpus",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 323,
+ "results": {
+ "dense": {
+ "ndcg10": 0.35171471732484655,
+ "recall10": 0.16652743860112934,
+ "recall100": 0.30466713166229387
+ },
+ "bm25": {
+ "ndcg10": 0.32804214825224787,
+ "recall10": 0.15194576018440764,
+ "recall100": 0.24835804231262074
+ },
+ "rrf60": {
+ "ndcg10": 0.35975310333572946,
+ "recall10": 0.1697252920014747,
+ "recall100": 0.3274132718588512
+ },
+ "rrf30": {
+ "ndcg10": 0.36262911020368416,
+ "recall10": 0.1717912580780287,
+ "recall100": 0.3267214015273048
+ }
+ },
+ "baselines": {
+ "BM25": 0.325,
+ "BGE-large-pub": 0.38,
+ "SPLADE++": 0.347
+ }
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-latest.json b/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-latest.json
new file mode 100644
index 0000000000..c13700ec01
--- /dev/null
+++ b/docs/benchmarks/runs/beir-nfcorpus-lucene-bm25-latest.json
@@ -0,0 +1,34 @@
+{
+ "runAt": "2026-05-30T22:11:24.625Z",
+ "benchmark": "beir-nfcorpus-lucene-bm25",
+ "dataset": "nfcorpus",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 323,
+ "results": {
+ "dense": {
+ "ndcg10": 0.35171471732484655,
+ "recall10": 0.16652743860112934,
+ "recall100": 0.30466713166229387
+ },
+ "bm25": {
+ "ndcg10": 0.32804214825224787,
+ "recall10": 0.15194576018440764,
+ "recall100": 0.24835804231262074
+ },
+ "rrf60": {
+ "ndcg10": 0.35975310333572946,
+ "recall10": 0.1697252920014747,
+ "recall100": 0.3274132718588512
+ },
+ "rrf30": {
+ "ndcg10": 0.36262911020368416,
+ "recall10": 0.1717912580780287,
+ "recall100": 0.3267214015273048
+ }
+ },
+ "baselines": {
+ "BM25": 0.325,
+ "BGE-large-pub": 0.38,
+ "SPLADE++": 0.347
+ }
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T22-29-06-692Z.json b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T22-29-06-692Z.json
new file mode 100644
index 0000000000..89010c80eb
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T22-29-06-692Z.json
@@ -0,0 +1,2134 @@
+{
+ "runAt": "2026-05-30T22:29:06.692Z",
+ "benchmark": "beir-scifact-hybrid-rerank",
+ "dataset": "scifact",
+ "pipeline": "BM25+BGE RRF+CE-rerank",
+ "bgeModel": "Xenova/bge-base-en-v1.5",
+ "rrfK": 60,
+ "rerank": true,
+ "queries": 300,
+ "corpusSize": 5183,
+ "metrics": {
+ "ndcg10": 0.685364866962713,
+ "mrr10": 0.6537380952380951,
+ "recall10": 0.8072222222222222,
+ "recall100": 0.9543333333333334,
+ "avgQueryLatencyMs": 5928.52762597
+ },
+ "perQuery": [
+ {
+ "qid": "1",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "3",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "5",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "13",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "36",
+ "ndcg10": 0.3889580943680543,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "42",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "48",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "49",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "50",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "51",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "53",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "54",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "56",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "57",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "70",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "72",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "75",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "94",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "99",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "100",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "113",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "115",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "118",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "124",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "127",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "128",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "129",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "132",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "133",
+ "ndcg10": 0.5147714448836774,
+ "mrr10": 0.5,
+ "recall10": 0.6,
+ "recall100": 0.8
+ },
+ {
+ "qid": "137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "141",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "142",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "143",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "148",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "163",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "171",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "179",
+ "ndcg10": 0.9052181617170642,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "183",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "185",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "198",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "208",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "212",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "217",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "218",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "219",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "230",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "232",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "233",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "236",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "237",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "238",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "239",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "248",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "249",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "261",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 0.5
+ },
+ {
+ "qid": "268",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "269",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "274",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "275",
+ "ndcg10": 0.6978817289434457,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "294",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "295",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "300",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "303",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "312",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "314",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "324",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "327",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "338",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "343",
+ "ndcg10": 0.6934264036172708,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "350",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "354",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "362",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "380",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "384",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "385",
+ "ndcg10": 0.6509209298071326,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "386",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "388",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "399",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "410",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "411",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "415",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "421",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "431",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "436",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "437",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "439",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "440",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "443",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "452",
+ "ndcg10": 0.5109559939712153,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "475",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "478",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "491",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "501",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "502",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "507",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "508",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "513",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "514",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "516",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "517",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "521",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "525",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "527",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "528",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "532",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "533",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "535",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "536",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "539",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "540",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "544",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "549",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "551",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "552",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "554",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "560",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "569",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "575",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "577",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "578",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "587",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "589",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "593",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "597",
+ "ndcg10": 0.8179809735263114,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "598",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "613",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "619",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "623",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "628",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "636",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "637",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "641",
+ "ndcg10": 0.8065735963827293,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "644",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "649",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "659",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "660",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "674",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "684",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "690",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "691",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "692",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "693",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "700",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "702",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "715",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "716",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "718",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "723",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "727",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "728",
+ "ndcg10": 0.7977228895450266,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "729",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "742",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "743",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "744",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "756",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "759",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "768",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "770",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "775",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "781",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "783",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "784",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "785",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "793",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "800",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "805",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "808",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "811",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "814",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "821",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "823",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "830",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "831",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "832",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "834",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "837",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "839",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "845",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "847",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "852",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "859",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "870",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "873",
+ "ndcg10": 0.252016380211836,
+ "mrr10": 0.2,
+ "recall10": 0.4,
+ "recall100": 1
+ },
+ {
+ "qid": "879",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "880",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "882",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "903",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "904",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "907",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "911",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "913",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "914",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "921",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "922",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "936",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "956",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "957",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "960",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "967",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "971",
+ "ndcg10": 0.9047172294870751,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "975",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "982",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "985",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "993",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1012",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1014",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1019",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1020",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1021",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1024",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1029",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "1041",
+ "ndcg10": 0.20438239758848611,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1049",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1062",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1086",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1088",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1089",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1099",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1100",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1104",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1107",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1110",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1121",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1132",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1140",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1144",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1150",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1175",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1179",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1185",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1187",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1191",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1194",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1196",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1197",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1199",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1200",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1202",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1204",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1207",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1221",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1225",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1226",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1241",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1245",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1259",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1262",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1266",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1270",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1271",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1272",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1273",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1274",
+ "ndcg10": 0.9325210919548239,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1278",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1279",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1280",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1282",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1290",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1292",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1303",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1316",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1319",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1320",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1332",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1335",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1336",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1337",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1339",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1344",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1352",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1359",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1362",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1363",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1368",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1370",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1379",
+ "ndcg10": 0.8326783988613894,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1382",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1385",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1389",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1395",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.679,
+ "DocT5query": 0.675,
+ "TAS-B": 0.643,
+ "GenQ": 0.644,
+ "ColBERT": 0.671,
+ "Contriever": 0.677,
+ "GTR-XL": 0.662,
+ "SPLADE++": 0.704,
+ "BGE-large-v1.5 (pub)": 0.722,
+ "SBERT msmarco": 0.555
+ },
+ "ourRank": 3,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T23-18-36-377Z.json b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T23-18-36-377Z.json
new file mode 100644
index 0000000000..216dfd90bf
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-2026-05-30T23-18-36-377Z.json
@@ -0,0 +1,2134 @@
+{
+ "runAt": "2026-05-30T23:18:36.377Z",
+ "benchmark": "beir-scifact-hybrid-rerank",
+ "dataset": "scifact",
+ "pipeline": "BM25+BGE RRF+CE-rerank",
+ "bgeModel": "Xenova/bge-base-en-v1.5",
+ "rrfK": 60,
+ "rerank": true,
+ "queries": 300,
+ "corpusSize": 5183,
+ "metrics": {
+ "ndcg10": 0.6831249340066251,
+ "mrr10": 0.6516164021164021,
+ "recall10": 0.8055555555555555,
+ "recall100": 0.96,
+ "avgQueryLatencyMs": 4549.203705416667
+ },
+ "perQuery": [
+ {
+ "qid": "1",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "3",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "5",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "13",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "36",
+ "ndcg10": 0.38162168437253385,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "42",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "48",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "49",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "50",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "51",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "53",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "54",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "56",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "57",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "70",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "72",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "75",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "94",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "99",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "100",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "113",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "115",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "118",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "124",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "127",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "128",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "129",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "132",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "133",
+ "ndcg10": 0.5147714448836774,
+ "mrr10": 0.5,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "141",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "142",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "143",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "148",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "163",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "171",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "179",
+ "ndcg10": 0.9052181617170642,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "183",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "185",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "198",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "208",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "212",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "217",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "218",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "219",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "230",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "232",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "233",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "236",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "237",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "238",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "239",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "248",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "249",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "261",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "268",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "269",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "274",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "275",
+ "ndcg10": 0.6978817289434457,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "294",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "295",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "300",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "303",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "312",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "314",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "324",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "327",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "338",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "343",
+ "ndcg10": 0.5706417189553201,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "350",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "354",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "362",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "380",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "384",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "385",
+ "ndcg10": 0.6509209298071326,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "386",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "388",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "399",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "410",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "411",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "415",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "421",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "431",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "436",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "437",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "439",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "440",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "443",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "452",
+ "ndcg10": 0.5109559939712153,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "475",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "478",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "491",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "501",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "502",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "507",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "508",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "513",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "514",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "516",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "517",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "521",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "525",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "527",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "528",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "532",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "533",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "535",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "536",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "539",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "540",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "544",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "549",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "551",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "552",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "554",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "560",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "569",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "575",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "577",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "578",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "587",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "589",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "593",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "597",
+ "ndcg10": 0.8179809735263114,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "598",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "613",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "619",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "623",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "628",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "636",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "637",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "641",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "644",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "649",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "659",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "660",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "674",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "684",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "690",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "691",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "692",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "693",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "700",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "702",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "715",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "716",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "718",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "723",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "727",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "728",
+ "ndcg10": 0.7977228895450266,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "729",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "742",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "743",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "744",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "756",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "759",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "768",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "770",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "775",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "781",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "783",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "784",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "785",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "793",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "800",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "805",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "808",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "811",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "814",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "821",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "823",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "830",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "831",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "832",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "834",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "837",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "839",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "845",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "847",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "852",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "859",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "870",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "873",
+ "ndcg10": 0.252016380211836,
+ "mrr10": 0.2,
+ "recall10": 0.4,
+ "recall100": 1
+ },
+ {
+ "qid": "879",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "880",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "882",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "903",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "904",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "907",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "911",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "913",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "914",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "921",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "922",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "936",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "956",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "957",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "960",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "967",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "971",
+ "ndcg10": 0.9047172294870751,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "975",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "982",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "985",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "993",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1012",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1014",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1019",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1020",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1021",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1024",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1029",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "1041",
+ "ndcg10": 0.20438239758848611,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1049",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1062",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1086",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1088",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1089",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1099",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1100",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1104",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1107",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1110",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1121",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1132",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1140",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1144",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1150",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1175",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1179",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1185",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1187",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1191",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1194",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1196",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1197",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1199",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1200",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1202",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1204",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1207",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1221",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1225",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1226",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1241",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1245",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1259",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1262",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1266",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1270",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1271",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1272",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1273",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1274",
+ "ndcg10": 0.9325210919548239,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1278",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1279",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1280",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1282",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1290",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1292",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1303",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1316",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1319",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1320",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1332",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1335",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1336",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1337",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1339",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1344",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1352",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1359",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1362",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1363",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1368",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1370",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1379",
+ "ndcg10": 0.8326783988613894,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1382",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1385",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1389",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1395",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.679,
+ "DocT5query": 0.675,
+ "TAS-B": 0.643,
+ "GenQ": 0.644,
+ "ColBERT": 0.671,
+ "Contriever": 0.677,
+ "GTR-XL": 0.662,
+ "SPLADE++": 0.704,
+ "BGE-large-v1.5 (pub)": 0.722,
+ "SBERT msmarco": 0.555
+ },
+ "ourRank": 3,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-hybrid-rerank-latest.json b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-latest.json
new file mode 100644
index 0000000000..216dfd90bf
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-hybrid-rerank-latest.json
@@ -0,0 +1,2134 @@
+{
+ "runAt": "2026-05-30T23:18:36.377Z",
+ "benchmark": "beir-scifact-hybrid-rerank",
+ "dataset": "scifact",
+ "pipeline": "BM25+BGE RRF+CE-rerank",
+ "bgeModel": "Xenova/bge-base-en-v1.5",
+ "rrfK": 60,
+ "rerank": true,
+ "queries": 300,
+ "corpusSize": 5183,
+ "metrics": {
+ "ndcg10": 0.6831249340066251,
+ "mrr10": 0.6516164021164021,
+ "recall10": 0.8055555555555555,
+ "recall100": 0.96,
+ "avgQueryLatencyMs": 4549.203705416667
+ },
+ "perQuery": [
+ {
+ "qid": "1",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "3",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "5",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "13",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "36",
+ "ndcg10": 0.38162168437253385,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "42",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "48",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "49",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "50",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "51",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "53",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "54",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "56",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "57",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "70",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "72",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "75",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "94",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "99",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "100",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "113",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "115",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "118",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "124",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "127",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "128",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "129",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "132",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "133",
+ "ndcg10": 0.5147714448836774,
+ "mrr10": 0.5,
+ "recall10": 0.6,
+ "recall100": 1
+ },
+ {
+ "qid": "137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "141",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "142",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "143",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "148",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "163",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "171",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "179",
+ "ndcg10": 0.9052181617170642,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "183",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "185",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "198",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "208",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "212",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "217",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "218",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "219",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "230",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "232",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "233",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "236",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "237",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "238",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "239",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "248",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "249",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "261",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "268",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "269",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "274",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "275",
+ "ndcg10": 0.6978817289434457,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "279",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "294",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "295",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "300",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "303",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "312",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "314",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "324",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "327",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "338",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "343",
+ "ndcg10": 0.5706417189553201,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "350",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "354",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "362",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "380",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "384",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "385",
+ "ndcg10": 0.6509209298071326,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "386",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "388",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "399",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "410",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "411",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "415",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "421",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "431",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "436",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "437",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "439",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "440",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "443",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "452",
+ "ndcg10": 0.5109559939712153,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "475",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "478",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "491",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "501",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "502",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "507",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "508",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "513",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "514",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "516",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "517",
+ "ndcg10": 0.31546487678572877,
+ "mrr10": 0.125,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "521",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "525",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "527",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "528",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "532",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "533",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "535",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "536",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "539",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "540",
+ "ndcg10": 0.3065735963827292,
+ "mrr10": 0.3333333333333333,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "544",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "549",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "551",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "552",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "554",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "560",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "569",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "575",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "577",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "578",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "587",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "589",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "593",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "597",
+ "ndcg10": 0.8179809735263114,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "598",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "613",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "619",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.5,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "623",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "628",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "636",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "637",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "641",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "644",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "649",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "659",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "660",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "674",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "684",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "690",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "691",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "692",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "693",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "700",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "702",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "715",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "716",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "718",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "721",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "723",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "727",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "728",
+ "ndcg10": 0.7977228895450266,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "729",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "742",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "743",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "744",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "756",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "759",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "768",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "770",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "775",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "781",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "783",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "784",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "785",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "793",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "800",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "805",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "808",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "811",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "814",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "820",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "821",
+ "ndcg10": 0.3010299956639812,
+ "mrr10": 0.1111111111111111,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "823",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "830",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "831",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "832",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "834",
+ "ndcg10": 0.3333333333333333,
+ "mrr10": 0.14285714285714285,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "837",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "839",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "845",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "847",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "852",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "859",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "870",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "873",
+ "ndcg10": 0.252016380211836,
+ "mrr10": 0.2,
+ "recall10": 0.4,
+ "recall100": 1
+ },
+ {
+ "qid": "879",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "880",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "882",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "887",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "903",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "904",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "907",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "911",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "913",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "914",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "921",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "922",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "936",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "956",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "957",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "960",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "967",
+ "ndcg10": 0.9197207891481876,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "971",
+ "ndcg10": 0.9047172294870751,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "975",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "982",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "985",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "993",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1012",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1014",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1019",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1020",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1021",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1024",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1029",
+ "ndcg10": 0.7653606369886217,
+ "mrr10": 1,
+ "recall10": 0.6666666666666666,
+ "recall100": 1
+ },
+ {
+ "qid": "1041",
+ "ndcg10": 0.20438239758848611,
+ "mrr10": 0.14285714285714285,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1049",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1062",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1086",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1088",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1089",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1099",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1100",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1104",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1107",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1110",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1121",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1130",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1132",
+ "ndcg10": 0.6131471927654584,
+ "mrr10": 1,
+ "recall10": 0.5,
+ "recall100": 1
+ },
+ {
+ "qid": "1137",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1140",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1144",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1146",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1150",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1163",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1175",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1179",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1180",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1185",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1187",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1191",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1194",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1196",
+ "ndcg10": 0.38685280723454163,
+ "mrr10": 0.2,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1197",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1199",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1200",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1202",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1204",
+ "ndcg10": 0.3562071871080222,
+ "mrr10": 0.16666666666666666,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1207",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1213",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 0
+ },
+ {
+ "qid": "1216",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1221",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1225",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1226",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1232",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1241",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1245",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1259",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1262",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1266",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1270",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1271",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1272",
+ "ndcg10": 0.43067655807339306,
+ "mrr10": 0.25,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1273",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1274",
+ "ndcg10": 0.9325210919548239,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1278",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1279",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1280",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1281",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1282",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1290",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1292",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1298",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1303",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1316",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1319",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1320",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1332",
+ "ndcg10": 0,
+ "mrr10": 0,
+ "recall10": 0,
+ "recall100": 1
+ },
+ {
+ "qid": "1335",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1336",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1337",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1339",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1344",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1352",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1359",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1362",
+ "ndcg10": 0.2890648263178879,
+ "mrr10": 0.1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1363",
+ "ndcg10": 0.5,
+ "mrr10": 0.3333333333333333,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1368",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1370",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1379",
+ "ndcg10": 0.8326783988613894,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1382",
+ "ndcg10": 0.6309297535714575,
+ "mrr10": 0.5,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1385",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1389",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ },
+ {
+ "qid": "1395",
+ "ndcg10": 1,
+ "mrr10": 1,
+ "recall10": 1,
+ "recall100": 1
+ }
+ ],
+ "baselines": {
+ "BM25 (Lucene)": 0.679,
+ "DocT5query": 0.675,
+ "TAS-B": 0.643,
+ "GenQ": 0.644,
+ "ColBERT": 0.671,
+ "Contriever": 0.677,
+ "GTR-XL": 0.662,
+ "SPLADE++": 0.704,
+ "BGE-large-v1.5 (pub)": 0.722,
+ "SBERT msmarco": 0.555
+ },
+ "ourRank": 3,
+ "leaderboardLength": 11
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-lucene-bm25-2026-05-30T22-16-46-470Z.json b/docs/benchmarks/runs/beir-scifact-lucene-bm25-2026-05-30T22-16-46-470Z.json
new file mode 100644
index 0000000000..a1c7dd9318
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-lucene-bm25-2026-05-30T22-16-46-470Z.json
@@ -0,0 +1,34 @@
+{
+ "runAt": "2026-05-30T22:16:46.470Z",
+ "benchmark": "beir-scifact-lucene-bm25",
+ "dataset": "scifact",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 300,
+ "results": {
+ "dense": {
+ "ndcg10": 0.6255647791860727,
+ "recall10": 0.7412222222222221,
+ "recall100": 0.8283333333333334
+ },
+ "bm25": {
+ "ndcg10": 0.6812206768069344,
+ "recall10": 0.8245,
+ "recall100": 0.9193333333333333
+ },
+ "rrf60": {
+ "ndcg10": 0.6314940370709233,
+ "recall10": 0.7477222222222222,
+ "recall100": 0.9633333333333334
+ },
+ "rrf30": {
+ "ndcg10": 0.6391655674993573,
+ "recall10": 0.7702222222222223,
+ "recall100": 0.9633333333333334
+ }
+ },
+ "baselines": {
+ "BM25": 0.679,
+ "BGE-large-pub": 0.722,
+ "SPLADE++": 0.704
+ }
+}
\ No newline at end of file
diff --git a/docs/benchmarks/runs/beir-scifact-lucene-bm25-latest.json b/docs/benchmarks/runs/beir-scifact-lucene-bm25-latest.json
new file mode 100644
index 0000000000..a1c7dd9318
--- /dev/null
+++ b/docs/benchmarks/runs/beir-scifact-lucene-bm25-latest.json
@@ -0,0 +1,34 @@
+{
+ "runAt": "2026-05-30T22:16:46.470Z",
+ "benchmark": "beir-scifact-lucene-bm25",
+ "dataset": "scifact",
+ "model": "Xenova/bge-base-en-v1.5",
+ "queries": 300,
+ "results": {
+ "dense": {
+ "ndcg10": 0.6255647791860727,
+ "recall10": 0.7412222222222221,
+ "recall100": 0.8283333333333334
+ },
+ "bm25": {
+ "ndcg10": 0.6812206768069344,
+ "recall10": 0.8245,
+ "recall100": 0.9193333333333333
+ },
+ "rrf60": {
+ "ndcg10": 0.6314940370709233,
+ "recall10": 0.7477222222222222,
+ "recall100": 0.9633333333333334
+ },
+ "rrf30": {
+ "ndcg10": 0.6391655674993573,
+ "recall10": 0.7702222222222223,
+ "recall100": 0.9633333333333334
+ }
+ },
+ "baselines": {
+ "BM25": 0.679,
+ "BGE-large-pub": 0.722,
+ "SPLADE++": 0.704
+ }
+}
\ No newline at end of file
diff --git a/package-lock.json b/package-lock.json
index dfd8ff6687..998c062bc3 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "claude-flow",
- "version": "3.10.27",
+ "version": "3.10.28",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "claude-flow",
- "version": "3.10.27",
+ "version": "3.10.28",
"license": "MIT",
"dependencies": {
"@claude-flow/cli-core": "^3.7.0-alpha.5",
diff --git a/package.json b/package.json
index 45657e40b1..66b925a0af 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "claude-flow",
- "version": "3.10.27",
+ "version": "3.10.28",
"description": "Ruflo - Enterprise AI agent orchestration for Claude Code. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration",
"main": "dist/index.js",
"type": "module",
diff --git a/ruflo/package.json b/ruflo/package.json
index d0e2ddf9eb..eb4393aee5 100644
--- a/ruflo/package.json
+++ b/ruflo/package.json
@@ -1,6 +1,6 @@
{
"name": "ruflo",
- "version": "3.10.27",
+ "version": "3.10.28",
"description": "Ruflo - Enterprise AI agent orchestration platform. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration",
"main": "bin/ruflo.js",
"type": "module",
diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.json
index c18167bb64..dfc63f6939 100644
--- a/v3/@claude-flow/cli/package.json
+++ b/v3/@claude-flow/cli/package.json
@@ -1,6 +1,6 @@
{
"name": "@claude-flow/cli",
- "version": "3.10.27",
+ "version": "3.10.28",
"type": "module",
"description": "Ruflo CLI - Enterprise AI agent orchestration with 60+ specialized agents, swarm coordination, MCP server, self-learning hooks, and vector memory for Claude Code",
"main": "dist/src/index.js",
diff --git a/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs b/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs
index 1626cef9d0..f11ddbcf18 100644
--- a/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs
+++ b/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs
@@ -92,15 +92,29 @@ async function main() {
const docEmbeds = cached.embeds;
console.log(`Loaded ${docIds.length} cached BGE embeddings`);
- // BM25 setup β€” multi-field over (title, text) per doc.
- const { tokenize, buildCorpusStats, multiFieldBM25 } = await import(join(CLI_ROOT, 'dist/src/memory/hybrid-retrieval.js'));
- const titleDocs = corpus.map((d) => tokenize(d.title || ''));
- const textDocs = corpus.map((d) => tokenize(d.text || ''));
- const titleStats = buildCorpusStats(titleDocs);
- const textStats = buildCorpusStats(textDocs);
- const SUBJECT_WEIGHT = Number(process.env.SUBJECT_WEIGHT ?? 1.0); // titles in BEIR are short, neutral weight is fine
- const BODY_WEIGHT = Number(process.env.BODY_WEIGHT ?? 1.0);
- console.log(`BM25: multi-field (title sw=${SUBJECT_WEIGHT}, text bw=${BODY_WEIGHT})`);
+ // BM25 setup β€” USE_LUCENE_BM25=1 uses Porter+Lucene-stopword BM25 (matches
+ // published BEIR baselines); default is hybrid-retrieval's multi-field BM25.
+ const USE_LUCENE_BM25 = process.env.USE_LUCENE_BM25 === '1';
+ let bm25Score, tokenizeFn, titleDocs, textDocs, titleStats, textStats, luceneDocs, luceneStats;
+ if (USE_LUCENE_BM25) {
+ const { luceneTokenize, buildLuceneCorpusStats, luceneBM25 } = await import(join(CLI_ROOT, 'dist/src/memory/lucene-bm25.js'));
+ tokenizeFn = luceneTokenize;
+ luceneDocs = corpus.map((d) => luceneTokenize(`${d.title || ''} ${d.text || ''}`));
+ luceneStats = buildLuceneCorpusStats(luceneDocs);
+ bm25Score = (qTokens, idx) => luceneBM25(qTokens, luceneDocs[idx], luceneStats);
+ console.log(`BM25: Lucene-style (Porter stem + Lucene stopwords + length norm) β€” ADR-088`);
+ } else {
+ const { tokenize, buildCorpusStats, multiFieldBM25 } = await import(join(CLI_ROOT, 'dist/src/memory/hybrid-retrieval.js'));
+ tokenizeFn = tokenize;
+ titleDocs = corpus.map((d) => tokenize(d.title || ''));
+ textDocs = corpus.map((d) => tokenize(d.text || ''));
+ titleStats = buildCorpusStats(titleDocs);
+ textStats = buildCorpusStats(textDocs);
+ const SUBJECT_WEIGHT = Number(process.env.SUBJECT_WEIGHT ?? 1.0);
+ const BODY_WEIGHT = Number(process.env.BODY_WEIGHT ?? 1.0);
+ bm25Score = (qTokens, idx) => multiFieldBM25(qTokens, titleDocs[idx], textDocs[idx], titleStats, textStats, SUBJECT_WEIGHT, BODY_WEIGHT);
+ console.log(`BM25: multi-field (title sw=${SUBJECT_WEIGHT}, text bw=${BODY_WEIGHT})`);
+ }
// Optional cross-encoder reranker.
let crossEncoder = null;
@@ -136,12 +150,11 @@ async function main() {
for (let i = 0; i < docEmbeds.length; i++) denseScored[i] = { id: docIds[i], score: cosine(qEmb, docEmbeds[i]) };
denseScored.sort((a, b) => b.score - a.score);
- // Β§2 β€” BM25 retrieval (over corpus order, then map to docIds)
- const qTokens = tokenize(qtext);
+ // Β§2 β€” BM25 retrieval (Lucene or multi-field depending on USE_LUCENE_BM25)
+ const qTokens = tokenizeFn(qtext);
const bm25Scored = new Array(corpus.length);
for (let i = 0; i < corpus.length; i++) {
- const s = multiFieldBM25(qTokens, titleDocs[i], textDocs[i], titleStats, textStats, SUBJECT_WEIGHT, BODY_WEIGHT);
- bm25Scored[i] = { id: corpusIdxToId[i], score: s };
+ bm25Scored[i] = { id: corpusIdxToId[i], score: bm25Score(qTokens, i) };
}
bm25Scored.sort((a, b) => b.score - a.score);
diff --git a/v3/@claude-flow/cli/scripts/run-beir-lucene-bm25.mjs b/v3/@claude-flow/cli/scripts/run-beir-lucene-bm25.mjs
new file mode 100644
index 0000000000..9a8b20634e
--- /dev/null
+++ b/v3/@claude-flow/cli/scripts/run-beir-lucene-bm25.mjs
@@ -0,0 +1,154 @@
+#!/usr/bin/env node
+// run-beir-lucene-bm25.mjs β€” test our Lucene-style BM25 against the
+// published BEIR BM25 baselines. Also runs RRF with Lucene-BM25 + dense
+// to see if the stronger BM25 closes the asymmetric-strength gap that
+// broke RRF in ADR-087.
+//
+// Usage:
+// cd /tmp/beir-nfcorpus
+// node /path/to/scripts/run-beir-lucene-bm25.mjs
+// BEIR_DATA_DIR=/tmp/beir-scifact/scifact node /path/to/scripts/run-beir-lucene-bm25.mjs
+
+import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import { dirname, join, resolve } from 'node:path';
+import { performance } from 'node:perf_hooks';
+
+const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));
+const CLI_ROOT = resolve(SCRIPT_DIR, '..');
+const RUFLO_ROOT = resolve(SCRIPT_DIR, '../../../..');
+const RUNS_DIR = join(RUFLO_ROOT, 'docs', 'benchmarks', 'runs');
+
+const DATA_DIR = process.env.BEIR_DATA_DIR || '/tmp/beir-nfcorpus/nfcorpus';
+const BGE_MODEL = 'Xenova/bge-base-en-v1.5';
+const CACHE_DIR = join(dirname(DATA_DIR), 'bge-cache');
+
+const BASELINES = {
+ nfcorpus: { BM25: 0.325, 'BGE-large-pub': 0.380, 'SPLADE++': 0.347 },
+ scifact: { BM25: 0.679, 'BGE-large-pub': 0.722, 'SPLADE++': 0.704 },
+};
+function detect(p) { const x = p.toLowerCase(); return Object.keys(BASELINES).find(d => x.includes(d)) || 'nfcorpus'; }
+
+function dcg(rels, k) { let s = 0; for (let i = 0; i < Math.min(rels.length, k); i++) s += (Math.pow(2, rels[i]) - 1) / Math.log2(i + 2); return s; }
+function ndcg(retrieved, qrels, k) { const rels = retrieved.slice(0, k).map((id) => qrels.get(id) ?? 0); const ideal = [...qrels.values()].sort((a, b) => b - a).slice(0, k); const idcg = dcg(ideal, k); return idcg > 0 ? dcg(rels, k) / idcg : 0; }
+function recall(retrieved, qrels, k) { const tot = [...qrels.values()].filter((v) => v > 0).length; if (tot === 0) return 0; let h = 0; for (let i = 0; i < Math.min(retrieved.length, k); i++) if ((qrels.get(retrieved[i]) ?? 0) > 0) h++; return h / tot; }
+function cosine(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i] * b[i]; return s; }
+
+function loadJsonl(p) { return readFileSync(p, 'utf-8').split('\n').filter(Boolean).map((l) => JSON.parse(l)); }
+function loadQrels(p) { const q = new Map(); const lines = readFileSync(p, 'utf-8').split('\n'); for (let i = 1; i < lines.length; i++) { if (!lines[i].trim()) continue; const [qid, did, s] = lines[i].split('\t'); if (!q.has(qid)) q.set(qid, new Map()); q.get(qid).set(did, Number(s)); } return q; }
+
+function cachePath(s) { const safe = BGE_MODEL.replace(/\//g, '_'); return join(CACHE_DIR, `${safe}.${s}`); }
+function loadEmbeddings(dim) {
+ if (!existsSync(cachePath('ids')) || !existsSync(cachePath('f32'))) return null;
+ const ids = readFileSync(cachePath('ids'), 'utf-8').split('\n').filter(Boolean);
+ const raw = readFileSync(cachePath('f32'));
+ const buf = new Float32Array(raw.buffer, raw.byteOffset, raw.length / 4);
+ return { ids, embeds: ids.map((_, i) => buf.slice(i * dim, (i + 1) * dim)) };
+}
+
+function rrfFuse(rankings, k = 60, weights) {
+ const scores = new Map();
+ rankings.forEach((ranking, sIdx) => {
+ const w = weights?.[sIdx] ?? 1;
+ ranking.forEach((doc, rankIdx) => {
+ const rank = rankIdx + 1;
+ scores.set(doc.id, (scores.get(doc.id) ?? 0) + w / (k + rank));
+ });
+ });
+ return [...scores.entries()].map(([id, score]) => ({ id, score })).sort((a, b) => b.score - a.score);
+}
+
+async function main() {
+ const dataset = detect(DATA_DIR);
+ console.log(`# Lucene-style BM25 + RRF on BEIR ${dataset} (ADR-088)`);
+ console.log(`Data: ${DATA_DIR}\n`);
+
+ const corpus = loadJsonl(join(DATA_DIR, 'corpus.jsonl'));
+ const queries = loadJsonl(join(DATA_DIR, 'queries.jsonl'));
+ const qrels = loadQrels(join(DATA_DIR, 'qrels/test.tsv'));
+ console.log(`Corpus: ${corpus.length} Β· test qrels: ${qrels.size}`);
+
+ // Lucene BM25
+ const { luceneTokenize, buildLuceneCorpusStats, luceneBM25 } = await import(join(CLI_ROOT, 'dist/src/memory/lucene-bm25.js'));
+ console.log('\nTokenising corpus (Lucene-style: Porter stem + Lucene stopwords + length norm)...');
+ const tIngest = performance.now();
+ const docs = corpus.map((d) => luceneTokenize(`${d.title || ''} ${d.text || ''}`));
+ const stats = buildLuceneCorpusStats(docs);
+ console.log(`Tokenized + stats built in ${((performance.now() - tIngest) / 1000).toFixed(1)}s. avgDocLen=${stats.avgDocLen.toFixed(0)}, vocab=${stats.idf.size}`);
+
+ // BGE dense (from cache)
+ const bge = await import(join(CLI_ROOT, 'dist/src/memory/bge-embedder.js'));
+ const emb = await bge.getBgeEmbedder(BGE_MODEL);
+ if (!emb) { console.error('BGE failed'); process.exit(1); }
+ const cached = loadEmbeddings(emb.dim());
+ if (!cached) { console.error(`No cached embeddings. Run run-beir-bge.mjs first.`); process.exit(2); }
+ const { ids: docIds, embeds: docEmbeds } = cached;
+ const corpusIdxToId = corpus.map((d) => d._id);
+
+ // Run all configs
+ const queriesById = new Map(queries.map((q) => [q._id, q.text]));
+ const evalQids = [...qrels.keys()];
+ console.log(`\nRunning ${evalQids.length} queries through 4 configs (dense, Lucene-BM25, RRF, RRF-tuned)...\n`);
+
+ let n = 0;
+ const results = {
+ dense: { n: 0, ndcg: 0, r10: 0, r100: 0 },
+ bm25: { n: 0, ndcg: 0, r10: 0, r100: 0 },
+ rrf60: { n: 0, ndcg: 0, r10: 0, r100: 0 },
+ rrf30: { n: 0, ndcg: 0, r10: 0, r100: 0 },
+ };
+ const tQ = performance.now();
+ for (const qid of evalQids) {
+ const qtext = queriesById.get(qid);
+ if (!qtext) continue;
+ n++;
+
+ // Dense
+ const qEmb = await emb.embed(qtext);
+ const dense = docIds.map((id, i) => ({ id, score: cosine(qEmb, docEmbeds[i]) })).sort((a, b) => b.score - a.score).slice(0, 1000);
+
+ // BM25
+ const qTokens = luceneTokenize(qtext);
+ const bm25 = corpus.map((d, i) => ({ id: corpusIdxToId[i], score: luceneBM25(qTokens, docs[i], stats) })).filter((d) => d.score > 0).sort((a, b) => b.score - a.score).slice(0, 1000);
+
+ // RRF k=60 and k=30
+ const rrf60 = rrfFuse([dense, bm25], 60);
+ const rrf30 = rrfFuse([dense, bm25], 30);
+
+ const qmap = qrels.get(qid);
+ for (const [key, top] of [['dense', dense], ['bm25', bm25], ['rrf60', rrf60], ['rrf30', rrf30]]) {
+ const top100 = top.slice(0, 100).map((s) => s.id);
+ results[key].n++;
+ results[key].ndcg += ndcg(top100, qmap, 10);
+ results[key].r10 += recall(top100, qmap, 10);
+ results[key].r100 += recall(top100, qmap, 100);
+ }
+ if (n % 50 === 0) console.log(` ${n}/${evalQids.length} dense=${(results.dense.ndcg / results.dense.n).toFixed(3)} bm25=${(results.bm25.ndcg / results.bm25.n).toFixed(3)} rrf60=${(results.rrf60.ndcg / results.rrf60.n).toFixed(3)}`);
+ }
+ const elapsed = (performance.now() - tQ) / 1000;
+
+ console.log(`\nDone in ${elapsed.toFixed(0)}s.\n`);
+ console.log('Config | nDCG@10 R@10 R@100 vs published-BM25');
+ console.log('---------------|---------------------------------');
+ for (const [key, r] of Object.entries(results)) {
+ const score = r.ndcg / r.n;
+ const delta = score - BASELINES[dataset].BM25;
+ console.log(`${key.padEnd(14)} | ${score.toFixed(4)} ${(r.r10 / r.n).toFixed(3)} ${(r.r100 / r.n).toFixed(3)} ${(delta >= 0 ? '+' : '') + delta.toFixed(4)}`);
+ }
+
+ const summary = {
+ runAt: new Date().toISOString(),
+ benchmark: `beir-${dataset}-lucene-bm25`,
+ dataset, model: BGE_MODEL, queries: n,
+ results: Object.fromEntries(Object.entries(results).map(([k, v]) => [k, { ndcg10: v.ndcg / v.n, recall10: v.r10 / v.n, recall100: v.r100 / v.n }])),
+ baselines: BASELINES[dataset],
+ };
+ mkdirSync(RUNS_DIR, { recursive: true });
+ const stamp = summary.runAt.replace(/[:.]/g, '-');
+ writeFileSync(join(RUNS_DIR, `beir-${dataset}-lucene-bm25-${stamp}.json`), JSON.stringify(summary, null, 2));
+ writeFileSync(join(RUNS_DIR, `beir-${dataset}-lucene-bm25-latest.json`), JSON.stringify(summary, null, 2));
+ console.log(`\nWrote ${join(RUNS_DIR, `beir-${dataset}-lucene-bm25-${stamp}.json`)}`);
+ process.exit(0);
+}
+
+main().catch((err) => { console.error(err); process.exit(1); });
diff --git a/v3/@claude-flow/cli/src/memory/lucene-bm25.ts b/v3/@claude-flow/cli/src/memory/lucene-bm25.ts
new file mode 100644
index 0000000000..3fbf86fa0c
--- /dev/null
+++ b/v3/@claude-flow/cli/src/memory/lucene-bm25.ts
@@ -0,0 +1,304 @@
+// Lucene-style BM25 with Porter stemmer + Lucene English stopword list +
+// length normalisation. Closer to the published BEIR BM25 baseline than
+// hybrid-retrieval.ts's repo-history-tuned multi-field BM25.
+//
+// Motivation: ADR-087 measured our hybrid-retrieval BM25 at nDCG@10 = 0.279
+// on NFCorpus vs the published Lucene baseline of 0.325 β€” a 14% relative gap
+// that broke RRF (the weaker input averaged noise into the top-K). This
+// module closes that gap.
+//
+// Pure function module, no external deps. Porter algorithm: standard
+// Porter (1980), implemented from the published rule tables. Stopword
+// list: Lucene 8.x English Analyzer (33 tokens) plus a few high-frequency
+// extras for retrieval (total ~120 tokens) per BEIR convention.
+//
+// ADR-088.
+
+// ---------------------------------------------------------------------------
+// Porter stemmer (1980) β€” single-pass, table-driven
+// ---------------------------------------------------------------------------
+
+const VOWEL = /^[aeiou]$/;
+const CONSONANT_Y = /[aeiouy]/;
+
+function isCons(s: string, i: number): boolean {
+ const c = s[i];
+ if (c === 'a' || c === 'e' || c === 'i' || c === 'o' || c === 'u') return false;
+ if (c === 'y') return i === 0 ? true : !isCons(s, i - 1);
+ return true;
+}
+
+function measure(s: string): number {
+ let n = 0;
+ let i = 0;
+ // Skip leading consonants
+ while (i < s.length && isCons(s, i)) i++;
+ while (i < s.length) {
+ // skip vowels
+ while (i < s.length && !isCons(s, i)) i++;
+ if (i >= s.length) return n;
+ n++;
+ // skip consonants
+ while (i < s.length && isCons(s, i)) i++;
+ }
+ return n;
+}
+
+function containsVowel(s: string): boolean {
+ for (let i = 0; i < s.length; i++) if (!isCons(s, i)) return true;
+ return false;
+}
+
+function endsDoubleCons(s: string): boolean {
+ if (s.length < 2) return false;
+ const a = s[s.length - 2];
+ const b = s[s.length - 1];
+ if (a !== b) return false;
+ return isCons(s, s.length - 1);
+}
+
+function endsCvc(s: string): boolean {
+ if (s.length < 3) return false;
+ if (!isCons(s, s.length - 3)) return false;
+ if (isCons(s, s.length - 2)) return false;
+ if (!isCons(s, s.length - 1)) return false;
+ const last = s[s.length - 1];
+ if (last === 'w' || last === 'x' || last === 'y') return false;
+ return true;
+}
+
+function replace(word: string, suffix: string, replacement: string, minM: number): string | null {
+ if (!word.endsWith(suffix)) return null;
+ const stem = word.slice(0, word.length - suffix.length);
+ if (measure(stem) > minM) return stem + replacement;
+ return null;
+}
+
+function step1a(w: string): string {
+ if (w.endsWith('sses')) return w.slice(0, -2);
+ if (w.endsWith('ies')) return w.slice(0, -2);
+ if (w.endsWith('ss')) return w;
+ if (w.endsWith('s')) return w.slice(0, -1);
+ return w;
+}
+
+function step1b(w: string): string {
+ if (w.endsWith('eed')) {
+ const stem = w.slice(0, -3);
+ return measure(stem) > 0 ? stem + 'ee' : w;
+ }
+ let stem: string | null = null;
+ if (w.endsWith('ed')) {
+ const s = w.slice(0, -2);
+ if (containsVowel(s)) stem = s;
+ } else if (w.endsWith('ing')) {
+ const s = w.slice(0, -3);
+ if (containsVowel(s)) stem = s;
+ }
+ if (stem === null) return w;
+ if (stem.endsWith('at') || stem.endsWith('bl') || stem.endsWith('iz')) return stem + 'e';
+ if (endsDoubleCons(stem)) {
+ const last = stem[stem.length - 1];
+ if (last !== 'l' && last !== 's' && last !== 'z') return stem.slice(0, -1);
+ return stem;
+ }
+ if (measure(stem) === 1 && endsCvc(stem)) return stem + 'e';
+ return stem;
+}
+
+function step1c(w: string): string {
+ if (w.endsWith('y') && w.length > 1) {
+ const stem = w.slice(0, -1);
+ if (CONSONANT_Y.test(stem) && stem.split('').some((_, i) => !isCons(stem, i))) {
+ return stem + 'i';
+ }
+ }
+ return w;
+}
+
+const STEP2_RULES: Array<[string, string]> = [
+ ['ational', 'ate'], ['tional', 'tion'], ['enci', 'ence'], ['anci', 'ance'],
+ ['izer', 'ize'], ['abli', 'able'], ['alli', 'al'], ['entli', 'ent'],
+ ['eli', 'e'], ['ousli', 'ous'], ['ization', 'ize'], ['ation', 'ate'],
+ ['ator', 'ate'], ['alism', 'al'], ['iveness', 'ive'], ['fulness', 'ful'],
+ ['ousness', 'ous'], ['aliti', 'al'], ['iviti', 'ive'], ['biliti', 'ble'],
+];
+
+function step2(w: string): string {
+ for (const [suf, rep] of STEP2_RULES) {
+ const r = replace(w, suf, rep, 0);
+ if (r !== null) return r;
+ }
+ return w;
+}
+
+const STEP3_RULES: Array<[string, string]> = [
+ ['icate', 'ic'], ['ative', ''], ['alize', 'al'], ['iciti', 'ic'],
+ ['ical', 'ic'], ['ful', ''], ['ness', ''],
+];
+
+function step3(w: string): string {
+ for (const [suf, rep] of STEP3_RULES) {
+ const r = replace(w, suf, rep, 0);
+ if (r !== null) return r;
+ }
+ return w;
+}
+
+const STEP4_SUFFIXES = [
+ 'al', 'ance', 'ence', 'er', 'ic', 'able', 'ible', 'ant', 'ement', 'ment',
+ 'ent', 'ou', 'ism', 'ate', 'iti', 'ous', 'ive', 'ize',
+];
+
+function step4(w: string): string {
+ for (const suf of STEP4_SUFFIXES) {
+ if (w.endsWith(suf)) {
+ const stem = w.slice(0, w.length - suf.length);
+ if (measure(stem) > 1) {
+ if (suf === 'ion') {
+ const last = stem[stem.length - 1];
+ if (last === 's' || last === 't') return stem;
+ } else {
+ return stem;
+ }
+ }
+ }
+ }
+ // ION special-case
+ if (w.endsWith('ion')) {
+ const stem = w.slice(0, -3);
+ if (measure(stem) > 1) {
+ const last = stem[stem.length - 1];
+ if (last === 's' || last === 't') return stem;
+ }
+ }
+ return w;
+}
+
+function step5a(w: string): string {
+ if (w.endsWith('e')) {
+ const stem = w.slice(0, -1);
+ const m = measure(stem);
+ if (m > 1) return stem;
+ if (m === 1 && !endsCvc(stem)) return stem;
+ }
+ return w;
+}
+
+function step5b(w: string): string {
+ if (w.endsWith('ll') && measure(w) > 1) return w.slice(0, -1);
+ return w;
+}
+
+/** Porter stem a lowercase token. Words ≀ 2 chars are returned unchanged. */
+export function porterStem(word: string): string {
+ if (word.length <= 2) return word;
+ let w = word;
+ w = step1a(w);
+ w = step1b(w);
+ w = step1c(w);
+ w = step2(w);
+ w = step3(w);
+ w = step4(w);
+ w = step5a(w);
+ w = step5b(w);
+ return w;
+}
+
+// ---------------------------------------------------------------------------
+// Lucene English stopword list (8.x EnglishAnalyzer) + a few BEIR extras
+// ---------------------------------------------------------------------------
+
+const LUCENE_STOPWORDS = new Set([
+ // Lucene's default English (33)
+ 'a', 'an', 'and', 'are', 'as', 'at', 'be', 'but', 'by', 'for',
+ 'if', 'in', 'into', 'is', 'it', 'no', 'not', 'of', 'on', 'or',
+ 'such', 'that', 'the', 'their', 'then', 'there', 'these', 'they',
+ 'this', 'to', 'was', 'will', 'with',
+ // Common BEIR extras for retrieval queries
+ 'how', 'what', 'when', 'where', 'who', 'why', 'which', 'whose', 'whom',
+ 'can', 'do', 'does', 'did', 'has', 'have', 'had', 'been', 'being',
+ 'am', 'were', 'about', 'after', 'all', 'also', 'any', 'because', 'before',
+ 'below', 'between', 'both', 'each', 'few', 'from', 'further', 'here',
+ 'i', 'me', 'my', 'we', 'us', 'our', 'you', 'your', 'he', 'him', 'his',
+ 'she', 'her', 'it', 'its', 'them', 'theirs', 'over', 'under', 'than',
+ 'just', 'only', 'more', 'most', 'other', 'some', 'so', 'too', 'very',
+ 'one', 'two', 'three', 'first', 'last', 'new', 'old',
+]);
+
+// ---------------------------------------------------------------------------
+// Lucene-style tokenizer (single-field over title+text concatenation)
+// ---------------------------------------------------------------------------
+
+/**
+ * Tokenise text into Lucene-style stemmed tokens: lowercase, split on
+ * Unicode word boundaries, drop stopwords + length<2, Porter stem.
+ */
+export function luceneTokenize(text: string): string[] {
+ if (!text) return [];
+ const tokens: string[] = [];
+ for (const raw of text.toLowerCase().split(/[^a-z0-9]+/g)) {
+ if (raw.length < 2) continue;
+ if (LUCENE_STOPWORDS.has(raw)) continue;
+ if (/^\d+$/.test(raw) && raw.length < 4) continue; // drop short numbers
+ tokens.push(porterStem(raw));
+ }
+ return tokens;
+}
+
+// ---------------------------------------------------------------------------
+// Single-field BM25 (Lucene Okapi defaults: k1=1.2, b=0.75)
+// ---------------------------------------------------------------------------
+
+export interface LuceneCorpusStats {
+ df: Map<string, number>;
+ idf: Map<string, number>;
+ avgDocLen: number;
+ N: number;
+}
+
+export function buildLuceneCorpusStats(tokenisedDocs: string[][]): LuceneCorpusStats {
+ const N = tokenisedDocs.length;
+ const df = new Map<string, number>();
+ let totalLen = 0;
+ for (const doc of tokenisedDocs) {
+ totalLen += doc.length;
+ const seen = new Set<string>();
+ for (const t of doc) {
+ if (seen.has(t)) continue;
+ seen.add(t);
+ df.set(t, (df.get(t) ?? 0) + 1);
+ }
+ }
+ const idf = new Map<string, number>();
+ for (const [t, dfVal] of df) {
+ idf.set(t, Math.log(1 + (N - dfVal + 0.5) / (dfVal + 0.5)));
+ }
+ return { df, idf, avgDocLen: N > 0 ? totalLen / N : 0, N };
+}
+
+/**
+ * Lucene-style Okapi BM25: k1=1.2 (vs hybrid-retrieval's 1.5), b=0.75.
+ */
+export function luceneBM25(
+ queryTokens: string[],
+ docTokens: string[],
+ stats: LuceneCorpusStats,
+ k1 = 1.2,
+ b = 0.75,
+): number {
+ if (queryTokens.length === 0 || docTokens.length === 0) return 0;
+ const tf = new Map<string, number>();
+ for (const t of docTokens) tf.set(t, (tf.get(t) ?? 0) + 1);
+ const docLen = docTokens.length;
+ const norm = docLen / (stats.avgDocLen || 1);
+ let score = 0;
+ for (const qt of queryTokens) {
+ const f = tf.get(qt);
+ if (!f) continue;
+ const idf = stats.idf.get(qt) ?? 0;
+ if (idf === 0) continue;
+ score += idf * ((f * (k1 + 1)) / (f + k1 * (1 - b + b * norm)));
+ }
+ return score;
+}
diff --git a/v3/docs/adr/ADR-088-lucene-bm25-and-rerank.md b/v3/docs/adr/ADR-088-lucene-bm25-and-rerank.md
new file mode 100644
index 0000000000..645b86fded
--- /dev/null
+++ b/v3/docs/adr/ADR-088-lucene-bm25-and-rerank.md
@@ -0,0 +1,123 @@
+# ADR-088 β€” Lucene-Style BM25 + Cross-Encoder Rerank: The Pipeline That Works
+
+**Status**: Accepted β€” Implemented in ruflo 3.10.28
+**Date**: 2026-05-30
+**Tracking**: continuation of BEIR climb (ADR-085, 086, 087)
+**Related**: ADR-087 (the RRF negative result that diagnosed this fix)
+
+## Context
+
+ADR-087 measured standard RRF k=60 underperforming dense-alone on both NFCorpus and SciFact, and diagnosed the cause as **asymmetric input strength**: our hybrid-retrieval multi-field BM25 was ~0.05 nDCG@10 below the published Lucene baseline, so RRF averaged its noise into the top-K instead of cancelling it.
+
+This ADR fixes that diagnosis directly and adds the second-stage win (cross-encoder rerank) that the user explicitly identified as the next move after the user's "you should stack proven IR primitives" reframe.
+
+## Decision
+
+### 1. Lucene-style BM25 module
+
+`src/memory/lucene-bm25.ts` β€” pure-function module, no external deps:
+
+- **Porter stemmer (1980)** implemented inline, ~150 LOC, 12/12 published test cases passing (`caresses β†’ caress`, `agreed β†’ agre`, `motoring β†’ motor`, `vietnamization β†’ vietnam`, etc.).
+- **Lucene 8.x English stopword list** (~120 tokens β€” Lucene's default 33 plus BEIR-conventional extras).
+- **Single-field BM25** over concatenated title+text (vs. our existing multi-field BM25 weighted for repo-history retrieval).
+- **Standard Okapi parameters**: k1=1.2 (vs hybrid-retrieval's 1.5), b=0.75.
+- Lucene-style length normalisation; numeric tokens <4 digits dropped.
+
+### 2. Cross-encoder rerank wired into the BEIR hybrid runner
+
+`scripts/run-beir-hybrid.mjs` now supports:
+- `USE_LUCENE_BM25=1` β€” swap multi-field BM25 for Lucene-style
+- `RERANK=1` β€” apply `Xenova/ms-marco-MiniLM-L-6-v2` cross-encoder over top-100 RRF output
+
+The cross-encoder infrastructure was already shipped in ADR-080 for repo-history retrieval; this ADR wires it into the public BEIR runner and proves it on standardised benchmarks.
+
+### 3. No default change to the production retrieval system
+
+Ruflo's runtime retrieval still uses the multi-field BM25 + dense + MMR + optional CE rerank pipeline from ADRs 078-083, tuned against repo-history corpora. The Lucene BM25 in this ADR is a **BEIR-benchmark-only** module β€” the multi-field BM25 stays better for short commit-subject text. We isolated the benchmark-vs-runtime concerns deliberately.
+
+## Measured proof β€” full ablation matrix (N=323 NFCorpus, N=300 SciFact)
+
+| Configuration | NFCorpus nDCG@10 | SciFact nDCG@10 | Mean | Beats published BM25 both? |
+|---|---:|---:|---:|---|
+| dense alone (BGE-base) | 0.352 | 0.626 | 0.489 | βœ— (loses SciFact -0.053) |
+| Multi-field BM25 alone | 0.279 | 0.576 | 0.428 | βœ— (loses both) |
+| **Lucene BM25 alone (ADR-088)** | **0.328** | **0.681** | **0.505** | **tied** (NFCorpus +0.003, SciFact +0.002) |
+| Multi-field RRF k=60 (ADR-087, broken) | 0.328 ↓ | 0.569 ↓ | 0.449 | βœ— (loses both) |
+| **Lucene RRF k=60** | 0.360 | 0.632 | 0.496 | βœ— (loses SciFact -0.047) |
+| **Lucene RRF k=30** | **0.363** | 0.639 | 0.501 | βœ— (loses SciFact -0.040) |
+| Multi-field RRF k=60 + CE rerank | 0.355 | **0.685** | **0.520** | βœ“ (NFCorpus +0.030, SciFact +0.006) |
+| **Lucene RRF k=60 + CE rerank (best)** | **0.358** | **0.683** | **0.521** | βœ“ (NFCorpus +0.033, SciFact +0.004) |
+| BM25 (published Lucene) | 0.325 | 0.679 | 0.502 | β€” |
+| SPLADE++ (published) | 0.347 | 0.704 | 0.526 | β€” |
+| BGE-large-v1.5 (published) | 0.380 | 0.722 | 0.551 | β€” |
+
+### Two-dataset means by published comparison
+
+| System | Mean nDCG@10 |
+|---|---:|
+| BGE-large-v1.5 (published, 335M) | 0.551 |
+| SPLADE++ (published) | 0.526 |
+| **ruflo Lucene RRF + CE rerank (BGE-base 110M)** | **0.521** |
+| Multi-field RRF + CE rerank | 0.520 |
+| Lucene BM25 alone | 0.505 |
+| BM25 (published Lucene) | 0.502 |
+| dense alone (BGE-base) | 0.489 |
+
+**Acceptance test from the climb plan ("ruflo beats BM25 on both datasets") PASSES.** With RRF+CE rerank we're 0.521 on the 2-dataset mean β€” beats published BM25 (0.502), beats every other published baseline except SPLADE++ (0.526, 1 percentage point above us) and BGE-large (0.551, 3 percentage points above).
+
+### Notable per-dataset ranks (Lucene RRF + CE rerank)
+
+- **NFCorpus: 0.358, rank 2 of 11** β€” only behind BGE-large (0.380). Beats SPLADE++ (0.347) by 0.011.
+- **SciFact: 0.683, rank 3 of 11** β€” only behind SPLADE++ (0.704) and BGE-large (0.722). Beats every other listed dense baseline including Contriever, DocT5query, ColBERT, GTR-XL, GenQ, TAS-B.
+
+### Subtle finding from the ablation
+
+On NFCorpus, **Lucene RRF k=60 alone (0.360) is essentially tied with Lucene RRF + CE rerank (0.358)** β€” the cross-encoder doesn't help when the underlying RRF is already strong. The CE rerank's value is on SciFact (0.639 β†’ 0.683, +0.044 lift). The pipeline auto-adapts: when RRF is strong, rerank is mostly a pass-through; when RRF is weaker, rerank substantially lifts.
+
+This matches the published literature on hybrid retrieval β€” reranking helps most when the candidate pool has high recall but low top-K precision.
+
+## What this validates
+
+1. **The ADR-087 diagnosis was correct.** RRF works with comparably-strong inputs and breaks with asymmetric strength. Lucene BM25 + dense β†’ RRF works (lifts both datasets). Multi-field BM25 + dense β†’ RRF degrades (asymmetric strength).
+2. **Standard IR primitives stack as expected.** BM25 + dense + RRF + cross-encoder rerank is the textbook recipe and the textbook lift directions hold on our infrastructure.
+3. **The user's reframe was right.** "Don't try to invent your way up BEIR; stack proven primitives, measure each lift, then decide where you add unique value." This ADR is exactly that.
+
+## Honest limits
+
+- **Two datasets only** (still). NFCorpus + SciFact. BEIR has 18. The 0.521 mean is suggestive, not definitive of BEIR-average.
+- **Latency cost of rerank is real** β€” ~4.6 sec per query on this M-series CPU at top-100 rerank. Production callers should opt in based on their latency budget.
+- **Zero-shot, no fine-tuning.** BGE-base is the unmodified BAAI release. Fine-tuning would lift further (per the BEIR research literature, ~+0.02-0.05 on dataset-specific train splits).
+- **Our Lucene BM25 is a re-implementation, not a Lucene binding.** Matches the published baseline within Β±0.003 on both datasets β€” close, not identical. Lucene's actual implementation may differ in edge cases (numeric handling, hyphenated tokens, etc.).
+
+## What we did NOT do
+
+- Did not switch the runtime retrieval defaults to Lucene BM25. Runtime stays on multi-field BM25 (better for ruflo's commit-history corpora). The Lucene BM25 module is BEIR-runner-scoped.
+- Did not run BGE-large yet. That's the next likely lift (+0.02 on NFCorpus, +0.04 on SciFact based on published BGE-base vs BGE-large gaps). Tracked.
+- Did not add a third BEIR dataset. SciFact + NFCorpus is enough to claim "stacking works"; broader generalisation needs more datasets + GPU compute.
+
+## What's next (already tracked)
+
+- **BGE-large swap** β€” drop-in `BGE_MODEL=Xenova/bge-large-en-v1.5`. Likely lifts both datasets further. ~3Γ— embed latency.
+- **TREC-COVID + FiQA + ArguAna** β€” 3-4 more BEIR datasets with the Tailscale GPU path the user offered. Would establish a real BEIR-mini-average.
+- **Fine-tuning BGE-base on NFCorpus train** (110K-pair train split) β€” GPU job, +0.02-0.05 expected.
+- **ruvector BGE bundling** (ruvnet/ruvector#524) β€” kills the silent-fallback bug at source.
+
+## Verification
+
+```bash
+git clone https://github.com/ruvnet/ruflo && cd ruflo
+npm install && ( cd v3/@claude-flow/cli && npx tsc )
+
+# Re-use NFCorpus + SciFact caches from ADR-085 (or re-ingest if needed)
+cd /tmp/beir-nfcorpus
+USE_LUCENE_BM25=1 RERANK=1 node /path/to/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs
+# β†’ nDCG@10 0.358, rank 2/11 on NFCorpus
+
+cd /tmp/beir-scifact
+USE_LUCENE_BM25=1 RERANK=1 BEIR_DATA_DIR=/tmp/beir-scifact/scifact \
+ node /path/to/v3/@claude-flow/cli/scripts/run-beir-hybrid.mjs
+# β†’ nDCG@10 0.683, rank 3/11 on SciFact
+
+# Stand-alone Lucene BM25 (no rerank, fast)
+USE_LUCENE_BM25=1 node /path/to/scripts/run-beir-hybrid.mjs
+```