#1 Persistent memory for AI coding agents based on real-world benchmarks
TypeScript agentmemoryagentsaiclaudeclaudecodecodexcopilotcursorgenaiharnesshermesmemoryompopenclawopencodepi
bench: add LongMemEval-S benchmark (95.2% R@5 hybrid, 86.2% BM25) (#91)
Rohit Ghumare565c2388a2ac
diff --git a/benchmark/LONGMEMEVAL.md b/benchmark/LONGMEMEVAL.mdnew file mode 100644index 00000000..00de938e--- /dev/null+++ b/benchmark/LONGMEMEVAL.md@@ -0,0 +1,79 @@+# LongMemEval-S Benchmark Results++[LongMemEval](https://arxiv.org/abs/2410.10813) (ICLR 2025) is an academic benchmark for evaluating long-term memory in chat assistants. It tests 5 core abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.++## Setup++- **Dataset**: LongMemEval-S (500 questions, ~48 sessions per question, ~115K tokens)+- **Source**: [xiaowu0162/longmemeval-cleaned](https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned)+- **Metric**: `recall_any@K` — does ANY gold session appear in top-K retrieved results?+- **Embedding model**: `all-MiniLM-L6-v2` (384 dimensions, local, no API key)+- **No LLM in the loop**: Pure retrieval evaluation, no answer generation or judge++## Results++| System | R@5 | R@10 | R@20 | NDCG@10 | MRR |+|---|---|---|---|---|---|+| **agentmemory BM25+Vector** | **95.2%** | **98.6%** | **99.4%** | **87.9%** | **88.2%** |+| agentmemory BM25-only | 86.2% | 94.6% | 98.6% | 73.0% | 71.5% |+| MemPalace raw (vector-only) | 96.6% | ~97.6% | — | — | — |++### By Question Type (BM25+Vector)++| Type | R@5 | R@10 | Count |+|---|---|---|---|+| knowledge-update | 98.7% | 100.0% | 78 |+| multi-session | 97.7% | 100.0% | 133 |+| single-session-assistant | 96.4% | 98.2% | 56 |+| temporal-reasoning | 95.5% | 97.7% | 133 |+| single-session-user | 90.0% | 97.1% | 70 |+| single-session-preference | 83.3% | 96.7% | 30 |++### By Question Type (BM25-only)++| Type | R@5 | R@10 | Count |+|---|---|---|---|+| knowledge-update | 92.3% | 98.7% | 78 |+| single-session-user | 91.4% | 95.7% | 70 |+| temporal-reasoning | 88.0% | 94.7% | 133 |+| multi-session | 86.5% | 96.2% | 133 |+| single-session-assistant | 80.4% | 91.1% | 56 |+| single-session-preference | 60.0% | 80.0% | 30 |++## Analysis++1. **BM25+Vector (95.2%) nearly matches pure vector search (96.6%)** with only a 1.4pp gap. Both use the same embedding model (all-MiniLM-L6-v2).++2. **BM25 alone gets 86.2%** — keyword search with Porter stemming and synonym expansion is surprisingly effective on conversational data.++3. **Adding vectors to BM25 gives +9pp** (86.2% → 95.2%), the largest improvement from any single component.++4. **Preferences are the hardest category** for both BM25 (60%) and hybrid (83.3%). These require understanding implicit/indirect statements.++5. **Multi-session and knowledge-update are strongest** (97.7%+ hybrid). The hybrid approach excels when facts are distributed across sessions.++6. **R@10 reaches 98.6%** — nearly all gold sessions are found within the top 10 results.++## Important Notes on Methodology++- These are **retrieval recall** scores, not end-to-end QA accuracy. The official LongMemEval metric is QA accuracy (retrieve + generate answer + GPT-4o judge).+- Systems on the actual LongMemEval QA leaderboard score 60-95% depending on the LLM reader (Oracle GPT-4o gets ~82.4%).+- We do NOT claim these as "LongMemEval scores" — they are retrieval-only evaluations on the LongMemEval-S haystack.+- Each question builds a fresh index from its ~48 sessions, searches with the question text, and checks if gold session IDs appear in results.++## Reproducibility++```bash+# Download dataset (264 MB)+pip install huggingface_hub+python3 -c "+from huggingface_hub import hf_hub_download+hf_hub_download(repo_id='xiaowu0162/longmemeval-cleaned', filename='longmemeval_s_cleaned.json', repo_type='dataset', local_dir='benchmark/data')+"++# Run BM25-only+npx tsx benchmark/longmemeval-bench.ts bm25++# Run BM25+Vector hybrid (requires @xenova/transformers)+npx tsx benchmark/longmemeval-bench.ts hybrid+```diff --git a/benchmark/data/.gitignore b/benchmark/data/.gitignorenew file mode 100644index 00000000..baae31f7--- /dev/null+++ b/benchmark/data/.gitignore@@ -0,0 +1 @@+longmemeval_s_cleaned.jsondiff --git a/benchmark/data/longmemeval_results_bm25.json b/benchmark/data/longmemeval_results_bm25.jsonnew file mode 100644index 00000000..eaaaba21--- /dev/null+++ b/benchmark/data/longmemeval_results_bm25.json@@ -0,0 +1,12491 @@+{+ "mode": "bm25",+ "questions": 500,+ "recall_any_at_5": 0.862,+ "recall_any_at_10": 0.946,+ "recall_any_at_20": 0.986,+ "ndcg_at_10": 0.7302373205176367,+ "mrr": 0.7153796961208726,+ "per_type": {+ "single-session-user": {+ "count": 70,+ "recall_any_at_5": 0.9142857142857143,+ "recall_any_at_10": 0.9571428571428572+ },+ "multi-session": {+ "count": 133,+ "recall_any_at_5": 0.8646616541353384,+ "recall_any_at_10": 0.9624060150375939+ },+ "single-session-preference": {+ "count": 30,+ "recall_any_at_5": 0.6,+ "recall_any_at_10": 0.8+ },+ "temporal-reasoning": {+ "count": 133,+ "recall_any_at_5": 0.8796992481203008,+ "recall_any_at_10": 0.9473684210526315+ },+ "knowledge-update": {+ "count": 78,+ "recall_any_at_5": 0.9230769230769231,+ "recall_any_at_10": 0.9871794871794872+ },+ "single-session-assistant": {+ "count": 56,+ "recall_any_at_5": 0.8035714285714286,+ "recall_any_at_10": 0.9107142857142857+ }+ },+ "per_question": [+ {+ "question_id": "e47becba",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_280352e9",+ "02bd2b90_3",+ "ultrachat_214101",+ "5dac7cc2_1",+ "ultrachat_336116",+ "f6859b48_2",+ "sharegpt_Cr2tc1f_0",+ "41abc171_4",+ "3b3a77d9",+ "aae4411b_2"+ ],+ "gold_session_ids": [+ "answer_280352e9"+ ]+ },+ {+ "question_id": "118b2229",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_40a90d51",+ "728deb4d_5",+ "5209e813_2",+ "db50c0f6",+ "db73b7e4_4",+ "628c958c_7",+ "5dce60dd",+ "ultrachat_311002",+ "sharegpt_dabXXRO_10",+ "ultrachat_392094"+ ],+ "gold_session_ids": [+ "answer_40a90d51"+ ]+ },+ {+ "question_id": "51a45a95",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d61669c7",+ "98c198fb",+ "5c44d9fe_1",+ "07ba9acd_2",+ "55e0c6db_2",+ "ultrachat_56839",+ "217debf7",+ "sharegpt_hChsWOp_128",+ "83addb61_4",+ "6e0b1800_2"+ ],+ "gold_session_ids": [+ "answer_d61669c7"+ ]+ },+ {+ "question_id": "58bf7951",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_355c48bb",+ "sharegpt_gmLF3PE_17",+ "ultrachat_304942",+ "ultrachat_268434",+ "793382f9_2",+ "ultrachat_221066",+ "ultrachat_278141",+ "sharegpt_CvVLg2J_18",+ "07b7a667_1",+ "1587bd78_1"+ ],+ "gold_session_ids": [+ "answer_355c48bb"+ ]+ },+ {+ "question_id": "1e043500",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e012175",+ "01bd9def",+ "be324b2e_1",+ "3548cae3",+ "d8a6c437",+ "4d0d7984",+ "sharegpt_6QUDIXG_24",+ "sharegpt_uQMdL9t_0",+ "sharegpt_85Y0l0J_365",+ "c285b705_2"+ ],+ "gold_session_ids": [+ "answer_3e012175"+ ]+ },+ {+ "question_id": "c5e8278d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6168136",+ "42e69993_2",+ "aaf71ce2_2",+ "sharegpt_DB8U4oH_9",+ "f58317c6",+ "d777e7da_1",+ "bdf735de_2",+ "dc832f73",+ "193c23bd_2",+ "e4881cbd_4"+ ],+ "gold_session_ids": [+ "answer_f6168136"+ ]+ },+ {+ "question_id": "6ade9755",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9398da02",+ "sharegpt_BghMUHh_0",+ "910ed479_3",+ "0b64c6cb_1",+ "ec8fa65d_2",+ "7c5ab02a_2",+ "ultrachat_329610",+ "1ddc89e9_3",+ "ultrachat_84599",+ "sharegpt_YsgacQi_0"+ ],+ "gold_session_ids": [+ "answer_9398da02"+ ]+ },+ {+ "question_id": "6f9b354f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_feb5200f",+ "sharegpt_BYHT3Xz_0",+ "ultrachat_549317",+ "sharegpt_VAELSB3_0",+ "60207b41_1",+ "sharegpt_IuLT3VI_9",+ "a79f1a04_1",+ "54d06eeb",+ "36828c66_2",+ "sharegpt_kRjBUsA_79"+ ],+ "gold_session_ids": [+ "answer_feb5200f"+ ]+ },+ {+ "question_id": "58ef2f1c",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_59547700",+ "13f3da23_3",+ "e6ab6a7b_2",+ "ultrachat_5212",+ "e82ce7cd_1",+ "ultrachat_246324",+ "b17c7f18",+ "ultrachat_483737",+ "ultrachat_152126",+ "sharegpt_eHZgwR0_9"+ ],+ "gold_session_ids": [+ "answer_59547700"+ ]+ },+ {+ "question_id": "f8c5f88b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c3567066",+ "sharegpt_TKj3yO8_0",+ "15232887_1",+ "68ace657_1",+ "91880423",+ "58de8f9b",+ "54e9197a",+ "7ff89315_3",+ "sharegpt_n2aGNgW_9",+ "121efb3b_1"+ ],+ "gold_session_ids": [+ "answer_c3567066"+ ]+ },+ {+ "question_id": "5d3d2817",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.0625,+ "retrieved_session_ids": [+ "85d16d01",+ "4ae45145",+ "sharegpt_mtS0qOD_65",+ "sharegpt_ipLglky_48",+ "e7b0637e_3",+ "7dc6f276_1",+ "72f053da_2",+ "4785e1f2_1",+ "25d1830a_1",+ "670ca40e"+ ],+ "gold_session_ids": [+ "answer_235eb6fb"+ ]+ },+ {+ "question_id": "7527f7e2",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7cb94507",+ "sharegpt_cRrfdsc_0",+ "18a06652_3",+ "d81d9846",+ "sharegpt_ToxFax1_535",+ "864a563d_3",+ "4e2acecc",+ "ultrachat_62919",+ "ultrachat_389020",+ "60147f90_1"+ ],+ "gold_session_ids": [+ "answer_7cb94507"+ ]+ },+ {+ "question_id": "c960da58",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e05e4612",+ "f7ffddbc",+ "124f5dc3_2",+ "sharegpt_9iUZNsL_0",+ "b65f51fb_5",+ "ultrachat_374640",+ "89d3053f",+ "sharegpt_ErOTMZ3_59",+ "52381716_1",+ "sharegpt_e328KPR_0"+ ],+ "gold_session_ids": [+ "answer_e05e4612"+ ]+ },+ {+ "question_id": "3b6f954b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "c271c071",+ "answer_94030872",+ "1f0c62de",+ "c566edbb_3",+ "6d3f5c68_4",+ "34590cab_1",+ "f999b05c_2",+ "sharegpt_opQbgPL_7",+ "e27891d3_2",+ "sharegpt_kpnwrGe_12"+ ],+ "gold_session_ids": [+ "answer_94030872"+ ]+ },+ {+ "question_id": "726462e0",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.09090909090909091,+ "retrieved_session_ids": [+ "9c05acfe_2",+ "af2d26f1_1",+ "741c6781_1",+ "29e76903_1",+ "ce144f08",+ "b84b7a2b_1",+ "082889ba_1",+ "5b83c26e_1",+ "6ef99651_2",+ "322a96e8"+ ],+ "gold_session_ids": [+ "answer_f38f679b"+ ]+ },+ {+ "question_id": "94f70d80",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c63c0458",+ "ultrachat_397420",+ "sharegpt_PCqCVmR_18",+ "2479cbae",+ "67045503",+ "09b651aa_3",+ "58de8f9b",+ "eee3f7d3",+ "sharegpt_WQr7EZT_0",+ "sharegpt_mJ7MnQS_31"+ ],+ "gold_session_ids": [+ "answer_c63c0458"+ ]+ },+ {+ "question_id": "66f24dbb",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fea2e4d3",+ "89452526_2",+ "8d015d9d_4",+ "7732b1cc",+ "ca3a4e4f_1",+ "7a10d327_2",+ "798e4ba3_2",+ "8d46b8fa_1",+ "b6858c2a_2",+ "ultrachat_428656"+ ],+ "gold_session_ids": [+ "answer_fea2e4d3"+ ]+ },+ {+ "question_id": "ad7109d1",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_679840f8",+ "a3d2af51_2",+ "sharegpt_RzNjBHR_0",+ "3dc02d26_2",+ "495cf518_1",+ "3fa35683_5",+ "ultrachat_74974",+ "ultrachat_202091",+ "7ad0bd25_1",+ "1d6744b5_1"+ ],+ "gold_session_ids": [+ "answer_679840f8"+ ]+ },+ {+ "question_id": "af8d2e46",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_82a04f59",+ "d5a6f455_2",+ "32a67b35",+ "ced213c6",+ "4ebc182b",+ "82a89303",+ "73d5f10c",+ "43668d77_1",+ "db6ddaba",+ "6f341f96_3"+ ],+ "gold_session_ids": [+ "answer_82a04f59"+ ]+ },+ {+ "question_id": "dccbc061",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8f276838",+ "0b8a4a9d_1",+ "25d1830a_2",+ "5fc843cd",+ "8d74c5f6",+ "ultrachat_243849",+ "0f417163",+ "4af27eab_2",+ "657ab20c_1",+ "ultrachat_520430"+ ],+ "gold_session_ids": [+ "answer_8f276838"+ ]+ },+ {+ "question_id": "c8c3f81d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_761acef8",+ "sharegpt_pRqHb1o_0",+ "19ec83c5_1",+ "66e94928",+ "2b16f3d6_1",+ "3c0ed3bf_4",+ "b818e2a1_1",+ "9bec5593_1",+ "25432e10",+ "e89b6be2_2"+ ],+ "gold_session_ids": [+ "answer_761acef8"+ ]+ },+ {+ "question_id": "8ebdbe50",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8ad8a34f",+ "b53f447a_1",+ "sharegpt_mJ7MnQS_31",+ "27d378b2_3",+ "90f16f14_2",+ "01e806f0_1",+ "d4f7a065_2",+ "03b4d5d8_2",+ "326f41d8",+ "ultrachat_194514"+ ],+ "gold_session_ids": [+ "answer_8ad8a34f"+ ]+ },+ {+ "question_id": "6b168ec8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "sharegpt_D9kTAIU_0",+ "dded1725_3",+ "answer_e623ae87",+ "ultrachat_414934",+ "fb2368b8_3",+ "ultrachat_473",+ "258645ba_1",+ "d58e8baa",+ "afffd8ec_3",+ "ultrachat_377741"+ ],+ "gold_session_ids": [+ "answer_e623ae87"+ ]+ },+ {+ "question_id": "75499fd8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_723bf11f",+ "5328c3c2_2",+ "ultrachat_428593",+ "ultrachat_360018",+ "3b028282",+ "168feef7_1",+ "ultrachat_189069",+ "sharegpt_jBafAKm_0",+ "sharegpt_mMNr1qm_0",+ "ultrachat_466236"+ ],+ "gold_session_ids": [+ "answer_723bf11f"+ ]+ },+ {+ "question_id": "21436231",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1e6d4567",+ "3fa35683_6",+ "7a7a4bf0_4",+ "0d0a89dc_1",+ "b307196d",+ "6bd0b406_1",+ "d0785ee9_1",+ "a0f8468c_3",+ "4bebc783_2",+ "2fd8bf73_1"+ ],+ "gold_session_ids": [+ "answer_1e6d4567"+ ]+ },+ {+ "question_id": "95bcc1c8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cb742a61",+ "99e5e380",+ "f64c4793_2",+ "sharegpt_hRba933_47",+ "abeda028",+ "e201572a",+ "1c1f5ccc_2",+ "c9763bff",+ "sharegpt_J3KJuOr_0",+ "56266c38_1"+ ],+ "gold_session_ids": [+ "answer_cb742a61"+ ]+ },+ {+ "question_id": "0862e8bf",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_MTUnqho_0",+ "answer_c6fd8ebd",+ "ultrachat_240637",+ "10541a2c_1",+ "ultrachat_518141",+ "sharegpt_WhKs0Le_0",+ "95c36d37",+ "sharegpt_2I1TxKW_0",+ "ultrachat_357044",+ "sharegpt_3BkB8g5_0"+ ],+ "gold_session_ids": [+ "answer_c6fd8ebd"+ ]+ },+ {+ "question_id": "853b0a1d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_69811d4a",+ "c70f9f9c",+ "1fdbdfff_5",+ "ebb5bc7c_2",+ "c3feb3bb",+ "sharegpt_i8xL0ih_109",+ "ultrachat_378413",+ "f71bf532_1",+ "sharegpt_n3MgsgL_0",+ "sharegpt_hRba933_47"+ ],+ "gold_session_ids": [+ "answer_69811d4a"+ ]+ },+ {+ "question_id": "a06e4cfe",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6fe9fb49",+ "84fb50bb_3",+ "af257b0b_2",+ "sharegpt_e1JmYDm_0",+ "0c891c0b_1",+ "9de2c795",+ "ultrachat_113982",+ "8dd2d3cc",+ "de64539a_2",+ "sharegpt_3swiY8I_36"+ ],+ "gold_session_ids": [+ "answer_6fe9fb49"+ ]+ },+ {+ "question_id": "37d43f65",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_55161935",+ "1cafd864_3",+ "fec04f53_2",+ "3ab79434_2",+ "sharegpt_brc2wJS_264",+ "4050ebff_5",+ "ultrachat_370801",+ "ultrachat_10500",+ "sharegpt_MOpCbrr_0",+ "ffe473f3_1"+ ],+ "gold_session_ids": [+ "answer_55161935"+ ]+ },+ {+ "question_id": "b86304ba",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "sharegpt_E0YL5SX_142",+ "e831a29f_1",+ "fe1e4351_1",+ "c734e31e_1",+ "ultrachat_328696",+ "91074ce7",+ "answer_645b0329",+ "8e9425ef_1",+ "sharegpt_xGoJZ6Z_0",+ "c1634164_2"+ ],+ "gold_session_ids": [+ "answer_645b0329"+ ]+ },+ {+ "question_id": "d52b4f67",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0df6aa4b",+ "62a6d083",+ "sharegpt_n6Ej34Y_75",+ "d31c3ec8_3",+ "ultrachat_258008",+ "18a06652_6",+ "ultrachat_127631",+ "c578da1f_1",+ "29545df8_1",+ "da1797c4_1"+ ],+ "gold_session_ids": [+ "answer_0df6aa4b"+ ]+ },+ {+ "question_id": "25e5aa4f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_986de8c3",+ "18807892_4",+ "8953c8c8_1",+ "91e581ad_1",+ "7a389bd1_2",+ "1de862d6_2",+ "a459d477",+ "sharegpt_8cFNcVG_0",+ "478e02c2",+ "b049dbb8_2"+ ],+ "gold_session_ids": [+ "answer_986de8c3"+ ]+ },+ {+ "question_id": "caf9ead2",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.31546487678572877,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "86974d49",+ "f8ba3f69",+ "7ae303b7_1",+ "58dd3f35",+ "ultrachat_532107",+ "fc167d00",+ "sharegpt_3D3oQC0_213",+ "answer_0714183a",+ "48a72204_1",+ "ultrachat_495546"+ ],+ "gold_session_ids": [+ "answer_0714183a"+ ]+ },+ {+ "question_id": "8550ddae",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c8354ae9",+ "a7189b75",+ "71cf5aeb_3",+ "9deebbc2_2",+ "8ec23b2c",+ "3b13915f",+ "sharegpt_H8FrXvr_25",+ "7b78417e_1",+ "4f234e2c",+ "e10dfffb_1"+ ],+ "gold_session_ids": [+ "answer_c8354ae9"+ ]+ },+ {+ "question_id": "60d45044",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9cddca88",+ "795a13a7",+ "a76e7e3c_2",+ "35a61ccd_1",+ "e5713180_1",+ "4ebf9095_1",+ "025c9e24_1",+ "af4d4ecd_2",+ "fec059c5",+ "f90c44e6_2"+ ],+ "gold_session_ids": [+ "answer_9cddca88"+ ]+ },+ {+ "question_id": "3f1e9474",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_57fc1954",+ "3aac691d_2",+ "e7497c07_2",+ "954d4451_2",+ "4d408e9d",+ "4f23a396_1",+ "cdf068b1_1",+ "ultrachat_393184",+ "ultrachat_177477",+ "ultrachat_462827"+ ],+ "gold_session_ids": [+ "answer_57fc1954"+ ]+ },+ {+ "question_id": "86b68151",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dc11c1eb",+ "0abaca24_1",+ "sharegpt_7rLq4MQ_9",+ "d1bd26fb_2",+ "3d72c0c0",+ "ultrachat_522663",+ "ultrachat_121159",+ "4e671700",+ "d8454317_4",+ "e200d96c_2"+ ],+ "gold_session_ids": [+ "answer_dc11c1eb"+ ]+ },+ {+ "question_id": "577d4d32",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ultrachat_125084",+ "a126eeab_3",+ "answer_0dd4d99a",+ "2ef53f61_2",+ "ultrachat_57579",+ "b283f9eb_1",+ "6e2cca63_2",+ "dc700e27",+ "92a6a6f7_2",+ "sharegpt_101i5OR_0"+ ],+ "gold_session_ids": [+ "answer_0dd4d99a"+ ]+ },+ {+ "question_id": "ec81a493",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ed1982fc",+ "ab4643a2_4",+ "d2fc150a_1",+ "ultrachat_27488",+ "6f55019d",+ "ultrachat_447961",+ "90fea364_3",+ "616fdd27",+ "ultrachat_413665",+ "6c29f175"+ ],+ "gold_session_ids": [+ "answer_ed1982fc"+ ]+ },+ {+ "question_id": "15745da0",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_586de428",+ "07d33eaa",+ "0d73afc8",+ "ultrachat_155656",+ "768544e1",+ "sharegpt_f28uI6i_0",+ "64a97d9b",+ "sharegpt_KUXG6gY_2",+ "a4f22269",+ "sharegpt_iF4hlNs_5"+ ],+ "gold_session_ids": [+ "answer_586de428"+ ]+ },+ {+ "question_id": "e01b8e2f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "d4cdb464",+ "97338ddd_1",+ "8acfa731",+ "answer_5ca6cd28",+ "1aaf8057_3",+ "ultrachat_185433",+ "ultrachat_556304",+ "sharegpt_9s78QGH_0",+ "b2543a58_3",+ "ultrachat_433104"+ ],+ "gold_session_ids": [+ "answer_5ca6cd28"+ ]+ },+ {+ "question_id": "bc8a6e93",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "f62bfe2b_1",+ "316dc9f8",+ "answer_e6143162",+ "sharegpt_qyWsgp8_13",+ "552c7fd0",+ "c6bed037_1",+ "sharegpt_Us25gXO_18",+ "ultrachat_133339",+ "413b57cb_4",+ "ultrachat_453950"+ ],+ "gold_session_ids": [+ "answer_e6143162"+ ]+ },+ {+ "question_id": "ccb36322",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "04a0b385",+ "bdcee74e_3",+ "ultrachat_395168",+ "940629b2_3",+ "answer_f1fbb330",+ "f79a48dd_1",+ "06d1a1a0",+ "facfdfe2",+ "162ff451",+ "c38b33ae"+ ],+ "gold_session_ids": [+ "answer_f1fbb330"+ ]+ },+ {+ "question_id": "001be529",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_530960c1",+ "0f71db75_1",+ "a244d459",+ "f067b0ee",+ "ultrachat_241736",+ "ed9dad6c_1",+ "sharegpt_6cz1Sq6_328",+ "de877349",+ "sharegpt_PKXxlMF_0",+ "fb793c87_2"+ ],+ "gold_session_ids": [+ "answer_530960c1"+ ]+ },+ {+ "question_id": "b320f3f8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5cc9b056",+ "aae4411b_2",+ "b89f68c4_4",+ "1b71c896_2",+ "40628271_1",+ "sharegpt_4FCMpJR_0",+ "ultrachat_151745",+ "f2797fac",+ "8bb8bc0d",+ "514a7856_2"+ ],+ "gold_session_ids": [+ "answer_5cc9b056"+ ]+ },+ {+ "question_id": "19b5f2b3",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "283e5c7e_1",+ "answer_5ff494b9",+ "275804a8_3",+ "9970ac13_2",+ "sharegpt_I0Wgney_11",+ "ultrachat_447485",+ "51adc57e_2",+ "ultrachat_302261",+ "ultrachat_268389",+ "b951f8c6_2"+ ],+ "gold_session_ids": [+ "answer_5ff494b9"+ ]+ },+ {+ "question_id": "4fd1909e",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2952aee4",+ "94c582bb_2",+ "sharegpt_hO4FoVt_0",+ "ad2135af",+ "2479cbae",+ "ultrachat_472809",+ "sharegpt_I2cfkPp_0",+ "sharegpt_xX1L0kg_29",+ "ultrachat_59418",+ "fef185b1"+ ],+ "gold_session_ids": [+ "answer_2952aee4"+ ]+ },+ {+ "question_id": "545bd2b5",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_47ffab4c",+ "b7f69a7d_1",+ "5fb3b5ac",+ "d1a1b9ea_1",+ "cc021f81_2",+ "cf8e352f_1",+ "a79f1a04_1",+ "1351505a_1",+ "9f9384ed_1",+ "700c16e2_1"+ ],+ "gold_session_ids": [+ "answer_47ffab4c"+ ]+ },+ {+ "question_id": "8a137a7f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_15d63a22",+ "77908e43_1",+ "64460e9d_1",+ "sharegpt_o1SCEXR_23",+ "e05e4612",+ "ultrachat_90882",+ "7738b24b",+ "ultrachat_44939",+ "e3e66b50",+ "sharegpt_W35IQm3_0"+ ],+ "gold_session_ids": [+ "answer_15d63a22"+ ]+ },+ {+ "question_id": "76d63226",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bbdc7b0a",+ "sharegpt_79vV0aT_377",+ "cb289226_1",+ "sharegpt_sRhcMiu_13",+ "sharegpt_Wi7Op1u_91",+ "6a2f9452_2",+ "b0729ec8_1",+ "5f73f2f3",+ "50136b31",+ "adc47df1_1"+ ],+ "gold_session_ids": [+ "answer_bbdc7b0a"+ ]+ },+ {+ "question_id": "86f00804",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96b8c9ee",+ "sharegpt_ErOTMZ3_277",+ "6e672b84_3",+ "35c5419d_abs_3",+ "137d8c6a_2",+ "sharegpt_sCFErnY_22",+ "6339aa1d_2",+ "0424a40a_2",+ "sharegpt_PQ71V9I_7",+ "d98f48f8_3"+ ],+ "gold_session_ids": [+ "answer_96b8c9ee"+ ]+ },+ {+ "question_id": "8e9d538c",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7bdcbd23",+ "sharegpt_tb5H6IH_3",+ "ultrachat_570726",+ "ultrachat_419107",+ "2deed26f",+ "ultrachat_169644",+ "6aaf298d",+ "654da2ba_1",+ "ultrachat_26072",+ "27d18d38"+ ],+ "gold_session_ids": [+ "answer_7bdcbd23"+ ]+ },+ {+ "question_id": "311778f1",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e40b054e",+ "sharegpt_vXNQZ2I_0",+ "d09360d3_1",+ "8e6e058b_3",+ "95cec590_1",+ "ultrachat_473538",+ "5db56d24_2",+ "4fe91719_1",+ "ultrachat_316997",+ "ultrachat_256989"+ ],+ "gold_session_ids": [+ "answer_e40b054e"+ ]+ },+ {+ "question_id": "c19f7a0b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f442ccbe",+ "sharegpt_6ZeW9Vb_0",+ "58ab5fc5",+ "9530353d_1",+ "0e829b8f",+ "0dde26d4_8",+ "c9292210_2",+ "2fd445a2_3",+ "1e739aef",+ "7bc2c7a3_1"+ ],+ "gold_session_ids": [+ "answer_f442ccbe"+ ]+ },+ {+ "question_id": "4100d0a0",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "8d068598",+ "ultrachat_130655",+ "answer_83c13ff9",+ "595902ff",+ "sharegpt_eV0krsR_0",+ "665c09b1_1",+ "4fd958dd_2",+ "fbfa105a",+ "ultrachat_16500",+ "ultrachat_187351"+ ],+ "gold_session_ids": [+ "answer_83c13ff9"+ ]+ },+ {+ "question_id": "29f2956b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7cc5362f",+ "00842aad_2",+ "639cec58",+ "a79db993",+ "6d6ffac5",+ "d50a8a33_1",+ "88361fdd",+ "ultrachat_238255",+ "90f7041a_3",+ "c4ed2287_3"+ ],+ "gold_session_ids": [+ "answer_7cc5362f"+ ]+ },+ {+ "question_id": "1faac195",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d01949bf",+ "f2ccf83b",+ "b2543a58_3",+ "48d385f0_1",+ "ultrachat_12436",+ "d2905eb6_2",+ "9d3d54f7",+ "bcf6953d_3",+ "8064b6ca",+ "70b9616e"+ ],+ "gold_session_ids": [+ "answer_d01949bf"+ ]+ },+ {+ "question_id": "faba32e5",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_39b12014",+ "d3da4592_2",+ "sharegpt_jxsIY6L_0",+ "05944d87_2",+ "sharegpt_OYMr8YY_44",+ "db301f3c_1",+ "sharegpt_3v69R59_0",+ "6b9e8346",+ "83cfab9f_2",+ "770d11f5"+ ],+ "gold_session_ids": [+ "answer_39b12014"+ ]+ },+ {+ "question_id": "f4f1d8a4",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "e45db473_3",+ "answer_f5b33470",+ "3aac691d_2",+ "sharegpt_pX51ZUz_0",+ "sharegpt_vXWOUqx_11",+ "64934c76",+ "76ea88ba_2",+ "sharegpt_7ATc6lt_63",+ "ultrachat_576354",+ "44837adb"+ ],+ "gold_session_ids": [+ "answer_f5b33470"+ ]+ },+ {+ "question_id": "c14c00dd",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_304511ce",+ "sharegpt_KznkJ2p_0",+ "sharegpt_gU0qvZu_0",+ "c77250d2",+ "cd1f5147",+ "8de3201f_1",+ "710f4e21_2",+ "e2cb1647",+ "541ecc45_1",+ "7cd7c296_3"+ ],+ "gold_session_ids": [+ "answer_304511ce"+ ]+ },+ {+ "question_id": "36580ce8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_93e1bd22",+ "ultrachat_149170",+ "ultrachat_270984",+ "9b38bda6_2",+ "ca0a84a3_1",+ "526354c8_2",+ "ultrachat_32917",+ "14076ea0_1",+ "sharegpt_5L3F069_69",+ "2aee3e70"+ ],+ "gold_session_ids": [+ "answer_93e1bd22"+ ]+ },+ {+ "question_id": "3d86fd0a",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_19c24c11",+ "7e4dab66_2",+ "cf8e352f_1",+ "1f643033_2",+ "06d1a1a0",+ "69674f25",+ "ultrachat_231397",+ "1c1a2b7f_2",+ "ultrachat_96218",+ "sharegpt_d7bP5fb_0"+ ],+ "gold_session_ids": [+ "answer_19c24c11"+ ]+ },+ {+ "question_id": "a82c026e",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_787e6a6d",+ "ccaabf0b_1",+ "sharegpt_MBVSMQO_45",+ "3009ba66_2",+ "d9727262_4",+ "752392bb_3",+ "daa32134",+ "sharegpt_vIkUTvg_0",+ "sharegpt_u71pVhw_0",+ "sharegpt_CMeL2P1_0"+ ],+ "gold_session_ids": [+ "answer_787e6a6d"+ ]+ },+ {+ "question_id": "0862e8bf_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.06666666666666667,+ "retrieved_session_ids": [+ "12cd736c",+ "f238edb0_1",+ "ultrachat_532726",+ "94bc18df_3",+ "32833e0b_5",+ "6a8f8453_2",+ "94f0d575_1",+ "sharegpt_PSjpANO_35",+ "ultrachat_210818",+ "3301f749_1"+ ],+ "gold_session_ids": [+ "answer_c6fd8ebd_abs"+ ]+ },+ {+ "question_id": "15745da0_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_586de428_abs",+ "565929a2_2",+ "e839253d",+ "375e27b9_2",+ "9b638f21_4",+ "66d120f4_2",+ "83dae08b",+ "ultrachat_375124",+ "sharegpt_JSxa86k_19",+ "ultrachat_360529"+ ],+ "gold_session_ids": [+ "answer_586de428_abs"+ ]+ },+ {+ "question_id": "bc8a6e93_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "55cf4ce0_2",+ "f51c7700",+ "375354d1_2",+ "answer_e6143162_abs",+ "5abba521",+ "sharegpt_MimvIAy_181",+ "13e9de13_4",+ "sharegpt_SF6KWw1_9",+ "ba944cdc",+ "sharegpt_hn3IS1q_0"+ ],+ "gold_session_ids": [+ "answer_e6143162_abs"+ ]+ },+ {+ "question_id": "19b5f2b3_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_159269",+ "answer_5ff494b9_abs",+ "66138e8d_1",+ "3e59ee68_1",+ "9f8fa5e4_2",+ "sharegpt_v11Tpg9_483",+ "f5cf8815_3",+ "e6640284_1",+ "45c29267",+ "eb134165_2"+ ],+ "gold_session_ids": [+ "answer_5ff494b9_abs"+ ]+ },+ {+ "question_id": "29f2956b_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "ultrachat_70054",+ "503b2761",+ "05b84f71_5",+ "answer_7cc5362f_abs",+ "cc50c6a9",+ "64b7d9cc_2",+ "f2fa97f5_3",+ "7363604d_2",+ "3b8f0d9d",+ "ultrachat_441324"+ ],+ "gold_session_ids": [+ "answer_7cc5362f_abs"+ ]+ },+ {+ "question_id": "f4f1d8a4_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3010299956639812,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "afe04238_1",+ "1dd13331_1",+ "ea3db78e_2",+ "080a4000_2",+ "f252001e",+ "ultrachat_317545",+ "sharegpt_jerQm7S_17",+ "402e0082_6",+ "answer_f5b33470_abs",+ "1a76dc7f_1"+ ],+ "gold_session_ids": [+ "answer_f5b33470_abs"+ ]+ },+ {+ "question_id": "0a995998",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_afa9873b_1",+ "answer_afa9873b_2",+ "sharegpt_igR6Bmm_0",+ "answer_afa9873b_3",+ "e6790684_1",+ "85846900_2",+ "25091584_1",+ "64b847a7_1",+ "ad3d0815_1",+ "439f9ed8_3"+ ],+ "gold_session_ids": [+ "answer_afa9873b_2",+ "answer_afa9873b_3",+ "answer_afa9873b_1"+ ]+ },+ {+ "question_id": "6d550036",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.34620964322385606,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "2e4430d8_2",+ "0bab76de",+ "answer_ec904b3c_1",+ "dae3906e_2",+ "answer_ec904b3c_2",+ "sharegpt_nVi6tIt_0",+ "b37a44e4_5",+ "a9981dc6_3",+ "187b4852_1",+ "5558a42e_2"+ ],+ "gold_session_ids": [+ "answer_ec904b3c_1",+ "answer_ec904b3c_4",+ "answer_ec904b3c_3",+ "answer_ec904b3c_2"+ ]+ },+ {+ "question_id": "gpt4_59c863d7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.502373740777944,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "cf7aad73_5",+ "eb47739f_2",+ "answer_593bdffd_4",+ "answer_593bdffd_1",+ "2f310ba0",+ "answer_593bdffd_2",+ "9ef698bc_2",+ "61d3fec4_2",+ "2f09d4c8",+ "a7d014e4_1"+ ],+ "gold_session_ids": [+ "answer_593bdffd_4",+ "answer_593bdffd_1",+ "answer_593bdffd_3",+ "answer_593bdffd_2"+ ]+ },+ {+ "question_id": "b5ef892d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.34870224750355494,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "sharegpt_hfWd2FK_0",+ "e552e1f9_2",+ "7294438c",+ "sharegpt_esZWTJv_8",+ "answer_a8b4290f_3",+ "answer_a8b4290f_1",+ "ultrachat_565056",+ "35dcacdc_2",+ "eb774a95",+ "25ec099e_1"+ ],+ "gold_session_ids": [+ "answer_a8b4290f_3",+ "answer_a8b4290f_1",+ "answer_a8b4290f_2"+ ]+ },+ {+ "question_id": "e831120c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "773aebbd_4",+ "answer_86c505e7_2",+ "a63ad8e3_2",+ "sharegpt_sDfu38Q_0",+ "answer_86c505e7_1",+ "sharegpt_CROow2Q_0",+ "ultrachat_485311",+ "8fb3fe3a_1",+ "7c82a6c3",+ "f07b6875_1"+ ],+ "gold_session_ids": [+ "answer_86c505e7_1",+ "answer_86c505e7_2"+ ]+ },+ {+ "question_id": "3a704032",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20210734650054757,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "sharegpt_HrxBbBK_0",+ "ultrachat_85818",+ "sharegpt_tzJ6yxo_0",+ "answer_c2204106_1",+ "ultrachat_418858",+ "aa41ac47",+ "ultrachat_357807",+ "f62bfe2b_2",+ "7712aded",+ "5cfdb3bc"+ ],+ "gold_session_ids": [+ "answer_c2204106_2",+ "answer_c2204106_3",+ "answer_c2204106_1"+ ]+ },+ {+ "question_id": "gpt4_d84a3211",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2638647595767405,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "4a32d58c_2",+ "ultrachat_234895",+ "sharegpt_M5zaN8c_0",+ "sharegpt_qHNCIfF_0",+ "answer_2880eb6c_4",+ "ultrachat_83526",+ "51ec01a4",+ "6acb2499",+ "8dd540f8_1",+ "answer_2880eb6c_1"+ ],+ "gold_session_ids": [+ "answer_2880eb6c_2",+ "answer_2880eb6c_4",+ "answer_2880eb6c_1",+ "answer_2880eb6c_3"+ ]+ },+ {+ "question_id": "aae3761f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.619668607939653,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "3d85d91e_2",+ "answer_526354c8_1",+ "38e81260_7",+ "sharegpt_bKkRqZc_0",+ "2a86d0da_2",+ "answer_526354c8_2",+ "answer_526354c8_3",+ "sharegpt_ynLoh9N_0",+ "715176f5_1",+ "d576152e_2"+ ],+ "gold_session_ids": [+ "answer_526354c8_1",+ "answer_526354c8_3",+ "answer_526354c8_2"+ ]+ },+ {+ "question_id": "gpt4_f2262a51",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6105456988825855,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_55a6940c_3",+ "sharegpt_hChsWOp_128",+ "sharegpt_eoEbthf_0",+ "0984a772",+ "ultrachat_268434",+ "dd81b163_1",+ "83e14506_1",+ "c0d6fa6f_2",+ "answer_55a6940c_2",+ "sharegpt_BWMyoNr_0"+ ],+ "gold_session_ids": [+ "answer_55a6940c_3",+ "answer_55a6940c_1",+ "answer_55a6940c_2"+ ]+ },+ {+ "question_id": "dd2973ad",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8175295903539445,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f9de4602_2",+ "sharegpt_O6NO7Vo_9",+ "e5f08e40",+ "6551be60_3",+ "98c198fb",+ "5b8ef613_4",+ "answer_f9de4602_1",+ "56266c38_1",+ "0f73eee7_2",+ "32c0dae1_3"+ ],+ "gold_session_ids": [+ "answer_f9de4602_2",+ "answer_f9de4602_1"+ ]+ },+ {+ "question_id": "c4a1ceb8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9058653015743079,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_56d02cab_3",+ "answer_56d02cab_1",+ "40c77045",+ "a09ae506_1",+ "d0091194",+ "answer_56d02cab_4",+ "answer_56d02cab_2",+ "ultrachat_441000",+ "ultrachat_433406",+ "44ba5b71_1"+ ],+ "gold_session_ids": [+ "answer_56d02cab_3",+ "answer_56d02cab_4",+ "answer_56d02cab_2",+ "answer_56d02cab_1"+ ]+ },+ {+ "question_id": "gpt4_a56e767c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8529278650606567,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf9e3940_3",+ "88c8df0e_3",+ "d75245ea",+ "answer_cf9e3940_1",+ "answer_cf9e3940_2",+ "ultrachat_104129",+ "71621be0",+ "ultrachat_257906",+ "7a819e6f_1",+ "f56e6152_1"+ ],+ "gold_session_ids": [+ "answer_cf9e3940_2",+ "answer_cf9e3940_1",+ "answer_cf9e3940_3"+ ]+ },+ {+ "question_id": "6cb6f249",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5012658353418871,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "ultrachat_404221",+ "sharegpt_IkYYvSd_79",+ "9b6db1a9_2",+ "answer_a4204937_2",+ "answer_a4204937_1",+ "ultrachat_374192",+ "89a1800c_3",+ "5edfe253_1",+ "sharegpt_JTaoEJT_23",+ "ultrachat_219372"+ ],+ "gold_session_ids": [+ "answer_a4204937_2",+ "answer_a4204937_1"+ ]+ },+ {+ "question_id": "46a3abf7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3295827480202853,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "e224317f_2",+ "09f2c6c5",+ "be133e38_1",+ "c1e598cd_1",+ "answer_c65042d7_2",+ "4fe91719_2",+ "ultrachat_42811",+ "answer_c65042d7_3",+ "a95bedca",+ "28eb86ab"+ ],+ "gold_session_ids": [+ "answer_c65042d7_3",+ "answer_c65042d7_2",+ "answer_c65042d7_1"+ ]+ },+ {+ "question_id": "36b9f61e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7039180890341347,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef74281f_1",+ "b283f9eb_5",+ "answer_ef74281f_2",+ "sharegpt_ba4V2an_2",+ "0de92c3f_2",+ "56911dc5_4",+ "d6f5639a_1",+ "93c5e61f",+ "e419b7c3_3",+ "889efc00_2"+ ],+ "gold_session_ids": [+ "answer_ef74281f_2",+ "answer_ef74281f_1",+ "answer_ef74281f_3"+ ]+ },+ {+ "question_id": "28dc39ac",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9669850519151657,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8d015d9d_1",+ "answer_8d015d9d_4",+ "answer_8d015d9d_3",+ "5854eebc_1",+ "answer_8d015d9d_5",+ "3f5df138_1",+ "answer_8d015d9d_2",+ "49b7a8b7_2",+ "1c1f5ccc_2",+ "6ccadc2b"+ ],+ "gold_session_ids": [+ "answer_8d015d9d_3",+ "answer_8d015d9d_5",+ "answer_8d015d9d_2",+ "answer_8d015d9d_4",+ "answer_8d015d9d_1"+ ]+ },+ {+ "question_id": "gpt4_2f8be40d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5307212739772434,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_L64rHOA_0",+ "answer_e7b0637e_2",+ "answer_e7b0637e_1",+ "1d6e01e8",+ "acda6a4e_4",+ "ultrachat_317479",+ "81b971b8_2",+ "62684a60",+ "f0e71553_4",+ "ultrachat_434633"+ ],+ "gold_session_ids": [+ "answer_e7b0637e_2",+ "answer_e7b0637e_1",+ "answer_e7b0637e_3"+ ]+ },+ {+ "question_id": "2e6d26dc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4042974959288707,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "sharegpt_vXNQZ2I_0",+ "ultrachat_121666",+ "e8bfacec_1",+ "14a520c8_2",+ "answer_fa526fc0_3",+ "f863a648_1",+ "answer_fa526fc0_4",+ "answer_fa526fc0_1",+ "sharegpt_h4ZC1fl_152",+ "73f14139"+ ],+ "gold_session_ids": [+ "answer_fa526fc0_4",+ "answer_fa526fc0_3",+ "answer_fa526fc0_1",+ "answer_fa526fc0_2"+ ]+ },+ {+ "question_id": "gpt4_15e38248",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2691828628414762,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "295fcaeb_2",+ "sharegpt_mm9tc5g_0",+ "ultrachat_515012",+ "263efe64_1",+ "0785de5a",+ "answer_8858d9dc_2",+ "answer_8858d9dc_4",+ "ultrachat_442952",+ "8c9bc932_2",+ "87e55e85_2"+ ],+ "gold_session_ids": [+ "answer_8858d9dc_3",+ "answer_8858d9dc_1",+ "answer_8858d9dc_4",+ "answer_8858d9dc_2"+ ]+ },+ {+ "question_id": "88432d0a",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.07142857142857142,+ "retrieved_session_ids": [+ "ultrachat_122782",+ "ca0553dd_1",+ "0e250059_2",+ "f777f641",+ "ultrachat_128882",+ "27e262d5_2",+ "ultrachat_414980",+ "ultrachat_528370",+ "c9d35c00_2",+ "1c8832b4_2"+ ],+ "gold_session_ids": [+ "answer_733e443a_3",+ "answer_733e443a_4",+ "answer_733e443a_2",+ "answer_733e443a_1"+ ]+ },+ {+ "question_id": "80ec1f4f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8179809735263114,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_990c8992_2",+ "f987da7e_3",+ "ultrachat_318441",+ "ultrachat_497987",+ "answer_990c8992_1",+ "answer_990c8992_3",+ "38aec47b_1",+ "2a95f614_1",+ "b6018747_3",+ "sharegpt_nhknQtd_29"+ ],+ "gold_session_ids": [+ "answer_990c8992_2",+ "answer_990c8992_1",+ "answer_990c8992_3"+ ]+ },+ {+ "question_id": "d23cf73b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.298254219601818,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "c328cd4f",+ "95b7c3cd_1",+ "ef71a35e_2",+ "answer_5a0d28f8_4",+ "b124e4a1_1",+ "9e21d6ab_2",+ "answer_5a0d28f8_2",+ "ad44d5c2_2",+ "ultrachat_275858",+ "ultrachat_553398"+ ],+ "gold_session_ids": [+ "answer_5a0d28f8_4",+ "answer_5a0d28f8_2",+ "answer_5a0d28f8_3",+ "answer_5a0d28f8_1"+ ]+ },+ {+ "question_id": "gpt4_7fce9456",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5592067393987519,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a679a86a_4",+ "1da409cd_1",+ "288e7d30",+ "191e1832",+ "04328d1c",+ "541ecc45_3",+ "answer_a679a86a_5",+ "answer_a679a86a_3",+ "76707a65_1",+ "sharegpt_tYc53EZ_13"+ ],+ "gold_session_ids": [+ "answer_a679a86a_5",+ "answer_a679a86a_4",+ "answer_a679a86a_2",+ "answer_a679a86a_3",+ "answer_a679a86a_1"+ ]+ },+ {+ "question_id": "d682f1a2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5323325778787926,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ultrachat_214872",+ "9de29219_1",+ "answer_c008e5df_3",+ "6e2cca63_8",+ "sharegpt_E7VUcfj_0",+ "sharegpt_LFPdgVp_22",+ "answer_c008e5df_2",+ "e30307d8_2",+ "answer_c008e5df_1",+ "58335f60_2"+ ],+ "gold_session_ids": [+ "answer_c008e5df_1",+ "answer_c008e5df_2",+ "answer_c008e5df_3"+ ]+ },+ {+ "question_id": "7024f17c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a21f3697_3",+ "answer_a21f3697_2",+ "answer_a21f3697_1",+ "d7281662_2",+ "8c652eb0_1",+ "3c12ff0c",+ "336b281b_1",+ "199a5225_2",+ "sharegpt_rshCiS5_1",+ "385683f0_1"+ ],+ "gold_session_ids": [+ "answer_a21f3697_1",+ "answer_a21f3697_2",+ "answer_a21f3697_3"+ ]+ },+ {+ "question_id": "gpt4_5501fe77",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8710785440003369,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_203bf3fa_3",+ "bc5827b5",+ "answer_203bf3fa_1",+ "542fa76e_1",+ "021c0d09_2",+ "answer_203bf3fa_2",+ "426e7403_3",+ "ultrachat_303597",+ "631e4016",+ "7a2c618c_1"+ ],+ "gold_session_ids": [+ "answer_203bf3fa_1",+ "answer_203bf3fa_3",+ "answer_203bf3fa_2"+ ]+ },+ {+ "question_id": "gpt4_2ba83207",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.13905617951077562,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "33028509_2",+ "325c2005",+ "sharegpt_JXREDs0_4",+ "131ff17e",+ "41dc5d45_4",+ "answer_6a3b5c13_2",+ "ultrachat_224880",+ "ultrachat_94578",+ "f379d356_2",+ "58d66fec"+ ],+ "gold_session_ids": [+ "answer_6a3b5c13_3",+ "answer_6a3b5c13_1",+ "answer_6a3b5c13_2",+ "answer_6a3b5c13_4"+ ]+ },+ {+ "question_id": "2318644b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_eaa8e3ef_2",+ "6ec23dc5_1",+ "sharegpt_iwcVyKX_0",+ "sharegpt_52wqzr7_0",+ "ultrachat_254829",+ "c5d99bff",+ "bff09560_1",+ "07a26dfa_2",+ "sharegpt_7NcBHFs_0",+ "e44e42dd"+ ],+ "gold_session_ids": [+ "answer_eaa8e3ef_1",+ "answer_eaa8e3ef_2"+ ]+ },+ {+ "question_id": "2ce6a0f2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.24630238874073,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "f257aed3_1",+ "answer_901a6763_4",+ "sharegpt_8W46coJ_3",+ "ultrachat_36844",+ "ultrachat_463514",+ "e4bacbcd_1",+ "a1714d2c_2",+ "89b2a0dc_1",+ "600077b0_3",+ "a267b11e"+ ],+ "gold_session_ids": [+ "answer_901a6763_2",+ "answer_901a6763_4",+ "answer_901a6763_1",+ "answer_901a6763_3"+ ]+ },+ {+ "question_id": "gpt4_d12ceb0e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3228087653971817,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "73f4798f",+ "0672cd7b_1",+ "33d16079_2",+ "ced213c6",+ "answer_2504635e_1",+ "31dc9d66_3",+ "9d97ff74",+ "f0c6ddb9_2",+ "answer_2504635e_2",+ "ee28793c"+ ],+ "gold_session_ids": [+ "answer_2504635e_3",+ "answer_2504635e_2",+ "answer_2504635e_1"+ ]+ },+ {+ "question_id": "00ca467f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.35853358850813305,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "8a70ec05_2",+ "e0fa8486_2",+ "4951c0b9_1",+ "answer_39900a0a_2",+ "7d1de770_2",+ "36828c66_1",+ "answer_39900a0a_3",+ "4a5ee697_1",+ "ultrachat_145200",+ "315a862c"+ ],+ "gold_session_ids": [+ "answer_39900a0a_3",+ "answer_39900a0a_2",+ "answer_39900a0a_1"+ ]+ },+ {+ "question_id": "b3c15d39",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.42177340954886444,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "sharegpt_gPmunOt_37",+ "sharegpt_inuIr9J_119",+ "7bb01253",+ "d298714d_1",+ "answer_05d808e6_1",+ "bd00a24d_2",+ "6be23caa",+ "5d4ff8db_1",+ "answer_05d808e6_2",+ "ultrachat_356582"+ ],+ "gold_session_ids": [+ "answer_05d808e6_1",+ "answer_05d808e6_2"+ ]+ },+ {+ "question_id": "gpt4_31ff4165",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4279411160210267,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "e2393b63_2",+ "sharegpt_WTYjzSr_0",+ "sharegpt_tKmxch5_0",+ "dc9bf721_1",+ "2c185a2b_1",+ "answer_02b63d04_4",+ "2d0b80f1",+ "answer_02b63d04_1",+ "answer_02b63d04_3",+ "answer_02b63d04_5"+ ],+ "gold_session_ids": [+ "answer_02b63d04_1",+ "answer_02b63d04_5",+ "answer_02b63d04_2",+ "answer_02b63d04_3",+ "answer_02b63d04_4"+ ]+ },+ {+ "question_id": "eeda8a6d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "a0f5d30b",+ "answer_3e5fea0e_1",+ "answer_3e5fea0e_2",+ "cf7aad73_3",+ "ultrachat_309574",+ "ultrachat_307733",+ "sharegpt_Caxpnd3_12",+ "b2a021c9_1",+ "sharegpt_khNFc2W_0",+ "034ce94a_1"+ ],+ "gold_session_ids": [+ "answer_3e5fea0e_1",+ "answer_3e5fea0e_2"+ ]+ },+ {+ "question_id": "2788b940",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.25657228426720274,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "sharegpt_dabXXRO_10",+ "ff6e824b_1",+ "544fe66c_2",+ "8de5b7cf_2",+ "sharegpt_vgBnRfE_0",+ "answer_8f6b938d_4",+ "ab483540",+ "sharegpt_hV2NEYj_2",+ "answer_8f6b938d_2",+ "3f863722"+ ],+ "gold_session_ids": [+ "answer_8f6b938d_1",+ "answer_8f6b938d_3",+ "answer_8f6b938d_4",+ "answer_8f6b938d_2"+ ]+ },+ {+ "question_id": "60bf93ed",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5802792108518124,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "d467e3fd",+ "answer_e0956e0a_1",+ "388c5534_3",+ "6414f676_4",+ "0e297c49",+ "4fe27c66_4",+ "ultrachat_7648",+ "answer_e0956e0a_2",+ "619e96bc",+ "84f80d64"+ ],+ "gold_session_ids": [+ "answer_e0956e0a_1",+ "answer_e0956e0a_2"+ ]+ },+ {+ "question_id": "9d25d4e0",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.1480409554829326,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "sharegpt_hRba933_47",+ "2366adbc",+ "464f3821",+ "82d6f027",+ "5f558dee_1",+ "46d9d476",+ "380a9d4d_1",+ "answer_fcff2dc4_2",+ "39911f94",+ "cd568521_3"+ ],+ "gold_session_ids": [+ "answer_fcff2dc4_2",+ "answer_fcff2dc4_1",+ "answer_fcff2dc4_3"+ ]+ },+ {+ "question_id": "129d1232",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.46927872602275644,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1de862d6_2",+ "sharegpt_De0lanR_0",+ "a0b8b0ad_7",+ "ultrachat_227373",+ "ultrachat_274599",+ "ultrachat_88574",+ "1ab255ba_1",+ "eaa8e3ef_2",+ "478c480d",+ "49b78a55_2"+ ],+ "gold_session_ids": [+ "answer_1de862d6_1",+ "answer_1de862d6_3",+ "answer_1de862d6_2"+ ]+ },+ {+ "question_id": "60472f9c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7653606369886217,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e7fe8c8b_1",+ "answer_e7fe8c8b_2",+ "ultrachat_373250",+ "b1815b81",+ "sharegpt_kF8hHMt_45",+ "e7e416f5_2",+ "7b74d32d_3",+ "fdd0f680_2",+ "33da50d0_1",+ "27f5e6e8_1"+ ],+ "gold_session_ids": [+ "answer_e7fe8c8b_1",+ "answer_e7fe8c8b_2",+ "answer_e7fe8c8b_3"+ ]+ },+ {+ "question_id": "gpt4_194be4b3",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5050778624218062,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "sharegpt_UbMVpdp_137",+ "ultrachat_474217",+ "ultrachat_58180",+ "answer_3826dc55_4",+ "answer_3826dc55_3",+ "answer_3826dc55_1",+ "sharegpt_VO7dTG3_0",+ "answer_3826dc55_5",+ "31403c51",+ "ultrachat_163834"+ ],+ "gold_session_ids": [+ "answer_3826dc55_1",+ "answer_3826dc55_3",+ "answer_3826dc55_5",+ "answer_3826dc55_2",+ "answer_3826dc55_4"+ ]+ },+ {+ "question_id": "a9f6b44c",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.16716045496620227,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "ultrachat_539849",+ "837f258b",+ "949d51ef",+ "e10dfffb_1",+ "bb107057_4",+ "answer_cc021f81_3",+ "ultrachat_468681",+ "3489fa38_2",+ "ultrachat_279504",+ "1126be1e_1"+ ],+ "gold_session_ids": [+ "answer_cc021f81_2",+ "answer_cc021f81_3",+ "answer_cc021f81_1"+ ]+ },+ {+ "question_id": "d851d5ba",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6198898455343711,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "561f3a6a_2",+ "2a28999b",+ "answer_5cdf9bd2_3",+ "answer_5cdf9bd2_2",+ "d6d8b854_4",+ "answer_5cdf9bd2_1",+ "9f8973ac",+ "sharegpt_x7apNZ0_0",+ "answer_5cdf9bd2_4",+ "66a6a097_3"+ ],+ "gold_session_ids": [+ "answer_5cdf9bd2_2",+ "answer_5cdf9bd2_1",+ "answer_5cdf9bd2_3",+ "answer_5cdf9bd2_4"+ ]+ },+ {+ "question_id": "5a7937c8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8385465274895063,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4cef8a3c_2",+ "b70ac29f_2",+ "3b38be11_2",+ "answer_4cef8a3c_3",+ "559ce9c2",+ "answer_4cef8a3c_1",+ "ultrachat_564068",+ "ultrachat_377263",+ "ultrachat_355239",+ "63c9e87c"+ ],+ "gold_session_ids": [+ "answer_4cef8a3c_3",+ "answer_4cef8a3c_1",+ "answer_4cef8a3c_2"+ ]+ },+ {+ "question_id": "gpt4_ab202e7f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9103075074400794,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_728deb4d_3",+ "answer_728deb4d_5",+ "29bc69b3",+ "answer_728deb4d_4",+ "a08fbe88_1",+ "dcafb5b3_5",+ "answer_728deb4d_2",+ "a394f6b5_2",+ "ultrachat_433876",+ "answer_728deb4d_1"+ ],+ "gold_session_ids": [+ "answer_728deb4d_5",+ "answer_728deb4d_2",+ "answer_728deb4d_3",+ "answer_728deb4d_1",+ "answer_728deb4d_4"+ ]+ },+ {+ "question_id": "gpt4_e05b82a6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9279611693743142,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6350aa4f_1",+ "answer_6350aa4f_2",+ "386f6df9",+ "answer_6350aa4f_4",+ "e2cd250e_2",+ "6172cc72_1",+ "503791f7_1",+ "answer_6350aa4f_3",+ "ultrachat_43272",+ "88e01130_1"+ ],+ "gold_session_ids": [+ "answer_6350aa4f_1",+ "answer_6350aa4f_2",+ "answer_6350aa4f_3",+ "answer_6350aa4f_4"+ ]+ },+ {+ "question_id": "gpt4_731e37d7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_826d51da_3",+ "answer_826d51da_1",+ "answer_826d51da_2",+ "answer_826d51da_4",+ "ultrachat_109542",+ "267a912c",+ "5eec1ec6",+ "65747d84_1",+ "baa5b468_3",+ "7e4aa7c2_1"+ ],+ "gold_session_ids": [+ "answer_826d51da_3",+ "answer_826d51da_4",+ "answer_826d51da_2",+ "answer_826d51da_1"+ ]+ },+ {+ "question_id": "edced276",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3706656904013186,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "1da409cd_1",+ "e4bbcd5f_4",+ "ultrachat_195214",+ "5726dc37_2",+ "b86848e2_3",+ "d89f6204",+ "2dc55862_1",+ "answer_60e8941a_2",+ "93f23301_1",+ "answer_60e8941a_1"+ ],+ "gold_session_ids": [+ "answer_60e8941a_2",+ "answer_60e8941a_1"+ ]+ },+ {+ "question_id": "10d9b85a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e0585cb5_2",+ "c51583cd_3",+ "3c8e7c0e_4",+ "ultrachat_143866",+ "5cbfaf3e_4",+ "d576152e_1",+ "ebe3b6bf_1",+ "ultrachat_501022",+ "9c5fa973",+ "sharegpt_Sionhxj_115"+ ],+ "gold_session_ids": [+ "answer_e0585cb5_2",+ "answer_e0585cb5_1"+ ]+ },+ {+ "question_id": "e3038f8c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4303349101522581,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "e3d4f89e_3",+ "d71c8b77_5",+ "85846900_2",+ "answer_b6018747_4",+ "a3d8e134_2",+ "answer_b6018747_3",+ "ultrachat_48888",+ "answer_b6018747_2",+ "941d586a",+ "6e2cca63_3"+ ],+ "gold_session_ids": [+ "answer_b6018747_2",+ "answer_b6018747_4",+ "answer_b6018747_1",+ "answer_b6018747_3"+ ]+ },+ {+ "question_id": "2b8f3739",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_23759615_3",+ "answer_23759615_1",+ "dd25eeb5_6",+ "answer_23759615_2",+ "b8ed5202_2",+ "ce6cae1b_3",+ "e9fb10e7_2",+ "ec8691f5",+ "52ccfdfa",+ "465c3970"+ ],+ "gold_session_ids": [+ "answer_23759615_2",+ "answer_23759615_3",+ "answer_23759615_1"+ ]+ },+ {+ "question_id": "1a8a66a6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3764290720714305,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "54aa4875",+ "answer_2bd23659_1",+ "sharegpt_mANdO1O_0",+ "22eebf8a_2",+ "d6792df8",+ "sharegpt_P01BTRw_0",+ "answer_2bd23659_2",+ "ultrachat_280864",+ "18fb3a22_3",+ "sharegpt_tyukJll_27"+ ],+ "gold_session_ids": [+ "answer_2bd23659_3",+ "answer_2bd23659_2",+ "answer_2bd23659_4",+ "answer_2bd23659_1"+ ]+ },+ {+ "question_id": "c2ac3c61",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5109559939712153,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "8e4e215b",+ "6009ba4e_5",+ "answer_923c0221_2",+ "0c483a3c",+ "ultrachat_412857",+ "sharegpt_z6pSZ19_0",+ "answer_923c0221_1",+ "sharegpt_gLb26GY_0",+ "b403d6f0_4",+ "97338ddd_1"+ ],+ "gold_session_ids": [+ "answer_923c0221_1",+ "answer_923c0221_2"+ ]+ },+ {+ "question_id": "bf659f65",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7726e7e9_1",+ "answer_7726e7e9_2",+ "f2835879_2",+ "answer_7726e7e9_3",+ "ultrachat_365864",+ "2917b130_2",+ "sharegpt_inuIr9J_119",+ "ultrachat_377765",+ "258645ba_3",+ "385a60c8_3"+ ],+ "gold_session_ids": [+ "answer_7726e7e9_1",+ "answer_7726e7e9_3",+ "answer_7726e7e9_2"+ ]+ },+ {+ "question_id": "gpt4_372c3eed",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6049306994552042,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c5419d_1",+ "10647770",+ "9a22ae03",+ "ultrachat_400797",+ "ultrachat_209842",+ "sharegpt_6dPUCcK_69",+ "8d969f3e_3",+ "b661e1d3_1",+ "c95f88fd_1",+ "answer_35c5419d_3"+ ],+ "gold_session_ids": [+ "answer_35c5419d_3",+ "answer_35c5419d_2",+ "answer_35c5419d_1"+ ]+ },+ {+ "question_id": "gpt4_2f91af09",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8710785440003369,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_669318cf_3",+ "e7e416f5_2",+ "answer_669318cf_2",+ "22b3af37_1",+ "d75245ea",+ "answer_669318cf_1",+ "9b85a31d_1",+ "51c82c8e_2",+ "fd6f60f0_2",+ "ultrachat_296153"+ ],+ "gold_session_ids": [+ "answer_669318cf_2",+ "answer_669318cf_1",+ "answer_669318cf_3"+ ]+ },+ {+ "question_id": "81507db6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7809509050749017,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da3c1266_5",+ "answer_da3c1266_1",+ "sharegpt_Caxpnd3_9",+ "7da6f798_1",+ "ultrachat_254248",+ "answer_da3c1266_4",+ "ultrachat_229430",+ "answer_da3c1266_2",+ "ultrachat_162129",+ "sharegpt_AZELQqX_0"+ ],+ "gold_session_ids": [+ "answer_da3c1266_3",+ "answer_da3c1266_4",+ "answer_da3c1266_1",+ "answer_da3c1266_5",+ "answer_da3c1266_2"+ ]+ },+ {+ "question_id": "88432d0a_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2476427880871277,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "1e01dbcc_1",+ "09d8b29e_1",+ "ultrachat_453194",+ "11f043cd_2",+ "8e5b0424",+ "4eff1baa",+ "answer_733e443a_abs_1",+ "9863a1c8_2",+ "answer_733e443a_abs_3",+ "1e215f06"+ ],+ "gold_session_ids": [+ "answer_733e443a_abs_1",+ "answer_733e443a_abs_3",+ "answer_733e443a_abs_2",+ "answer_733e443a_abs_4"+ ]+ },+ {+ "question_id": "80ec1f4f_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7122630665145961,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "b304b283_1",+ "answer_990c8992_abs_2",+ "answer_990c8992_abs_1",+ "b1c6d332",+ "answer_990c8992_abs_3",+ "31829c1b_1",+ "ultrachat_187094",+ "ultrachat_544442",+ "b341cf37_2",+ "e5f9453d"+ ],+ "gold_session_ids": [+ "answer_990c8992_abs_2",+ "answer_990c8992_abs_3",+ "answer_990c8992_abs_1"+ ]+ },+ {+ "question_id": "eeda8a6d_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "a0b8b0ad_5",+ "answer_3e5fea0e_abs_2",+ "ultrachat_544853",+ "answer_3e5fea0e_abs_1",+ "sharegpt_l7gyE1o_17",+ "f4eaef47",+ "ultrachat_199093",+ "d15d2899_4",+ "ultrachat_124888",+ "sharegpt_YpleI8A_285"+ ],+ "gold_session_ids": [+ "answer_3e5fea0e_abs_2",+ "answer_3e5fea0e_abs_1"+ ]+ },+ {+ "question_id": "60bf93ed_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "ultrachat_490706",+ "755c2d32_1",+ "1e91cdf0",+ "3e82fae8_1",+ "0944933f_1",+ "2946bf12",+ "sharegpt_7mz4EV7_0",+ "sharegpt_6cz1Sq6_130",+ "9ef698bc_2",+ "cdf068b1_3"+ ],+ "gold_session_ids": [+ "answer_e0956e0a_abs_2",+ "answer_e0956e0a_abs_1"+ ]+ },+ {+ "question_id": "edced276_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.19342640361727081,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "sharegpt_u1AM5RT_88",+ "66138e8d_1",+ "ultrachat_410709",+ "7a2bdd15_2",+ "ultrachat_305072",+ "7f4cd11e_1",+ "sharegpt_1ZrnYv8_2",+ "answer_60e8941a_abs_1",+ "8b8e7cf8",+ "sharegpt_YP2Ynd9_0"+ ],+ "gold_session_ids": [+ "answer_60e8941a_abs_1",+ "answer_60e8941a_abs_2"+ ]+ },+ {+ "question_id": "gpt4_372c3eed_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.46927872602275644,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c5419d_abs_1",+ "b0a9de7f_1",+ "6d3f5c68_4",+ "54aa4875",+ "9282ea7c_3",+ "85dec0b4_1",+ "3214139b_1",+ "39948bcf_1",+ "e6cc6157_4",+ "48021f90_1"+ ],+ "gold_session_ids": [+ "answer_35c5419d_abs_3",+ "answer_35c5419d_abs_1",+ "answer_35c5419d_abs_2"+ ]+ },+ {+ "question_id": "8a2466db",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "6a5b5a78",+ "answer_edb03329",+ "6be54739_3",+ "3d0c9f89_1",+ "sharegpt_MKMWjX0_25",+ "e3cc9931_1",+ "ultrachat_10500",+ "a2e2cb72_3",+ "b67748d1_1",+ "6dcf5fa0_1"+ ],+ "gold_session_ids": [+ "answer_edb03329"+ ]+ },+ {+ "question_id": "06878be2",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "465e7ef9",+ "8fcfbe43_2",+ "a6074da9_1",+ "sharegpt_GSC090N_6",+ "ultrachat_556731",+ "ultrachat_319509",+ "answer_555dfb94",+ "a67c1862_4",+ "sharegpt_5cxEGva_147",+ "ca929779_1"+ ],+ "gold_session_ids": [+ "answer_555dfb94"+ ]+ },+ {+ "question_id": "75832dbd",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.05,+ "retrieved_session_ids": [+ "1ef63c66",+ "sharegpt_xni9yYO_6",+ "6172cc72_1",+ "47ec1674_2",+ "c51583cd_1",+ "d49b1a24_1",+ "ultrachat_165893",+ "e765a945_2",+ "c884b67a_1",+ "c7270a1b_1"+ ],+ "gold_session_ids": [+ "answer_d87a6ef8"+ ]+ },+ {+ "question_id": "0edc2aef",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "8464304d_2",+ "6a747f2e",+ "f20e72e4_1",+ "53dc1394",+ "08ca1f31_2",+ "answer_d586e9cd",+ "25b1afbc",+ "sharegpt_aVExml0_92",+ "3f4611f4",+ "33a813dd_2"+ ],+ "gold_session_ids": [+ "answer_d586e9cd"+ ]+ },+ {+ "question_id": "35a27287",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "880ee13e",+ "58ae7c4f_3",+ "answer_9b182436",+ "203bf3fa_2",+ "a98ff421_1",+ "ultrachat_152094",+ "e4a1b565_2",+ "64071e19_2",+ "c9b1c309_4",+ "02f2ab8e_2"+ ],+ "gold_session_ids": [+ "answer_9b182436"+ ]+ },+ {+ "question_id": "32260d93",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "sharegpt_q2uAW7H_0",+ "3f787a78_4",+ "573dc24b_1",+ "6d3f5c68_4",+ "sharegpt_zciCXP1_0",+ "1b26bdd5_5",+ "answer_0250ae1c",+ "ultrachat_136348",+ "ultrachat_171173",+ "60ff3a8c"+ ],+ "gold_session_ids": [+ "answer_0250ae1c"+ ]+ },+ {+ "question_id": "195a1a1b",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.0625,+ "retrieved_session_ids": [+ "32833e0b_3",+ "sharegpt_WVr7Jiq_0",+ "ce79a004",+ "89a1800c_2",+ "sharegpt_O3mtxv4_0",+ "cc50c6a9",+ "9f12daa1_1",+ "af0ab6ab",+ "ultrachat_510178",+ "d03b6a05_2"+ ],+ "gold_session_ids": [+ "answer_6dc4305e"+ ]+ },+ {+ "question_id": "afdc33df",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8549e5e0",+ "d03b6a05_3",+ "ultrachat_214872",+ "43b53ac4_1",+ "d5128b51",+ "ad8d9f0f",+ "22f9f163_3",+ "ultrachat_399348",+ "96b6e5e8_2",+ "b6558bfb"+ ],+ "gold_session_ids": [+ "answer_8549e5e0"+ ]+ },+ {+ "question_id": "caf03d32",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2fc6aabb",+ "a52e072b",+ "ec806228",+ "6b137bce_4",+ "eb0d8dc1_6",+ "bd6d0687",+ "ultrachat_200859",+ "4b1af06f_1",+ "78369dbb",+ "6f0e184f"+ ],+ "gold_session_ids": [+ "answer_2fc6aabb"+ ]+ },+ {+ "question_id": "54026fce",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "670c49e6",+ "1c95e152_1",+ "answer_f7b22c66",+ "518d26d3_6",+ "6e92e417",+ "5600db68_1",+ "e68d6ffc_1",+ "1fcb4134_3",+ "52c34859_2",+ "d60381f2_1"+ ],+ "gold_session_ids": [+ "answer_f7b22c66"+ ]+ },+ {+ "question_id": "06f04340",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "sharegpt_Daapfhj_0",+ "be2cb665",+ "42924d15",+ "14f9ee3c",+ "b459f888_3",+ "91223fd5_1",+ "66bfa1db",+ "0844dea6",+ "728deb4d_4",+ "4fba4832_1"+ ],+ "gold_session_ids": [+ "answer_92d5f7cd"+ ]+ },+ {+ "question_id": "6b7dfb22",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6502d0f",+ "56859d37",+ "574df152_1",+ "77a26018",+ "8489e4ce_1",+ "377f8fd9_1",+ "e7b0637e_3",+ "fc69fd44_3",+ "e2e29a09",+ "ultrachat_148325"+ ],+ "gold_session_ids": [+ "answer_f6502d0f"+ ]+ },+ {+ "question_id": "1a1907b4",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "e42e7876_5",+ "743f03a1_abs_1",+ "4f3b36d4_5",+ "d6f5639a_1",+ "78dbd237",+ "e622a1bd",+ "61d3fec4_1",+ "8e78fa70_1",+ "4e3da326_1",+ "aa930b56_1"+ ],+ "gold_session_ids": [+ "answer_719502eb"+ ]+ },+ {+ "question_id": "09d032c9",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "d3971322_1",+ "2a17ea02_1",+ "3fc2244f",+ "sharegpt_Wd2Dps4_0",+ "26d9aaaf",+ "21ef2d05_1",+ "46bab85b",+ "16ebc8f8_4",+ "af631aa3_2",+ "1fe34a92"+ ],+ "gold_session_ids": [+ "answer_b10dce5e"+ ]+ },+ {+ "question_id": "38146c39",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "d22c2f04_5",+ "a86b30e4",+ "42234f98",+ "15ed03f0_1",+ "e5b1bc37",+ "answer_772472c8",+ "5806662d_1",+ "87472dfd_3",+ "286aa281_5",+ "0908a2e6"+ ],+ "gold_session_ids": [+ "answer_772472c8"+ ]+ },+ {+ "question_id": "d24813b1",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7c0ade93",+ "c2a34674_1",+ "b7e64b5c_1",+ "3124bb28_1",+ "084802f9_3",+ "22ce6625_2",+ "9530353d_1",+ "e08bf81f_4",+ "61a46ff7_3",+ "128f4e4d_3"+ ],+ "gold_session_ids": [+ "answer_7c0ade93"+ ]+ },+ {+ "question_id": "57f827a0",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "f64c2982_1",+ "answer_1bde8d3b",+ "ultrachat_18651",+ "d05d6391_1",+ "1bff1675",+ "6e110a53_1",+ "762d288d_2",+ "e85e0eaf_1",+ "e9ad5914_1",+ "2ef55f49_1"+ ],+ "gold_session_ids": [+ "answer_1bde8d3b"+ ]+ },+ {+ "question_id": "95228167",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e613445",+ "9f80607d",+ "99e0fd9f_3",+ "f75238a5_2",+ "sharegpt_fwXicqL_262",+ "aa71db11_1",+ "0bac2789",+ "3d2e66f2_2",+ "7e17ae56_3",+ "c34b6a1c_1"+ ],+ "gold_session_ids": [+ "answer_5e613445"+ ]+ },+ {+ "question_id": "505af2f5",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "66bf4b03_1",+ "answer_f3164f2c",+ "4f2d6be4_2",+ "9216c4d3_4",+ "ultrachat_342931",+ "56cdb8b8_2",+ "69a42467_1",+ "sharegpt_i59HUGf_0",+ "ebeb4c51",+ "629f17b4_2"+ ],+ "gold_session_ids": [+ "answer_f3164f2c"+ ]+ },+ {+ "question_id": "75f70248",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8ee04a2e",+ "dcc8fc7f_2",+ "ff67236f_3",+ "adb1f904",+ "95c9666f_5",+ "2d74df23_4",+ "15e366a1_1",+ "6c5ef9fe",+ "c1d13b9e",+ "6ac43c5b_4"+ ],+ "gold_session_ids": [+ "answer_8ee04a2e"+ ]+ },+ {+ "question_id": "d6233ab6",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "2c81cdf8_3",+ "sharegpt_k4i6Hmp_8",+ "87cfeb28_2",+ "0e726047",+ "cb05ed10",+ "7af38385_2",+ "e419b7c3_4",+ "e064120a",+ "426e7403_2",+ "c927ffbb"+ ],+ "gold_session_ids": [+ "answer_b0fac439"+ ]+ },+ {+ "question_id": "1da05512",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "f5a2e179_1",+ "sharegpt_mANdO1O_66",+ "ae1cd5f3",+ "answer_4d3be2ab",+ "ce3ad11f_2",+ "aea71da3_2",+ "da1ea197_1",+ "sharegpt_eSTYwMj_14",+ "402e0082_4",+ "6b93018e_2"+ ],+ "gold_session_ids": [+ "answer_4d3be2ab"+ ]+ },+ {+ "question_id": "fca70973",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "f504b269",+ "312bf939",+ "fc005760",+ "5d5e80c5_1",+ "sharegpt_lSeiu11_0",+ "2f2884ad_1",+ "answer_a1e169b1",+ "83ea6306_1",+ "4e3f0831_1",+ "09cff8ac"+ ],+ "gold_session_ids": [+ "answer_a1e169b1"+ ]+ },+ {+ "question_id": "b6025781",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8414cc57",+ "f20c73f5_3",+ "6446f6e6",+ "2ef53f61_1",+ "ddb3ab46",+ "bbca6598_3",+ "1a693999_4",+ "22db3cc3_5",+ "1d6744b5_1",+ "89d49715_2"+ ],+ "gold_session_ids": [+ "answer_8414cc57"+ ]+ },+ {+ "question_id": "a89d7624",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "a479a7d6_4",+ "082fc07f_1",+ "99f2f5b1_1",+ "sharegpt_G3RxeFJ_94",+ "answer_8f15ac24",+ "39a056ae",+ "685d0b0d",+ "ultrachat_30787",+ "84f13d49_2",+ "02f70006_1"+ ],+ "gold_session_ids": [+ "answer_8f15ac24"+ ]+ },+ {+ "question_id": "b0479f84",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_30f63ddb",+ "4f37e5ec",+ "abc2ac45",+ "6af947bf_1",+ "1cde7ee4_1",+ "79019170_1",+ "ultrachat_263918",+ "457b1adb_1",+ "ultrachat_232424",+ "94cffbd2_2"+ ],+ "gold_session_ids": [+ "answer_30f63ddb"+ ]+ },+ {+ "question_id": "1d4e3b97",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "809cabca_1",+ "fb2368b8_3",+ "5cd6ab1b",+ "243ca32d",+ "c7772f2d_2",+ "ecb24dd8_2",+ "answer_e6b6353d",+ "fd26f9f4_1",+ "c51583cd_3",+ "8c7ae899_2"+ ],+ "gold_session_ids": [+ "answer_e6b6353d"+ ]+ },+ {+ "question_id": "07b6f563",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "5e7b1c98_1",+ "answer_d03098f9",+ "55c267aa",+ "fd18e0dd_2",+ "ddf41a24",+ "sharegpt_72kPQPm_11",+ "0ac1a6f9_2",+ "66ab6260",+ "dcc7f430_1",+ "ultrachat_480403"+ ],+ "gold_session_ids": [+ "answer_d03098f9"+ ]+ },+ {+ "question_id": "1c0ddc50",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8da8c7e0",+ "b33e89b5_1",+ "2f0c1f4e",+ "bdf33c73_2",+ "2aa70c9c_1",+ "c7ca6dff",+ "a0aa5035",+ "3d006c58_2",+ "2566382f_2",+ "0a7d5bf6_2"+ ],+ "gold_session_ids": [+ "answer_8da8c7e0"+ ]+ },+ {+ "question_id": "0a34ad58",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9ca2a195_1",+ "answer_cebb7159",+ "1e65e79a",+ "90342357_3",+ "f849dd04",+ "4b495a04_1",+ "58ad78d6_2",+ "51fe163e_2",+ "b869d7ed_2",+ "dd3d8e73_1"+ ],+ "gold_session_ids": [+ "answer_cebb7159"+ ]+ },+ {+ "question_id": "d3ab962e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5802792108518124,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "166d4800",+ "answer_5237bb0b_2",+ "1980bbfa_2",+ "ultrachat_164834",+ "ultrachat_155769",+ "b66e4f25_2",+ "013b320f",+ "answer_5237bb0b_1",+ "ultrachat_503039",+ "ae15e8b6_3"+ ],+ "gold_session_ids": [+ "answer_5237bb0b_2",+ "answer_5237bb0b_1"+ ]+ },+ {+ "question_id": "2311e44b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bf633415_1",+ "9b3611d7_1",+ "answer_bf633415_2",+ "16bc615e_1",+ "ultrachat_458191",+ "5854eebc_2",+ "1e53c034",+ "77753dd9_1",+ "81ab8aec_1",+ "sharegpt_hi2xiJT_14"+ ],+ "gold_session_ids": [+ "answer_bf633415_2",+ "answer_bf633415_1"+ ]+ },+ {+ "question_id": "cc06de0d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4fb01417_1",+ "670c49e6",+ "01551a39_2",+ "answer_4fb01417_2",+ "91c71874_2",+ "d5eab084_2",+ "ccb9811f",+ "9863a1c8_3",+ "92d74e73",+ "55033f6f"+ ],+ "gold_session_ids": [+ "answer_4fb01417_2",+ "answer_4fb01417_1"+ ]+ },+ {+ "question_id": "a11281a2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c69ee1f9_2",+ "answer_c69ee1f9_1",+ "7a88956e",+ "4b1af06f_2",+ "9cda33e5",+ "bb2180e9_3",+ "c4c4d85a",+ "ultrachat_55839",+ "9208d3be",+ "14feb919"+ ],+ "gold_session_ids": [+ "answer_c69ee1f9_2",+ "answer_c69ee1f9_1"+ ]+ },+ {+ "question_id": "4f54b7c9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_50940cb7_1",+ "answer_50940cb7_2",+ "b89f68c4_1",+ "22eebf8a_2",+ "8a834aee_1",+ "bcfd4ab7",+ "663337ce_1",+ "dde142b2_1",+ "sharegpt_NPdQIWB_0",+ "ultrachat_576691"+ ],+ "gold_session_ids": [+ "answer_50940cb7_2",+ "answer_50940cb7_1"+ ]+ },+ {+ "question_id": "85fa3a3f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dcb18b9b_2",+ "answer_dcb18b9b_1",+ "b7fd3b48",+ "4f52229e",+ "83f89c02",+ "0d66562c_1",+ "71110dbd_1",+ "93ff2f73",+ "8f707521_1",+ "c6c06e6e"+ ],+ "gold_session_ids": [+ "answer_dcb18b9b_2",+ "answer_dcb18b9b_1"+ ]+ },+ {+ "question_id": "9aaed6a3",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8175295903539445,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_353d3c6d_2",+ "c846422a_1",+ "11e942da_2",+ "3b715074",+ "sharegpt_HdWyKUy_0",+ "c95f2a09_2",+ "answer_353d3c6d_1",+ "ae332229_2",+ "3f9f66f1_1",+ "2e89bd03_1"+ ],+ "gold_session_ids": [+ "answer_353d3c6d_2",+ "answer_353d3c6d_1"+ ]+ },+ {+ "question_id": "1f2b8d4f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8454588_1",+ "9ee4da12_1",+ "answer_d8454588_2",+ "d3ee5958_2",+ "be1c6872",+ "76a39abc",+ "ultrachat_401506",+ "ultrachat_391569",+ "c8c64252",+ "ultrachat_413177"+ ],+ "gold_session_ids": [+ "answer_d8454588_1",+ "answer_d8454588_2"+ ]+ },+ {+ "question_id": "e6041065",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4eb6d671_1",+ "answer_4eb6d671_2",+ "d79173aa_1",+ "bdcee74e_3",+ "4a006b77",+ "ultrachat_407355",+ "sharegpt_YV9bnz5_0",+ "832e4ecf",+ "4c053e7e_1",+ "b73a4d04_1"+ ],+ "gold_session_ids": [+ "answer_4eb6d671_2",+ "answer_4eb6d671_1"+ ]+ },+ {+ "question_id": "51c32626",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_58820c75_1",+ "answer_58820c75_2",+ "sharegpt_hzwc0HI_13",+ "c051e740_2",+ "b38766c1_3",+ "809cbce9_1",+ "sharegpt_jBafAKm_0",+ "932a97c6",+ "932ea3bf_1",+ "fb62ade8"+ ],+ "gold_session_ids": [+ "answer_58820c75_1",+ "answer_58820c75_2"+ ]+ },+ {+ "question_id": "d905b33f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_85a77c48_1",+ "8cfa91c3_2",+ "8b69d1ae",+ "6423d6a4_2",+ "answer_85a77c48_2",+ "ultrachat_381881",+ "2d364fb0",+ "3301f749_1",+ "368276ab_3",+ "dd4a0e15"+ ],+ "gold_session_ids": [+ "answer_85a77c48_2",+ "answer_85a77c48_1"+ ]+ },+ {+ "question_id": "7405e8b1",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_80323f3f_1",+ "answer_80323f3f_2",+ "f2f2a606_3",+ "a6b00510_4",+ "c578da1f_1",+ "6e983235_1",+ "8eeb4704",+ "6c1c5bc3_2",+ "643d285a_3",+ "e40d733b_2"+ ],+ "gold_session_ids": [+ "answer_80323f3f_1",+ "answer_80323f3f_2"+ ]+ },+ {+ "question_id": "f35224e0",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "fd7becaf_1",+ "answer_e9bb9500_2",+ "8e608386",+ "2ea6fda4_1",+ "answer_e9bb9500_1",+ "96da07f9_4",+ "4b285f83",+ "e184e4c3_1",+ "sharegpt_0DiDwpY_23",+ "sharegpt_Pqhvz5n_0"+ ],+ "gold_session_ids": [+ "answer_e9bb9500_2",+ "answer_e9bb9500_1"+ ]+ },+ {+ "question_id": "6456829e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_743f03a1_2",+ "c44d1533_2",+ "answer_743f03a1_1",+ "ultrachat_247641",+ "a8c73552",+ "97d3d674",+ "fe7b6b40_1",+ "385683f0_3",+ "ultrachat_494325",+ "sharegpt_LxQSGQ9_0"+ ],+ "gold_session_ids": [+ "answer_743f03a1_2",+ "answer_743f03a1_1"+ ]+ },+ {+ "question_id": "a4996e51",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_feb5f98a_2",+ "answer_feb5f98a_1",+ "sharegpt_LYRqOF7_9",+ "28a64f28_2",+ "4cc9270f",+ "b1d9eb66_3",+ "16bd5ea5_1",+ "252715f8_3",+ "600077b0_1",+ "2ca99347_2"+ ],+ "gold_session_ids": [+ "answer_feb5f98a_1",+ "answer_feb5f98a_2"+ ]+ },+ {+ "question_id": "3c1045c8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_20893",+ "answer_c8cc60d6_2",+ "ecd24d5e_1",+ "sharegpt_jUUvPID_0",+ "b52f03a1_1",+ "sharegpt_NTHvrGV_0",+ "sharegpt_oIB2JG9_0",+ "50b705ab_2",+ "sharegpt_jSnesqJ_43",+ "be32ad25_2"+ ],+ "gold_session_ids": [+ "answer_c8cc60d6_2",+ "answer_c8cc60d6_1"+ ]+ },+ {+ "question_id": "60036106",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e552e1f9_1",+ "answer_e552e1f9_2",+ "sharegpt_moARQa7_0",+ "sharegpt_mtS0qOD_65",+ "763f028a",+ "c575291e_2",+ "fcc6d66d_2",+ "10857212_2",+ "e2cdd030",+ "43283702_1"+ ],+ "gold_session_ids": [+ "answer_e552e1f9_1",+ "answer_e552e1f9_2"+ ]+ },+ {+ "question_id": "681a1674",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3be95d43_2",+ "answer_3be95d43_1",+ "ultrachat_436833",+ "ultrachat_64650",+ "9e411500_1",+ "7d33fcb4_2",+ "sharegpt_8Kzmn95_0",+ "ultrachat_11300",+ "50e8f2b5_1",+ "a21c4257"+ ],+ "gold_session_ids": [+ "answer_3be95d43_1",+ "answer_3be95d43_2"+ ]+ },+ {+ "question_id": "e25c3b8d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_gQaSoiF_13",+ "answer_6702277b_2",+ "ff201786_1",+ "90fea364_1",+ "answer_6702277b_1",+ "35b030f3_2",+ "746d0c00_1",+ "5237d8c6",+ "sharegpt_Nk10TjE_29",+ "20fa75e8_3"+ ],+ "gold_session_ids": [+ "answer_6702277b_1",+ "answer_6702277b_2"+ ]+ },+ {+ "question_id": "4adc0475",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_463810",+ "answer_6efce493_2",+ "f82637ed_1",+ "ultrachat_186558",+ "debc34d7_2",+ "fa0fa74d_2",+ "bb27df5e_2",+ "sharegpt_7nYPy2P_29",+ "ultrachat_170415",+ "183d2822_1"+ ],+ "gold_session_ids": [+ "answer_6efce493_1",+ "answer_6efce493_2"+ ]+ },+ {+ "question_id": "4bc144e2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9ef115d4_1",+ "42bcee92_2",+ "answer_9ef115d4_2",+ "60207b41_3",+ "2e2ae792_1",+ "d75245ea",+ "a177e66a_2",+ "bf9c39e9_1",+ "1fd71f84",+ "9416c7de"+ ],+ "gold_session_ids": [+ "answer_9ef115d4_1",+ "answer_9ef115d4_2"+ ]+ },+ {+ "question_id": "ef66a6e5",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "sharegpt_bitlG4U_0",+ "3f05a03e_1",+ "147ab7e9_6",+ "ultrachat_428073",+ "ultrachat_61361",+ "answer_f7fd1029_1",+ "4c08398b_1",+ "502b6b52_4",+ "41e778e4_2",+ "295645f5_1"+ ],+ "gold_session_ids": [+ "answer_f7fd1029_2",+ "answer_f7fd1029_1"+ ]+ },+ {+ "question_id": "5025383b",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20438239758848611,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "2d21f921",+ "0e349efa",+ "sharegpt_SwxX91i_5",+ "2b8fe437_1",+ "a268827b_2",+ "f40fa3ed",+ "answer_688f9a3f_2",+ "22f9af5f",+ "042c55a5",+ "baf0243b_2"+ ],+ "gold_session_ids": [+ "answer_688f9a3f_1",+ "answer_688f9a3f_2"+ ]+ },+ {+ "question_id": "a1cc6108",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_17dc2f5b_2",+ "001cefa7_2",+ "a841ef89_1",+ "5aa88956_2",+ "sharegpt_9eEV9T3_8",+ "sharegpt_jpiaPoJ_29",+ "298a07de_5",+ "59b87367",+ "fb328ace_1",+ "31e254b5"+ ],+ "gold_session_ids": [+ "answer_17dc2f5b_2",+ "answer_17dc2f5b_1"+ ]+ },+ {+ "question_id": "9ee3ecd6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_66c23110_2",+ "answer_66c23110_1",+ "fd56c5bd_2",+ "fa0fa74d_2",+ "9b1b9069_1",+ "6577a6d9_1",+ "f550a835_1",+ "42d9467c_1",+ "a9eda378",+ "ultrachat_185685"+ ],+ "gold_session_ids": [+ "answer_66c23110_1",+ "answer_66c23110_2"+ ]+ },+ {+ "question_id": "3fdac837",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "09d021e7_1",+ "a98ff421_7",+ "d1165be3_2",+ "30c92c0c_1",+ "ultrachat_293546",+ "c85ef24e_1",+ "92b06116_1",+ "5298db7c_1",+ "a1334327_1",+ "9b446113_2"+ ],+ "gold_session_ids": [+ "answer_419d21d5_2",+ "answer_419d21d5_1"+ ]+ },+ {+ "question_id": "91b15a6e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5404a208_1",+ "answer_5404a208_2",+ "ultrachat_42513",+ "ultrachat_326780",+ "sharegpt_zS41R74_0",+ "6169ef55_1",+ "sharegpt_0ktU0jP_16",+ "sharegpt_7r7RS6F_0",+ "7cfeb89c_2",+ "b28cdea2"+ ],+ "gold_session_ids": [+ "answer_5404a208_1",+ "answer_5404a208_2"+ ]+ },+ {+ "question_id": "27016adc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a37bdf22_1",+ "ff5e0db0_1",+ "answer_a37bdf22_2",+ "sharegpt_ZbXRfgJ_0",+ "28ea20bf",+ "199a5225_2",+ "46548fab_3",+ "ultrachat_566359",+ "dd25eeb5_8",+ "fa858208_2"+ ],+ "gold_session_ids": [+ "answer_a37bdf22_2",+ "answer_a37bdf22_1"+ ]+ },+ {+ "question_id": "720133ac",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c9dfeaea_2",+ "answer_c9dfeaea_1",+ "4c128d97_2",+ "sharegpt_Km6yx9W_13",+ "de43030f_2",+ "ultrachat_479833",+ "0cf86cbe_2",+ "sharegpt_sG53ibg_116",+ "ultrachat_508711",+ "e200d96c_3"+ ],+ "gold_session_ids": [+ "answer_c9dfeaea_1",+ "answer_c9dfeaea_2"+ ]+ },+ {+ "question_id": "77eafa52",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_33c251f0_1",+ "7b13b4d6_2",+ "sharegpt_UUFCRmF_0",+ "answer_33c251f0_2",+ "sharegpt_qEukY6n_14",+ "ultrachat_309057",+ "41a87f13_1",+ "fc9a62c0_1",+ "174cccbc",+ "sharegpt_C9GjDsF_0"+ ],+ "gold_session_ids": [+ "answer_33c251f0_2",+ "answer_33c251f0_1"+ ]+ },+ {+ "question_id": "8979f9ec",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35e36341_2",+ "answer_35e36341_1",+ "bcc9382a_2",+ "8540055e_1",+ "1064ed80_2",+ "1980bbfa_2",+ "052d7def",+ "a6b4c4c2_1",+ "ultrachat_466891",+ "ultrachat_562471"+ ],+ "gold_session_ids": [+ "answer_35e36341_1",+ "answer_35e36341_2"+ ]+ },+ {+ "question_id": "0100672e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c9798c_2",+ "answer_35c9798c_1",+ "07313722_2",+ "sharegpt_sDfu38Q_0",+ "326e86e0_3",+ "b167a4a8",+ "f60c9de5",+ "ultrachat_372868",+ "c694fe7b_1",+ "0f53a05e_4"+ ],+ "gold_session_ids": [+ "answer_35c9798c_2",+ "answer_35c9798c_1"+ ]+ },+ {+ "question_id": "a96c20ee",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7903864795495061,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef84b994_1",+ "32e292f2",+ "dff89d9d_1",+ "9ca3534f_1",+ "ultrachat_259222",+ "sharegpt_00qHEQ6_0",+ "sharegpt_lWLBUhQ_539",+ "c69eb8ee_4",+ "3d2e66f2_2",+ "answer_ef84b994_2"+ ],+ "gold_session_ids": [+ "answer_ef84b994_1",+ "answer_ef84b994_2"+ ]+ },+ {+ "question_id": "92a0aa75",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.09090909090909091,+ "retrieved_session_ids": [+ "ultrachat_402091",+ "sharegpt_hPax8pw_11",+ "a5854a8d_2",+ "bd00a24d_2",+ "sharegpt_xkWMnwQ_0",+ "edd89480_1",+ "6da0bba2_1",+ "2bdd78cb_1",+ "1c0921ee_3",+ "173e2e47"+ ],+ "gold_session_ids": [+ "answer_6cb8f792_1",+ "answer_6cb8f792_2"+ ]+ },+ {+ "question_id": "3fe836c9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.48381288316677695,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "c81897cd_2",+ "0a42783a",+ "answer_1bb63ea5_1",+ "f0d7d364",+ "8129a7a8",+ "c4293e27_2",+ "a3cfa973",+ "ba184a34",+ "e94b6708",+ "answer_1bb63ea5_2"+ ],+ "gold_session_ids": [+ "answer_1bb63ea5_2",+ "answer_1bb63ea5_1"+ ]+ },+ {+ "question_id": "1c549ce4",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4cb841a8_1",+ "answer_4cb841a8_2",+ "d743153b_2",+ "f999b05c_1",+ "ultrachat_441412",+ "f1e4daa8_2",+ "sharegpt_DwfIyV9_0",+ "3d006c58_2",+ "35f052f9_2",+ "bd4110bc"+ ],+ "gold_session_ids": [+ "answer_4cb841a8_1",+ "answer_4cb841a8_2"+ ]+ },+ {+ "question_id": "6c49646a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.23719771276929622,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "1a24ef7c_5",+ "ultrachat_565061",+ "fffa6e59",+ "313404c6_1",+ "answer_cc1ced21_1",+ "8acb29f9_2",+ "403ee298_3",+ "9f220c66_1",+ "6a2f306a",+ "6155addd_2"+ ],+ "gold_session_ids": [+ "answer_cc1ced21_2",+ "answer_cc1ced21_1"+ ]+ },+ {+ "question_id": "1192316e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e184e4c3_1",+ "answer_e184e4c3_2",+ "c5db849c_1",+ "69b90254",+ "sharegpt_pZ3I356_11",+ "66e58c90_3",+ "55f7cc4f_3",+ "199a5225_1",+ "6f8cd952",+ "52c6d59a_4"+ ],+ "gold_session_ids": [+ "answer_e184e4c3_2",+ "answer_e184e4c3_1"+ ]+ },+ {+ "question_id": "0ea62687",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "5dce60dd",+ "answer_dc5e537d_2",+ "answer_dc5e537d_1",+ "92ee65e1",+ "eb336de0_2",+ "d1ea55c0_1",+ "c5e9cbd8_1",+ "2fd28490_1",+ "fba28a35_3",+ "4a5ee697_1"+ ],+ "gold_session_ids": [+ "answer_dc5e537d_1",+ "answer_dc5e537d_2"+ ]+ },+ {+ "question_id": "67e0d0f2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3a5010af_2",+ "75a48603_1",+ "e89b6be2_2",+ "answer_3a5010af_1",+ "sharegpt_uD7CBng_30",+ "78e40ca2_2",+ "ultrachat_170247",+ "ultrachat_203993",+ "2252d3dd",+ "93bed3d8_2"+ ],+ "gold_session_ids": [+ "answer_3a5010af_2",+ "answer_3a5010af_1"+ ]+ },+ {+ "question_id": "bb7c3b45",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_de64539a_1",+ "answer_de64539a_2",+ "7a4e1601",+ "sharegpt_I0uNTjm_36",+ "b9220c40",+ "752b10e6_1",+ "07548ed6",+ "f3d43a12_3",+ "3891baea_4",+ "ultrachat_139470"+ ],+ "gold_session_ids": [+ "answer_de64539a_1",+ "answer_de64539a_2"+ ]+ },+ {+ "question_id": "ba358f49",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3065735963827292,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "9cc709cb",+ "ultrachat_317048",+ "answer_cbd08e3c_1",+ "91c71874_2",+ "d964ed3b",+ "95cb552e_1",+ "sharegpt_bZQ9094_0",+ "f6859b48_2",+ "385683f0_2",+ "4f838497_2"+ ],+ "gold_session_ids": [+ "answer_cbd08e3c_1",+ "answer_cbd08e3c_2"+ ]+ },+ {+ "question_id": "61f8c8f8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_872e8da2_1",+ "a3107e2a_1",+ "answer_872e8da2_2",+ "1a24ef7c_5",+ "7d5d9dd3",+ "7af38385_2",+ "2154c810_1",+ "sharegpt_7jGo66i_99",+ "7b88c38b_2",+ "ultrachat_211983"+ ],+ "gold_session_ids": [+ "answer_872e8da2_2",+ "answer_872e8da2_1"+ ]+ },+ {+ "question_id": "60159905",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.36181498356361597,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "sharegpt_RtCgJK2_508",+ "b3ef2cd4_1",+ "15998e39_4",+ "5faf5962_3",+ "ultrachat_190672",+ "afe04238_1",+ "eef7c3fa_2",+ "ultrachat_168392",+ "answer_75eca223_2",+ "answer_75eca223_1"+ ],+ "gold_session_ids": [+ "answer_75eca223_2",+ "answer_75eca223_1"+ ]+ },+ {+ "question_id": "ef9cf60a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_87e3a1cb_2",+ "answer_87e3a1cb_1",+ "7b88c38b_1",+ "sharegpt_mvtmr4e_19",+ "2357de36_2",+ "sharegpt_6fU0WOe_56",+ "sharegpt_NelPQLT_0",+ "0275bceb_2",+ "d677eb06",+ "3cd4a679"+ ],+ "gold_session_ids": [+ "answer_87e3a1cb_1",+ "answer_87e3a1cb_2"+ ]+ },+ {+ "question_id": "73d42213",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_275858",+ "answer_1881e7db_1",+ "answer_1881e7db_2",+ "sharegpt_ab6IEma_0",+ "ultrachat_3128",+ "sharegpt_lNG1bAS_0",+ "a1251fef_2",+ "7c2ec9e9_2",+ "sharegpt_GKjb2be_0",+ "3cb41ab8_1"+ ],+ "gold_session_ids": [+ "answer_1881e7db_1",+ "answer_1881e7db_2"+ ]+ },+ {+ "question_id": "bc149d6b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.45560514958746035,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "4f435d20_2",+ "451ecb23_3",+ "7211c69e",+ "ultrachat_461193",+ "answer_92147866_1",+ "answer_92147866_2",+ "27d378b2_3",+ "sharegpt_HullwHG_71",+ "86935301_2",+ "1b474144"+ ],+ "gold_session_ids": [+ "answer_92147866_1",+ "answer_92147866_2"+ ]+ },+ {+ "question_id": "099778bb",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_80d6d664_1",+ "answer_80d6d664_2",+ "6bb8af95_1",+ "b1793bba",+ "sharegpt_s1I3UkG_0",+ "5b014f9b_1",+ "d00ba6d0_2",+ "sharegpt_O8djord_90",+ "sharegpt_qyWsgp8_13",+ "0a042aef_2"+ ],+ "gold_session_ids": [+ "answer_80d6d664_2",+ "answer_80d6d664_1"+ ]+ },+ {+ "question_id": "09ba9854",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96c743d0_1",+ "answer_96c743d0_2",+ "ultrachat_447648",+ "8682e696_2",+ "df963ea2_1",+ "32e6c308",+ "8c63238e",+ "5faf5962_3",+ "ae32e467_2",+ "f7644391_2"+ ],+ "gold_session_ids": [+ "answer_96c743d0_2",+ "answer_96c743d0_1"+ ]+ },+ {+ "question_id": "d6062bb9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_23f3a657_1",+ "answer_23f3a657_2",+ "sharegpt_xuhzAgG_19",+ "sharegpt_hcaZN94_647",+ "ultrachat_485311",+ "cd4ae1ad",+ "f7644391_2",+ "65d28641_2",+ "59c704ad_2",+ "b35610f4_4"+ ],+ "gold_session_ids": [+ "answer_23f3a657_2",+ "answer_23f3a657_1"+ ]+ },+ {+ "question_id": "157a136e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.48247555939075504,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "377f8fd9_4",+ "9b3a7f2c_4",+ "ultrachat_375609",+ "answer_8de18468_2",+ "ultrachat_255132",+ "answer_8de18468_1",+ "sharegpt_xQQ30VB_0",+ "298bf7a3_1",+ "a2e2cb72_5",+ "d161536e"+ ],+ "gold_session_ids": [+ "answer_8de18468_2",+ "answer_8de18468_1"+ ]+ },+ {+ "question_id": "c18a7dc8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2e2085fa_1",+ "9d5a389d",+ "f716720c_3",+ "5022812c",+ "fc6c3985",+ "805528d6_1",+ "3421bdbe_1",+ "4f52229e",+ "1c177942_3",+ "ultrachat_301566"+ ],+ "gold_session_ids": [+ "answer_2e2085fa_2",+ "answer_2e2085fa_1"+ ]+ },+ {+ "question_id": "a3332713",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_16ece55f_2",+ "answer_16ece55f_1",+ "1c8a264a_4",+ "67388dfc_2",+ "117098d9_3",+ "29fbbc99",+ "f6859b48_3",+ "a864e7aa_1",+ "ef2400e0_1",+ "sharegpt_bYymtlW_0"+ ],+ "gold_session_ids": [+ "answer_16ece55f_2",+ "answer_16ece55f_1"+ ]+ },+ {+ "question_id": "55241a1f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fa08bf49_1",+ "answer_fa08bf49_2",+ "9c9a22d8",+ "76491ee4_1",+ "sharegpt_dUoAhvI_71",+ "2aeb1268",+ "d117e2da",+ "sharegpt_TzwvePR_0",+ "46e4a8eb_2",+ "60153a02_4"+ ],+ "gold_session_ids": [+ "answer_fa08bf49_1",+ "answer_fa08bf49_2"+ ]+ },+ {+ "question_id": "a08a253f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_47152166_1",+ "answer_47152166_2",+ "6e5130ed_2",+ "28209b6a_4",+ "798fa5f2",+ "4524c250_2",+ "d79173aa_3",+ "771570c5_1",+ "83ca9337",+ "a79f1a04_1"+ ],+ "gold_session_ids": [+ "answer_47152166_2",+ "answer_47152166_1"+ ]+ },+ {+ "question_id": "f0e564bc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cfcf5340_2",+ "ultrachat_384067",+ "55da3cbf_3",+ "answer_cfcf5340_1",+ "ultrachat_330960",+ "932a97c6",+ "1788a980",+ "65ca99fb_2",+ "sharegpt_W3YFpVK_7",+ "6dfc99bf_2"+ ],+ "gold_session_ids": [+ "answer_cfcf5340_1",+ "answer_cfcf5340_2"+ ]+ },+ {+ "question_id": "078150f1",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5706417189553201,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "e75d8a9c",+ "46e4a8eb_1",+ "answer_254d8b09_1",+ "answer_254d8b09_2",+ "0e218647_2",+ "57bf2079",+ "f54d3a48",+ "61b27d67_3",+ "b18ba8d0_1",+ "ultrachat_202549"+ ],+ "gold_session_ids": [+ "answer_254d8b09_1",+ "answer_254d8b09_2"+ ]+ },+ {+ "question_id": "8cf4d046",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e2278b24_2",+ "ultrachat_548964",+ "a30f52a4",+ "sharegpt_P1Okivz_9",+ "c36f4196",+ "sharegpt_WjpyexI_15",+ "961b965c_2",+ "6c6c8df7",+ "410ec447",+ "c7270a1b_1"+ ],+ "gold_session_ids": [+ "answer_e2278b24_1",+ "answer_e2278b24_2"+ ]+ },+ {+ "question_id": "a346bb18",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20438239758848611,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "c51572bb_2",+ "5726dc37_2",+ "84fb50bb_1",+ "b4003083",+ "53949711_2",+ "59e96888_1",+ "answer_4934b2d7_2",+ "ultrachat_49611",+ "ultrachat_300801",+ "d3a4eb00_1"+ ],+ "gold_session_ids": [+ "answer_4934b2d7_2",+ "answer_4934b2d7_1"+ ]+ },+ {+ "question_id": "37f165cf",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6b9b2b1e_2",+ "answer_6b9b2b1e_1",+ "20b1b65f_2",+ "c85d1682_1",+ "cdcf791b_1",+ "7e8e790a",+ "ultrachat_319568",+ "50dd6bc1_1",+ "c41e97c9_2",+ "ultrachat_453124"+ ],+ "gold_session_ids": [+ "answer_6b9b2b1e_2",+ "answer_6b9b2b1e_1"+ ]+ },+ {+ "question_id": "8e91e7d9",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "2f980ae0_2",+ "ultrachat_147574",+ "c3023a45_4",+ "4e3da326_1",+ "4f8caea3_2",+ "7a389bd1_7",+ "c2381e1a_2",+ "42e69993_2",+ "sharegpt_PZrBCF2_0",+ "44fc2b1e"+ ],+ "gold_session_ids": [+ "answer_477ae455_1",+ "answer_477ae455_2"+ ]+ },+ {+ "question_id": "87f22b4a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_182225",+ "answer_f56e6152_1",+ "02b99ec3_1",+ "ea28bb75",+ "answer_f56e6152_2",+ "c6015a0c_3",+ "889076d6",+ "ultrachat_350147",+ "0320f558_2",+ "28741edc_1"+ ],+ "gold_session_ids": [+ "answer_f56e6152_2",+ "answer_f56e6152_1"+ ]+ },+ {+ "question_id": "e56a43b9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_430d0a87_2",+ "answer_430d0a87_1",+ "7596ba6a",+ "83e14506_2",+ "38e81260_7",+ "57a6a404_3",+ "b28cdea2",+ "sharegpt_iylDztx_14",+ "ultrachat_267137",+ "f684ac4c_3"+ ],+ "gold_session_ids": [+ "answer_430d0a87_1",+ "answer_430d0a87_2"+ ]+ },+ {+ "question_id": "efc3f7c2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fc81d1a2_1",+ "ultrachat_38938",+ "fc005760",+ "bbca6598_1",+ "bbdc7b0a",+ "88a7faa7_1",+ "1ea13fbd_2",+ "8a7df17e_1",+ "answer_fc81d1a2_2",+ "0f013a55"+ ],+ "gold_session_ids": [+ "answer_fc81d1a2_1",+ "answer_fc81d1a2_2"+ ]+ },+ {+ "question_id": "21d02d0d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "42d9467c_2",+ "answer_2c637141_1",+ "ultrachat_71299",+ "sharegpt_adeEi7y_7",+ "answer_2c637141_2",+ "2e28f3db",+ "cf021b36_2",+ "ultrachat_270892",+ "sharegpt_B9jlITg_0",+ "a15fa6eb"+ ],+ "gold_session_ids": [+ "answer_2c637141_2",+ "answer_2c637141_1"+ ]+ },+ {+ "question_id": "2311e44b_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bf633415_abs_1",+ "313404c6_2",+ "answer_bf633415_abs_2",+ "4c8afa9c_1",+ "90342357_2",+ "5e906cb0_2",+ "09d021e7_2",+ "e9e9027c_2",+ "35915bf5",+ "393045ae_3"+ ],+ "gold_session_ids": [+ "answer_bf633415_abs_1",+ "answer_bf633415_abs_2"+ ]+ },+ {+ "question_id": "6456829e_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_743f03a1_abs_2",+ "b61cc91b_2",+ "answer_743f03a1_abs_1",+ "3d8d4828_2",+ "sharegpt_Hc0Zm6i_0",+ "ultrachat_417298",+ "d20bb5aa_1",+ "1b26bdd5_5",+ "sharegpt_ZxGSwYN_7",+ "4015360f_2"+ ],+ "gold_session_ids": [+ "answer_743f03a1_abs_1",+ "answer_743f03a1_abs_2"+ ]+ },+ {+ "question_id": "e5ba910e_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e49ed9d3_abs_2",+ "ultrachat_282969",+ "answer_e49ed9d3_abs_1",+ "6339aa1d_2",+ "7e0fa6a6",+ "a239515b",+ "ultrachat_402969",+ "6d013bad_3",+ "8e8c87fe_1",+ "ultrachat_165962"+ ],+ "gold_session_ids": [+ "answer_e49ed9d3_abs_1",+ "answer_e49ed9d3_abs_2"+ ]+ },+ {+ "question_id": "a96c20ee_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef84b994_abs_1",+ "4163e709_3",+ "answer_ef84b994_abs_2",+ "fee68529_2",+ "47103d08",+ "sharegpt_jM9e92F_0",+ "ultrachat_231397",+ "ultrachat_293509",+ "1917d461",+ "ultrachat_24513"+ ],+ "gold_session_ids": [+ "answer_ef84b994_abs_1",+ "answer_ef84b994_abs_2"+ ]+ },+ {+ "question_id": "ba358f49_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "730a1d1d_2",+ "answer_cbd08e3c_abs_1",+ "1162611c",+ "5829ab38",+ "9844f740",+ "sharegpt_7kFao5h_0",+ "ultrachat_387251",+ "9b3a7f2c_2",+ "11f043cd_2",+ "sharegpt_XWLA0IO_0"+ ],+ "gold_session_ids": [+ "answer_cbd08e3c_abs_1",+ "answer_cbd08e3c_abs_2"+ ]+ },+ {+ "question_id": "09ba9854_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96c743d0_abs_1",+ "85ca9420",+ "answer_96c743d0_abs_2",+ "66e58c90_2",+ "a50b0d73_2",+ "ultrachat_342361",+ "338ac116_2",+ "9beab022_1",+ "25021ded",+ "d8e9a53a_3"+ ],+ "gold_session_ids": [+ "answer_96c743d0_abs_1",+ "answer_96c743d0_abs_2"+ ]+ },+ {+ "question_id": "gpt4_59149c77",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d00ba6d0_2",+ "9e2c2a6c_2",+ "answer_d00ba6d0_1",+ "9242fbe3_1",+ "sharegpt_XlNs8Lz_199",+ "148f87a0_2",+ "54a05e72_2",+ "31829c1b_1",+ "sharegpt_ic0CCIq_0",+ "c07ae000_2"+ ],+ "gold_session_ids": [+ "answer_d00ba6d0_1",+ "answer_d00ba6d0_2"+ ]+ },+ {+ "question_id": "gpt4_f49edff3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.431733884398313,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "4376945f",+ "answer_3e9fce53_3",+ "9f8fa5e4_2",+ "acfdbdcb",+ "9f091256_2",+ "c73cebb3_2",+ "40a289b7_2",+ "dc7c53fa_4",+ "7d33fcb4_1",+ "answer_3e9fce53_2"+ ],+ "gold_session_ids": [+ "answer_3e9fce53_1",+ "answer_3e9fce53_2",+ "answer_3e9fce53_3"+ ]+ },+ {+ "question_id": "71017276",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0b4a8adc_1",+ "4a545031_2",+ "sharegpt_CjoQMV6_0",+ "sharegpt_qevDNvo_0",+ "sharegpt_tCLMKZY_0",+ "58ec258f_2",+ "398f23e7",+ "ultrachat_491808",+ "10889cf0_1",+ "a80c8ed4"+ ],+ "gold_session_ids": [+ "answer_0b4a8adc_1"+ ]+ },+ {+ "question_id": "b46e15ed",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2856436410275446,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "a410bab2_3",+ "47311a4c_1",+ "9d754f54",+ "answer_4bfcc250_4",+ "53949711_2",+ "ultrachat_265690",+ "940629b2_2",+ "ultrachat_228593",+ "answer_4bfcc250_2",+ "fa34119a_1"+ ],+ "gold_session_ids": [+ "answer_4bfcc250_4",+ "answer_4bfcc250_3",+ "answer_4bfcc250_2",+ "answer_4bfcc250_1"+ ]+ },+ {+ "question_id": "gpt4_fa19884c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ff201786_1",+ "answer_ff201786_2",+ "cbf48918_1",+ "f4f4b504_2",+ "6414f676_2",+ "e8de3056_2",+ "8414cc57",+ "a7b1b27c",+ "7ea7741b_2",+ "128082f8_1"+ ],+ "gold_session_ids": [+ "answer_ff201786_1",+ "answer_ff201786_2"+ ]+ },+ {+ "question_id": "0bc8ad92",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9325210919548239,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f4ea84fb_3",+ "answer_f4ea84fb_1",+ "ultrachat_349938",+ "eca613fc",+ "sharegpt_vClQxDr_21",+ "answer_f4ea84fb_2",+ "sharegpt_ntu1Cds_0",+ "d3cc5bdc_1",+ "0868982f_1",+ "bcc37468_2"+ ],+ "gold_session_ids": [+ "answer_f4ea84fb_3",+ "answer_f4ea84fb_2",+ "answer_f4ea84fb_1"+ ]+ },+ {+ "question_id": "af082822",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "6cf1848e_3",+ "bc5827b5",+ "b28990c8_7",+ "c285b705_1",+ "6bb8af95_2",+ "answer_b51b6115_1",+ "1dfef591_2",+ "fbe6fa2c_2",+ "facef6e0_1",+ "09c3d921"+ ],+ "gold_session_ids": [+ "answer_b51b6115_1"+ ]+ },+ {+ "question_id": "gpt4_4929293a",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_add9b012_2",+ "answer_add9b012_1",+ "ultrachat_139245",+ "73353aee_1",+ "ultrachat_512282",+ "ultrachat_329640",+ "c1e598cd_1",+ "8cfe5096_1",+ "ultrachat_345984",+ "sharegpt_oQacgfY_0"+ ],+ "gold_session_ids": [+ "answer_add9b012_2",+ "answer_add9b012_1"+ ]+ },+ {+ "question_id": "gpt4_b5700ca9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a17423e7_1",+ "44c62f26",+ "33bdd89a_1",+ "65698aa9_7",+ "ultrachat_383774",+ "ultrachat_366301",+ "5053474c_2",+ "31829c1b_1",+ "ultrachat_346810",+ "47382adf_1"+ ],+ "gold_session_ids": [+ "answer_a17423e7_1"+ ]+ },+ {+ "question_id": "9a707b81",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "4abbeb2a_2",+ "answer_dba89487_2",+ "5a13d047_1",+ "answer_dba89487_1",+ "a8e24e22",+ "e1d11615_1",+ "3fcbb58c_4",+ "ultrachat_233519",+ "990f3ef9_6",+ "ultrachat_550996"+ ],+ "gold_session_ids": [+ "answer_dba89487_2",+ "answer_dba89487_1"+ ]+ },+ {+ "question_id": "gpt4_1d4ab0c9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_febde667_2",+ "answer_febde667_1",+ "7257bd15_1",+ "b3e0a3ab_3",+ "sharegpt_Caxpnd3_12",+ "4ff83df8",+ "3f1d1224_3",+ "5b83c26e_5",+ "sharegpt_MIYBRJz_27",+ "ab855b34_2"+ ],+ "gold_session_ids": [+ "answer_febde667_1",+ "answer_febde667_2"+ ]+ },+ {+ "question_id": "gpt4_e072b769",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c19bd2bf_1",+ "5558a42e_1",+ "b9e32ff8_1",+ "0dd54b7c_1",+ "4d6d013c",+ "f39f7655_1",+ "66d3fdb7_2",+ "4f7b5dc9_1",+ "78413cce_4",+ "2fc9ce79_2"+ ],+ "gold_session_ids": [+ "answer_c19bd2bf_1"+ ]+ },+ {+ "question_id": "0db4c65d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b9e32ff8_2",+ "answer_b9e32ff8_1",+ "953d5a4b_1",+ "71dc2037_2",+ "f78a22b6_3",+ "e6c3a50a",+ "sharegpt_5NoXULS_0",+ "8d1f9505",+ "ultrachat_324557",+ "6461402b"+ ],+ "gold_session_ids": [+ "answer_b9e32ff8_1",+ "answer_b9e32ff8_2"+ ]+ },+ {+ "question_id": "gpt4_1d80365e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_661b711f_1",+ "answer_661b711f_2",+ "805e571c_1",+ "sharegpt_yerbusu_1",+ "0bad887e_1",+ "1ecc86e7",+ "554b054e_2",+ "d6b17438_1",+ "ef0341d4_2",+ "sharegpt_5uZ7IdY_0"+ ],+ "gold_session_ids": [+ "answer_661b711f_1",+ "answer_661b711f_2"+ ]+ },+ {+ "question_id": "gpt4_7f6b06db",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "64266f80_1",+ "6f051087_2",+ "4bf66c38_2",+ "39c1bf6a",+ "252715f8_2",+ "da42526b_1",+ "70dc7d08_1",+ "42165950_8",+ "sharegpt_v11Tpg9_255",+ "ultrachat_6893"+ ],+ "gold_session_ids": [+ "answer_5d8c99d3_1",+ "answer_5d8c99d3_2",+ "answer_5d8c99d3_3"+ ]+ },+ {+ "question_id": "gpt4_6dc9b45b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c4df007f_1",+ "ultrachat_384387",+ "9adc81d7_2",+ "afa0c2e7_5",+ "sharegpt_jyynvvk_0",+ "cfcf5340_2",+ "ultrachat_35147",+ "9752e18d",+ "d8f48b0a",+ "56911dc5_2"+ ],+ "gold_session_ids": [+ "answer_c4df007f_1"+ ]+ },+ {+ "question_id": "gpt4_8279ba02",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "85f19a30_1",+ "answer_56521e65_1",+ "86c5d31d",+ "ultrachat_302826",+ "2fd445a2_5",+ "ultrachat_520053",+ "1a037873_2",+ "ultrachat_78027",+ "b909542d_1",+ "e1c807f9"+ ],+ "gold_session_ids": [+ "answer_56521e65_1"+ ]+ },+ {+ "question_id": "gpt4_18c2b244",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c862f65a_2",+ "answer_c862f65a_3",+ "answer_c862f65a_1",+ "dfbc46bd_1",+ "84896d13",+ "d9e1d5fd",+ "ultrachat_503659",+ "ffd66315_2",+ "746ae854_2",+ "7e2c6065"+ ],+ "gold_session_ids": [+ "answer_c862f65a_2",+ "answer_c862f65a_3",+ "answer_c862f65a_1"+ ]+ },+ {+ "question_id": "gpt4_a1b77f9c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.971476156593264,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e9ad5914_5",+ "answer_e9ad5914_3",+ "answer_e9ad5914_2",+ "answer_e9ad5914_4",+ "47ec1674_2",+ "199a5225_1",+ "answer_e9ad5914_6",+ "answer_e9ad5914_1",+ "d52d478b",+ "e09b6396"+ ],+ "gold_session_ids": [+ "answer_e9ad5914_1",+ "answer_e9ad5914_2",+ "answer_e9ad5914_3",+ "answer_e9ad5914_4",+ "answer_e9ad5914_5",+ "answer_e9ad5914_6"+ ]+ },+ {+ "question_id": "gpt4_1916e0ea",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_447052a5_2",+ "answer_447052a5_1",+ "ultrachat_306167",+ "7a36e820_2",+ "280352e9",+ "ddf41a24",+ "d3fe25ff",+ "b9890c15_2",+ "987b5d00",+ "fa1ce406_2"+ ],+ "gold_session_ids": [+ "answer_447052a5_2",+ "answer_447052a5_1"+ ]+ },+ {+ "question_id": "gpt4_7a0daae1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4d5490f1_1",+ "answer_4d5490f1_2",+ "014a5d36",+ "7c08d74b_2",+ "ultrachat_49899",+ "bf3aebdb_2",+ "445e6a7a_2",+ "2ff297a8_1",+ "8c68df0d_1",+ "819dfad7_2"+ ],+ "gold_session_ids": [+ "answer_4d5490f1_1",+ "answer_4d5490f1_2"+ ]+ },+ {+ "question_id": "gpt4_468eb063",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "e60a93ff_2",+ "sharegpt_Sf51OjE_8",+ "ultrachat_91161",+ "answer_9b09d95b_1",+ "ultrachat_108278",+ "sharegpt_1rIk8tS_0",+ "sharegpt_gfhUw40_17",+ "d97db962_2",+ "23ecf657",+ "982514b2_1"+ ],+ "gold_session_ids": [+ "answer_9b09d95b_1"+ ]+ },+ {+ "question_id": "gpt4_7abb270c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4296937870297496,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "087d2b0a_2",+ "answer_7093d898_6",+ "answer_7093d898_5",+ "6e42386d_1",+ "5808284e",+ "85056d5f_1",+ "3a7cdf9c",+ "a211539c",+ "cb787bfa_1",+ "answer_7093d898_2"+ ],+ "gold_session_ids": [+ "answer_7093d898_1",+ "answer_7093d898_2",+ "answer_7093d898_3",+ "answer_7093d898_4",+ "answer_7093d898_5",+ "answer_7093d898_6"+ ]+ },+ {+ "question_id": "gpt4_1e4a8aeb",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_16bd5ea5_1",+ "answer_16bd5ea5_2",+ "63b51f8a_5",+ "f7838a91_1",+ "fd18e0dd_1",+ "28621d6a_2",+ "bc5827b5",+ "6a7d7055_1",+ "faad7d7a_1",+ "ultrachat_239720"+ ],+ "gold_session_ids": [+ "answer_16bd5ea5_1",+ "answer_16bd5ea5_2"+ ]+ },+ {+ "question_id": "gpt4_4fc4f797",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_be07688f_1",+ "answer_be07688f_2",+ "sharegpt_pRqHb1o_0",+ "10d7d2b4_2",+ "39566615_1",+ "sharegpt_V2PULWC_13",+ "47c5482a",+ "0c783e4a_1",+ "463ad596_2",+ "86298245_2"+ ],+ "gold_session_ids": [+ "answer_be07688f_1",+ "answer_be07688f_2"+ ]+ },+ {+ "question_id": "4dfccbf7",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4bebc782_1",+ "answer_4bebc782_2",+ "8fa624b2_6",+ "e1b7b676",+ "c2f86a02",+ "b197b7fb",+ "sharegpt_Dcshjlv_0",+ "36f8b380_1",+ "f2b8e6a8_1",+ "491ebc5d_2"+ ],+ "gold_session_ids": [+ "answer_4bebc782_1",+ "answer_4bebc782_2"+ ]+ },+ {+ "question_id": "gpt4_61e13b3c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e831a29f_1",+ "answer_e831a29f_2",+ "sharegpt_uD7CBng_4",+ "cc380119_3",+ "f3fcf4fd_2",+ "ultrachat_441289",+ "a5c89dac",+ "b4923852",+ "d4439279_1",+ "e9f4f027_4"+ ],+ "gold_session_ids": [+ "answer_e831a29f_1",+ "answer_e831a29f_2"+ ]+ },+ {+ "question_id": "gpt4_45189cb4",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2960819109658652,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "dc3ee1d1_1",+ "answer_e6c20e52_3",+ "93395e5f_3",+ "6593cb8b_1",+ "22452c22",+ "ultrachat_560291",+ "sharegpt_AnH3ftB_0",+ "ultrachat_307270",+ "e789afdb_1",+ "0eb4cb14"+ ],+ "gold_session_ids": [+ "answer_e6c20e52_3",+ "answer_e6c20e52_2",+ "answer_e6c20e52_1"+ ]+ },+ {+ "question_id": "2ebe6c90",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "1850bfb1",+ "answer_c9d35c00_1",+ "answer_c9d35c00_2",+ "9ccea2f0_2",+ "sharegpt_jyMHY1J_31",+ "ultrachat_27073",+ "90e55108_3",+ "c3c1f84a_1",+ "c6bd5277",+ "a4e89c45"+ ],+ "gold_session_ids": [+ "answer_c9d35c00_1",+ "answer_c9d35c00_2"+ ]+ },+ {+ "question_id": "gpt4_e061b84f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.13565197343244778,+ "mrr": 0.1,+ "retrieved_session_ids": [+ "77974d02",+ "539c2346_3",+ "ultrachat_308277",+ "ultrachat_18218",+ "4cc9270f",+ "0a6bf5e4_1",+ "773287dc_1",+ "ultrachat_436674",+ "bab3f409",+ "answer_8c64ce25_3"+ ],+ "gold_session_ids": [+ "answer_8c64ce25_2",+ "answer_8c64ce25_1",+ "answer_8c64ce25_3"+ ]+ },+ {+ "question_id": "370a8ff4",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_61d1be50_1",+ "answer_61d1be50_2",+ "sharegpt_UbMVpdp_93",+ "cfde748a",+ "ultrachat_26182",+ "2b4eaee0_1",+ "sharegpt_JRQtEcd_115",+ "9f8fa5e4_5",+ "df53e1b7_2",+ "sharegpt_jGPXxpI_17"+ ],+ "gold_session_ids": [+ "answer_61d1be50_1",+ "answer_61d1be50_2"+ ]+ },+ {+ "question_id": "gpt4_d6585ce8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8687949224876582,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f999b05b_2",+ "answer_f999b05b_5",+ "answer_f999b05b_4",+ "answer_f999b05b_3",+ "a2b2d331_2",+ "62f24315_1",+ "23901329_2",+ "sharegpt_Asf5ayG_41",+ "6ec9bed9",+ "sharegpt_JlSnb71_23"+ ],+ "gold_session_ids": [+ "answer_f999b05b_1",+ "answer_f999b05b_4",+ "answer_f999b05b_2",+ "answer_f999b05b_5",+ "answer_f999b05b_3"+ ]+ },+ {+ "question_id": "gpt4_4ef30696",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5714285040141098,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "8fa64936_2",+ "answer_f964cea3_2",+ "d9868305_2",+ "sharegpt_ELYlA30_25",+ "80b3f093",+ "4067bede",+ "f829b530_2",+ "3aa868c7_1",+ "answer_f964cea3_1",+ "4b29957c_1"+ ],+ "gold_session_ids": [+ "answer_f964cea3_1",+ "answer_f964cea3_2"+ ]+ },+ {+ "question_id": "gpt4_ec93e27f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e3aa84c4_1",+ "answer_e3aa84c4_2",+ "b76006cb_3",+ "2a1d13f4",+ "85846900_2",+ "sharegpt_Sg0ctDe_14",+ "6b63bdc5_2",+ "5de5c1ed_1",+ "f11b88d8_3",+ "522dd987_1"+ ],+ "gold_session_ids": [+ "answer_e3aa84c4_2",+ "answer_e3aa84c4_1"+ ]+ },+ {+ "question_id": "6e984301",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_402356",+ "answer_88841f26_2",+ "b38766c1_2",+ "b500d9be",+ "answer_88841f26_1",+ "2bc4ae7d",+ "sharegpt_gVjorGx_0",+ "c285b705_2",+ "55c23a23_2",+ "0daaefa7_1"+ ],+ "gold_session_ids": [+ "answer_88841f26_1",+ "answer_88841f26_2"+ ]+ },+ {+ "question_id": "8077ef71",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "f7ffddbc",+ "ultrachat_468198",+ "answer_0dd54b7c_1",+ "5365840b_1",+ "sharegpt_9EM9xb8_46",+ "sharegpt_11WxPMZ_0",+ "1da90ded",+ "sharegpt_Wi7Op1u_39",+ "d87a6ef8",+ "sharegpt_hoQsVGz_51"+ ],+ "gold_session_ids": [+ "answer_0dd54b7c_1"+ ]+ },+ {+ "question_id": "gpt4_f420262c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3347975674340217,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_B0riiep_0",+ "answer_d8a1af6b_2",+ "a823dc7c",+ "ultrachat_58198",+ "ultrachat_133350",+ "answer_d8a1af6b_4",+ "116bae09",+ "94464f25",+ "3418b277_5",+ "3ea07188_2"+ ],+ "gold_session_ids": [+ "answer_d8a1af6b_1",+ "answer_d8a1af6b_2",+ "answer_d8a1af6b_3",+ "answer_d8a1af6b_4",+ "answer_d8a1af6b_5"+ ]+ },+ {+ "question_id": "gpt4_8e165409",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d97db962_2",+ "answer_d97db962_1",+ "sharegpt_B7mSlfQ_8",+ "sharegpt_JRQtEcd_115",+ "ultrachat_77564",+ "4c2bd0fe_2",+ "sharegpt_DrLV0RL_25",+ "09599d45_1",+ "aee13015_3",+ "ultrachat_175639"+ ],+ "gold_session_ids": [+ "answer_d97db962_1",+ "answer_d97db962_2"+ ]+ },+ {+ "question_id": "gpt4_74aed68e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aed8cf17_2",+ "answer_aed8cf17_1",+ "sharegpt_O3FeoHQ_153",+ "470f29c0_1",+ "ultrachat_176592",+ "ultrachat_337472",+ "50d66391_6",+ "3f05a03e_1",+ "34d446d0",+ "db2f2ed2"+ ],+ "gold_session_ids": [+ "answer_aed8cf17_1",+ "answer_aed8cf17_2"+ ]+ },+ {+ "question_id": "bcbe585f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_43ba3965_1",+ "a98ff421_4",+ "7fa8099b_1",+ "d8a1af6c_5",+ "5dbe11b9",+ "8953c8c8_2",+ "43c487f2_1",+ "sharegpt_WXCvFir_31",+ "a158b7e7",+ "8166673a"+ ],+ "gold_session_ids": [+ "answer_43ba3965_1"+ ]+ },+ {+ "question_id": "gpt4_21adecb5",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.23719771276929622,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "128082f8_2",+ "e1f37e24",+ "sharegpt_ipLglky_42",+ "0f0f9d80_1",+ "answer_1e2369c9_1",+ "b99bd2df",+ "928890e4_1",+ "ultrachat_444790",+ "2880eb6c_1",+ "1206a46c"+ ],+ "gold_session_ids": [+ "answer_1e2369c9_1",+ "answer_1e2369c9_2"+ ]+ },+ {+ "question_id": "5e1b23de",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "1380576d_1",+ "1a5344e9_3",+ "answer_c18d480b_1",+ "c3f945f3_1",+ "0ba46bdc_2",+ "a13abca5_2",+ "ultrachat_153798",+ "c28aed33",+ "8278312e_1",+ "sharegpt_z2MpvCJ_0"+ ],+ "gold_session_ids": [+ "answer_c18d480b_1"+ ]+ },+ {+ "question_id": "gpt4_98f46fc6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5850de18_1",+ "answer_5850de18_2",+ "ultrachat_45511",+ "8e3b479f_1",+ "28897044",+ "c285b705_2",+ "sharegpt_rnL5VQO_0",+ "sharegpt_PLN69pg_19",+ "29e1426c_2",+ "6172cc72_3"+ ],+ "gold_session_ids": [+ "answer_5850de18_2",+ "answer_5850de18_1"+ ]+ },+ {+ "question_id": "gpt4_af6db32f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "aee13015_6",+ "sharegpt_hDjMGr7_70",+ "answer_184c8f56_1",+ "e0b8c937_1",+ "sharegpt_PJmlduQ_0",+ "sharegpt_Np41szj_13",+ "b283f9eb_2",+ "ec1b557a_1",+ "43b449ce",+ "f26846e1_3"+ ],+ "gold_session_ids": [+ "answer_184c8f56_1"+ ]+ },+ {+ "question_id": "eac54adc",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0d4d0347_2",+ "answer_0d4d0347_1",+ "bf234051_2",+ "b40f0a7a",+ "ae051325_3",+ "sharegpt_0ktU0jP_16",+ "9ee69ca6_1",+ "32fc393b_2",+ "490bb46d_3",+ "sharegpt_KzDwaf1_253"+ ],+ "gold_session_ids": [+ "answer_0d4d0347_1",+ "answer_0d4d0347_2"+ ]+ },+ {+ "question_id": "gpt4_7ddcf75f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ad8a76cb_1",+ "sharegpt_CWiYdQ8_152",+ "a0b8b0ad_7",+ "1781acb1",+ "ultrachat_172010",+ "9bfccfdc_2",+ "9f8c432e_3",+ "ultrachat_74113",+ "8c652eb0_2",+ "7ba84ed3_2"+ ],+ "gold_session_ids": [+ "answer_ad8a76cb_1"+ ]+ },+ {+ "question_id": "gpt4_a2d1d1f6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6d6e33f_1",+ "sharegpt_gI1U1eK_4",+ "2c31d8be_3",+ "894dd0c2_3",+ "f5cf8815_4",+ "a1bfb382_1",+ "3b8076bf_1",+ "bbe42a92",+ "a21f3697_3",+ "ultrachat_224894"+ ],+ "gold_session_ids": [+ "answer_f6d6e33f_1"+ ]+ },+ {+ "question_id": "gpt4_85da3956",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_581ab834_1",+ "ultrachat_373754",+ "14b000d9_1",+ "97f8a545_2",+ "fcc7fe77_2",+ "8da8c7e0",+ "sharegpt_h1UXagU_1",+ "sharegpt_ewad5Sr_0",+ "2fd445a2_4",+ "8d952ece_2"+ ],+ "gold_session_ids": [+ "answer_581ab834_1"+ ]+ },+ {+ "question_id": "gpt4_b0863698",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "b2fa108b_1",+ "91e581ad_3",+ "ce8b2a9e_2",+ "answer_550bb2d1_1",+ "5b014f9b_1",+ "4a79d078_2",+ "3b7c6429",+ "sharegpt_FPbCOuq_30",+ "sharegpt_T377N30_0",+ "005ec7cf_1"+ ],+ "gold_session_ids": [+ "answer_550bb2d1_1"+ ]+ },+ {+ "question_id": "gpt4_68e94287",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.19342640361727081,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "43502c3c_2",+ "52c6d59a_2",+ "50f136f2_3",+ "edfa358d_1",+ "64460e9d_1",+ "ultrachat_297774",+ "50ca8a86",+ "answer_9793daa3_2",+ "sharegpt_35udPn0_1",+ "sharegpt_Wm2ElWB_0"+ ],+ "gold_session_ids": [+ "answer_9793daa3_2",+ "answer_9793daa3_1"+ ]+ },+ {+ "question_id": "gpt4_e414231e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e28c1f0d_1",+ "answer_e28c1f0d_2",+ "d3065d85",+ "1bab001f",+ "ultrachat_485311",+ "7e3fa056_2",+ "c87e6c56",+ "c12d5181_3",+ "642d1256",+ "a459d477"+ ],+ "gold_session_ids": [+ "answer_e28c1f0d_1",+ "answer_e28c1f0d_2"+ ]+ },+ {+ "question_id": "gpt4_7ca326fa",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf021b36_1",+ "answer_cf021b36_3",+ "answer_cf021b36_2",+ "8d0ca228",+ "sharegpt_7SCdbX2_0",+ "16d79242_1",+ "604de0a3_2",+ "1301fbd1",+ "sharegpt_CYVtC0w_0",+ "b4025421"+ ],+ "gold_session_ids": [+ "answer_cf021b36_1",+ "answer_cf021b36_2",+ "answer_cf021b36_3"+ ]+ },+ {+ "question_id": "gpt4_7bc6cf22",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e22d6aef_1",+ "09d021e7_1",+ "3e43bf49_1",+ "e6cc6157_1",+ "225f3ed6_2",+ "sharegpt_yzmMl34_0",+ "eca3b71c_2",+ "39566615_1",+ "ultrachat_215356",+ "9f793d6f_3"+ ],+ "gold_session_ids": [+ "answer_e22d6aef_1"+ ]+ },+ {+ "question_id": "2ebe6c92",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "1ba79ea8_1",+ "fb303dd2_2",+ "16756728_1",+ "9d4312f6_3",+ "ca3a4e4f_1",+ "answer_c9d35c00_2",+ "d9e023f2_2",+ "719a18dd_2",+ "84048eba_5",+ "b30a47be_2"+ ],+ "gold_session_ids": [+ "answer_c9d35c00_1",+ "answer_c9d35c00_2"+ ]+ },+ {+ "question_id": "gpt4_e061b84g",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.18154179253735267,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "9dac9e37_1",+ "de43030f_1",+ "68705467_1",+ "ultrachat_547960",+ "answer_8c64ce26_3",+ "ultrachat_213300",+ "ae15e8b6_1",+ "sharegpt_rB6IP4G_0",+ "3929e6cc_1",+ "ultrachat_391029"+ ],+ "gold_session_ids": [+ "answer_8c64ce26_2",+ "answer_8c64ce26_1",+ "answer_8c64ce26_3"+ ]+ },+ {+ "question_id": "71017277",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0b4a8adc_1",+ "976ca0d9",+ "85a1be56_2",+ "f8de4e92_4",+ "5854eebc_2",+ "e64f9553_1",+ "e5b86922_1",+ "0aacd15f_5",+ "dd99d002",+ "40a1e681_1"+ ],+ "gold_session_ids": [+ "answer_0b4a8adc_1"+ ]+ },+ {+ "question_id": "b46e15ee",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4517623092632536,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "sharegpt_1ZakfXH_14",+ "495cf518_2",+ "answer_4bfcc251_4",+ "918c4788_1",+ "f3fcf4fd_1",+ "answer_4bfcc251_1",+ "ultrachat_322794",+ "c92239bc",+ "answer_4bfcc251_2",+ "d4446e25_2"+ ],+ "gold_session_ids": [+ "answer_4bfcc251_4",+ "answer_4bfcc251_3",+ "answer_4bfcc251_2",+ "answer_4bfcc251_1"+ ]+ },+ {+ "question_id": "gpt4_d6585ce9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.4973199614801841,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "58dd3f35",+ "bb107057_4",+ "a554ed79_4",+ "answer_f999b05c_5",+ "answer_f999b05c_2",+ "17a561cb_2",+ "answer_f999b05c_1",+ "answer_f999b05c_4",+ "a15fa6eb",+ "b3763b6b_1"+ ],+ "gold_session_ids": [+ "answer_f999b05c_1",+ "answer_f999b05c_4",+ "answer_f999b05c_2",+ "answer_f999b05c_5",+ "answer_f999b05c_3"+ ]+ },+ {+ "question_id": "gpt4_1e4a8aec",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "b1265fd7_1",+ "9f1f3bf4",+ "b34a52aa_3",+ "e675c69a_1",+ "1c662b7b_1",+ "a8086137",+ "sharegpt_9JZCfee_0",+ "0b635369_2",+ "49576fd6",+ "e41b78c7_2"+ ],+ "gold_session_ids": [+ "answer_16bd5ea6_1",+ "answer_16bd5ea6_2"+ ]+ },+ {+ "question_id": "gpt4_f420262d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8550961823035497,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8a1af6c_2",+ "27639dd8_3",+ "answer_d8a1af6c_4",+ "0dcfb511_1",+ "answer_d8a1af6c_5",+ "1032cbcf_1",+ "answer_d8a1af6c_1",+ "3c0ed3bf_4",+ "answer_d8a1af6c_3",+ "e1023f28"+ ],+ "gold_session_ids": [+ "answer_d8a1af6c_1",+ "answer_d8a1af6c_2",+ "answer_d8a1af6c_5",+ "answer_d8a1af6c_4",+ "answer_d8a1af6c_3"+ ]+ },+ {+ "question_id": "gpt4_59149c78",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5249810332008933,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "415657e0",+ "sharegpt_GI6737T_2",+ "answer_d00ba6d1_2",+ "afec1061_2",+ "9848a1e3_2",+ "answer_d00ba6d1_1",+ "ultrachat_463998",+ "23754665",+ "765ce8a7_2",+ "2c282c7c"+ ],+ "gold_session_ids": [+ "answer_d00ba6d1_1",+ "answer_d00ba6d1_2"+ ]+ },+ {+ "question_id": "gpt4_e414231f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e28c1f0e_1",+ "ultrachat_136119",+ "ultrachat_150948",+ "0bd59f15_2",+ "deb5bb60",+ "answer_e28c1f0e_2",+ "752c438c_3",+ "sharegpt_K8mXaIT_0",+ "07a26dfa_5",+ "135f761b_1"+ ],+ "gold_session_ids": [+ "answer_e28c1f0e_1",+ "answer_e28c1f0e_2"+ ]+ },+ {+ "question_id": "gpt4_4929293b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.0625,+ "retrieved_session_ids": [+ "b4ed7223_3",+ "ultrachat_511609",+ "e76430ea_3",+ "7fa98614_1",+ "bda611f6_3",+ "c0dbabb8",+ "ultrachat_156176",+ "11d959b3_1",+ "628c958c_1",+ "ea8c38f8"+ ],+ "gold_session_ids": [+ "answer_add9b013_2",+ "answer_add9b013_1"+ ]+ },+ {+ "question_id": "gpt4_468eb064",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.05555555555555555,+ "retrieved_session_ids": [+ "ultrachat_337251",+ "1d74a734",+ "db1aefb6_3",+ "5e2b7e01_1",+ "sharegpt_wrwD6bG_0",+ "a2f0054f",+ "ultrachat_59629",+ "21ab8a2c_1",+ "c3023a45_1",+ "25e5b797_1"+ ],+ "gold_session_ids": [+ "answer_9b09d95b_1"+ ]+ },+ {+ "question_id": "gpt4_fa19884d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "588a2123",+ "answer_ff201787_2",+ "answer_ff201787_1",+ "f910dc31_4",+ "b35fadaa",+ "ccc87da9_1",+ "040615bc_1",+ "f7a61595_3",+ "55da3cbf_2",+ "ultrachat_424557"+ ],+ "gold_session_ids": [+ "answer_ff201787_1",+ "answer_ff201787_2"+ ]+ },+ {+ "question_id": "9a707b82",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.07142857142857142,+ "retrieved_session_ids": [+ "d33dbfd5_2",+ "fab41c07",+ "e8df5350",+ "990f3ef9_2",+ "8e78fa70_1",+ "7966888b_2",+ "f8e9d4fa",+ "ed5c4c33_1",+ "09e97c6f_2",+ "778bedd4"+ ],+ "gold_session_ids": [+ "answer_dba89488_2",+ "answer_dba89488_1"+ ]+ },+ {+ "question_id": "eac54add",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.06666666666666667,+ "retrieved_session_ids": [+ "2ea6fda4_1",+ "7966888b_2",+ "eb336de0_3",+ "7bb01253",+ "773aebbd_3",+ "2946bf12",+ "072fac63_1",+ "4d578deb",+ "bdb49a42_2",+ "1fb3bbed_1"+ ],+ "gold_session_ids": [+ "answer_0d4d0348_1",+ "answer_0d4d0348_2"+ ]+ },+ {+ "question_id": "4dfccbf8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "sharegpt_uD7CBng_2",+ "sharegpt_3D3oQC0_213",+ "953aaad5_3",+ "0ae4c9b0_2",+ "7db28e68_1",+ "answer_4bebc783_1",+ "b4f63a70_3",+ "edd89480_1",+ "a63ad8e3_3",+ "6c16c3ec_2"+ ],+ "gold_session_ids": [+ "answer_4bebc783_1",+ "answer_4bebc783_2"+ ]+ },+ {+ "question_id": "0bc8ad93",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f4ea84fc_1",+ "answer_f4ea84fc_3",+ "answer_f4ea84fc_2",+ "sharegpt_HOO59Ue_99",+ "2c185a2b_1",+ "ultrachat_260376",+ "sharegpt_afCogMg_34",+ "ef23ecbc_2",+ "85736a07_1",+ "97338ddd_1"+ ],+ "gold_session_ids": [+ "answer_f4ea84fc_3",+ "answer_f4ea84fc_2",+ "answer_f4ea84fc_1"+ ]+ },+ {+ "question_id": "6e984302",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.430624116386567,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "c03fc56c_1",+ "41dc5d45_1",+ "7285299a_8",+ "2ed9532f",+ "answer_88841f27_1",+ "cf3dc165_1",+ "sharegpt_4Wjx3pH_0",+ "answer_88841f27_2",+ "b7702ee4_1",+ "7cd7c296_1"+ ],+ "gold_session_ids": [+ "answer_88841f27_1",+ "answer_88841f27_2"+ ]+ },+ {+ "question_id": "gpt4_8279ba03",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.07142857142857142,+ "retrieved_session_ids": [+ "864a563d_5",+ "b357fb8b_2",+ "606d0c4a_2",+ "084bb880_2",+ "518d26d3_4",+ "d1a1b9ea_1",+ "652c0717_3",+ "dea400f8_2",+ "50d66391_4",+ "2ef55f49_3"+ ],+ "gold_session_ids": [+ "answer_56521e66_1"+ ]+ },+ {+ "question_id": "gpt4_b5700ca0",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a17423e8_1",+ "109e07e8_2",+ "ultrachat_540666",+ "b403d6f0_4",+ "e789afdb_2",+ "5829ab38",+ "fd6f60f0_1",+ "064f42ee_3",+ "ultrachat_418858",+ "sharegpt_FpTfRvR_0"+ ],+ "gold_session_ids": [+ "answer_a17423e8_1"+ ]+ },+ {+ "question_id": "gpt4_68e94288",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.23719771276929622,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "4c49e37f",+ "6551be60_1",+ "767b1b89",+ "d2606fe9",+ "answer_9793daa4_1",+ "d8454317_7",+ "0a7c4f1c_1",+ "c99dcd81",+ "128f4e4d_2",+ "ultrachat_74747"+ ],+ "gold_session_ids": [+ "answer_9793daa4_2",+ "answer_9793daa4_1"+ ]+ },+ {+ "question_id": "gpt4_2655b836",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4be1b6b4_3",+ "answer_4be1b6b4_2",+ "answer_4be1b6b4_1",+ "sharegpt_KzDwaf1_237",+ "d8a1af6b_4",+ "b876911c",+ "be32ad25_2",+ "d82cefd3",+ "3bc55308",+ "e6120ff9_2"+ ],+ "gold_session_ids": [+ "answer_4be1b6b4_1",+ "answer_4be1b6b4_2",+ "answer_4be1b6b4_3"+ ]+ },+ {+ "question_id": "gpt4_2487a7cb",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1c6b85ea_2",+ "4f5880c6_6",+ "answer_1c6b85ea_1",+ "sharegpt_QN26oUg_0",+ "f87fea58_4",+ "sharegpt_CHaU4ax_10",+ "67a8ef5f",+ "sharegpt_gLb26GY_0",+ "ultrachat_295777",+ "b10c041b_2"+ ],+ "gold_session_ids": [+ "answer_1c6b85ea_2",+ "answer_1c6b85ea_1"+ ]+ },+ {+ "question_id": "gpt4_76048e76",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b535969f_1",+ "409dfa26_1",+ "answer_b535969f_2",+ "33630c9e_2",+ "8fcaf3a9_2",+ "sharegpt_vx14orQ_11",+ "sharegpt_inuIr9J_119",+ "e2d9bcfe",+ "6a78e959_2",+ "ultrachat_185750"+ ],+ "gold_session_ids": [+ "answer_b535969f_1",+ "answer_b535969f_2"+ ]+ },+ {+ "question_id": "gpt4_2312f94c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5328c3c2_2",+ "answer_5328c3c2_1",+ "e831a29f_2",+ "sharegpt_XYtuANl_9",+ "ultrachat_327449",+ "ultrachat_500682",+ "f3745025_2",+ "a764b677_2",+ "ultrachat_319398",+ "b9f59a03_2"+ ],+ "gold_session_ids": [+ "answer_5328c3c2_2",+ "answer_5328c3c2_1"+ ]+ },+ {+ "question_id": "0bb5a684",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8065735963827293,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e936197f_1",+ "3c8e7c0e_3",+ "1a14d6cb_1",+ "ultrachat_62919",+ "826d51da_3",+ "565929a2_2",+ "b2d46bf2",+ "answer_e936197f_2",+ "fce8bb8a_2",+ "79a20d73"+ ],+ "gold_session_ids": [+ "answer_e936197f_1",+ "answer_e936197f_2"+ ]+ },+ {+ "question_id": "08f4fc43",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6ea1541e_1",+ "answer_6ea1541e_2",+ "ultrachat_359609",+ "9216c4d3_4",+ "762a4ba1_3",+ "sharegpt_JJu53iW_0",+ "e295eb5f",+ "ultrachat_157004",+ "f3bd00ec_2",+ "cdbd6862_1"+ ],+ "gold_session_ids": [+ "answer_6ea1541e_2",+ "answer_6ea1541e_1"+ ]+ },+ {+ "question_id": "2c63a862",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20438239758848611,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "93a5cfe6",+ "33baaaab_1",+ "ad175dc4_2",+ "1854abc7_2",+ "64eea322_1",+ "0fccc817_1",+ "answer_d39b7977_2",+ "83c13ff9",+ "acbc1954",+ "9c549466_2"+ ],+ "gold_session_ids": [+ "answer_d39b7977_1",+ "answer_d39b7977_2"+ ]+ },+ {+ "question_id": "gpt4_385a5000",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7a4a93f1_2",+ "answer_7a4a93f1_1",+ "8062f4ab_2",+ "7a4e1601",+ "28f74b09_1",+ "6380e5d6",+ "cae65795_1",+ "29d27eaa_2",+ "dc832f73",+ "sharegpt_Dcoc4IC_17"+ ],+ "gold_session_ids": [+ "answer_7a4a93f1_2",+ "answer_7a4a93f1_1"+ ]+ },+ {+ "question_id": "2a1811e2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1cc3cd0c_2",+ "answer_1cc3cd0c_1",+ "f5a4d792_1",+ "901a6763_2",+ "1aaf8057_3",+ "ultrachat_187883",+ "e9fb10e7_1",+ "4019b1a8_4",+ "2ac6f050_1",+ "89e05490"+ ],+ "gold_session_ids": [+ "answer_1cc3cd0c_1",+ "answer_1cc3cd0c_2"+ ]+ },+ {+ "question_id": "bbf86515",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5249810332008933,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "592cb8d2",+ "75b98d2a_1",+ "answer_b3763b6b_1",+ "05260a4b_2",+ "sharegpt_GrLzv0V_432",+ "answer_b3763b6b_2",+ "ba85548f",+ "df003c93_2",+ "ultrachat_320650",+ "29194fbe_1"+ ],+ "gold_session_ids": [+ "answer_b3763b6b_1",+ "answer_b3763b6b_2"+ ]+ },+ {+ "question_id": "gpt4_5dcc0aab",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9277589908435726,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_099c1b6c_3",+ "answer_099c1b6c_1",+ "answer_099c1b6c_5",+ "353d3c6d_1",+ "3891baea_3",+ "ultrachat_156812",+ "3f787a78_6",+ "answer_099c1b6c_2",+ "sharegpt_5Rpp72b_87",+ "answer_099c1b6c_4"+ ],+ "gold_session_ids": [+ "answer_099c1b6c_5",+ "answer_099c1b6c_2",+ "answer_099c1b6c_4",+ "answer_099c1b6c_3",+ "answer_099c1b6c_1"+ ]+ },+ {+ "question_id": "gpt4_0b2f1d21",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c4e5d969_2",+ "answer_c4e5d969_1",+ "1cafd864_3",+ "f31a0422_1",+ "sharegpt_hDjMGr7_30",+ "sharegpt_ME35OD8_167",+ "sharegpt_uD7CBng_36",+ "0418a4b1_1",+ "0df69ec5",+ "sharegpt_KBe10PZ_0"+ ],+ "gold_session_ids": [+ "answer_c4e5d969_2",+ "answer_c4e5d969_1"+ ]+ },+ {+ "question_id": "f0853d11",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_932ea3bf_1",+ "answer_932ea3bf_2",+ "b16a8656_2",+ "10c0752c",+ "sharegpt_Dcshjlv_33",+ "sharegpt_9upOSlE_27",+ "bd8708e2_2",+ "6e436d54",+ "sharegpt_hBX7Nqo_25",+ "8c68df0d_2"+ ],+ "gold_session_ids": [+ "answer_932ea3bf_2",+ "answer_932ea3bf_1"+ ]+ },+ {+ "question_id": "gpt4_6ed717ea",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d50a8a33_2",+ "answer_d50a8a33_1",+ "15ed03f0_1",+ "637bc32a_2",+ "ultrachat_33224",+ "d4b6ec0b",+ "ultrachat_400856",+ "e0585cb5_1",+ "sharegpt_HQMYIkt_0",+ "50940cb7_1"+ ],+ "gold_session_ids": [+ "answer_d50a8a33_1",+ "answer_d50a8a33_2"+ ]+ },+ {+ "question_id": "gpt4_70e84552",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b3070ec4_2",+ "01f61248_1",+ "answer_b3070ec4_1",+ "2f8a3459_2",+ "ultrachat_573894",+ "30491f6f_1",+ "67a8ef5f",+ "298a07de_5",+ "sharegpt_d7bP5fb_0",+ "2b9b6224_2"+ ],+ "gold_session_ids": [+ "answer_b3070ec4_2",+ "answer_b3070ec4_1"+ ]+ },+ {+ "question_id": "a3838d2b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3482538302146053,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "sharegpt_GDFyxeT_0",+ "sharegpt_JXREDs0_4",+ "ultrachat_462884",+ "answer_4ffa04a2_1",+ "answer_4ffa04a2_6",+ "8cf0d2a9",+ "answer_4ffa04a2_4",+ "ultrachat_95340",+ "bd4110bc",+ "ae73b0f7_2"+ ],+ "gold_session_ids": [+ "answer_4ffa04a2_1",+ "answer_4ffa04a2_6",+ "answer_4ffa04a2_4",+ "answer_4ffa04a2_3",+ "answer_4ffa04a2_5",+ "answer_4ffa04a2_2"+ ]+ },+ {+ "question_id": "gpt4_93159ced",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e5131a1b_2",+ "263efe64_1",+ "17d91635_1",+ "ultrachat_387032",+ "answer_e5131a1b_1",+ "460be54a",+ "c81d1a7b_2",+ "e9ad5914_6",+ "09b815e7_2",+ "ultrachat_324774"+ ],+ "gold_session_ids": [+ "answer_e5131a1b_2",+ "answer_e5131a1b_1"+ ]+ },+ {+ "question_id": "gpt4_2d58bcd6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e11e0ae_2",+ "0bd0a01c",+ "answer_3e11e0ae_1",+ "1bf8b212",+ "sharegpt_XUxvLWu_0",+ "be32ad25_1",+ "sharegpt_DvZqBYf_93",+ "343cbb3b",+ "5d4bfa9d",+ "c48a1e21_2"+ ],+ "gold_session_ids": [+ "answer_3e11e0ae_1",+ "answer_3e11e0ae_2"+ ]+ },+ {+ "question_id": "gpt4_65aabe59",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "6a7e3aec",+ "answer_30dfe889_1",+ "answer_30dfe889_2",+ "77b8fcd9_1",+ "78bcce6a",+ "ultrachat_190669",+ "52fdad30_4",+ "sharegpt_v2b1TzW_0",+ "05a37787",+ "9b1a5fb0"+ ],+ "gold_session_ids": [+ "answer_30dfe889_1",+ "answer_30dfe889_2"+ ]+ },+ {+ "question_id": "982b5123",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ab603dd5_1",+ "04e05522_2",+ "answer_ab603dd5_2",+ "9a955705",+ "6ff77d45_2",+ "359a2a0d",+ "bf8ab267",+ "07ba9acd_2",+ "1bfd5a8b_3",+ "ultrachat_35755"+ ],+ "gold_session_ids": [+ "answer_ab603dd5_1",+ "answer_ab603dd5_2"+ ]+ },+ {+ "question_id": "b9cfe692",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e3bb940_1",+ "answer_5e3bb940_2",+ "answer_5e3bb940_3",+ "91d847c1_6",+ "da650f3b",+ "sharegpt_oXgiN7q_53",+ "0c27c633_2",+ "ultrachat_265626",+ "sharegpt_JRQtEcd_115",+ "2c7bff3a_1"+ ],+ "gold_session_ids": [+ "answer_5e3bb940_3",+ "answer_5e3bb940_2",+ "answer_5e3bb940_1"+ ]+ },+ {+ "question_id": "gpt4_4edbafa2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.48247555939075504,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "7533d546_5",+ "3f5249ee_3",+ "22be78fc_2",+ "answer_0a00c163_2",+ "6c68d9b4_1",+ "answer_0a00c163_1",+ "a65d076a",+ "4a006b77",+ "725f95e1",+ "18c0af9c"+ ],+ "gold_session_ids": [+ "answer_0a00c163_1",+ "answer_0a00c163_2"+ ]+ },+ {+ "question_id": "c8090214",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.44130740935663865,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "ultrachat_518687",+ "005ec7cf_1",+ "162ad3bf_1",+ "answer_70dc7d08_2",+ "c21c9d97",+ "sharegpt_xOzzaof_0",+ "e2d9bcfe",+ "ultrachat_381881",+ "ultrachat_415426",+ "answer_70dc7d08_1"+ ],+ "gold_session_ids": [+ "answer_70dc7d08_2",+ "answer_70dc7d08_1"+ ]+ },+ {+ "question_id": "gpt4_483dd43c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fb793c87_1",+ "answer_fb793c87_2",+ "sharegpt_I6IzvEF_0",+ "29695e1c_3",+ "sharegpt_ItFPuha_0",+ "4243ce01_1",+ "52161c9e_1",+ "19c24c11",+ "sharegpt_E0YL5SX_145",+ "73353aee_3"+ ],+ "gold_session_ids": [+ "answer_fb793c87_1",+ "answer_fb793c87_2"+ ]+ },+ {+ "question_id": "e4e14d04",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf425855_2",+ "answer_cf425855_1",+ "sharegpt_HO1gGmm_0",+ "ultrachat_542420",+ "ffc627f2",+ "sharegpt_zX5iazJ_19",+ "1c1f5ccc_3",+ "5944b36a_1",+ "f8de4e92_2",+ "30f7f504"+ ],+ "gold_session_ids": [+ "answer_cf425855_1",+ "answer_cf425855_2"+ ]+ },+ {+ "question_id": "c9f37c46",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cdba3d9f_1",+ "answer_cdba3d9f_2",+ "a8e24e22",+ "0cf86cbe_1",+ "ultrachat_418541",+ "02df1225_1",+ "b38766c1_3",+ "211bb1c2_1",+ "4ffa04a2_1",+ "c3d6a282_1"+ ],+ "gold_session_ids": [+ "answer_cdba3d9f_1",+ "answer_cdba3d9f_2"+ ]+ },+ {+ "question_id": "gpt4_2c50253f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_4Su6bUM_11",+ "answer_9af4e346_2",+ "a8fae4b8_2",+ "answer_9af4e346_1",+ "57c8bf62_1",+ "sharegpt_o0jtbUO_8",+ "4b9ed528_2",+ "4cef8a3c_3",+ "462ce3bc_2",+ "ultrachat_340289"+ ],+ "gold_session_ids": [+ "answer_9af4e346_1",+ "answer_9af4e346_2"+ ]+ },+ {+ "question_id": "dcfa8644",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e3eeb12_1",+ "answer_5e3eeb12_2",+ "b2243528_1",+ "58dd3f35",+ "0d129c99",+ "7de51ffe_1",+ "41f94af6",+ "ultrachat_431004",+ "sharegpt_oG3LXXo_6",+ "595902ff"+ ],+ "gold_session_ids": [+ "answer_5e3eeb12_2",+ "answer_5e3eeb12_1"+ ]+ },+ {+ "question_id": "gpt4_b4a80587",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b9d9150e_2",+ "answer_b9d9150e_1",+ "sharegpt_inuIr9J_119",+ "f79a48dd_1",+ "3cb41ab8_1",+ "ultrachat_91937",+ "866de400",+ "017fa8a9",+ "b9c4fa1b",+ "4ecbe1f0_1"+ ],+ "gold_session_ids": [+ "answer_b9d9150e_2",+ "answer_b9d9150e_1"+ ]+ },+ {+ "question_id": "gpt4_9a159967",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6871475159848289,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "b2cae160_1",+ "answer_8a42fedf_1",+ "answer_8a42fedf_2",+ "4ea94f85",+ "253742b4_3",+ "ultrachat_484212",+ "answer_8a42fedf_3",+ "0df54428_4",+ "0ad608b3",+ "bff09560_2"+ ],+ "gold_session_ids": [+ "answer_8a42fedf_1",+ "answer_8a42fedf_3",+ "answer_8a42fedf_2"+ ]+ },+ {+ "question_id": "cc6d1ec1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_be73098b_2",+ "answer_be73098b_1",+ "33baaaab_3",+ "ultrachat_282969",+ "ultrachat_95254",+ "ultrachat_508911",+ "ultrachat_436945",+ "sharegpt_uD7CBng_20",+ "sharegpt_j4070wP_0",+ "3cb41ab8_1"+ ],+ "gold_session_ids": [+ "answer_be73098b_2",+ "answer_be73098b_1"+ ]+ },+ {+ "question_id": "gpt4_8c8961ae",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_72d9aa58_1",+ "answer_72d9aa58_2",+ "3b22d17b",+ "1cde7ee4_2",+ "b77d0b58",+ "05fad1cb_2",+ "43a2c8af_1",+ "37f0ce6b_1",+ "02f9bedb_2",+ "sharegpt_Ms1yTte_64"+ ],+ "gold_session_ids": [+ "answer_72d9aa58_1",+ "answer_72d9aa58_2"+ ]+ },+ {+ "question_id": "gpt4_d9af6064",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da704e79_1",+ "answer_da704e79_2",+ "03b4d5d8_2",+ "ff49b5a5_2",+ "bcebee27_3",+ "sharegpt_VbIxRWJ_33",+ "sharegpt_WjW4qki_0",+ "sharegpt_w9YUs4c_0",+ "ultrachat_182531",+ "98c83fe4"+ ],+ "gold_session_ids": [+ "answer_da704e79_2",+ "answer_da704e79_1"+ ]+ },+ {+ "question_id": "gpt4_7de946e7",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "caa00337_2",+ "answer_6a78e959_1",+ "answer_6a78e959_2",+ "c4d370d3_1",+ "58bec5fb_3",+ "6155addd_2",+ "ultrachat_220429",+ "sharegpt_6Yfj9A3_5",+ "3023b36f",+ "ultrachat_105249"+ ],+ "gold_session_ids": [+ "answer_6a78e959_2",+ "answer_6a78e959_1"+ ]+ },+ {+ "question_id": "d01c6aa8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.17723928678404774,+ "mrr": 0.1,+ "retrieved_session_ids": [+ "f2fa97f5_1",+ "fc9a62c0_1",+ "51fe163e_2",+ "6ebd9e18_1",+ "5024758e",+ "fbd67491_2",+ "64460e9d_1",+ "sharegpt_CWiYdQ8_13",+ "debc34d7_2",+ "answer_991d55e5_1"+ ],+ "gold_session_ids": [+ "answer_991d55e5_1",+ "answer_991d55e5_2"+ ]+ },+ {+ "question_id": "993da5e2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_54f0d6f9_2",+ "b16a8656_2",+ "12cd736c",+ "answer_54f0d6f9_1",+ "c756d93e",+ "a6c0f032_3",+ "8953c8c8_3",+ "13f3da23_2",+ "c69a28f0",+ "ultrachat_217815"+ ],+ "gold_session_ids": [+ "answer_54f0d6f9_1",+ "answer_54f0d6f9_2"+ ]+ },+ {+ "question_id": "a3045048",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "aaf71ce2_2",+ "answer_016f6bd4_1",+ "answer_016f6bd4_2",+ "5cb58cc6",+ "d12d5a98_1",+ "9cdb028a",+ "2def525b_1",+ "536a334d",+ "sharegpt_asoSnhq_0",+ "752b10e6_2"+ ],+ "gold_session_ids": [+ "answer_016f6bd4_2",+ "answer_016f6bd4_1"+ ]+ },+ {+ "question_id": "gpt4_d31cdae3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8464304d_2",+ "answer_8464304d_1",+ "20a5c39b_3",+ "55033f6f",+ "de695a4e",+ "41743aae_2",+ "9f8c432e_1",+ "sharegpt_HDF4645_0",+ "ultrachat_413288",+ "ultrachat_169591"+ ],+ "gold_session_ids": [+ "answer_8464304d_1",+ "answer_8464304d_2"+ ]+ },+ {+ "question_id": "gpt4_cd90e484",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aa930b56_2",+ "answer_aa930b56_1",+ "6043dcf6_1",+ "018eac29",+ "83fb74bf_1",+ "866de400",+ "4bfcc251_1",+ "sharegpt_obCC1BP_14",+ "cc5a6057_2",+ "af42ae9c_1"+ ],+ "gold_session_ids": [+ "answer_aa930b56_1",+ "answer_aa930b56_2"+ ]+ },+ {+ "question_id": "gpt4_88806d6e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5912352048230277,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_562852",+ "answer_e60a93ff_1",+ "9ac3fa82_1",+ "sharegpt_j4PmNxu_11",+ "ultrachat_258611",+ "c11fb634_1",+ "answer_e60a93ff_2",+ "24bfc009_5",+ "ultrachat_220071",+ "ef0341d4_2"+ ],+ "gold_session_ids": [+ "answer_e60a93ff_1",+ "answer_e60a93ff_2"+ ]+ },+ {+ "question_id": "gpt4_4cd9eba1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5fcca8bc_1",+ "answer_5fcca8bc_2",+ "1b576f6e_2",+ "dafc2440_2",+ "f54d04cc",+ "456860e7_2",+ "sharegpt_3hCiKrk_0",+ "d36a862c_1",+ "3be2cefb_1",+ "ultrachat_221041"+ ],+ "gold_session_ids": [+ "answer_5fcca8bc_2",+ "answer_5fcca8bc_1"+ ]+ },+ {+ "question_id": "gpt4_93f6379c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9010126104210695,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_544fe66c_2",+ "answer_544fe66c_3",+ "c07b8034",+ "d3a34811",+ "1b9b2960_3",+ "b8a688f6",+ "e06c6d99_5",+ "sharegpt_cfESFSo_0",+ "5024758e",+ "answer_544fe66c_1"+ ],+ "gold_session_ids": [+ "answer_544fe66c_2",+ "answer_544fe66c_1",+ "answer_544fe66c_3"+ ]+ },+ {+ "question_id": "b29f3365",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_436d4309_2",+ "answer_436d4309_1",+ "sharegpt_VhAV1pG_0",+ "c279c578",+ "ultrachat_58020",+ "66b56e10",+ "6b7605d1_1",+ "ultrachat_533198",+ "ultrachat_243663",+ "95c9666f_5"+ ],+ "gold_session_ids": [+ "answer_436d4309_2",+ "answer_436d4309_1"+ ]+ },+ {+ "question_id": "gpt4_2f56ae70",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.34870224750355494,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "3aac691d_2",+ "dc3ee1d1_1",+ "83cd701b_3",+ "e61d966a_1",+ "answer_7a36e820_2",+ "answer_7a36e820_3",+ "ultrachat_452651",+ "sharegpt_KzDwaf1_165",+ "647f5851",+ "sharegpt_BZ0hFLo_0"+ ],+ "gold_session_ids": [+ "answer_7a36e820_2",+ "answer_7a36e820_3",+ "answer_7a36e820_1"+ ]+ },+ {+ "question_id": "6613b389",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7653606369886217,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aaf71ce2_2",+ "answer_aaf71ce2_3",+ "e3d8da67",+ "f1cc1ddb_1",+ "2780323d_3",+ "86c8f348_1",+ "e87aa998_1",+ "ultrachat_444602",+ "ultrachat_31390",+ "55bc2e1e"+ ],+ "gold_session_ids": [+ "answer_aaf71ce2_3",+ "answer_aaf71ce2_2",+ "answer_aaf71ce2_1"+ ]+ },+ {+ "question_id": "gpt4_78cf46a3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5a78688d_2",+ "sharegpt_rx39ipj_0",+ "7fd52e1c_2",+ "answer_5a78688d_1",+ "sharegpt_89M8Aiz_0",+ "b89a3ec3",+ "sharegpt_Fakxkr4_10",+ "42b45f6e_2",+ "e8e65a1f",+ "52b19cba_1"+ ],+ "gold_session_ids": [+ "answer_5a78688d_1",+ "answer_5a78688d_2"+ ]+ },+ {+ "question_id": "gpt4_0a05b494",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a68db5db_2",+ "56911dc5_6",+ "answer_a68db5db_1",+ "ultrachat_273213",+ "0abe3e51",+ "8fb3fe3a_2",+ "sharegpt_eWQFoct_17",+ "sharegpt_PfiDxfU_9",+ "30677a32",+ "05b84f71_1"+ ],+ "gold_session_ids": [+ "answer_a68db5db_2",+ "answer_a68db5db_1"+ ]+ },+ {+ "question_id": "gpt4_1a1dc16d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "87f1f950_2",+ "answer_faad7d7a_2",+ "answer_faad7d7a_1",+ "ultrachat_480665",+ "c12d5181_1",+ "sharegpt_6cz1Sq6_328",+ "ultrachat_63236",+ "ultrachat_41485",+ "ultrachat_428346",+ "ultrachat_523153"+ ],+ "gold_session_ids": [+ "answer_faad7d7a_2",+ "answer_faad7d7a_1"+ ]+ },+ {+ "question_id": "gpt4_2f584639",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_11a8f823_2",+ "answer_11a8f823_1",+ "ultrachat_430445",+ "ultrachat_229781",+ "c33707b0",+ "sharegpt_bq9G6bT_2",+ "13947b64_1",+ "821242cd_2",+ "68131948",+ "d441f4c5"+ ],+ "gold_session_ids": [+ "answer_11a8f823_2",+ "answer_11a8f823_1"+ ]+ },+ {+ "question_id": "gpt4_213fd887",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_53582e7e_2",+ "answer_53582e7e_1",+ "ultrachat_165091",+ "dc1f9713_1",+ "9bdbfc62_2",+ "3b38bf46",+ "dee39195_2",+ "8aa17385",+ "368276ab_3",+ "6d734945"+ ],+ "gold_session_ids": [+ "answer_53582e7e_2",+ "answer_53582e7e_1"+ ]+ },+ {+ "question_id": "gpt4_5438fa52",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.39780880120575696,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "a8e24e22",+ "ab8dce45",+ "d68bab3c_1",+ "0a699029_4",+ "528f481d_6",+ "ultrachat_69599",+ "answer_b10f3828_2",+ "answer_b10f3828_1",+ "sharegpt_LVUZee8_35",+ "290fe09a"+ ],+ "gold_session_ids": [+ "answer_b10f3828_1",+ "answer_b10f3828_2"+ ]+ },+ {+ "question_id": "gpt4_c27434e8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8e33f5c_2",+ "answer_d8e33f5c_1",+ "ultrachat_70335",+ "4d578deb",+ "sharegpt_TRck0h4_0",+ "sharegpt_9XlbWvL_0",+ "e6c3a50a",+ "5d1c7bf7",+ "11cd43df",+ "16933961"+ ],+ "gold_session_ids": [+ "answer_d8e33f5c_1",+ "answer_d8e33f5c_2"+ ]+ },+ {+ "question_id": "gpt4_fe651585",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5012658353418871,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "ff6b8624",+ "e30c4a7b_2",+ "9f220c66_1",+ "answer_65600ff6_2",+ "answer_65600ff6_1",+ "231c60aa",+ "ultrachat_576354",+ "987b36db_3",+ "da828711",+ "cee0b8d5"+ ],+ "gold_session_ids": [+ "answer_65600ff6_2",+ "answer_65600ff6_1"+ ]+ },+ {+ "question_id": "8c18457d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_124f5dc3_2",+ "answer_124f5dc3_1",+ "e8e65a1f",+ "3c8e7c0e_2",+ "98cd882c_2",+ "c3023a45_2",+ "ultrachat_417305",+ "c8dcbe7f_1",+ "9f803df5_1",+ "ultrachat_166506"+ ],+ "gold_session_ids": [+ "answer_124f5dc3_2",+ "answer_124f5dc3_1"+ ]+ },+ {+ "question_id": "gpt4_70e84552_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b3070ec4_abs_1",+ "answer_b3070ec4_abs_2",+ "e2cd250e_2",+ "f6ba44fe_1",+ "0a699029_4",+ "4e0da76d",+ "ultrachat_96694",+ "sharegpt_3hJWfne_11",+ "51d338f6",+ "1572cf8a"+ ],+ "gold_session_ids": [+ "answer_b3070ec4_abs_2",+ "answer_b3070ec4_abs_1"+ ]+ },+ {+ "question_id": "gpt4_93159ced_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5912352048230277,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "156988fe",+ "answer_e5131a1b_abs_1",+ "76385c25",+ "f5eb52d6",+ "168feef7_2",+ "eb80877b_1",+ "answer_e5131a1b_abs_2",+ "7b9ee249_1",+ "2f097fba_1",+ "56bba837"+ ],+ "gold_session_ids": [+ "answer_e5131a1b_abs_1",+ "answer_e5131a1b_abs_2"+ ]+ },+ {+ "question_id": "982b5123_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.18457569677956817,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "ultrachat_447209",+ "a8179676",+ "sharegpt_kaEoitr_15",+ "sharegpt_jxsIY6L_0",+ "b1793bba",+ "ultrachat_461193",+ "bee1aec6_2",+ "fe82a033_1",+ "answer_ab603dd5_abs_1",+ "5080119d_1"+ ],+ "gold_session_ids": [+ "answer_ab603dd5_abs_1",+ "answer_ab603dd5_abs_2"+ ]+ },+ {+ "question_id": "c8090214_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3065735963827292,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ca3aa07f_1",+ "d6f5639a_3",+ "answer_70dc7d08_abs_1",+ "ultrachat_393699",+ "64c71782",+ "ultrachat_129606",+ "687f4717",+ "f10be626",+ "2cfd748e_5",+ "b192ae00_1"+ ],+ "gold_session_ids": [+ "answer_70dc7d08_abs_1",+ "answer_70dc7d08_abs_2"+ ]+ },+ {+ "question_id": "gpt4_c27434e8_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8e33f5c_abs_1",+ "answer_d8e33f5c_abs_2",+ "ultrachat_552313",+ "25d1830a_1",+ "5e0e1fd8_3",+ "d2335d72",+ "5c184f9f_5",+ "59a6150d_3",+ "9b8dea8c_1",+ "sharegpt_hGtkNGL_12"+ ],+ "gold_session_ids": [+ "answer_d8e33f5c_abs_1",+ "answer_d8e33f5c_abs_2"+ ]+ },+ {+ "question_id": "gpt4_fe651585_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_65600ff6_abs_2",+ "d9bce149_2",+ "sharegpt_7PLgNll_0",+ "adc9bd10",+ "ultrachat_238311",+ "sharegpt_TBRPvuE_0",+ "e4bacbcd_1",+ "sharegpt_kTBVvbN_0",+ "caa00337_1",+ "6b7abcd9"+ ],+ "gold_session_ids": [+ "answer_65600ff6_abs_1",+ "answer_65600ff6_abs_2"+ ]+ },+ {+ "question_id": "6a1eabeb",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5640920940185894,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "afe04238_7",+ "answer_a25d4a91_2",+ "51a529d2_2",+ "d3ee5958_2",+ "4aaf678f_3",+ "9f80607d",+ "131ff17e",+ "d03995c6",+ "9b3a7f2c_1",+ "answer_a25d4a91_1"+ ],+ "gold_session_ids": [+ "answer_a25d4a91_1",+ "answer_a25d4a91_2"+ ]+ },+ {+ "question_id": "6aeb4375",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3f9693b7_1",+ "answer_3f9693b7_2",+ "sharegpt_a9ZjGW8_9",+ "dc9bf721_1",+ "29621a34",+ "08256f84_2",+ "ultrachat_115445",+ "402e0082_5",+ "8a5e1497_3",+ "ultrachat_123028"+ ],+ "gold_session_ids": [+ "answer_3f9693b7_1",+ "answer_3f9693b7_2"+ ]+ },+ {+ "question_id": "830ce83f",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ad5a7d50_2",+ "7a389bd1_7",+ "answer_0b1a0942_2",+ "a7f88bb3_2",+ "dfbda5e9",+ "639cec58",+ "020149a5_1",+ "answer_0b1a0942_1",+ "8bc305e4_2",+ "ultrachat_346816"+ ],+ "gold_session_ids": [+ "answer_0b1a0942_1",+ "answer_0b1a0942_2"+ ]+ },+ {+ "question_id": "852ce960",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_FYhsZ0Q_0",+ "answer_3a6f1e82_1",+ "answer_3a6f1e82_2",+ "010a28ab_3",+ "d36d11b9_2",+ "e8bfacec_1",+ "a110c4dc_1",+ "a76e7e3c_1",+ "aaf22693",+ "f55fee5b_2"+ ],+ "gold_session_ids": [+ "answer_3a6f1e82_1",+ "answer_3a6f1e82_2"+ ]+ },+ {+ "question_id": "945e3d21",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6a4f8626_1",+ "dc880372_3",+ "ef0341d4_2",+ "answer_6a4f8626_2",+ "9bbd38a6",+ "5fcca8bc_1",+ "6da0bba2_1",+ "b916da64_1",+ "4cbe4e20_4",+ "c7847b66_1"+ ],+ "gold_session_ids": [+ "answer_6a4f8626_1",+ "answer_6a4f8626_2"+ ]+ },+ {+ "question_id": "d7c942c3",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_eecb10d9_2",+ "answer_eecb10d9_1",+ "sharegpt_LOF3smB_25",+ "577da76d_2",+ "f86a2788",+ "8843878f_2",+ "sharegpt_zxm712D_0",+ "d15c9567_2",+ "ddf0f116_3",+ "b2243528_1"+ ],+ "gold_session_ids": [+ "answer_eecb10d9_1",+ "answer_eecb10d9_2"+ ]+ },+ {+ "question_id": "71315a70",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c44b9df4_1",+ "answer_c44b9df4_2",+ "ultrachat_399278",+ "3d82ba51_3",+ "d1d3fd12_1",+ "1a43e6cf",+ "bcd5c30f_1",+ "d8a74286",+ "37d634c0_1",+ "sharegpt_rhjZmCm_0"+ ],+ "gold_session_ids": [+ "answer_c44b9df4_1",+ "answer_c44b9df4_2"+ ]+ },+ {+ "question_id": "89941a93",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e1403127_2",+ "answer_e1403127_1",+ "sharegpt_ZzCRfPu_30",+ "44202c74_1",+ "sharegpt_bn0XI2B_0",+ "169213ae",+ "dde142b2_2",+ "ultrachat_62543",+ "3581c390_3",+ "sharegpt_ni8QFie_0"+ ],+ "gold_session_ids": [+ "answer_e1403127_1",+ "answer_e1403127_2"+ ]+ },+ {+ "question_id": "ce6d2d27",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.23719771276929622,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "ultrachat_553022",+ "48ad2a93",+ "4f5880c6_5",+ "ultrachat_573386",+ "answer_73540165_2",+ "78dc3b29",+ "a46d10a2_1",+ "9f6bec4f",+ "0e250059_1",+ "7a4d00b3_2"+ ],+ "gold_session_ids": [+ "answer_73540165_1",+ "answer_73540165_2"+ ]+ },+ {+ "question_id": "9ea5eabc",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "418c27fa",+ "answer_02e66dec_1",+ "answer_02e66dec_2",+ "6e2cca63_1",+ "fcb08a46_2",+ "sharegpt_KwbJJ66_18",+ "sharegpt_3v69R59_0",+ "390acb55_3",+ "62e6593e_1",+ "8c8b180e"+ ],+ "gold_session_ids": [+ "answer_02e66dec_1",+ "answer_02e66dec_2"+ ]+ },+ {+ "question_id": "07741c44",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7e9ad7b4_2",+ "answer_7e9ad7b4_1",+ "8d969f3e_2",+ "98ae7eef",+ "ultrachat_541598",+ "af4d4ecd_4",+ "sharegpt_mUiNDhv_0",+ "a4b136d1_2",+ "sharegpt_vWO8y7i_35",+ "f6b80ebe"+ ],+ "gold_session_ids": [+ "answer_7e9ad7b4_1",+ "answer_7e9ad7b4_2"+ ]+ },+ {+ "question_id": "a1eacc2a",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0eb23770_1",+ "answer_0eb23770_2",+ "521255f7_2",+ "1f035408",+ "7daa121d_2",+ "1ccb1920",+ "93f23301_1",+ "c910dd2c_3",+ "ultrachat_187728",+ "fde0ca18"+ ],+ "gold_session_ids": [+ "answer_0eb23770_1",+ "answer_0eb23770_2"+ ]+ },+ {+ "question_id": "184da446",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e2f4f947_2",+ "85a49fe0_2",+ "fcddf7d1_2",+ "bf633415_2",+ "ultrachat_195877",+ "answer_e2f4f947_1",+ "82a89303",+ "7ee6a6c0_1",+ "sharegpt_nsHU229_0",+ "ultrachat_116760"+ ],+ "gold_session_ids": [+ "answer_e2f4f947_1",+ "answer_e2f4f947_2"+ ]+ },+ {+ "question_id": "031748ae",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8748f791_1",+ "eff063bb_2",+ "answer_8748f791_2",+ "5ac7f9c5_3",+ "d0488050",+ "e0f71050_2",+ "31702c20_3",+ "f9e54d9c",+ "bc660b5f_1",+ "70764af1_4"+ ],+ "gold_session_ids": [+ "answer_8748f791_1",+ "answer_8748f791_2"+ ]+ },+ {+ "question_id": "4d6b87c8",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8175295903539445,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_766ab8da_2",+ "4dd737a9_1",+ "d87e86f6_1",+ "sharegpt_yPCVYhX_0",+ "5c85023a_1",+ "9af4e346_1",+ "answer_766ab8da_1",+ "cffde796",+ "85da7827",+ "ultrachat_411499"+ ],+ "gold_session_ids": [+ "answer_766ab8da_1",+ "answer_766ab8da_2"+ ]+ },+ {+ "question_id": "0f05491a",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d6d2eba8_2",+ "answer_d6d2eba8_1",+ "9b6db1a9_4",+ "057cbbc4_1",+ "9760f4ca",+ "bb914c5c_1",+ "d81d9846",+ "9f4bec78",+ "ee0921d4",+ "0c260a71_3"+ ],+ "gold_session_ids": [+ "answer_d6d2eba8_1",+ "answer_d6d2eba8_2"+ ]+ },+ {+ "question_id": "08e075c7",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "25b3e36c_1",+ "answer_cdbe2250_2",+ "answer_cdbe2250_1",+ "a2fa5ba3",+ "b1d9d555_1",+ "5ff94163_1",+ "ultrachat_265107",+ "ae77c245",+ "77827bdf",+ "sharegpt_Ah0tRO5_0"+ ],+ "gold_session_ids": [+ "answer_cdbe2250_1",+ "answer_cdbe2250_2"+ ]+ },+ {+ "question_id": "f9e8c073",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b191df5b_2",+ "sharegpt_GDFyxeT_0",+ "answer_b191df5b_1",+ "ultrachat_88473",+ "22db3cc3_1",+ "669318cf_3",+ "0a042aef_2",+ "50b32eed_2",+ "ultrachat_219619",+ "4604bc73_2"+ ],+ "gold_session_ids": [+ "answer_b191df5b_1",+ "answer_b191df5b_2"+ ]+ },+ {+ "question_id": "41698283",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c7ddc051_1",+ "answer_c7ddc051_2",+ "431ae25c",+ "28e948f5",+ "0e8ba03b_1",+ "631e4016",+ "ff3f5781",+ "47a2bd5a_1",+ "f35e3cb7_1",+ "f9d0bc67"+ ],+ "gold_session_ids": [+ "answer_c7ddc051_1",+ "answer_c7ddc051_2"+ ]+ },+ {+ "question_id": "2698e78f",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9282283d_1",+ "answer_9282283d_2",+ "69674f25",+ "sharegpt_DrLV0RL_0",+ "d76030af_1",+ "0fffb38f_2",+ "5ac7f9c5_3",+ "a0fcf806",+ "ebd61d29_2",+ "sharegpt_4qCDHY4_17"+ ],+ "gold_session_ids": [+ "answer_9282283d_1",+ "answer_9282283d_2"+ ]+ },+ {+ "question_id": "b6019101",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.39564672360221187,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "sharegpt_CvVLg2J_18",+ "de43030f_2",+ "sharegpt_v9WEVZG_0",+ "f5f1ff92_3",+ "ultrachat_338597",+ "answer_67074b4b_1",+ "7b0dea50_2",+ "ultrachat_331484",+ "sharegpt_3u9F9yx_0",+ "answer_67074b4b_2"+ ],+ "gold_session_ids": [+ "answer_67074b4b_1",+ "answer_67074b4b_2"+ ]+ },+ {+ "question_id": "45dc21b6",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_07664d43_1",+ "5934b91f",+ "answer_07664d43_2",+ "28ad789a_1",+ "0da2772d",+ "5754046b",+ "sharegpt_mMdyu3t_0",+ "1c0921ee_3",+ "de459c4b",+ "b7f69a7d_2"+ ],+ "gold_session_ids": [+ "answer_07664d43_1",+ "answer_07664d43_2"+ ]+ },+ {+ "question_id": "5a4f22c0",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b0f3dfff_2",+ "answer_b0f3dfff_1",+ "af778ece_1",+ "5a1164c9_1",+ "acb77ba4",+ "007e7d81_2",+ "sharegpt_I0uNTjm_22",+ "b10f3828_1",+ "c3f945f3_1",+ "sharegpt_cXkL3cR_77"+ ],+ "gold_session_ids": [+ "answer_b0f3dfff_1",+ "answer_b0f3dfff_2"+ ]+ },+ {+ "question_id": "6071bd76",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5437713091520254,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "5c18cb32_1",+ "9de53d68_1",+ "answer_4dac77cb_2",+ "sharegpt_3D3oQC0_12",+ "answer_4dac77cb_1",+ "ultrachat_174856",+ "e6b6353d",+ "ultrachat_69243",+ "6f2c01fc_3",+ "710833f3"+ ],+ "gold_session_ids": [+ "answer_4dac77cb_1",+ "answer_4dac77cb_2"+ ]+ },+ {+ "question_id": "e493bb7c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1a374afa_2",+ "answer_1a374afa_1",+ "ultrachat_185447",+ "0aacd15f_1",+ "9161500f_2",+ "sharegpt_9CnlN2a_0",+ "ultrachat_415114",+ "ultrachat_287713",+ "ultrachat_392200",+ "ultrachat_513900"+ ],+ "gold_session_ids": [+ "answer_1a374afa_1",+ "answer_1a374afa_2"+ ]+ },+ {+ "question_id": "618f13b2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_caf5b52e_1",+ "answer_caf5b52e_2",+ "sharegpt_PvScEsS_0",+ "b3619c2c",+ "01a120b5",+ "d2366c37",+ "dda70510_2",+ "6172cc72_3",+ "c51583cd_3",+ "4632c2c6_1"+ ],+ "gold_session_ids": [+ "answer_caf5b52e_1",+ "answer_caf5b52e_2"+ ]+ },+ {+ "question_id": "72e3ee87",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d7de9a6a_2",+ "answer_d7de9a6a_1",+ "ultrachat_195333",+ "1c1f5ccc_4",+ "73a2f346_3",+ "sharegpt_KdbXhTW_9",+ "ultrachat_523550",+ "be8c3970_2",+ "080a4000_2",+ "ultrachat_421360"+ ],+ "gold_session_ids": [+ "answer_d7de9a6a_1",+ "answer_d7de9a6a_2"+ ]+ },+ {+ "question_id": "c4ea545c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d3bf812b_2",+ "5be74579",+ "a2a8e2c9",+ "sharegpt_71QV6Zs_7",+ "sharegpt_I0uNTjm_16",+ "b2d284b1",+ "sharegpt_JJu53iW_0",+ "9f9b402a_5",+ "answer_d3bf812b_1",+ "85a49fe0_1"+ ],+ "gold_session_ids": [+ "answer_d3bf812b_1",+ "answer_d3bf812b_2"+ ]+ },+ {+ "question_id": "01493427",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a7b44747_2",+ "answer_a7b44747_1",+ "88e73f02",+ "077f7bcb_3",+ "291d8c82",+ "5a840928_1",+ "55e6ff53",+ "3e8f07c9_1",+ "087ac53a_2",+ "a4d6c239_2"+ ],+ "gold_session_ids": [+ "answer_a7b44747_1",+ "answer_a7b44747_2"+ ]+ },+ {+ "question_id": "6a27ffc2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_77f32504_2",+ "4604bc73_3",+ "answer_77f32504_1",+ "6ae451d1_2",+ "ultrachat_161210",+ "sharegpt_KSCqghj_0",+ "9ded9e01_2",+ "sharegpt_kkqlRlo_13",+ "eb87aced_1",+ "ultrachat_505209"+ ],+ "gold_session_ids": [+ "answer_77f32504_1",+ "answer_77f32504_2"+ ]+ },+ {+ "question_id": "2133c1b5",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2640681225725909,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "a63ad8e3_3",+ "sharegpt_bSlIUYf_32",+ "sharegpt_Olp439q_19",+ "answer_52382508_2",+ "3b655772_1",+ "9a4c74d3_1",+ "sharegpt_MvUpG5V_5",+ "add9b012_1",+ "ultrachat_460074",+ "ultrachat_502724"+ ],+ "gold_session_ids": [+ "answer_52382508_1",+ "answer_52382508_2"+ ]+ },+ {+ "question_id": "18bc8abd",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fff743f5_2",+ "answer_fff743f5_1",+ "a35679d5_2",+ "2714885f_3",+ "951df45e",+ "ultrachat_529103",+ "sharegpt_afCogMg_34",+ "4ad82121_2",+ "c6d55c42_2",+ "ba6b67af_3"+ ],+ "gold_session_ids": [+ "answer_fff743f5_1",+ "answer_fff743f5_2"+ ]+ },+ {+ "question_id": "db467c8c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5714285040141098,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_ocf8Lud_39",+ "answer_611b6e83_1",+ "77cf4551_1",+ "76967d22",+ "sharegpt_7nYPy2P_29",+ "sharegpt_HBmf6q0_0",+ "ultrachat_553961",+ "daac1a76",+ "answer_611b6e83_2",+ "sharegpt_TAL32at_0"+ ],+ "gold_session_ids": [+ "answer_611b6e83_1",+ "answer_611b6e83_2"+ ]+ },+ {+ "question_id": "7a87bd0c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d08a934d_1",+ "answer_d08a934d_2",+ "e250791e_3",+ "3d98b7e6_2",+ "15519944_1",+ "9aaeecbf",+ "93e9e82c_2",+ "62369c56_1",+ "81f318a2_1",+ "sharegpt_AOimCZD_0"+ ],+ "gold_session_ids": [+ "answer_d08a934d_1",+ "answer_d08a934d_2"+ ]+ },+ {+ "question_id": "e61a7584",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f25c32f5_1",+ "answer_f25c32f5_2",+ "ultrachat_367590",+ "82159da4_1",+ "285525b2_1",+ "574df152_2",+ "afa96a40",+ "18a06652_4",+ "6a4b4b7e_2",+ "e1416816_2"+ ],+ "gold_session_ids": [+ "answer_f25c32f5_1",+ "answer_f25c32f5_2"+ ]+ },+ {+ "question_id": "1cea1afa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_79c395a9_1",+ "answer_79c395a9_2",+ "e9ad5914_5",+ "9c479e89_1",+ "7063fad2",+ "sharegpt_inuIr9J_119",+ "e131bcd4",+ "7af38385_1",+ "ultrachat_51727",+ "ultrachat_455127"+ ],+ "gold_session_ids": [+ "answer_79c395a9_1",+ "answer_79c395a9_2"+ ]+ },+ {+ "question_id": "ed4ddc30",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_babbaccb_2",+ "4ffa04a2_4",+ "01a120b5",+ "c279c578",+ "answer_babbaccb_1",+ "57c91b14_2",+ "e4969569_2",+ "4ed9cd1b_2",+ "8b726b64_1",+ "aa31c397_2"+ ],+ "gold_session_ids": [+ "answer_babbaccb_1",+ "answer_babbaccb_2"+ ]+ },+ {+ "question_id": "8fb83627",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_966cecbb_1",+ "answer_966cecbb_2",+ "4019b1a8_5",+ "9315f268_1",+ "ultrachat_172298",+ "be050ab3_2",+ "02b81ad9",+ "ultrachat_44939",+ "b184aab6",+ "88e01130_1"+ ],+ "gold_session_ids": [+ "answer_966cecbb_1",+ "answer_966cecbb_2"+ ]+ },+ {+ "question_id": "b01defab",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8c0712af_2",+ "answer_8c0712af_1",+ "sharegpt_VbIxRWJ_33",+ "sharegpt_zMUpvkc_57",+ "59dc866c_1",+ "ultrachat_76442",+ "639cec58",+ "fc69fd44_7",+ "9aa07e25",+ "6408b911"+ ],+ "gold_session_ids": [+ "answer_8c0712af_1",+ "answer_8c0712af_2"+ ]+ },+ {+ "question_id": "22d2cb42",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bcce0b73_1",+ "answer_bcce0b73_2",+ "ultrachat_227551",+ "74932466_1",+ "060aeced",+ "57c91b14_2",+ "ff898925",+ "sharegpt_ZXM9kAl_0",+ "eec72a82_4",+ "ultrachat_18212"+ ],+ "gold_session_ids": [+ "answer_bcce0b73_1",+ "answer_bcce0b73_2"+ ]+ },+ {+ "question_id": "0e4e4c46",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f2f998c7_1",+ "b42cbaf0_1",+ "71d1e644",+ "sharegpt_XOT5vdc_6",+ "17822471",+ "answer_f2f998c7_2",+ "cbd08e3c_abs_2",+ "0773943b",+ "55e0c6db_1",+ "e82ce7cd_3"+ ],+ "gold_session_ids": [+ "answer_f2f998c7_1",+ "answer_f2f998c7_2"+ ]+ },+ {+ "question_id": "4b24c848",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.36181498356361597,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "ultrachat_244540",+ "16e094b3_1",+ "b0855671_2",+ "sharegpt_Lj1CaQL_0",+ "306cb175",+ "f5cf8815_1",+ "ultrachat_421733",+ "1815d1b7",+ "answer_2cec623b_2",+ "answer_2cec623b_1"+ ],+ "gold_session_ids": [+ "answer_2cec623b_1",+ "answer_2cec623b_2"+ ]+ },+ {+ "question_id": "7e974930",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c9f5693c_1",+ "answer_c9f5693c_2",+ "705b5399_3",+ "ultrachat_251049",+ "898ce7a5_1",+ "ultrachat_113430",+ "sharegpt_EuvEEnD_0",+ "a1e169b1",+ "sharegpt_cXkL3cR_77",+ "ultrachat_380252"+ ],+ "gold_session_ids": [+ "answer_c9f5693c_1",+ "answer_c9f5693c_2"+ ]+ },+ {+ "question_id": "603deb26",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "c4661b3c_1",+ "88611fff",+ "a55a5afc_2",+ "93191d30_2",+ "81f6b12f_2",+ "answer_8afdebac_2",+ "ede65c84_2",+ "f0e71553_4",+ "5e867c2f_2",+ "37f66729"+ ],+ "gold_session_ids": [+ "answer_8afdebac_1",+ "answer_8afdebac_2"+ ]+ },+ {+ "question_id": "59524333",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b28f2c7a_1",+ "ultrachat_150151",+ "ultrachat_458180",+ "638e7094_1",+ "sharegpt_Clr6oyu_0",+ "answer_b28f2c7a_2",+ "sharegpt_NYFuCmY_0",+ "c9e7e793",+ "sharegpt_KhxXaow_0",+ "sharegpt_S6bD7yp_0"+ ],+ "gold_session_ids": [+ "answer_b28f2c7a_1",+ "answer_b28f2c7a_2"+ ]+ },+ {+ "question_id": "5831f84d",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6052602440527057,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "7a88956e",+ "answer_8d63a897_1",+ "ef84b994_abs_2",+ "41c90f4c_2",+ "0a1e90d1_1",+ "answer_8d63a897_2",+ "sharegpt_6I57AvU_67",+ "6a471944_1",+ "b70358bc_6",+ "0d66562c_4"+ ],+ "gold_session_ids": [+ "answer_8d63a897_1",+ "answer_8d63a897_2"+ ]+ },+ {+ "question_id": "eace081b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8a791264_1",+ "answer_8a791264_2",+ "sharegpt_HOO59Ue_137",+ "9316aae3_1",+ "4d04b866",+ "4af27eab_2",+ "413b57cb_3",+ "1dcfb9a0_2",+ "a9ab2024",+ "f340a3db"+ ],+ "gold_session_ids": [+ "answer_8a791264_1",+ "answer_8a791264_2"+ ]+ },+ {+ "question_id": "affe2881",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "fb5dd87f_7",+ "answer_90de9b4d_1",+ "4604bc73_3",+ "answer_90de9b4d_2",+ "377f8fd9_2",+ "0018b628",+ "33994682_3",+ "sharegpt_P0Qe2Gx_0",+ "7badf4f3_1",+ "3d895af0_3"+ ],+ "gold_session_ids": [+ "answer_90de9b4d_1",+ "answer_90de9b4d_2"+ ]+ },+ {+ "question_id": "50635ada",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dcd74827_1",+ "answer_dcd74827_2",+ "b6f35f3a_1",+ "42bcee92_2",+ "03f76b6b",+ "add7cc1e",+ "sharegpt_eSTYwMj_10",+ "2e53911b_1",+ "419099c7_1",+ "sharegpt_BVhcEnF_0"+ ],+ "gold_session_ids": [+ "answer_dcd74827_1",+ "answer_dcd74827_2"+ ]+ },+ {+ "question_id": "e66b632c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ac0140ce_1",+ "b876911c",+ "84098c1b",+ "answer_ac0140ce_2",+ "259d58da_2",+ "c2808c05_2",+ "ultrachat_306973",+ "47d18b61_1",+ "ultrachat_223509",+ "ultrachat_533755"+ ],+ "gold_session_ids": [+ "answer_ac0140ce_1",+ "answer_ac0140ce_2"+ ]+ },+ {+ "question_id": "0ddfec37",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a22b654d_2",+ "e184e4c3_2",+ "0a699029_2",+ "3422786d_2",+ "answer_a22b654d_1",+ "147ab7e9_7",+ "f2199726_2",+ "ultrachat_180221",+ "e4881cbd_1",+ "sharegpt_ckrlzN2_35"+ ],+ "gold_session_ids": [+ "answer_a22b654d_1",+ "answer_a22b654d_2"+ ]+ },+ {+ "question_id": "f685340e",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_25df025b_1",+ "a4815e8e_2",+ "142e3203_1",+ "answer_25df025b_2",+ "e7dcd5df",+ "e862f726_1",+ "992eb1f3_2",+ "eef7c3fa_1",+ "a8ef396f_1",+ "c07ae000_2"+ ],+ "gold_session_ids": [+ "answer_25df025b_1",+ "answer_25df025b_2"+ ]+ },+ {+ "question_id": "cc5ded98",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a5b68517_1",+ "answer_a5b68517_2",+ "c63b2c8f",+ "ef785bb4_2",+ "b13d5e4e_1",+ "d973fa59_1",+ "0a140d49_1",+ "c937aab6",+ "sharegpt_bTDMt3m_80",+ "f14a4532_1"+ ],+ "gold_session_ids": [+ "answer_a5b68517_1",+ "answer_a5b68517_2"+ ]+ },+ {+ "question_id": "dfde3500",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35d6c0be_1",+ "ad1c7ac6",+ "59c704ad_1",+ "answer_35d6c0be_2",+ "86e218e3_2",+ "5c18cb32_2",+ "779c0465",+ "5fcca8bc_2",+ "134127c2_2",+ "5d5e80c5_1"+ ],+ "gold_session_ids": [+ "answer_35d6c0be_1",+ "answer_35d6c0be_2"+ ]+ },+ {+ "question_id": "69fee5aa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d6028d6e_1",+ "answer_d6028d6e_2",+ "a0b8b0ad_7",+ "5d284d7d_2",+ "991d55e5_1",+ "42adb80d_2",+ "ultrachat_327452",+ "sharegpt_QxC97cL_0",+ "sharegpt_hpWyNVd_0",+ "6ff77d45_2"+ ],+ "gold_session_ids": [+ "answer_d6028d6e_1",+ "answer_d6028d6e_2"+ ]+ },+ {+ "question_id": "7401057b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_94650bfa_1",+ "answer_94650bfa_2",+ "c3757c5b",+ "ultrachat_389374",+ "sharegpt_Ah0tRO5_0",+ "42e69993_1",+ "7be23135",+ "1450ffad",+ "30f20967_2",+ "8de5b7cf_2"+ ],+ "gold_session_ids": [+ "answer_94650bfa_1",+ "answer_94650bfa_2"+ ]+ },+ {+ "question_id": "cf22b7bf",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ae3a122b_1",+ "answer_ae3a122b_2",+ "c8fcdf5a_1",+ "e201572a",+ "ultrachat_346016",+ "36d5bbde_1",+ "22fe1c93",+ "457e26bf",+ "c3c334ff_1",+ "8ab0292c_1"+ ],+ "gold_session_ids": [+ "answer_ae3a122b_1",+ "answer_ae3a122b_2"+ ]+ },+ {+ "question_id": "a2f3aa27",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5126c02d_1",+ "c98b3349_3",+ "b65f51fb_5",+ "c11b64f3_1",+ "f188b48d_2",+ "4cf2ea34",+ "b4f28f96_1",+ "01a467ee_2",+ "answer_5126c02d_2",+ "57bf4122_1"+ ],+ "gold_session_ids": [+ "answer_5126c02d_1",+ "answer_5126c02d_2"+ ]+ },+ {+ "question_id": "c7dc5443",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0cdbca92_2",+ "c2fb8e86_2",+ "answer_0cdbca92_1",+ "debc34d7_2",+ "e200d96c_3",+ "42d9467c_2",+ "ultrachat_494538",+ "sharegpt_hXYFkwv_35",+ "9f0f59d1_3",+ "728fc45a"+ ],+ "gold_session_ids": [+ "answer_0cdbca92_1",+ "answer_0cdbca92_2"+ ]+ },+ {+ "question_id": "06db6396",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da72b1b4_1",+ "answer_da72b1b4_2",+ "ultrachat_333096",+ "5bd9f1e6_1",+ "sharegpt_2pwdGxJ_16",+ "84d7650f_2",+ "6ebdc1fe_2",+ "da94a327_2",+ "35038174_4",+ "8235b87d"+ ],+ "gold_session_ids": [+ "answer_da72b1b4_1",+ "answer_da72b1b4_2"+ ]+ },+ {+ "question_id": "3ba21379",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "49576fd6",+ "answer_cd345582_2",+ "93aff0d7_2",+ "sharegpt_QQQAWUd_19",+ "answer_cd345582_1",+ "f64c4793_1",+ "ultrachat_224436",+ "sharegpt_DPJevuk_0",+ "5e50c81b",+ "0f78155a_1"+ ],+ "gold_session_ids": [+ "answer_cd345582_1",+ "answer_cd345582_2"+ ]+ },+ {+ "question_id": "9bbe84a2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c6a0c6c2_1",+ "answer_c6a0c6c2_2",+ "ultrachat_148297",+ "sharegpt_hcaZN94_546",+ "e6e4fd83_2",+ "cd81a69b",+ "6b7605d1_2",+ "1fc31387",+ "ff0bb7fd_2",+ "114c1fb7_6"+ ],+ "gold_session_ids": [+ "answer_c6a0c6c2_1",+ "answer_c6a0c6c2_2"+ ]+ },+ {+ "question_id": "10e09553",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_67be2c38_2",+ "answer_67be2c38_1",+ "7d1f5395",+ "ff67236f_2",+ "e5f785de",+ "ultrachat_196381",+ "3021992b_2",+ "959e6cb0_7",+ "58ab5fc5",+ "918c4788_1"+ ],+ "gold_session_ids": [+ "answer_67be2c38_1",+ "answer_67be2c38_2"+ ]+ },+ {+ "question_id": "dad224aa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4a97ae40_1",+ "6de8645d_1",+ "answer_4a97ae40_2",+ "756a4a02_2",+ "56b5b760",+ "b971ed75_3",+ "91e581ad_1",+ "5c690047_2",+ "sharegpt_2Qvu6BM_34",+ "d0bef272_1"+ ],+ "gold_session_ids": [+ "answer_4a97ae40_1",+ "answer_4a97ae40_2"+ ]+ },+ {+ "question_id": "ba61f0b9",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f377cda7_1",+ "answer_f377cda7_2",+ "sharegpt_s8Opwwu_0",+ "495cf518_1",+ "c9213a03_1",+ "221f2754_1",+ "daca659d",+ "ultrachat_421876",+ "9316aae3_2",+ "8c1da8f9_1"+ ],+ "gold_session_ids": [+ "answer_f377cda7_1",+ "answer_f377cda7_2"+ ]+ },+ {+ "question_id": "42ec0761",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e3892371_2",+ "answer_e3892371_1",+ "53acd215",+ "493bd421_2",+ "812ccbd8_4",+ "30aa70c6",+ "sharegpt_JHGwlee_33",+ "5dc74ba7",+ "27d0d3e7",+ "sharegpt_OLc5cYf_653"+ ],+ "gold_session_ids": [+ "answer_e3892371_1",+ "answer_e3892371_2"+ ]+ },+ {+ "question_id": "5c40ec5b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1cb52d0a_2",+ "answer_1cb52d0a_1",+ "be050ab3_2",+ "d9868305_1",+ "9eb7bb35",+ "sharegpt_Ah0tRO5_0",+ "0e8c2a62",+ "5cc9e1ed_3",+ "692075ef",+ "25a2002c"+ ],+ "gold_session_ids": [+ "answer_1cb52d0a_1",+ "answer_1cb52d0a_2"+ ]+ },+ {+ "question_id": "c6853660",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5012658353418871,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "ultrachat_435535",+ "a4863bcb_2",+ "33994682_3",+ "answer_626e93c4_2",+ "answer_626e93c4_1",+ "sharegpt_PQ71V9I_7",+ "0e931802_2",+ "6ed1c85e",+ "6474cfcc",+ "71dc2037_2"+ ],+ "gold_session_ids": [+ "answer_626e93c4_1",+ "answer_626e93c4_2"+ ]+ },+ {+ "question_id": "26bdc477",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f762ad8d_1",+ "answer_f762ad8d_2",+ "27ce9f13_2",+ "37b3c75d",+ "a223e307",+ "8aa17385",+ "2d79c1c3_2",+ "sharegpt_bq9G6bT_2",+ "551eae69_1",+ "f684ac4c_1"+ ],+ "gold_session_ids": [+ "answer_f762ad8d_1",+ "answer_f762ad8d_2"+ ]+ },+ {+ "question_id": "0977f2af",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "e255d6fc_2",+ "bdf45964_3",+ "b24eddd9_1",+ "ultrachat_554962",+ "104be29f_1",+ "answer_3bf5b73b_2",+ "ultrachat_143766",+ "sharegpt_l6aWsPc_0",+ "9b083158_1",+ "sharegpt_qDHHlQO_27"+ ],+ "gold_session_ids": [+ "answer_3bf5b73b_1",+ "answer_3bf5b73b_2"+ ]+ },+ {+ "question_id": "6aeb4375_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "f21e11f1_2",+ "answer_3f9693b7_abs_1",+ "1cf6c966_2",+ "ultrachat_209842",+ "answer_3f9693b7_abs_2",+ "710f4e21_1",+ "fcff2f22",+ "ea83c1fe_1",+ "53c4fdd7_2",+ "a5286353_1"+ ],+ "gold_session_ids": [+ "answer_3f9693b7_abs_1",+ "answer_3f9693b7_abs_2"+ ]+ },+ {+ "question_id": "031748ae_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8748f791_abs_1",+ "1521a6c3_1",+ "ultrachat_341706",+ "sharegpt_EX0p0tj_19",+ "answer_8748f791_abs_2",+ "169cd54f",+ "4d2c2c88_1",+ "feba9363_1",+ "210d7fd8_2",+ "ultrachat_473482"+ ],+ "gold_session_ids": [+ "answer_8748f791_abs_1",+ "answer_8748f791_abs_2"+ ]+ },+ {+ "question_id": "2698e78f_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6240505200038379,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "7ebf92f6_2",+ "answer_9282283d_abs_1",+ "3d94eae8",+ "9bd6beaa_1",+ "answer_9282283d_abs_2",+ "9316aae3_1",+ "f607e932_1",+ "9e21d6ab_1",+ "2e18c90b_1",+ "ce291e98"+ ],+ "gold_session_ids": [+ "answer_9282283d_abs_1",+ "answer_9282283d_abs_2"+ ]+ },+ {+ "question_id": "2133c1b5_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "6c7079a1",+ "sharegpt_g3YFWLT_0",+ "ef0341d4_1",+ "fe6d8377_1",+ "ef84b994_1",+ "d0cfe938_1",+ "18c1f00c_1",+ "sharegpt_LmxRyal_9",+ "sharegpt_BgSFVRW_0",+ "ultrachat_543717"+ ],+ "gold_session_ids": [+ "answer_52382508_abs_1",+ "answer_52382508_abs_2"+ ]+ },+ {+ "question_id": "0ddfec37_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3706656904013186,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "6de70f46_1",+ "3a11533e",+ "4bd8d215_1",+ "36a06ee8",+ "25572732",+ "a6b6067b",+ "e3e061c4",+ "answer_a22b654d_abs_1",+ "a35959ba_2",+ "answer_a22b654d_abs_2"+ ],+ "gold_session_ids": [+ "answer_a22b654d_abs_1",+ "answer_a22b654d_abs_2"+ ]+ },+ {+ "question_id": "f685340e_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_25df025b_abs_1",+ "6b895848_1",+ "answer_25df025b_abs_2",+ "bc6c5757_2",+ "11fa2b8f",+ "be4dc764_1",+ "79bb1298_5",+ "1970789e",+ "sharegpt_oF0gxsp_0",+ "7b49beea"+ ],+ "gold_session_ids": [+ "answer_25df025b_abs_1",+ "answer_25df025b_abs_2"+ ]+ },+ {+ "question_id": "89941a94",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5437713091520254,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "f7a61595_5",+ "85b9848f",+ "answer_e1403127_2",+ "sharegpt_bvzjXyv_0",+ "answer_e1403127_1",+ "fe1e4351_2",+ "0d758215_2",+ "1f035408",+ "de275215_3",+ "ultrachat_556968"+ ],+ "gold_session_ids": [+ "answer_e1403127_1",+ "answer_e1403127_2"+ ]+ },+ {+ "question_id": "07741c45",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7e9ad7b4_2",+ "5300bfba",+ "604de0a3_2",+ "answer_7e9ad7b4_1",+ "af4a2fd1",+ "df846730",+ "4c2bd0fe_1",+ "37d65d40_3",+ "sharegpt_hHGVWPU_4",+ "465bc6a2"+ ],+ "gold_session_ids": [+ "answer_7e9ad7b4_1",+ "answer_7e9ad7b4_2"+ ]+ },+ {+ "question_id": "7161e7e2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_5Lzox6N_0",+ "3c3fee41",+ "5aed819f",+ "b4f63a70_3",+ "5850de18_2",+ "de93e859",+ "e89b6be2_2",+ "ultrachat_541021",+ "6b17c99b_2",+ "3070419a_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_5Lzox6N_0"+ ]+ },+ {+ "question_id": "c4f10528",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_234453",+ "51c89181_1",+ "8b1b7007",+ "11248f50",+ "c5c309b9_1",+ "dc1f9713_2",+ "e12cfbcd",+ "sharegpt_yMwEvl7_557",+ "7157bff1_2",+ "5ecb3466_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_234453"+ ]+ },+ {+ "question_id": "89527b6b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.0625,+ "retrieved_session_ids": [+ "9fa1d135_1",+ "ultrachat_117600",+ "1c6b85ea_1",+ "8c1da8f9_1",+ "4e147c90",+ "48d385f0_1",+ "689b46e2_4",+ "51c262b6_2",+ "9c722816",+ "07f3f0e0_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_YkWn1Ne_0"+ ]+ },+ {+ "question_id": "e9327a54",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_480665",+ "c58fdf1e_2",+ "c38fb85b",+ "c0d099e6_2",+ "2d79c1c3_1",+ "8221810f_3",+ "c81897cd_2",+ "b65f51fb_5",+ "ba944cdc",+ "28e0ea23_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_480665"+ ]+ },+ {+ "question_id": "4c36ccef",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "f18b6c03_1",+ "sharegpt_1PRjwYK_11",+ "answer_ultrachat_448704",+ "89749c78_1",+ "940629b2_1",+ "988baea9",+ "5412d059",+ "7b1d1f43_2",+ "89d49715_1",+ "ultrachat_149723"+ ],+ "gold_session_ids": [+ "answer_ultrachat_448704"+ ]+ },+ {+ "question_id": "6ae235be",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_IUWQYGQ_0",+ "0a1e90d1_2",+ "ec8fa65d_1",+ "99188307",+ "41b6e78c",+ "ca3a4e4f_3",+ "cc5a6057_1",+ "809cabca_1",+ "43811b62",+ "c25b95c5_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_IUWQYGQ_0"+ ]+ },+ {+ "question_id": "7e00a6cb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "4dd1ba8d_2",+ "24f78a31_2",+ "answer_ultrachat_370515",+ "9998814e_2",+ "b7dca8f1_1",+ "5534efcb_2",+ "a9981dc6_2",+ "39358a85_3",+ "ultrachat_564942",+ "e2cd250e_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_370515"+ ]+ },+ {+ "question_id": "1903aded",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "9f8c432e_2",+ "sharegpt_64onasJ_0",+ "ultrachat_138198",+ "9b3611d7_1",+ "8adb232f_1",+ "f8246a66_1",+ "184efb91_3",+ "ultrachat_134914",+ "b163d176",+ "sharegpt_MCI1EVK_39"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hA7AkP3_0"+ ]+ },+ {+ "question_id": "ceb54acb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3010299956639812,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "de43030f_1",+ "dddce60a_1",+ "0840765f_1",+ "c60d5c8a_1",+ "2f23dd1a",+ "7aa2a9af_2",+ "0bbd7094_1",+ "sharegpt_hUhgRkO_0",+ "answer_sharegpt_cGdjmYo_0",+ "197d99cc"+ ],+ "gold_session_ids": [+ "answer_sharegpt_cGdjmYo_0"+ ]+ },+ {+ "question_id": "f523d9fe",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_m2xJfjo_0",+ "ultrachat_391251",+ "fbfda981",+ "636e0e32_1",+ "193c23bd_1",+ "44165708",+ "bed5fbab",+ "73f75ab4",+ "sharegpt_TF6GLMw_26",+ "fd18e0dd_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_m2xJfjo_0"+ ]+ },+ {+ "question_id": "0e5e2d1a",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_113156",+ "f84b4266",+ "sharegpt_btv4SkQ_0",+ "c756c34e",+ "c4e5d969_2",+ "5d79fd65_2",+ "147ab7e9_7",+ "22f9f163_1",+ "ultrachat_222974",+ "a803f50b_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_113156"+ ]+ },+ {+ "question_id": "fea54f57",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_187684",+ "5ac7f9c5_1",+ "a98f8672_1",+ "a1e90def_1",+ "34590cab_1",+ "c8854b28",+ "d5f14d5f_1",+ "0aa99e2e_1",+ "3210adc8",+ "d81c92cb"+ ],+ "gold_session_ids": [+ "answer_ultrachat_187684"+ ]+ },+ {+ "question_id": "cc539528",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "15a8ffb7_2",+ "answer_ultrachat_374124",+ "9e2e32c1_4",+ "faa194ea",+ "sharegpt_WBeb7tv_29",+ "fb760626",+ "18807892_4",+ "ultrachat_490706",+ "38e81260_5",+ "3a1ea336_3"+ ],+ "gold_session_ids": [+ "answer_ultrachat_374124"+ ]+ },+ {+ "question_id": "dc439ea3",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_459954",+ "49d8a382",+ "1bfd5a8b_2",+ "5a81277c",+ "ultrachat_186512",+ "76ecc9b9_1",+ "6ef99651_1",+ "sharegpt_q2uAW7H_0",+ "cb5637b3",+ "ultrachat_4166"+ ],+ "gold_session_ids": [+ "answer_ultrachat_459954"+ ]+ },+ {+ "question_id": "18dcd5a5",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.06666666666666667,+ "retrieved_session_ids": [+ "sharegpt_d1tE544_5",+ "33d16079_2",+ "6ea1541e_1",+ "47727b14",+ "9844f740",+ "sharegpt_MW0whoh_79",+ "5e906cb0_2",+ "aceffaab",+ "sharegpt_2yaKUmC_0",+ "959e6cb0_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hn3IS1q_0"+ ]+ },+ {+ "question_id": "488d3006",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_275993",+ "a68090b0_2",+ "ce19df54_3",+ "9a57d65b",+ "82b82901",+ "62c00fbc",+ "1af2c0fb_1",+ "b0f8fc7b_2",+ "a394f6b5_1",+ "3eb40515"+ ],+ "gold_session_ids": [+ "answer_ultrachat_275993"+ ]+ },+ {+ "question_id": "58470ed2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9f8973ac",+ "answer_sharegpt_U4oCSfU_7",+ "ultrachat_478785",+ "4050ebff_6",+ "4a324955",+ "8ca0085a_3",+ "5c0574b1",+ "66bfa1db",+ "69bd3d4a_1",+ "ultrachat_173621"+ ],+ "gold_session_ids": [+ "answer_sharegpt_U4oCSfU_7"+ ]+ },+ {+ "question_id": "8cf51dda",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_HFMn2ZX_0",+ "daa09295",+ "9beab022_1",+ "e862f726_3",+ "sharegpt_jpiaPoJ_738",+ "7073a9cd_2",+ "b500d9be",+ "5df2f487",+ "3358b5ca_2",+ "ed383233_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_HFMn2ZX_0"+ ]+ },+ {+ "question_id": "1d4da289",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "517ad0f0",+ "sharegpt_AfruXDt_0",+ "answer_ultrachat_348449",+ "bf85791d",+ "a05c7d7d",+ "f01df2d2_1",+ "59c5f97e",+ "ultrachat_213759",+ "252715f8_4",+ "76e56e2c_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_348449"+ ]+ },+ {+ "question_id": "8464fc84",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_467053",+ "8c65c687",+ "1d5998ca_1",+ "69a105f9_1",+ "21d86ebb_2",+ "98d39a6d",+ "9ac77ac9_2",+ "2d003fe4_2",+ "602b4074_3",+ "6ac43c5b_4"+ ],+ "gold_session_ids": [+ "answer_ultrachat_467053"+ ]+ },+ {+ "question_id": "8aef76bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_563222",+ "146dfabe",+ "44b1595a",+ "27d378b2_2",+ "08e084f1_1",+ "c7f6643e_2",+ "05e258f4_2",+ "014a5d36",+ "5209e813_2",+ "c80b8b74_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_563222"+ ]+ },+ {+ "question_id": "71a3fd6b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "b7146673_1",+ "d8f4b0f4",+ "7f7e26d2_2",+ "a29e9311",+ "8419b385_3",+ "f21fc078_1",+ "answer_ultrachat_417348",+ "eee3f7d3",+ "36743359_3",+ "d7a98679_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_417348"+ ]+ },+ {+ "question_id": "2bf43736",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "8897a2e5",+ "answer_sharegpt_2kpncbX_13",+ "16c3be68_1",+ "1a5fb2cd",+ "2e89bd03_1",+ "d79173aa_3",+ "5ca6cd28",+ "fa858208_2",+ "a93750ef_2",+ "ultrachat_49111"+ ],+ "gold_session_ids": [+ "answer_sharegpt_2kpncbX_13"+ ]+ },+ {+ "question_id": "70b3e69b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_334948",+ "40afb91c_1",+ "ultrachat_47234",+ "sharegpt_O8qJclh_0",+ "09e1a929_2",+ "28cd2f40",+ "1388b204_2",+ "dd92723f_1",+ "ultrachat_146029",+ "35c5419d_3"+ ],+ "gold_session_ids": [+ "answer_ultrachat_334948"+ ]+ },+ {+ "question_id": "8752c811",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_6pWK9yx_0",+ "9db30f1c_3",+ "93f23301_2",+ "56745369_2",+ "9272a6a2_2",+ "d0a41222_1",+ "02b63d04_1",+ "31f7c649_2",+ "f0e71553_5",+ "87afa197_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_6pWK9yx_0"+ ]+ },+ {+ "question_id": "3249768e",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2890648263178879,+ "mrr": 0.1,+ "retrieved_session_ids": [+ "689b46e2_2",+ "268e8532_2",+ "71f2b8ba",+ "0868982f_1",+ "ec00a866_2",+ "172b47cb_2",+ "c6c5bb8b_2",+ "11e9d591_2",+ "779c0465",+ "answer_sharegpt_CaxTGYP_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_CaxTGYP_0"+ ]+ },+ {+ "question_id": "1b9b7252",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "0e726047",+ "answer_ultrachat_115151",+ "4f6e1b77_1",+ "af4d4ecd_6",+ "sharegpt_1AhFMpw_5",+ "66e8a6dd",+ "330c05e7_1",+ "e8212ed9",+ "064f42ee_1",+ "3abaa836"+ ],+ "gold_session_ids": [+ "answer_ultrachat_115151"+ ]+ },+ {+ "question_id": "1568498a",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "sharegpt_h87NLyS_49",+ "f964cea3_1",+ "70fed904",+ "answer_sharegpt_d6JJiqH_76",+ "28eb86ab",+ "1fef8ffe_1",+ "6ad46850_2",+ "936b0969_1",+ "a05cd79f_1",+ "71e0db57"+ ],+ "gold_session_ids": [+ "answer_sharegpt_d6JJiqH_76"+ ]+ },+ {+ "question_id": "6222b6eb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_H9PiM5G_0",+ "25ffc571",+ "caf7480d_1",+ "5209e813_2",+ "7b2523d3",+ "ab603dd5_abs_2",+ "723bf11f",+ "b35fadaa",+ "40a90d51",+ "sharegpt_cyc2xbb_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_H9PiM5G_0"+ ]+ },+ {+ "question_id": "e8a79c70",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.058823529411764705,+ "retrieved_session_ids": [+ "c0d099e6_2",+ "539778d4_2",+ "c068abb2",+ "486705ce",+ "sharegpt_ha8kUEr_0",+ "fc706dc5_2",+ "c4c504d2_6",+ "f731656b_2",+ "45c29267",+ "89aed484_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_13075"+ ]+ },+ {+ "question_id": "d596882b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "71888aff_2",+ "0dc2efcc_2",+ "answer_ultrachat_252214",+ "5558a42e_3",+ "456807b7_1",+ "dc9bf721_1",+ "53582e7e_1",+ "bef3abcd",+ "a864e7aa_4",+ "692075ef"+ ],+ "gold_session_ids": [+ "answer_ultrachat_252214"+ ]+ },+ {+ "question_id": "e3fc4d6e",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_5m7gg5F_0",+ "d31c3ec8_1",+ "ultrachat_442052",+ "42adb80d_3",+ "4d261642",+ "9b72b151",+ "ultrachat_161238",+ "bec86aec",+ "05be3ff0_4",+ "sharegpt_ltqfPQc_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_5m7gg5F_0"+ ]+ },+ {+ "question_id": "51b23612",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_427265",+ "409015ef_1",+ "fd78c83d_2",+ "f35cb519_2",+ "8672f398_2",+ "255c753b_1",+ "cbd1fe79_2",+ "74b0227e_2",+ "17f444f3",+ "ultrachat_343"+ ],+ "gold_session_ids": [+ "answer_ultrachat_427265"+ ]+ },+ {+ "question_id": "3e321797",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_94624",+ "07805a44",+ "57144028_2",+ "e77fc218_2",+ "5558a42e_3",+ "193c23bd_1",+ "9252145e_2",+ "04b7b396_1",+ "11302335_4",+ "sharegpt_EZmhz8p_0"+ ],+ "gold_session_ids": [+ "answer_ultrachat_94624"+ ]+ },+ {+ "question_id": "e982271f",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_195444",+ "sharegpt_BdpBz7Q_0",+ "eb0d8dc1_4",+ "ultrachat_564244",+ "8a7c65c1_1",+ "sharegpt_f83u8ds_59",+ "ultrachat_383774",+ "ultrachat_380780",+ "7b4bce3b_2",+ "db65997e"+ ],+ "gold_session_ids": [+ "answer_ultrachat_195444"+ ]+ },+ {+ "question_id": "352ab8bd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_NoDZzot_7",+ "eee803c7",+ "3826dc55_5",+ "337cbfc8_2",+ "9bafe9a2",+ "d6d87935_2",+ "sharegpt_UCVg3jO_0",+ "3b0f28f6_2",+ "ultrachat_1052",+ "df08e7a0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_NoDZzot_7"+ ]+ },+ {+ "question_id": "fca762bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "a8bb9614_2",+ "d0cdfddb_2",+ "3ac6fe4c",+ "d5b3b034_1",+ "42d9467c_1",+ "2bc4ae7d",+ "answer_ultrachat_39395",+ "9beab022_1",+ "3b7a6ecd_1",+ "99eec551"+ ],+ "gold_session_ids": [+ "answer_ultrachat_39395"+ ]+ },+ {+ "question_id": "7a8d0b71",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_i0tMT9q_9",+ "d5eab084_2",+ "147ab7e9_5",+ "ccc6ab06_2",+ "f07752be_1",+ "ultrachat_136480",+ "f20e72e4_2",+ "4aaf678f_4",+ "c14e3f09_2",+ "14725f4a_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_i0tMT9q_9"+ ]+ },+ {+ "question_id": "a40e080f",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "e63fbbeb_1",+ "answer_ultrachat_269020",+ "sharegpt_fyAkcGP_81",+ "ef917f63",+ "sharegpt_oXgiN7q_327",+ "cd38b6ee",+ "eb18a1b7",+ "sharegpt_G3RxeFJ_94",+ "528f481d_6",+ "85736a07_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_269020"+ ]+ },+ {+ "question_id": "8b9d4367",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_289157",+ "7babf05b",+ "c83bbe71_1",+ "0bad887e_1",+ "93d4bb28",+ "20cf06b8",+ "45d1c5c4",+ "5880cdab",+ "031061e6_4",+ "ultrachat_321805"+ ],+ "gold_session_ids": [+ "answer_ultrachat_289157"+ ]+ },+ {+ "question_id": "5809eb10",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_4aJsGCH_0",+ "sharegpt_GDFyxeT_0",+ "fc3d6c80",+ "922c819f_2",+ "5237bb0b_1",+ "sharegpt_fQhexkP_38",+ "616b8721",+ "35ee7421",+ "sharegpt_n3MgsgL_0",+ "sharegpt_QaiMtpK_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_4aJsGCH_0"+ ]+ },+ {+ "question_id": "41275add",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2890648263178879,+ "mrr": 0.1,+ "retrieved_session_ids": [+ "a68db5db_1",+ "eee803c7",+ "acc3913b",+ "sharegpt_dUoAhvI_71",+ "9345f7dc_1",+ "c07ae000_1",+ "b3619c2c",+ "53fe2138",+ "2e0ff38d_2",+ "answer_sharegpt_81riySf_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_81riySf_0"+ ]+ },+ {+ "question_id": "4388e9dd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_qTi81nS_0",+ "0826def6_1",+ "sharegpt_Tun3KJ4_0",+ "313404c6_2",+ "a6c0f032_3",+ "cf9eda76",+ "b6506028_2",+ "3197d998_7",+ "47311a4c_2",+ "1ebd753f_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_qTi81nS_0"+ ]+ },+ {+ "question_id": "4baee567",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_i9adwQn_0",+ "sharegpt_wduYbsX_0",+ "sharegpt_28Mwwk9_0",+ "f6ec94b4_1",+ "74711358_2",+ "e00298a6_2",+ "sharegpt_lWLBUhQ_547",+ "ed3504e1",+ "d5415748",+ "4509e9fc"+ ],+ "gold_session_ids": [+ "answer_sharegpt_i9adwQn_0"+ ]+ },+ {+ "question_id": "561fabcd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_hChsWOp_97",+ "aee13015_3",+ "63c75b13_2",+ "1c1f5ccc_6",+ "cb442986",+ "33da50d0_5",+ "83fb74bf_1",+ "sharegpt_QaiMtpK_0",+ "864a563d_4",+ "d5d7eff7"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hChsWOp_97"+ ]+ },+ {+ "question_id": "b759caee",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_2BSXlAr_0",+ "eae516cd",+ "fcae5c39_1",+ "d3575920",+ "f55fee5b_4",+ "191f4c9b_2",+ "3a667084_2",+ "bab41bb6_5",+ "22aa7cca_1",+ "6be2f065_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_2BSXlAr_0"+ ]+ },+ {+ "question_id": "ac031881",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_GYqnAhC_190",+ "449b918c_3",+ "sharegpt_cIj2dFl_4",+ "e30307d8_1",+ "eb95f93c",+ "6608c1da",+ "f43d9ae5",+ "010a28ab_3",+ "c8c3084d_3",+ "57c91b14_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_GYqnAhC_190"+ ]+ },+ {+ "question_id": "28bcfaac",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "8d410160",+ "answer_ultrachat_446979",+ "aafe556c",+ "135a62d7_2",+ "485f4604_2",+ "774da34f_1",+ "510b98f3",+ "e7cc239c_2",+ "7cd7c296_1",+ "17cb8d2f_5"+ ],+ "gold_session_ids": [+ "answer_ultrachat_446979"+ ]+ },+ {+ "question_id": "16c90bf4",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_294807",+ "89d49715_1",+ "afa0c2e7_5",+ "cc830ab4_1",+ "d576152e_4",+ "101636c0_1",+ "fa2714ed_1",+ "b0da5097_2",+ "8464304d_2",+ "d5f4d9fa_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_294807"+ ]+ },+ {+ "question_id": "c8f1aeed",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_519486",+ "f7cd21e5",+ "sharegpt_ebYKddM_87",+ "ed8ba4b0_1",+ "48a72204_3",+ "f8a9cd01_2",+ "b8620ffe_2",+ "sharegpt_9ra36HI_32",+ "12cd736c",+ "199a5225_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_519486"+ ]+ },+ {+ "question_id": "eaca4986",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "931c521e_1",+ "f987da7e_3",+ "3477c37c_1",+ "sharegpt_yC6nsDU_0",+ "1c1a2b7f_2",+ "564761aa",+ "sharegpt_zY4q7i0_5",+ "0e543e9e_2",+ "sharegpt_kNi3QzP_24",+ "07fb2f15_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_SS141vi_0"+ ]+ },+ {+ "question_id": "c7cf7dfd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_456407",+ "96b8c9ee",+ "285a29e7_1",+ "9db6d988_2",+ "sharegpt_gQaSoiF_13",+ "e24943fc",+ "1382d6fb_1",+ "1341f0b9",+ "780b6c7c_1",+ "e27fe923_5"+ ],+ "gold_session_ids": [+ "answer_ultrachat_456407"+ ]+ },+ {+ "question_id": "e48988bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "f84b4266",+ "6e110a53_1",+ "answer_ultrachat_174360",+ "476b1cbb",+ "8fb3fe3a_3",+ "ff6bccaa_3",+ "ultrachat_494373",+ "e388463b_1",+ "ultrachat_283303",+ "ultrachat_58258"+ ],+ "gold_session_ids": [+ "answer_ultrachat_174360"+ ]+ },+ {+ "question_id": "1de5cff2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "848d721f_1",+ "answer_ultrachat_440262",+ "001cefa7_2",+ "4c6775ed_1",+ "64048e35",+ "42d9467c_1",+ "e7418361_2",+ "28031c43_2",+ "1c1f5ccc_1",+ "c14dbe17"+ ],+ "gold_session_ids": [+ "answer_ultrachat_440262"+ ]+ },+ {+ "question_id": "65240037",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_403752",+ "f8dc1273",+ "37141183",+ "92466e8e_1",+ "7df4c735_1",+ "0504a710_3",+ "9139676d",+ "sharegpt_ME35OD8_167",+ "d3d299a4",+ "46f07bd0_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_403752"+ ]+ },+ {+ "question_id": "778164c6",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "b605741a_1",+ "490bb46d_1",+ "05b551b6",+ "35201d43",+ "483ddd90",+ "81e406dd_2",+ "answer_ultrachat_399000",+ "ddaa5a15_2",+ "1340826c",+ "sharegpt_ki9IVDq_6"+ ],+ "gold_session_ids": [+ "answer_ultrachat_399000"+ ]+ }+ ]+}\ No newline at end of filediff --git a/benchmark/data/longmemeval_results_hybrid.json b/benchmark/data/longmemeval_results_hybrid.jsonnew file mode 100644index 00000000..21f1ad41--- /dev/null+++ b/benchmark/data/longmemeval_results_hybrid.json@@ -0,0 +1,12491 @@+{+ "mode": "hybrid",+ "questions": 500,+ "recall_any_at_5": 0.952,+ "recall_any_at_10": 0.986,+ "recall_any_at_20": 0.994,+ "ndcg_at_10": 0.8785881269713454,+ "mrr": 0.8821431318681319,+ "per_type": {+ "single-session-user": {+ "count": 70,+ "recall_any_at_5": 0.9,+ "recall_any_at_10": 0.9714285714285714+ },+ "multi-session": {+ "count": 133,+ "recall_any_at_5": 0.9774436090225563,+ "recall_any_at_10": 1+ },+ "single-session-preference": {+ "count": 30,+ "recall_any_at_5": 0.8333333333333334,+ "recall_any_at_10": 0.9666666666666667+ },+ "temporal-reasoning": {+ "count": 133,+ "recall_any_at_5": 0.9548872180451128,+ "recall_any_at_10": 0.9774436090225563+ },+ "knowledge-update": {+ "count": 78,+ "recall_any_at_5": 0.9871794871794872,+ "recall_any_at_10": 1+ },+ "single-session-assistant": {+ "count": 56,+ "recall_any_at_5": 0.9642857142857143,+ "recall_any_at_10": 0.9821428571428571+ }+ },+ "per_question": [+ {+ "question_id": "e47becba",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_280352e9",+ "f6859b48_2",+ "sharegpt_T1EiHWI_13",+ "41abc171_4",+ "ultrachat_336116",+ "02bd2b90_3",+ "sharegpt_UnjngE7_65",+ "ultrachat_214101",+ "ultrachat_486410",+ "ultrachat_514031"+ ],+ "gold_session_ids": [+ "answer_280352e9"+ ]+ },+ {+ "question_id": "118b2229",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_40a90d51",+ "628c958c_7",+ "db73b7e4_4",+ "db50c0f6",+ "7d8b5834_1",+ "0afa904a",+ "728deb4d_5",+ "79f74d4e_1",+ "sharegpt_dabXXRO_10",+ "004edb32"+ ],+ "gold_session_ids": [+ "answer_40a90d51"+ ]+ },+ {+ "question_id": "51a45a95",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d61669c7",+ "07ba9acd_2",+ "55e0c6db_2",+ "98c198fb",+ "sharegpt_MKMWjX0_25",+ "6e0b1800_2",+ "5c44d9fe_1",+ "sharegpt_3vxz2Zr_0",+ "07d1915d_1",+ "dfe646a7_1"+ ],+ "gold_session_ids": [+ "answer_d61669c7"+ ]+ },+ {+ "question_id": "58bf7951",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_355c48bb",+ "ultrachat_304942",+ "793382f9_2",+ "ultrachat_268434",+ "5b83c26e_1",+ "ultrachat_278141",+ "ultrachat_71618",+ "1587bd78_1",+ "ultrachat_143341",+ "ultrachat_155546"+ ],+ "gold_session_ids": [+ "answer_355c48bb"+ ]+ },+ {+ "question_id": "1e043500",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "01bd9def",+ "8020d945_2",+ "sharegpt_bph7DTk_7",+ "answer_3e012175",+ "sharegpt_uQMdL9t_0",+ "c285b705_2",+ "sharegpt_Clr6oyu_0",+ "4d0d7984",+ "2d0d0004",+ "b28990c8_1"+ ],+ "gold_session_ids": [+ "answer_3e012175"+ ]+ },+ {+ "question_id": "c5e8278d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6168136",+ "f58317c6",+ "sharegpt_DB8U4oH_9",+ "e4881cbd_4",+ "89b1028d_1",+ "aaf71ce2_2",+ "0fe848c5_3",+ "dc832f73",+ "e01eaa19_2",+ "03362ec8_2"+ ],+ "gold_session_ids": [+ "answer_f6168136"+ ]+ },+ {+ "question_id": "6ade9755",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9398da02",+ "a200b713_4",+ "ultrachat_270925",+ "sharegpt_BghMUHh_0",+ "5df2f487",+ "7c5ab02a_2",+ "de5634fc_2",+ "ultrachat_462",+ "ec8fa65d_2",+ "5d049e3c"+ ],+ "gold_session_ids": [+ "answer_9398da02"+ ]+ },+ {+ "question_id": "6f9b354f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_feb5200f",+ "ultrachat_549317",+ "de877349",+ "0c0f2449_2",+ "a79f1a04_1",+ "c22eca8c_1",+ "sharegpt_BYHT3Xz_0",+ "a729934a_2",+ "40643340_2",+ "2c275178_1"+ ],+ "gold_session_ids": [+ "answer_feb5200f"+ ]+ },+ {+ "question_id": "58ef2f1c",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_59547700",+ "e6ab6a7b_2",+ "ultrachat_5212",+ "e82ce7cd_1",+ "sharegpt_eHZgwR0_9",+ "13f3da23_3",+ "b546cb39_2",+ "sharegpt_RFw4j0W_0",+ "ultrachat_246324",+ "ultrachat_477344"+ ],+ "gold_session_ids": [+ "answer_59547700"+ ]+ },+ {+ "question_id": "f8c5f88b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c3567066",+ "91880423",+ "15232887_1",+ "sharegpt_TKj3yO8_0",+ "54e9197a",+ "68ace657_1",+ "66c23110_1",+ "sharegpt_n2aGNgW_9",+ "5520ca18",+ "0a6bf5e4_2"+ ],+ "gold_session_ids": [+ "answer_c3567066"+ ]+ },+ {+ "question_id": "5d3d2817",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3010299956639812,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "670ca40e",+ "sharegpt_ebWP6Hz_0",+ "sharegpt_ipLglky_48",+ "sharegpt_bTDMt3m_64",+ "sharegpt_BKl952A_23",+ "49b78a55_1",+ "e00298a6_1",+ "e93efd9a_1",+ "answer_235eb6fb",+ "85d16d01"+ ],+ "gold_session_ids": [+ "answer_235eb6fb"+ ]+ },+ {+ "question_id": "7527f7e2",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7cb94507",+ "18a06652_3",+ "44bbd8cd_2",+ "4e2acecc",+ "afbd7193_1",+ "sharegpt_xIDRgaY_0",+ "sharegpt_cRrfdsc_0",+ "sharegpt_ToxFax1_535",+ "bee872da_2",+ "88fed3ed_2"+ ],+ "gold_session_ids": [+ "answer_7cb94507"+ ]+ },+ {+ "question_id": "c960da58",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e05e4612",+ "sharegpt_ErOTMZ3_59",+ "sharegpt_CdnyHep_0",+ "ultrachat_198948",+ "f7ffddbc",+ "sharegpt_OAUjazp_0",+ "ultrachat_374640",+ "b76006cb_3",+ "sharegpt_SoyhGNj_0",+ "52381716_1"+ ],+ "gold_session_ids": [+ "answer_e05e4612"+ ]+ },+ {+ "question_id": "3b6f954b",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.07692307692307693,+ "retrieved_session_ids": [+ "c271c071",+ "ultrachat_230611",+ "ultrachat_64326",+ "ultrachat_572251",+ "sharegpt_cD2Kgkn_33",+ "4ed9cd1b_1",+ "e27891d3_2",+ "ultrachat_294369",+ "f999b05c_2",+ "ultrachat_308865"+ ],+ "gold_session_ids": [+ "answer_94030872"+ ]+ },+ {+ "question_id": "726462e0",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "741c6781_1",+ "6ef99651_2",+ "ce144f08",+ "cdd9704d_1",+ "9c05acfe_2",+ "b84b7a2b_1",+ "ultrachat_450587",+ "7e4dab66_1",+ "5b83c26e_1",+ "sharegpt_IxOWCIE_9"+ ],+ "gold_session_ids": [+ "answer_f38f679b"+ ]+ },+ {+ "question_id": "94f70d80",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c63c0458",+ "sharegpt_PCqCVmR_18",+ "96eac086",+ "sharegpt_ZvjRGRN_0",+ "2479cbae",+ "ultrachat_526427",+ "b015c3c3_1",+ "4e77dbbe_5",+ "2ae4f277_1",+ "09b651aa_3"+ ],+ "gold_session_ids": [+ "answer_c63c0458"+ ]+ },+ {+ "question_id": "66f24dbb",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "89452526_2",+ "answer_fea2e4d3",+ "798e4ba3_2",+ "8d015d9d_4",+ "ca3a4e4f_1",+ "7a10d327_2",+ "8d46b8fa_1",+ "aeb3818c_2",+ "sharegpt_BvuMJIh_0",+ "fce8bb8a_2"+ ],+ "gold_session_ids": [+ "answer_fea2e4d3"+ ]+ },+ {+ "question_id": "ad7109d1",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "3dc02d26_2",+ "answer_679840f8",+ "22aa7cca_3",+ "06121f7f_1",+ "3030efec_2",+ "7ad0bd25_1",+ "0982a1fa_5",+ "1d6744b5_1",+ "sharegpt_zdDMlRC_0",+ "02f0738e_4"+ ],+ "gold_session_ids": [+ "answer_679840f8"+ ]+ },+ {+ "question_id": "af8d2e46",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_82a04f59",+ "ced213c6",+ "d5a6f455_2",+ "43668d77_1",+ "33c3b457",+ "7f61409f",+ "f75238a5_1",+ "ultrachat_164840",+ "38e4b795_2",+ "73d5f10c"+ ],+ "gold_session_ids": [+ "answer_82a04f59"+ ]+ },+ {+ "question_id": "dccbc061",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8f276838",+ "25d1830a_2",+ "ultrachat_343",+ "ultrachat_519037",+ "0b8a4a9d_1",+ "79bb1298_3",+ "4af27eab_2",+ "sharegpt_buGcwO5_33",+ "sharegpt_MKMWjX0_25",+ "657ab20c_1"+ ],+ "gold_session_ids": [+ "answer_8f276838"+ ]+ },+ {+ "question_id": "c8c3f81d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_761acef8",+ "19ec83c5_1",+ "b818e2a1_1",+ "2b16f3d6_1",+ "eb403c80_2",+ "25432e10",+ "sharegpt_pRqHb1o_0",+ "sharegpt_6CsjCEy_0",+ "eb5d5a93_1",+ "e89b6be2_2"+ ],+ "gold_session_ids": [+ "answer_761acef8"+ ]+ },+ {+ "question_id": "8ebdbe50",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8ad8a34f",+ "90f16f14_2",+ "b53f447a_1",+ "39911f94",+ "27d378b2_3",+ "03b4d5d8_2",+ "sharegpt_c0PdB3l_0",+ "66369fde_4",+ "sharegpt_M84blrA_53",+ "407dd888"+ ],+ "gold_session_ids": [+ "answer_8ad8a34f"+ ]+ },+ {+ "question_id": "6b168ec8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e623ae87",+ "ultrachat_414934",+ "sharegpt_1kE6haW_52",+ "afffd8ec_3",+ "388c5534_1",+ "32cf110f_1",+ "443688cc_2",+ "33dff20c_3",+ "sharegpt_ba4V2an_2",+ "518d26d3_5"+ ],+ "gold_session_ids": [+ "answer_e623ae87"+ ]+ },+ {+ "question_id": "75499fd8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_723bf11f",+ "ultrachat_428593",+ "5328c3c2_2",+ "533372c8_3",+ "sharegpt_v11Tpg9_195",+ "sharegpt_WetAHhj_0",+ "ddf0f116_3",+ "sharegpt_u1AM5RT_273",+ "ultrachat_371964",+ "ultrachat_466236"+ ],+ "gold_session_ids": [+ "answer_723bf11f"+ ]+ },+ {+ "question_id": "21436231",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1e6d4567",+ "7a7a4bf0_4",+ "1b2b30d3",+ "a0f8468c_3",+ "a3118ef0_1",+ "4bebc783_2",+ "d0785ee9_1",+ "a3058382_4",+ "sharegpt_DGiqvVv_169",+ "0d0a89dc_1"+ ],+ "gold_session_ids": [+ "answer_1e6d4567"+ ]+ },+ {+ "question_id": "95bcc1c8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cb742a61",+ "abeda028",+ "56266c38_1",+ "c9763bff",+ "sharegpt_uATA5m5_41",+ "ultrachat_320154",+ "d59a335d",+ "99e5e380",+ "396238f9",+ "ultrachat_203404"+ ],+ "gold_session_ids": [+ "answer_cb742a61"+ ]+ },+ {+ "question_id": "0862e8bf",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c6fd8ebd",+ "sharegpt_MTUnqho_0",+ "ultrachat_357044",+ "sharegpt_A7KJApk_21",+ "sharegpt_2I1TxKW_0",+ "sharegpt_WhKs0Le_0",+ "10541a2c_1",+ "sharegpt_oLlUxYj_40",+ "ultrachat_240637",+ "ultrachat_518141"+ ],+ "gold_session_ids": [+ "answer_c6fd8ebd"+ ]+ },+ {+ "question_id": "853b0a1d",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "c70f9f9c",+ "answer_69811d4a",+ "sharegpt_n3MgsgL_0",+ "ultrachat_376940",+ "sharegpt_hRba933_47",+ "f71bf532_1",+ "ultrachat_378413",+ "sharegpt_F1lIiep_0",+ "sharegpt_UnjngE7_65",+ "sharegpt_oh3eExx_8"+ ],+ "gold_session_ids": [+ "answer_69811d4a"+ ]+ },+ {+ "question_id": "a06e4cfe",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "af257b0b_2",+ "answer_6fe9fb49",+ "84fb50bb_3",+ "b66c6397_1",+ "ultrachat_442201",+ "6dfb33f1_1",+ "0c891c0b_1",+ "de64539a_2",+ "9de2c795",+ "15519944_2"+ ],+ "gold_session_ids": [+ "answer_6fe9fb49"+ ]+ },+ {+ "question_id": "37d43f65",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_55161935",+ "1cafd864_3",+ "782c0f84",+ "3ab79434_2",+ "ultrachat_415674",+ "sharegpt_uNuzBLK_0",+ "sharegpt_Bm91yJg_0",+ "63587769",+ "3dcf1e0b_2",+ "ultrachat_107321"+ ],+ "gold_session_ids": [+ "answer_55161935"+ ]+ },+ {+ "question_id": "b86304ba",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "ultrachat_328696",+ "e831a29f_1",+ "65f3c006_1",+ "sharegpt_E0YL5SX_142",+ "8e9425ef_1",+ "answer_645b0329",+ "ae00973d_4",+ "sharegpt_xGoJZ6Z_0",+ "5cc9b056",+ "ea8bb4f8_2"+ ],+ "gold_session_ids": [+ "answer_645b0329"+ ]+ },+ {+ "question_id": "d52b4f67",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0df6aa4b",+ "d31c3ec8_3",+ "62a6d083",+ "cd1f21d0_3",+ "0840765f_1",+ "sharegpt_n6Ej34Y_75",+ "sharegpt_BVfC7JV_0",+ "f999b05b_5",+ "ultrachat_481735",+ "9b17bb98"+ ],+ "gold_session_ids": [+ "answer_0df6aa4b"+ ]+ },+ {+ "question_id": "25e5aa4f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_986de8c3",+ "18807892_4",+ "ultrachat_225537",+ "sharegpt_8cFNcVG_0",+ "sharegpt_6MPnc5F_9",+ "sharegpt_rqvR1Ep_0",+ "8953c8c8_1",+ "ultrachat_373294",+ "sharegpt_SwdaeDm_15",+ "1de862d6_2"+ ],+ "gold_session_ids": [+ "answer_986de8c3"+ ]+ },+ {+ "question_id": "caf9ead2",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0714183a",+ "7ae303b7_1",+ "86974d49",+ "ultrachat_137778",+ "cd568521_2",+ "fcc6d66d_1",+ "e18a6f0b_4",+ "58dd3f35",+ "f8ba3f69",+ "5e014cd6_2"+ ],+ "gold_session_ids": [+ "answer_0714183a"+ ]+ },+ {+ "question_id": "8550ddae",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c8354ae9",+ "71cf5aeb_3",+ "a7189b75",+ "9deebbc2_2",+ "e10dfffb_1",+ "c7a15bdc",+ "3b13915f",+ "d2cb1f8c",+ "ultrachat_365941",+ "38bbc2b7_2"+ ],+ "gold_session_ids": [+ "answer_c8354ae9"+ ]+ },+ {+ "question_id": "60d45044",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9cddca88",+ "a76e7e3c_2",+ "fec059c5",+ "446173bb_1",+ "26bc645b_5",+ "f90c44e6_2",+ "af4d4ecd_2",+ "0c0f2449_1",+ "de34d3f7",+ "fcc6d66d_1"+ ],+ "gold_session_ids": [+ "answer_9cddca88"+ ]+ },+ {+ "question_id": "3f1e9474",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_57fc1954",+ "205f348b_2",+ "e7497c07_2",+ "ultrachat_393184",+ "ultrachat_477598",+ "sharegpt_hl7jQnw_0",+ "ultrachat_489379",+ "ultrachat_177477",+ "3aac691d_2",+ "sharegpt_5clkQjb_12"+ ],+ "gold_session_ids": [+ "answer_57fc1954"+ ]+ },+ {+ "question_id": "86b68151",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "0abaca24_1",+ "answer_dc11c1eb",+ "4e671700",+ "sharegpt_beV15ZO_0",+ "d1bd26fb_2",+ "sharegpt_rnL5VQO_0",+ "d8454317_4",+ "sharegpt_KdbXhTW_13",+ "ultrachat_182718",+ "159ab103"+ ],+ "gold_session_ids": [+ "answer_dc11c1eb"+ ]+ },+ {+ "question_id": "577d4d32",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "6e2cca63_2",+ "f60245ed",+ "ultrachat_125084",+ "dc700e27",+ "26f9a255_1",+ "2ef53f61_2",+ "answer_0dd4d99a",+ "sharegpt_101i5OR_0",+ "b283f9eb_1",+ "sharegpt_1ByFFwm_0"+ ],+ "gold_session_ids": [+ "answer_0dd4d99a"+ ]+ },+ {+ "question_id": "ec81a493",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ed1982fc",+ "6f55019d",+ "ab4643a2_4",+ "616fdd27",+ "ultrachat_27488",+ "90fea364_3",+ "ffbd49a4",+ "bb2180e9_1",+ "d22a462a_2",+ "ultrachat_74904"+ ],+ "gold_session_ids": [+ "answer_ed1982fc"+ ]+ },+ {+ "question_id": "15745da0",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_586de428",+ "07d33eaa",+ "ultrachat_446016",+ "1f5370f7_4",+ "0d73afc8",+ "ultrachat_155656",+ "1c662b7b_1",+ "768544e1",+ "c1f03307_2",+ "4cbb6dd2"+ ],+ "gold_session_ids": [+ "answer_586de428"+ ]+ },+ {+ "question_id": "e01b8e2f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5ca6cd28",+ "97338ddd_1",+ "86c5d31d",+ "8acfa731",+ "b2543a58_3",+ "1aaf8057_3",+ "d31c3ec8_1",+ "8be2c3f1",+ "sharegpt_4ANhVKP_0",+ "d4cdb464"+ ],+ "gold_session_ids": [+ "answer_5ca6cd28"+ ]+ },+ {+ "question_id": "bc8a6e93",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e6143162",+ "ultrachat_453950",+ "b869d7ed_2",+ "ultrachat_184007",+ "a1b3dd27_1",+ "6f8c03ab",+ "f62bfe2b_1",+ "05594106_1",+ "1cb7a6c4_4",+ "sharegpt_W9DAhiE_0"+ ],+ "gold_session_ids": [+ "answer_e6143162"+ ]+ },+ {+ "question_id": "ccb36322",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f1fbb330",+ "facfdfe2",+ "bdcee74e_3",+ "162ff451",+ "04a0b385",+ "06d1a1a0",+ "940629b2_3",+ "28a45124",+ "ultrachat_203880",+ "46d9d476"+ ],+ "gold_session_ids": [+ "answer_f1fbb330"+ ]+ },+ {+ "question_id": "001be529",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_530960c1",+ "sharegpt_6cz1Sq6_328",+ "sharegpt_L5SCdrp_0",+ "dffe8d2d_2",+ "f067b0ee",+ "752a01fe_3",+ "a244d459",+ "de877349",+ "2c1537ff",+ "ultrachat_241736"+ ],+ "gold_session_ids": [+ "answer_530960c1"+ ]+ },+ {+ "question_id": "b320f3f8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5cc9b056",+ "aae4411b_2",+ "b89f68c4_4",+ "sharegpt_4FCMpJR_0",+ "1b71c896_2",+ "5e4bb245_2",+ "66ffbb8b_1",+ "e28f75f4",+ "sharegpt_7QD86cC_20",+ "ultrachat_562554"+ ],+ "gold_session_ids": [+ "answer_5cc9b056"+ ]+ },+ {+ "question_id": "19b5f2b3",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5ff494b9",+ "ultrachat_302261",+ "sharegpt_I0Wgney_11",+ "ultrachat_447485",+ "ultrachat_138083",+ "275804a8_3",+ "ultrachat_8325",+ "5214491c",+ "51adc57e_2",+ "fbe6fa2c_1"+ ],+ "gold_session_ids": [+ "answer_5ff494b9"+ ]+ },+ {+ "question_id": "4fd1909e",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2952aee4",+ "ad2135af",+ "ultrachat_186237",+ "94c582bb_2",+ "ultrachat_472809",+ "502b6b52_2",+ "f716720c_1",+ "ultrachat_162975",+ "65bc93cf_7",+ "2479cbae"+ ],+ "gold_session_ids": [+ "answer_2952aee4"+ ]+ },+ {+ "question_id": "545bd2b5",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "b7f69a7d_1",+ "answer_47ffab4c",+ "d1a1b9ea_1",+ "1351505a_1",+ "32615aa6_3",+ "6e672b84_2",+ "700c16e2_1",+ "9f9384ed_1",+ "10889cf0_1",+ "cc021f81_2"+ ],+ "gold_session_ids": [+ "answer_47ffab4c"+ ]+ },+ {+ "question_id": "8a137a7f",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_15d63a22",+ "ac583170_2",+ "e3e66b50",+ "ultrachat_293136",+ "sharegpt_tR1SJyb_0",+ "ultrachat_44939",+ "e2cb1647",+ "sharegpt_CMbaRkD_37",+ "77908e43_1",+ "sharegpt_o1SCEXR_23"+ ],+ "gold_session_ids": [+ "answer_15d63a22"+ ]+ },+ {+ "question_id": "76d63226",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "cb289226_1",+ "50136b31",+ "6a2f9452_2",+ "ultrachat_180710",+ "sharegpt_Wi7Op1u_91",+ "answer_bbdc7b0a",+ "11566271_2",+ "sharegpt_sRhcMiu_13",+ "sharegpt_jsjbgCR_0",+ "372544ba"+ ],+ "gold_session_ids": [+ "answer_bbdc7b0a"+ ]+ },+ {+ "question_id": "86f00804",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96b8c9ee",+ "6e672b84_3",+ "35c5419d_abs_3",+ "sharegpt_sCFErnY_22",+ "sharegpt_BAHXiar_14",+ "sharegpt_ErOTMZ3_277",+ "6339aa1d_2",+ "sharegpt_K3jwzdd_0",+ "830c739e",+ "cf9010c2_1"+ ],+ "gold_session_ids": [+ "answer_96b8c9ee"+ ]+ },+ {+ "question_id": "8e9d538c",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7bdcbd23",+ "27d18d38",+ "ultrachat_570726",+ "2deed26f",+ "4b5568a5",+ "sharegpt_MoCDhww_0",+ "ultrachat_185241",+ "sharegpt_cADPA4R_0",+ "sharegpt_tb5H6IH_3",+ "ultrachat_419107"+ ],+ "gold_session_ids": [+ "answer_7bdcbd23"+ ]+ },+ {+ "question_id": "311778f1",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e40b054e",+ "sharegpt_vXNQZ2I_0",+ "4fe91719_1",+ "8e6e058b_3",+ "5db56d24_2",+ "31903f54",+ "414b9122",+ "ultrachat_316997",+ "ultrachat_514341",+ "sharegpt_1kE6haW_276"+ ],+ "gold_session_ids": [+ "answer_e40b054e"+ ]+ },+ {+ "question_id": "c19f7a0b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f442ccbe",+ "9530353d_1",+ "58ab5fc5",+ "sharegpt_6ZeW9Vb_0",+ "7bc2c7a3_1",+ "ultrachat_352431",+ "c9292210_2",+ "5558a42e_1",+ "ultrachat_80803",+ "sharegpt_RjNU9sC_20"+ ],+ "gold_session_ids": [+ "answer_f442ccbe"+ ]+ },+ {+ "question_id": "4100d0a0",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_83c13ff9",+ "8d068598",+ "eb888fcf_1",+ "sharegpt_8w0MhIA_0",+ "22a97fec",+ "sharegpt_eV0krsR_0",+ "sharegpt_Viy08ou_0",+ "sharegpt_t3f69h7_23",+ "4fd958dd_2",+ "2b86336d_1"+ ],+ "gold_session_ids": [+ "answer_83c13ff9"+ ]+ },+ {+ "question_id": "29f2956b",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7cc5362f",+ "00842aad_2",+ "90f7041a_3",+ "46548fab_3",+ "a79db993",+ "88361fdd",+ "6d6ffac5",+ "715b1b28",+ "c4ed2287_3",+ "d50a8a33_1"+ ],+ "gold_session_ids": [+ "answer_7cc5362f"+ ]+ },+ {+ "question_id": "1faac195",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "b2543a58_3",+ "answer_d01949bf",+ "d2905eb6_2",+ "2def525b_2",+ "f2ccf83b",+ "94f460be_1",+ "sharegpt_wgTTTPq_0",+ "bcf6953d_3",+ "75f97838_1",+ "bce91ca6"+ ],+ "gold_session_ids": [+ "answer_d01949bf"+ ]+ },+ {+ "question_id": "faba32e5",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_39b12014",+ "sharegpt_3v69R59_0",+ "770d11f5",+ "83cfab9f_2",+ "sharegpt_benxw0S_9",+ "f0ecaea5_2",+ "69f3fc12_1",+ "sharegpt_jxsIY6L_0",+ "70323809_2",+ "sharegpt_PdnvIns_0"+ ],+ "gold_session_ids": [+ "answer_39b12014"+ ]+ },+ {+ "question_id": "f4f1d8a4",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "3aac691d_2",+ "76ea88ba_2",+ "eb682e52_2",+ "answer_f5b33470",+ "6f051087_2",+ "64934c76",+ "5944b36a_2",+ "04d84aa8_2",+ "sharegpt_pX51ZUz_0",+ "e45db473_3"+ ],+ "gold_session_ids": [+ "answer_f5b33470"+ ]+ },+ {+ "question_id": "c14c00dd",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "c77250d2",+ "answer_304511ce",+ "5dac7cc2_1",+ "sharegpt_XCNeusQ_9",+ "sharegpt_zs9nI2N_0",+ "f91cac51_1",+ "cd1f5147",+ "sharegpt_vXWOUqx_11",+ "sharegpt_gU0qvZu_0",+ "sharegpt_FfqJ2xI_0"+ ],+ "gold_session_ids": [+ "answer_304511ce"+ ]+ },+ {+ "question_id": "36580ce8",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ultrachat_149170",+ "ultrachat_32917",+ "answer_93e1bd22",+ "ultrachat_270984",+ "ultrachat_318891",+ "ultrachat_53784",+ "sharegpt_5L3F069_69",+ "2aee3e70",+ "2865bf22_2",+ "526354c8_2"+ ],+ "gold_session_ids": [+ "answer_93e1bd22"+ ]+ },+ {+ "question_id": "3d86fd0a",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_19c24c11",+ "cf8e352f_1",+ "06d1a1a0",+ "6227a491_2",+ "5a13d047_1",+ "7e4dab66_2",+ "ultrachat_394611",+ "1c1a2b7f_2",+ "d1d3fd12_2",+ "4ff568ba_2"+ ],+ "gold_session_ids": [+ "answer_19c24c11"+ ]+ },+ {+ "question_id": "a82c026e",+ "question_type": "single-session-user",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "ccaabf0b_1",+ "752392bb_3",+ "sharegpt_u71pVhw_0",+ "daa32134",+ "1a693999_4",+ "answer_787e6a6d",+ "516b0aae_1",+ "a6c41826_3",+ "sharegpt_vIkUTvg_0",+ "c71e1c7a_2"+ ],+ "gold_session_ids": [+ "answer_787e6a6d"+ ]+ },+ {+ "question_id": "0862e8bf_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "f238edb0_1",+ "answer_c6fd8ebd_abs",+ "sharegpt_xLtmCeT_0",+ "sharegpt_StUzdLK_227",+ "sharegpt_PSjpANO_35",+ "12cd736c",+ "3301f749_1",+ "6a8f8453_2",+ "ultrachat_460804",+ "sharegpt_8T6d65A_83"+ ],+ "gold_session_ids": [+ "answer_c6fd8ebd_abs"+ ]+ },+ {+ "question_id": "15745da0_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_586de428_abs",+ "375e27b9_2",+ "565929a2_2",+ "9b638f21_4",+ "ultrachat_360529",+ "7dfe0b9d_1",+ "sharegpt_bmEgv3O_5",+ "sharegpt_JSxa86k_19",+ "be976c82_1",+ "sharegpt_Ql85pW6_0"+ ],+ "gold_session_ids": [+ "answer_586de428_abs"+ ]+ },+ {+ "question_id": "bc8a6e93_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e6143162_abs",+ "ba944cdc",+ "375354d1_2",+ "31f3f40d_4",+ "sharegpt_hn3IS1q_0",+ "b3763b6b_2",+ "9c68c22f",+ "sharegpt_MimvIAy_181",+ "de1f4aec_1",+ "22356a93_4"+ ],+ "gold_session_ids": [+ "answer_e6143162_abs"+ ]+ },+ {+ "question_id": "19b5f2b3_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5ff494b9_abs",+ "ultrachat_159269",+ "sharegpt_oPOTyid_28",+ "56cdb8b8_1",+ "e44a5733_1",+ "sharegpt_v11Tpg9_483",+ "d0b0dabe",+ "f5cf8815_3",+ "9f8fa5e4_2",+ "8fa624b2_3"+ ],+ "gold_session_ids": [+ "answer_5ff494b9_abs"+ ]+ },+ {+ "question_id": "29f2956b_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7cc5362f_abs",+ "64b7d9cc_2",+ "f2fa97f5_3",+ "503b2761",+ "ultrachat_13007",+ "3b8f0d9d",+ "1516d4c8",+ "05b84f71_5",+ "01ecd4fa_3",+ "ultrachat_70054"+ ],+ "gold_session_ids": [+ "answer_7cc5362f_abs"+ ]+ },+ {+ "question_id": "f4f1d8a4_abs",+ "question_type": "single-session-user",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "1dd13331_1",+ "ea3db78e_2",+ "answer_f5b33470_abs",+ "080a4000_2",+ "f252001e",+ "56745369_1",+ "31299b8e_2",+ "793382f9_2",+ "sharegpt_n7xJvjp_0",+ "ultrachat_317545"+ ],+ "gold_session_ids": [+ "answer_f5b33470_abs"+ ]+ },+ {+ "question_id": "0a995998",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_afa9873b_2",+ "answer_afa9873b_1",+ "answer_afa9873b_3",+ "85846900_2",+ "25091584_1",+ "439f9ed8_3",+ "ultrachat_331531",+ "fa46a10e",+ "ad3d0815_1",+ "64b847a7_1"+ ],+ "gold_session_ids": [+ "answer_afa9873b_2",+ "answer_afa9873b_3",+ "answer_afa9873b_1"+ ]+ },+ {+ "question_id": "6d550036",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.123151194370365,+ "mrr": 0.125,+ "retrieved_session_ids": [+ "2e4430d8_2",+ "sharegpt_zciCXP1_12",+ "sharegpt_nVi6tIt_0",+ "0bab76de",+ "sharegpt_UbMVpdp_93",+ "sharegpt_J7ZAFLd_0",+ "dae3906e_2",+ "answer_ec904b3c_1",+ "sharegpt_oXgiN7q_53",+ "25b2b29d"+ ],+ "gold_session_ids": [+ "answer_ec904b3c_1",+ "answer_ec904b3c_4",+ "answer_ec904b3c_3",+ "answer_ec904b3c_2"+ ]+ },+ {+ "question_id": "gpt4_59c863d7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9447176050778178,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_593bdffd_4",+ "answer_593bdffd_1",+ "answer_593bdffd_2",+ "eb47739f_2",+ "61d3fec4_2",+ "a7d014e4_1",+ "9ef698bc_2",+ "sharegpt_Opm4kTV_0",+ "ultrachat_233858",+ "answer_593bdffd_3"+ ],+ "gold_session_ids": [+ "answer_593bdffd_4",+ "answer_593bdffd_1",+ "answer_593bdffd_3",+ "answer_593bdffd_2"+ ]+ },+ {+ "question_id": "b5ef892d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9325210919548239,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a8b4290f_1",+ "answer_a8b4290f_3",+ "35dcacdc_2",+ "7294438c",+ "eb774a95",+ "answer_a8b4290f_2",+ "1a44346c_1",+ "ultrachat_565056",+ "ultrachat_367439",+ "dc968254_1"+ ],+ "gold_session_ids": [+ "answer_a8b4290f_3",+ "answer_a8b4290f_1",+ "answer_a8b4290f_2"+ ]+ },+ {+ "question_id": "e831120c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_86c505e7_2",+ "answer_86c505e7_1",+ "8fb3fe3a_1",+ "63388005",+ "sharegpt_CROow2Q_0",+ "a63ad8e3_2",+ "ba6b67af_2",+ "sharegpt_sDfu38Q_0",+ "db6ab13b_1",+ "773aebbd_4"+ ],+ "gold_session_ids": [+ "answer_86c505e7_1",+ "answer_86c505e7_2"+ ]+ },+ {+ "question_id": "3a704032",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c2204106_1",+ "answer_c2204106_2",+ "answer_c2204106_3",+ "0babf5b6_2",+ "7712aded",+ "ultrachat_501733",+ "014a5d36",+ "sharegpt_tzJ6yxo_0",+ "sharegpt_jlDfWk7_17",+ "be24e3db_2"+ ],+ "gold_session_ids": [+ "answer_c2204106_2",+ "answer_c2204106_3",+ "answer_c2204106_1"+ ]+ },+ {+ "question_id": "gpt4_d84a3211",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9619991470595832,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2880eb6c_4",+ "answer_2880eb6c_1",+ "answer_2880eb6c_2",+ "18d68208_1",+ "sharegpt_qHNCIfF_0",+ "4889cd54_1",+ "answer_2880eb6c_3",+ "700cc131_2",+ "83fa816b",+ "ultrachat_83526"+ ],+ "gold_session_ids": [+ "answer_2880eb6c_2",+ "answer_2880eb6c_4",+ "answer_2880eb6c_1",+ "answer_2880eb6c_3"+ ]+ },+ {+ "question_id": "aae3761f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_526354c8_1",+ "answer_526354c8_2",+ "38e81260_7",+ "answer_526354c8_3",+ "b192ae00_2",+ "2a86d0da_2",+ "sharegpt_ynLoh9N_0",+ "ultrachat_120599",+ "sharegpt_bKkRqZc_0",+ "41e778e4_1"+ ],+ "gold_session_ids": [+ "answer_526354c8_1",+ "answer_526354c8_3",+ "answer_526354c8_2"+ ]+ },+ {+ "question_id": "gpt4_f2262a51",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8385465274895063,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_55a6940c_3",+ "sharegpt_hChsWOp_128",+ "dd81b163_1",+ "answer_55a6940c_2",+ "ultrachat_139167",+ "answer_55a6940c_1",+ "ultrachat_109996",+ "ultrachat_268434",+ "f2ffaf25",+ "6f46ba40_1"+ ],+ "gold_session_ids": [+ "answer_55a6940c_3",+ "answer_55a6940c_1",+ "answer_55a6940c_2"+ ]+ },+ {+ "question_id": "dd2973ad",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f9de4602_2",+ "56266c38_1",+ "answer_f9de4602_1",+ "sharegpt_O6NO7Vo_9",+ "5b8ef613_4",+ "41743aae_1",+ "6551be60_3",+ "bc542ef7_2",+ "ultrachat_478199",+ "98c198fb"+ ],+ "gold_session_ids": [+ "answer_f9de4602_2",+ "answer_f9de4602_1"+ ]+ },+ {+ "question_id": "c4a1ceb8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_56d02cab_3",+ "answer_56d02cab_1",+ "answer_56d02cab_4",+ "answer_56d02cab_2",+ "ultrachat_441000",+ "40c77045",+ "1dc2b670_2",+ "cef33b28_1",+ "ba9f938b_2",+ "d0091194"+ ],+ "gold_session_ids": [+ "answer_56d02cab_3",+ "answer_56d02cab_4",+ "answer_56d02cab_2",+ "answer_56d02cab_1"+ ]+ },+ {+ "question_id": "gpt4_a56e767c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf9e3940_3",+ "answer_cf9e3940_1",+ "answer_cf9e3940_2",+ "d75245ea",+ "ultrachat_104129",+ "15998e39_2",+ "dac54d1e_1",+ "ultrachat_21691",+ "88c8df0e_3",+ "32da578d_2"+ ],+ "gold_session_ids": [+ "answer_cf9e3940_2",+ "answer_cf9e3940_1",+ "answer_cf9e3940_3"+ ]+ },+ {+ "question_id": "6cb6f249",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_404221",+ "answer_a4204937_1",+ "answer_a4204937_2",+ "9b6db1a9_2",+ "67620dbc_2",+ "32fc393b_2",+ "sharegpt_JTaoEJT_23",+ "f0e71553_5",+ "579b27c1",+ "sharegpt_WjpyexI_8"+ ],+ "gold_session_ids": [+ "answer_a4204937_2",+ "answer_a4204937_1"+ ]+ },+ {+ "question_id": "46a3abf7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9325210919548239,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c65042d7_2",+ "answer_c65042d7_3",+ "28eb86ab",+ "2917b130_1",+ "a95bedca",+ "answer_c65042d7_1",+ "sharegpt_v11Tpg9_58",+ "8acfa731",+ "e224317f_2",+ "ultrachat_558017"+ ],+ "gold_session_ids": [+ "answer_c65042d7_3",+ "answer_c65042d7_2",+ "answer_c65042d7_1"+ ]+ },+ {+ "question_id": "36b9f61e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef74281f_1",+ "answer_ef74281f_2",+ "answer_ef74281f_3",+ "a4ad48d9",+ "fc6f9c59",+ "85cd56c7_1",+ "sharegpt_Dvl0iLL_0",+ "3159942d_2",+ "56911dc5_4",+ "ultrachat_18673"+ ],+ "gold_session_ids": [+ "answer_ef74281f_2",+ "answer_ef74281f_1",+ "answer_ef74281f_3"+ ]+ },+ {+ "question_id": "28dc39ac",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8d015d9d_4",+ "answer_8d015d9d_1",+ "answer_8d015d9d_5",+ "answer_8d015d9d_2",+ "answer_8d015d9d_3",+ "42bcee92_2",+ "df003c93_3",+ "53985dc3",+ "ultrachat_452868",+ "5854eebc_1"+ ],+ "gold_session_ids": [+ "answer_8d015d9d_3",+ "answer_8d015d9d_5",+ "answer_8d015d9d_2",+ "answer_8d015d9d_4",+ "answer_8d015d9d_1"+ ]+ },+ {+ "question_id": "gpt4_2f8be40d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e7b0637e_1",+ "answer_e7b0637e_2",+ "answer_e7b0637e_3",+ "acda6a4e_4",+ "1d6e01e8",+ "a3332b4c_2",+ "81b971b8_2",+ "ultrachat_217382",+ "sharegpt_sScldBY_0",+ "dfe646a7_2"+ ],+ "gold_session_ids": [+ "answer_e7b0637e_2",+ "answer_e7b0637e_1",+ "answer_e7b0637e_3"+ ]+ },+ {+ "question_id": "2e6d26dc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fa526fc0_3",+ "answer_fa526fc0_4",+ "answer_fa526fc0_1",+ "answer_fa526fc0_2",+ "ultrachat_393167",+ "ultrachat_541982",+ "ultrachat_121666",+ "73f14139",+ "14a520c8_2",+ "sharegpt_IigLRfw_0"+ ],+ "gold_session_ids": [+ "answer_fa526fc0_4",+ "answer_fa526fc0_3",+ "answer_fa526fc0_1",+ "answer_fa526fc0_2"+ ]+ },+ {+ "question_id": "gpt4_15e38248",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7668091220635322,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8858d9dc_2",+ "answer_8858d9dc_4",+ "8c9bc932_2",+ "263efe64_1",+ "7784236e_2",+ "295fcaeb_2",+ "answer_8858d9dc_1",+ "1e7f3172_5",+ "sharegpt_XK3KWPT_0",+ "e697b2dd_4"+ ],+ "gold_session_ids": [+ "answer_8858d9dc_3",+ "answer_8858d9dc_1",+ "answer_8858d9dc_4",+ "answer_8858d9dc_2"+ ]+ },+ {+ "question_id": "88432d0a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7827141072039429,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_733e443a_3",+ "1c8832b4_2",+ "fa460b49_2",+ "ultrachat_359327",+ "f777f641",+ "answer_733e443a_2",+ "answer_733e443a_4",+ "answer_733e443a_1",+ "0934d99a",+ "0e250059_2"+ ],+ "gold_session_ids": [+ "answer_733e443a_3",+ "answer_733e443a_4",+ "answer_733e443a_2",+ "answer_733e443a_1"+ ]+ },+ {+ "question_id": "80ec1f4f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_990c8992_2",+ "answer_990c8992_3",+ "answer_990c8992_1",+ "ultrachat_318441",+ "b909542d_1",+ "sharegpt_nzDNOrq_29",+ "32387419_4",+ "f987da7e_3",+ "9a584663_2",+ "38aec47b_1"+ ],+ "gold_session_ids": [+ "answer_990c8992_2",+ "answer_990c8992_1",+ "answer_990c8992_3"+ ]+ },+ {+ "question_id": "d23cf73b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9267582364714126,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5a0d28f8_4",+ "answer_5a0d28f8_2",+ "95b7c3cd_1",+ "9705c90b",+ "answer_5a0d28f8_1",+ "answer_5a0d28f8_3",+ "c328cd4f",+ "be0962fb_1",+ "20ea9554_2",+ "9e21d6ab_2"+ ],+ "gold_session_ids": [+ "answer_5a0d28f8_4",+ "answer_5a0d28f8_2",+ "answer_5a0d28f8_3",+ "answer_5a0d28f8_1"+ ]+ },+ {+ "question_id": "gpt4_7fce9456",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9896062251871525,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a679a86a_4",+ "answer_a679a86a_5",+ "answer_a679a86a_3",+ "answer_a679a86a_1",+ "1da409cd_1",+ "answer_a679a86a_2",+ "541ecc45_3",+ "04328d1c",+ "288e7d30",+ "3753e55f_2"+ ],+ "gold_session_ids": [+ "answer_a679a86a_5",+ "answer_a679a86a_4",+ "answer_a679a86a_2",+ "answer_a679a86a_3",+ "answer_a679a86a_1"+ ]+ },+ {+ "question_id": "d682f1a2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c008e5df_3",+ "answer_c008e5df_1",+ "9de29219_1",+ "answer_c008e5df_2",+ "sharegpt_LFPdgVp_22",+ "sharegpt_ysMparQ_41",+ "6e2cca63_8",+ "sharegpt_WjpyexI_15",+ "7332a779_2",+ "sharegpt_JTaoEJT_23"+ ],+ "gold_session_ids": [+ "answer_c008e5df_1",+ "answer_c008e5df_2",+ "answer_c008e5df_3"+ ]+ },+ {+ "question_id": "7024f17c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a21f3697_2",+ "answer_a21f3697_3",+ "answer_a21f3697_1",+ "d7281662_2",+ "385683f0_1",+ "f8476198",+ "b5ef63be_2",+ "8c652eb0_1",+ "b7f69a7d_1",+ "445e06ce_2"+ ],+ "gold_session_ids": [+ "answer_a21f3697_1",+ "answer_a21f3697_2",+ "answer_a21f3697_3"+ ]+ },+ {+ "question_id": "gpt4_5501fe77",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_203bf3fa_3",+ "answer_203bf3fa_1",+ "answer_203bf3fa_2",+ "021c0d09_2",+ "9b083158_2",+ "ultrachat_163385",+ "542fa76e_1",+ "ultrachat_46240",+ "ultrachat_337168",+ "ultrachat_43091"+ ],+ "gold_session_ids": [+ "answer_203bf3fa_1",+ "answer_203bf3fa_3",+ "answer_203bf3fa_2"+ ]+ },+ {+ "question_id": "gpt4_2ba83207",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9047172294870751,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6a3b5c13_2",+ "33028509_2",+ "answer_6a3b5c13_1",+ "answer_6a3b5c13_3",+ "answer_6a3b5c13_4",+ "ea336da0",+ "ultrachat_224880",+ "41dc5d45_4",+ "325c2005",+ "ultrachat_568233"+ ],+ "gold_session_ids": [+ "answer_6a3b5c13_3",+ "answer_6a3b5c13_1",+ "answer_6a3b5c13_2",+ "answer_6a3b5c13_4"+ ]+ },+ {+ "question_id": "2318644b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_eaa8e3ef_2",+ "answer_eaa8e3ef_1",+ "c4ea8219_1",+ "ultrachat_135444",+ "sharegpt_iwcVyKX_0",+ "7ff89315_3",+ "6ec23dc5_1",+ "fc733b20_2",+ "sharegpt_36VQ7Bq_0",+ "7d02ef5c_5"+ ],+ "gold_session_ids": [+ "answer_eaa8e3ef_1",+ "answer_eaa8e3ef_2"+ ]+ },+ {+ "question_id": "2ce6a0f2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7365896932159578,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_901a6763_4",+ "600077b0_3",+ "answer_901a6763_1",+ "e4bacbcd_1",+ "answer_901a6763_2",+ "a267b11e",+ "ultrachat_36844",+ "f257aed3_1",+ "ultrachat_192754",+ "sharegpt_8W46coJ_3"+ ],+ "gold_session_ids": [+ "answer_901a6763_2",+ "answer_901a6763_4",+ "answer_901a6763_1",+ "answer_901a6763_3"+ ]+ },+ {+ "question_id": "gpt4_d12ceb0e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6546154994739983,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "73f4798f",+ "answer_2504635e_1",+ "0672cd7b_1",+ "answer_2504635e_2",+ "f0c6ddb9_2",+ "d8e33f5c_abs_2",+ "answer_2504635e_3",+ "a1166ecc_3",+ "31dc9d66_3",+ "33d16079_2"+ ],+ "gold_session_ids": [+ "answer_2504635e_3",+ "answer_2504635e_2",+ "answer_2504635e_1"+ ]+ },+ {+ "question_id": "00ca467f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9325210919548239,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_39900a0a_2",+ "answer_39900a0a_3",+ "7d1de770_2",+ "10857212_2",+ "4951c0b9_1",+ "answer_39900a0a_1",+ "e0fa8486_2",+ "4a5ee697_1",+ "ultrachat_360972",+ "426e7403_3"+ ],+ "gold_session_ids": [+ "answer_39900a0a_3",+ "answer_39900a0a_2",+ "answer_39900a0a_1"+ ]+ },+ {+ "question_id": "b3c15d39",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_gPmunOt_37",+ "answer_05d808e6_1",+ "answer_05d808e6_2",+ "d298714d_1",+ "sharegpt_inuIr9J_119",+ "sharegpt_V2j1zkI_0",+ "7bb01253",+ "5d4ff8db_1",+ "6be23caa",+ "bd00a24d_2"+ ],+ "gold_session_ids": [+ "answer_05d808e6_1",+ "answer_05d808e6_2"+ ]+ },+ {+ "question_id": "gpt4_31ff4165",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_02b63d04_1",+ "answer_02b63d04_3",+ "answer_02b63d04_4",+ "answer_02b63d04_5",+ "answer_02b63d04_2",+ "63f8f5a8",+ "2c185a2b_1",+ "2d0b80f1",+ "e2393b63_2",+ "35201d43"+ ],+ "gold_session_ids": [+ "answer_02b63d04_1",+ "answer_02b63d04_5",+ "answer_02b63d04_2",+ "answer_02b63d04_3",+ "answer_02b63d04_4"+ ]+ },+ {+ "question_id": "eeda8a6d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e5fea0e_2",+ "answer_3e5fea0e_1",+ "sharegpt_Caxpnd3_12",+ "sharegpt_an5DTCa_0",+ "cf7aad73_3",+ "be6eb800",+ "ultrachat_309574",+ "sharegpt_JsCcnH4_0",+ "a2609501_2",+ "ultrachat_313541"+ ],+ "gold_session_ids": [+ "answer_3e5fea0e_1",+ "answer_3e5fea0e_2"+ ]+ },+ {+ "question_id": "2788b940",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9550236580992476,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8f6b938d_4",+ "answer_8f6b938d_1",+ "answer_8f6b938d_2",+ "ff6e824b_1",+ "8de5b7cf_2",+ "sharegpt_dabXXRO_10",+ "ultrachat_410014",+ "answer_8f6b938d_3",+ "69f3386e",+ "544fe66c_2"+ ],+ "gold_session_ids": [+ "answer_8f6b938d_1",+ "answer_8f6b938d_3",+ "answer_8f6b938d_4",+ "answer_8f6b938d_2"+ ]+ },+ {+ "question_id": "60bf93ed",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6052602440527057,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "6414f676_4",+ "answer_e0956e0a_2",+ "ultrachat_7648",+ "388c5534_3",+ "4fe27c66_4",+ "answer_e0956e0a_1",+ "0e297c49",+ "a200b713_2",+ "3197603c",+ "ultrachat_259496"+ ],+ "gold_session_ids": [+ "answer_e0956e0a_1",+ "answer_e0956e0a_2"+ ]+ },+ {+ "question_id": "9d25d4e0",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fcff2dc4_2",+ "answer_fcff2dc4_1",+ "cd568521_3",+ "answer_fcff2dc4_3",+ "5f558dee_1",+ "380a9d4d_1",+ "464f3821",+ "9b38bda6_1",+ "sharegpt_eOKax2Z_0",+ "sharegpt_hRba933_47"+ ],+ "gold_session_ids": [+ "answer_fcff2dc4_2",+ "answer_fcff2dc4_1",+ "answer_fcff2dc4_3"+ ]+ },+ {+ "question_id": "129d1232",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5937751258332797,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_YfIEYo1_0",+ "answer_1de862d6_3",+ "ultrachat_274599",+ "sharegpt_V2j1zkI_0",+ "5e4bb245_2",+ "69a9211c_2",+ "answer_1de862d6_2",+ "ultrachat_88574",+ "answer_1de862d6_1",+ "bf3aebdb_1"+ ],+ "gold_session_ids": [+ "answer_1de862d6_1",+ "answer_1de862d6_3",+ "answer_1de862d6_2"+ ]+ },+ {+ "question_id": "60472f9c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e7fe8c8b_1",+ "answer_e7fe8c8b_2",+ "answer_e7fe8c8b_3",+ "33da50d0_1",+ "ultrachat_373250",+ "b1815b81",+ "ultrachat_5271",+ "sharegpt_LQbL4dQ_0",+ "7b74d32d_3",+ "sharegpt_paNxxAc_0"+ ],+ "gold_session_ids": [+ "answer_e7fe8c8b_1",+ "answer_e7fe8c8b_2",+ "answer_e7fe8c8b_3"+ ]+ },+ {+ "question_id": "gpt4_194be4b3",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9330795243082165,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3826dc55_3",+ "answer_3826dc55_4",+ "ultrachat_474217",+ "answer_3826dc55_5",+ "sharegpt_VO7dTG3_0",+ "answer_3826dc55_1",+ "answer_3826dc55_2",+ "212ce1f4_4",+ "407031a9_2",+ "ultrachat_385467"+ ],+ "gold_session_ids": [+ "answer_3826dc55_1",+ "answer_3826dc55_3",+ "answer_3826dc55_5",+ "answer_3826dc55_2",+ "answer_3826dc55_4"+ ]+ },+ {+ "question_id": "a9f6b44c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8854598815714874,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cc021f81_3",+ "d4912cd9",+ "answer_cc021f81_2",+ "1126be1e_1",+ "answer_cc021f81_1",+ "ca896bdc_1",+ "35bc3132_2",+ "bb107057_4",+ "ec8d10c7_2",+ "fdd0f680_1"+ ],+ "gold_session_ids": [+ "answer_cc021f81_2",+ "answer_cc021f81_3",+ "answer_cc021f81_1"+ ]+ },+ {+ "question_id": "d851d5ba",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5cdf9bd2_1",+ "answer_5cdf9bd2_3",+ "answer_5cdf9bd2_2",+ "answer_5cdf9bd2_4",+ "70764af1_4",+ "68ace657_2",+ "2a28999b",+ "sharegpt_dxirwR4_25",+ "561f3a6a_2",+ "47183c60_3"+ ],+ "gold_session_ids": [+ "answer_5cdf9bd2_2",+ "answer_5cdf9bd2_1",+ "answer_5cdf9bd2_3",+ "answer_5cdf9bd2_4"+ ]+ },+ {+ "question_id": "5a7937c8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4cef8a3c_2",+ "answer_4cef8a3c_1",+ "answer_4cef8a3c_3",+ "63c9e87c",+ "9bd49ce9_1",+ "40d799e8_2",+ "55cc1ba3_1",+ "ultrachat_288747",+ "ultrachat_564068",+ "df54d2be"+ ],+ "gold_session_ids": [+ "answer_4cef8a3c_3",+ "answer_4cef8a3c_1",+ "answer_4cef8a3c_2"+ ]+ },+ {+ "question_id": "gpt4_ab202e7f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8539316501572937,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_728deb4d_5",+ "answer_728deb4d_3",+ "answer_728deb4d_2",+ "dcafb5b3_5",+ "answer_728deb4d_4",+ "a08fbe88_1",+ "29bc69b3",+ "sharegpt_nc62Spr_7",+ "10541a2c_2",+ "ultrachat_464603"+ ],+ "gold_session_ids": [+ "answer_728deb4d_5",+ "answer_728deb4d_2",+ "answer_728deb4d_3",+ "answer_728deb4d_1",+ "answer_728deb4d_4"+ ]+ },+ {+ "question_id": "gpt4_e05b82a6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9550236580992476,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6350aa4f_2",+ "answer_6350aa4f_4",+ "answer_6350aa4f_3",+ "386f6df9",+ "ultrachat_43272",+ "aed6b1b6_1",+ "6a0d566a_2",+ "answer_6350aa4f_1",+ "503791f7_1",+ "88e01130_1"+ ],+ "gold_session_ids": [+ "answer_6350aa4f_1",+ "answer_6350aa4f_2",+ "answer_6350aa4f_3",+ "answer_6350aa4f_4"+ ]+ },+ {+ "question_id": "gpt4_731e37d7",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_826d51da_1",+ "answer_826d51da_4",+ "answer_826d51da_3",+ "answer_826d51da_2",+ "7e4aa7c2_1",+ "a93750ef_1",+ "b52f03a1_2",+ "ultrachat_109542",+ "267a912c",+ "sharegpt_vonEwUo_17"+ ],+ "gold_session_ids": [+ "answer_826d51da_3",+ "answer_826d51da_4",+ "answer_826d51da_2",+ "answer_826d51da_1"+ ]+ },+ {+ "question_id": "edced276",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_60e8941a_2",+ "answer_60e8941a_1",+ "1da409cd_1",+ "ultrachat_475154",+ "ultrachat_421943",+ "sharegpt_IFLajLc_0",+ "d89f6204",+ "59a6150d_3",+ "ce1abc42_1",+ "f584ba36_2"+ ],+ "gold_session_ids": [+ "answer_60e8941a_2",+ "answer_60e8941a_1"+ ]+ },+ {+ "question_id": "10d9b85a",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.21840743681816419,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "c51583cd_3",+ "84889496_1",+ "4e59fb02_2",+ "d576152e_1",+ "ultrachat_143866",+ "answer_e0585cb5_2",+ "cbd1fe79_2",+ "45823393_2",+ "3c8e7c0e_4",+ "aa6afba8"+ ],+ "gold_session_ids": [+ "answer_e0585cb5_2",+ "answer_e0585cb5_1"+ ]+ },+ {+ "question_id": "e3038f8c",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6084170171049966,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "a3d8e134_2",+ "85846900_2",+ "answer_b6018747_3",+ "sharegpt_hgGAUvu_0",+ "answer_b6018747_2",+ "answer_b6018747_1",+ "e3d4f89e_3",+ "answer_b6018747_4",+ "f2df24cd_4",+ "sharegpt_WxCMnMO_5"+ ],+ "gold_session_ids": [+ "answer_b6018747_2",+ "answer_b6018747_4",+ "answer_b6018747_1",+ "answer_b6018747_3"+ ]+ },+ {+ "question_id": "2b8f3739",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_23759615_1",+ "answer_23759615_2",+ "answer_23759615_3",+ "e9fb10e7_2",+ "ce6cae1b_3",+ "c075835c",+ "ec8691f5",+ "2154c810_1",+ "67d6f18f_1",+ "285525b2_1"+ ],+ "gold_session_ids": [+ "answer_23759615_2",+ "answer_23759615_3",+ "answer_23759615_1"+ ]+ },+ {+ "question_id": "1a8a66a6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5855700749881525,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2bd23659_1",+ "d6792df8",+ "answer_2bd23659_3",+ "22eebf8a_2",+ "5eed6833_2",+ "f829b530_3",+ "b01aafcb_2",+ "ebb5d262",+ "ultrachat_310819",+ "sharegpt_lsiJM0Y_0"+ ],+ "gold_session_ids": [+ "answer_2bd23659_3",+ "answer_2bd23659_2",+ "answer_2bd23659_4",+ "answer_2bd23659_1"+ ]+ },+ {+ "question_id": "c2ac3c61",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_923c0221_2",+ "answer_923c0221_1",+ "ultrachat_412857",+ "6009ba4e_5",+ "9f803df5_1",+ "sharegpt_z6pSZ19_0",+ "8e4e215b",+ "80dc42f7_1",+ "ultrachat_46891",+ "sharegpt_CQ3HZO9_53"+ ],+ "gold_session_ids": [+ "answer_923c0221_1",+ "answer_923c0221_2"+ ]+ },+ {+ "question_id": "bf659f65",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8278123145308894,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7726e7e9_1",+ "f2835879_2",+ "ultrachat_365864",+ "answer_7726e7e9_2",+ "258645ba_3",+ "sharegpt_inuIr9J_119",+ "answer_7726e7e9_3",+ "ultrachat_167088",+ "71619051_1",+ "32833e0b_3"+ ],+ "gold_session_ids": [+ "answer_7726e7e9_1",+ "answer_7726e7e9_3",+ "answer_7726e7e9_2"+ ]+ },+ {+ "question_id": "gpt4_372c3eed",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9134015924715543,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c5419d_1",+ "answer_35c5419d_3",+ "sharegpt_DRsWWMJ_0",+ "8d969f3e_3",+ "8dd672a3_3",+ "sharegpt_6VqlSoL_25",+ "sharegpt_6dPUCcK_69",+ "answer_35c5419d_2",+ "9a22ae03",+ "sharegpt_2hMUduQ_1"+ ],+ "gold_session_ids": [+ "answer_35c5419d_3",+ "answer_35c5419d_2",+ "answer_35c5419d_1"+ ]+ },+ {+ "question_id": "gpt4_2f91af09",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_669318cf_3",+ "answer_669318cf_2",+ "answer_669318cf_1",+ "51c82c8e_2",+ "sharegpt_d7bP5fb_0",+ "22b3af37_1",+ "ultrachat_296153",+ "49b78a55_1",+ "sharegpt_YmzANeT_0",+ "e7e416f5_2"+ ],+ "gold_session_ids": [+ "answer_669318cf_2",+ "answer_669318cf_1",+ "answer_669318cf_3"+ ]+ },+ {+ "question_id": "81507db6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8495981447350542,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da3c1266_5",+ "9071ba70_3",+ "answer_da3c1266_2",+ "ultrachat_254248",+ "2ca99347_2",+ "answer_da3c1266_3",+ "answer_da3c1266_4",+ "answer_da3c1266_1",+ "0f6b9698_2",+ "sharegpt_Caxpnd3_9"+ ],+ "gold_session_ids": [+ "answer_da3c1266_3",+ "answer_da3c1266_4",+ "answer_da3c1266_1",+ "answer_da3c1266_5",+ "answer_da3c1266_2"+ ]+ },+ {+ "question_id": "88432d0a_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9349366583346498,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_733e443a_abs_3",+ "answer_733e443a_abs_1",+ "752b10e6_2",+ "answer_733e443a_abs_4",+ "b2ecce8d_1",+ "31ca5871_1",+ "answer_733e443a_abs_2",+ "8e5b0424",+ "11f043cd_2",+ "sharegpt_conu8MS_14"+ ],+ "gold_session_ids": [+ "answer_733e443a_abs_1",+ "answer_733e443a_abs_3",+ "answer_733e443a_abs_2",+ "answer_733e443a_abs_4"+ ]+ },+ {+ "question_id": "80ec1f4f_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_990c8992_abs_1",+ "answer_990c8992_abs_3",+ "answer_990c8992_abs_2",+ "b341cf37_2",+ "b304b283_1",+ "sharegpt_7uSzi6w_0",+ "7eda4f2a_1",+ "ultrachat_169512",+ "ultrachat_330542",+ "sharegpt_Xf63XgQ_0"+ ],+ "gold_session_ids": [+ "answer_990c8992_abs_2",+ "answer_990c8992_abs_3",+ "answer_990c8992_abs_1"+ ]+ },+ {+ "question_id": "eeda8a6d_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e5fea0e_abs_2",+ "answer_3e5fea0e_abs_1",+ "9530353d_2",+ "d15d2899_4",+ "44ba5b71_2",+ "c8cc60d6_1",+ "628c958c_5",+ "da71721f",+ "ultrachat_124888",+ "sharegpt_l7gyE1o_17"+ ],+ "gold_session_ids": [+ "answer_3e5fea0e_abs_2",+ "answer_3e5fea0e_abs_1"+ ]+ },+ {+ "question_id": "60bf93ed_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.18457569677956817,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "cdf068b1_3",+ "a95d014c_3",+ "1e91cdf0",+ "sharegpt_6cz1Sq6_130",+ "6cd203f7_2",+ "0944933f_1",+ "755c2d32_1",+ "9a6c8d3c",+ "answer_e0956e0a_abs_2",+ "36d5bbde_1"+ ],+ "gold_session_ids": [+ "answer_e0956e0a_abs_2",+ "answer_e0956e0a_abs_1"+ ]+ },+ {+ "question_id": "edced276_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_60e8941a_abs_1",+ "answer_60e8941a_abs_2",+ "a7a64da9_3",+ "66138e8d_1",+ "76a6b5fb_1",+ "8b8e7cf8",+ "8e8288b3",+ "sharegpt_u1AM5RT_88",+ "b68a96c0_2",+ "7a2bdd15_2"+ ],+ "gold_session_ids": [+ "answer_60e8941a_abs_1",+ "answer_60e8941a_abs_2"+ ]+ },+ {+ "question_id": "gpt4_372c3eed_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8278123145308894,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c5419d_abs_1",+ "b0a9de7f_1",+ "31e254b5",+ "answer_35c5419d_abs_3",+ "ultrachat_563740",+ "85dec0b4_1",+ "answer_35c5419d_abs_2",+ "f002a741_2",+ "sharegpt_xEpi2SM_55",+ "sharegpt_JuqFCEz_71"+ ],+ "gold_session_ids": [+ "answer_35c5419d_abs_3",+ "answer_35c5419d_abs_1",+ "answer_35c5419d_abs_2"+ ]+ },+ {+ "question_id": "8a2466db",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_edb03329",+ "6dcf5fa0_1",+ "b67748d1_1",+ "6a5b5a78",+ "e3cc9931_1",+ "3392c0c7",+ "sharegpt_osTHjYi_0",+ "3d0c9f89_1",+ "6be54739_3",+ "898bef66"+ ],+ "gold_session_ids": [+ "answer_edb03329"+ ]+ },+ {+ "question_id": "06878be2",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_555dfb94",+ "3e9fce53_1",+ "ca929779_1",+ "a6074da9_1",+ "e58109ed_1",+ "465e7ef9",+ "be1c6872",+ "a67c1862_4",+ "sharegpt_cn3iUbp_7",+ "9863a1c8_3"+ ],+ "gold_session_ids": [+ "answer_555dfb94"+ ]+ },+ {+ "question_id": "75832dbd",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3562071871080222,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "sharegpt_xni9yYO_6",+ "c51583cd_1",+ "47ec1674_2",+ "6172cc72_1",+ "d49b1a24_1",+ "answer_d87a6ef8",+ "be753d18",+ "ultrachat_165893",+ "c7270a1b_1",+ "c884b67a_1"+ ],+ "gold_session_ids": [+ "answer_d87a6ef8"+ ]+ },+ {+ "question_id": "0edc2aef",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "8464304d_2",+ "f20e72e4_1",+ "answer_d586e9cd",+ "08ca1f31_2",+ "3c3a9042_3",+ "c3023a45_1",+ "6a747f2e",+ "c6c3a982_1",+ "53dc1394",+ "ultrachat_166065"+ ],+ "gold_session_ids": [+ "answer_d586e9cd"+ ]+ },+ {+ "question_id": "35a27287",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "58ae7c4f_3",+ "880ee13e",+ "a98ff421_1",+ "answer_9b182436",+ "02f2ab8e_2",+ "f863a648_3",+ "d20be4f3_1",+ "203bf3fa_2",+ "ultrachat_267380",+ "c9b1c309_4"+ ],+ "gold_session_ids": [+ "answer_9b182436"+ ]+ },+ {+ "question_id": "32260d93",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "573dc24b_1",+ "answer_0250ae1c",+ "6d3f5c68_4",+ "b84b7a2b_2",+ "1b26bdd5_5",+ "04bef53b",+ "d6bbe94d_3",+ "2d6a38fe_1",+ "5ace29d7_1",+ "a33cf36a_1"+ ],+ "gold_session_ids": [+ "answer_0250ae1c"+ ]+ },+ {+ "question_id": "195a1a1b",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "89a1800c_2",+ "87fff4b4_1",+ "ed9dad6c_2",+ "9f12daa1_1",+ "1b71c896_2",+ "ce79a004",+ "answer_6dc4305e",+ "sharegpt_WVr7Jiq_0",+ "d03b6a05_2",+ "eaebcc81_2"+ ],+ "gold_session_ids": [+ "answer_6dc4305e"+ ]+ },+ {+ "question_id": "afdc33df",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8549e5e0",+ "ad8d9f0f",+ "b6558bfb",+ "22f9f163_3",+ "d03b6a05_3",+ "d5128b51",+ "e58109ed_1",+ "96b6e5e8_2",+ "43b53ac4_1",+ "afa0c2e7_2"+ ],+ "gold_session_ids": [+ "answer_8549e5e0"+ ]+ },+ {+ "question_id": "caf03d32",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2fc6aabb",+ "bd6d0687",+ "ec806228",+ "a52e072b",+ "af631aa3_1",+ "6f0e184f",+ "sharegpt_kLAPfLd_0",+ "0bab76de",+ "sharegpt_3D7Qpuq_14",+ "4b1af06f_1"+ ],+ "gold_session_ids": [+ "answer_2fc6aabb"+ ]+ },+ {+ "question_id": "54026fce",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f7b22c66",+ "670c49e6",+ "6e92e417",+ "e6195229",+ "1c95e152_1",+ "1fcb4134_3",+ "ef5d4aa7_1",+ "1a9f5723_6",+ "eb91c46b",+ "ultrachat_491226"+ ],+ "gold_session_ids": [+ "answer_f7b22c66"+ ]+ },+ {+ "question_id": "06f04340",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3010299956639812,+ "mrr": 0.1111111111111111,+ "retrieved_session_ids": [+ "91223fd5_1",+ "42924d15",+ "728deb4d_4",+ "be2cb665",+ "b459f888_3",+ "66bfa1db",+ "8b156015_2",+ "6e6fbb6b",+ "answer_92d5f7cd",+ "de1f4aec_2"+ ],+ "gold_session_ids": [+ "answer_92d5f7cd"+ ]+ },+ {+ "question_id": "6b7dfb22",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6502d0f",+ "574df152_1",+ "56859d37",+ "9ee69ca6_3",+ "fc69fd44_3",+ "377f8fd9_1",+ "1c1f5ccc_4",+ "8489e4ce_1",+ "e7b0637e_3",+ "d8454317_3"+ ],+ "gold_session_ids": [+ "answer_f6502d0f"+ ]+ },+ {+ "question_id": "1a1907b4",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "e42e7876_5",+ "743f03a1_abs_1",+ "8e78fa70_1",+ "ae77c245",+ "4f3b36d4_5",+ "4e3da326_1",+ "answer_719502eb",+ "9585e0c6_1",+ "61d3fec4_1",+ "65b01f7c"+ ],+ "gold_session_ids": [+ "answer_719502eb"+ ]+ },+ {+ "question_id": "09d032c9",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "21ef2d05_1",+ "answer_b10dce5e",+ "e8bfacec_2",+ "af631aa3_2",+ "c7f7949e_3",+ "1fe34a92",+ "87e8ec02",+ "ultrachat_274584",+ "54c5a89c_2",+ "16ebc8f8_4"+ ],+ "gold_session_ids": [+ "answer_b10dce5e"+ ]+ },+ {+ "question_id": "38146c39",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "5806662d_1",+ "answer_772472c8",+ "42234f98",+ "15ed03f0_1",+ "d22c2f04_5",+ "28116d8a_1",+ "5263736d_1",+ "87aaee77",+ "467db12a",+ "381b80a3"+ ],+ "gold_session_ids": [+ "answer_772472c8"+ ]+ },+ {+ "question_id": "d24813b1",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "e08bf81f_4",+ "answer_7c0ade93",+ "3124bb28_1",+ "61a46ff7_3",+ "c2a34674_1",+ "084802f9_3",+ "b7e64b5c_1",+ "446173bb_1",+ "29e25716_1",+ "22ce6625_2"+ ],+ "gold_session_ids": [+ "answer_7c0ade93"+ ]+ },+ {+ "question_id": "57f827a0",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1bde8d3b",+ "e85e0eaf_1",+ "6e110a53_1",+ "d0f42e3f",+ "670ca40e",+ "2ef55f49_1",+ "f1dd798a_1",+ "762d288d_2",+ "ultrachat_18651",+ "f64c2982_1"+ ],+ "gold_session_ids": [+ "answer_1bde8d3b"+ ]+ },+ {+ "question_id": "95228167",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e613445",+ "3d2e66f2_2",+ "5d8c99d3_3",+ "b916da64_2",+ "7e17ae56_3",+ "8199fb3a_4",+ "084802f9_2",+ "9f80607d",+ "57d6e39b_2",+ "c34b6a1c_1"+ ],+ "gold_session_ids": [+ "answer_5e613445"+ ]+ },+ {+ "question_id": "505af2f5",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f3164f2c",+ "4f2d6be4_2",+ "9216c4d3_4",+ "af631aa3_1",+ "sharegpt_i59HUGf_0",+ "66bf4b03_1",+ "sharegpt_kjeGJvK_11",+ "0e193841_8",+ "7d52ca48_2",+ "ebeb4c51"+ ],+ "gold_session_ids": [+ "answer_f3164f2c"+ ]+ },+ {+ "question_id": "75f70248",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8ee04a2e",+ "2d74df23_4",+ "6ac43c5b_4",+ "95c9666f_5",+ "adb1f904",+ "c6ec5d11",+ "a394f6b5_1",+ "sharegpt_KdbXhTW_13",+ "ultrachat_32122",+ "c1d13b9e"+ ],+ "gold_session_ids": [+ "answer_8ee04a2e"+ ]+ },+ {+ "question_id": "d6233ab6",+ "question_type": "single-session-preference",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "e419b7c3_4",+ "2c81cdf8_3",+ "0e726047",+ "sharegpt_xni9yYO_6",+ "7af38385_2",+ "f916c63a_2",+ "87cfeb28_2",+ "cf7aad73_3",+ "32f28c7b_1",+ "ultrachat_237324"+ ],+ "gold_session_ids": [+ "answer_b0fac439"+ ]+ },+ {+ "question_id": "1da05512",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4d3be2ab",+ "aea71da3_2",+ "sharegpt_eSTYwMj_14",+ "f5a2e179_1",+ "f2565b3b_2",+ "6de70f46_1",+ "da1ea197_1",+ "e4adef2a",+ "6b93018e_2",+ "58de8f9b"+ ],+ "gold_session_ids": [+ "answer_4d3be2ab"+ ]+ },+ {+ "question_id": "fca70973",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "fc005760",+ "answer_a1e169b1",+ "5d5e80c5_1",+ "2f2884ad_1",+ "99b061c5_1",+ "4e3f0831_1",+ "312bf939",+ "f504b269",+ "09cff8ac",+ "762a8b45_2"+ ],+ "gold_session_ids": [+ "answer_a1e169b1"+ ]+ },+ {+ "question_id": "b6025781",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "f20c73f5_3",+ "answer_8414cc57",+ "2ef53f61_1",+ "1a693999_4",+ "612c8368_2",+ "d15d2899_5",+ "6446f6e6",+ "02592f97_1",+ "89d49715_2",+ "3a1ea336_1"+ ],+ "gold_session_ids": [+ "answer_8414cc57"+ ]+ },+ {+ "question_id": "a89d7624",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.43067655807339306,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "a479a7d6_4",+ "99f2f5b1_1",+ "082fc07f_1",+ "answer_8f15ac24",+ "39a056ae",+ "ultrachat_30787",+ "3cb41ab8_4",+ "ultrachat_383054",+ "b14d9f4e_2",+ "168776ed_2"+ ],+ "gold_session_ids": [+ "answer_8f15ac24"+ ]+ },+ {+ "question_id": "b0479f84",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_30f63ddb",+ "4f37e5ec",+ "386f6df9",+ "79019170_1",+ "0cf86cbe_2",+ "4b563ae6_1",+ "abc2ac45",+ "fbfda981",+ "818545a6_2",+ "ddf41a24"+ ],+ "gold_session_ids": [+ "answer_30f63ddb"+ ]+ },+ {+ "question_id": "1d4e3b97",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "fb2368b8_3",+ "answer_e6b6353d",+ "ecb24dd8_2",+ "809cabca_1",+ "044e200f_2",+ "5cd6ab1b",+ "8c7ae899_2",+ "03633a51",+ "c51583cd_3",+ "fd26f9f4_1"+ ],+ "gold_session_ids": [+ "answer_e6b6353d"+ ]+ },+ {+ "question_id": "07b6f563",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d03098f9",+ "55c267aa",+ "4bebc782_1",+ "5e7b1c98_1",+ "fd18e0dd_2",+ "66ab6260",+ "9b2f4dd7_2",+ "0ac1a6f9_2",+ "6af947bf_1",+ "bbccfe79"+ ],+ "gold_session_ids": [+ "answer_d03098f9"+ ]+ },+ {+ "question_id": "1c0ddc50",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8da8c7e0",+ "2f0c1f4e",+ "2aa70c9c_1",+ "b33e89b5_1",+ "c7ca6dff",+ "a0aa5035",+ "bdf33c73_2",+ "0a7d5bf6_2",+ "2566382f_2",+ "b0c6fffa"+ ],+ "gold_session_ids": [+ "answer_8da8c7e0"+ ]+ },+ {+ "question_id": "0a34ad58",+ "question_type": "single-session-preference",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cebb7159",+ "9ca2a195_1",+ "58ad78d6_2",+ "90342357_3",+ "5c0574b1",+ "b4c5414d_2",+ "51fe163e_2",+ "f7e682c3_5",+ "dd3d8e73_1",+ "f849dd04"+ ],+ "gold_session_ids": [+ "answer_cebb7159"+ ]+ },+ {+ "question_id": "d3ab962e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5237bb0b_2",+ "answer_5237bb0b_1",+ "ultrachat_248741",+ "3329e5e8_1",+ "be679e09",+ "ultrachat_564044",+ "ultrachat_164834",+ "aa235649_1",+ "92f1ea4d_1",+ "24e44a64"+ ],+ "gold_session_ids": [+ "answer_5237bb0b_2",+ "answer_5237bb0b_1"+ ]+ },+ {+ "question_id": "2311e44b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bf633415_1",+ "answer_bf633415_2",+ "ultrachat_458191",+ "9b3611d7_1",+ "1e53c034",+ "sharegpt_hi2xiJT_14",+ "eb3eb62b_2",+ "sharegpt_ELyN256_13",+ "77753dd9_1",+ "2714885f_1"+ ],+ "gold_session_ids": [+ "answer_bf633415_2",+ "answer_bf633415_1"+ ]+ },+ {+ "question_id": "cc06de0d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4fb01417_1",+ "answer_4fb01417_2",+ "91c71874_2",+ "01551a39_2",+ "4ad63a03_3",+ "92d74e73",+ "55033f6f",+ "53ae82be_3",+ "9863a1c8_3",+ "d5eab084_2"+ ],+ "gold_session_ids": [+ "answer_4fb01417_2",+ "answer_4fb01417_1"+ ]+ },+ {+ "question_id": "a11281a2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c69ee1f9_2",+ "answer_c69ee1f9_1",+ "bb2180e9_3",+ "4b1af06f_2",+ "ultrachat_55839",+ "d95e64a7_4",+ "ultrachat_103691",+ "sharegpt_u3MVGYG_3",+ "ultrachat_234016",+ "7a88956e"+ ],+ "gold_session_ids": [+ "answer_c69ee1f9_2",+ "answer_c69ee1f9_1"+ ]+ },+ {+ "question_id": "4f54b7c9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_50940cb7_2",+ "answer_50940cb7_1",+ "dde142b2_1",+ "22eebf8a_2",+ "663337ce_1",+ "sharegpt_NPdQIWB_0",+ "8a834aee_1",+ "aafe556c",+ "b89f68c4_1",+ "a6c0f032_3"+ ],+ "gold_session_ids": [+ "answer_50940cb7_2",+ "answer_50940cb7_1"+ ]+ },+ {+ "question_id": "85fa3a3f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dcb18b9b_2",+ "answer_dcb18b9b_1",+ "sharegpt_O1gqpsW_0",+ "a4b223a7_1",+ "421605fc_1",+ "989ad9e6_1",+ "22b3af37_2",+ "93ff2f73",+ "0d66562c_1",+ "e4881cbd_1"+ ],+ "gold_session_ids": [+ "answer_dcb18b9b_2",+ "answer_dcb18b9b_1"+ ]+ },+ {+ "question_id": "9aaed6a3",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_353d3c6d_2",+ "c846422a_1",+ "answer_353d3c6d_1",+ "3b715074",+ "c95f2a09_2",+ "819dfad7_2",+ "sharegpt_Le00ERV_0",+ "ultrachat_216636",+ "sharegpt_HpdNUlz_9",+ "a861a671_1"+ ],+ "gold_session_ids": [+ "answer_353d3c6d_2",+ "answer_353d3c6d_1"+ ]+ },+ {+ "question_id": "1f2b8d4f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8454588_1",+ "answer_d8454588_2",+ "b5c147c7_2",+ "be1c6872",+ "8d015d9d_5",+ "9ee4da12_1",+ "628c958c_1",+ "ultrachat_401506",+ "0bd59f15_3",+ "ultrachat_571561"+ ],+ "gold_session_ids": [+ "answer_d8454588_1",+ "answer_d8454588_2"+ ]+ },+ {+ "question_id": "e6041065",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4eb6d671_2",+ "answer_4eb6d671_1",+ "b73a4d04_1",+ "2a86d0da_2",+ "bdcee74e_3",+ "ultrachat_407355",+ "ec616e7e_4",+ "49f2e396_2",+ "4a006b77",+ "3da98832_2"+ ],+ "gold_session_ids": [+ "answer_4eb6d671_2",+ "answer_4eb6d671_1"+ ]+ },+ {+ "question_id": "51c32626",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_58820c75_2",+ "answer_58820c75_1",+ "sharegpt_TRck0h4_0",+ "c051e740_2",+ "sharegpt_hzwc0HI_13",+ "fb62ade8",+ "ultrachat_260793",+ "ultrachat_301579",+ "ultrachat_488942",+ "f29704ac_2"+ ],+ "gold_session_ids": [+ "answer_58820c75_1",+ "answer_58820c75_2"+ ]+ },+ {+ "question_id": "d905b33f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_85a77c48_2",+ "answer_85a77c48_1",+ "3301f749_1",+ "8cfa91c3_2",+ "368276ab_3",+ "7613d5ec_2",+ "70d0ae5d",+ "2d364fb0",+ "6423d6a4_2",+ "a9b7f49e_1"+ ],+ "gold_session_ids": [+ "answer_85a77c48_2",+ "answer_85a77c48_1"+ ]+ },+ {+ "question_id": "7405e8b1",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_80323f3f_2",+ "answer_80323f3f_1",+ "f2f2a606_3",+ "a6b00510_4",+ "6c1c5bc3_2",+ "sharegpt_uiECyCD_93",+ "sharegpt_2pwdGxJ_1",+ "89568a93_1",+ "8eeb4704",+ "7d7f11d0_1"+ ],+ "gold_session_ids": [+ "answer_80323f3f_1",+ "answer_80323f3f_2"+ ]+ },+ {+ "question_id": "f35224e0",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e9bb9500_2",+ "answer_e9bb9500_1",+ "8e608386",+ "96da07f9_4",+ "42234f98",+ "176594fa_3",+ "sharegpt_gGriMhQ_181",+ "sharegpt_d8dUtMG_0",+ "fd7becaf_1",+ "4b285f83"+ ],+ "gold_session_ids": [+ "answer_e9bb9500_2",+ "answer_e9bb9500_1"+ ]+ },+ {+ "question_id": "6456829e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_743f03a1_2",+ "answer_743f03a1_1",+ "c44d1533_2",+ "ultrachat_555576",+ "5bed6828",+ "sharegpt_LxQSGQ9_0",+ "sharegpt_3xXQihW_45",+ "385683f0_3",+ "793ff37a",+ "47b924c1_1"+ ],+ "gold_session_ids": [+ "answer_743f03a1_2",+ "answer_743f03a1_1"+ ]+ },+ {+ "question_id": "a4996e51",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_feb5f98a_1",+ "answer_feb5f98a_2",+ "252715f8_3",+ "b1d9eb66_3",+ "726e9a1f_3",+ "ef0341d4_2",+ "4f2d6be4_2",+ "sharegpt_tvcY0lQ_0",+ "2ca99347_2",+ "ultrachat_295714"+ ],+ "gold_session_ids": [+ "answer_feb5f98a_1",+ "answer_feb5f98a_2"+ ]+ },+ {+ "question_id": "3c1045c8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c8cc60d6_2",+ "ultrachat_20893",+ "answer_c8cc60d6_1",+ "ultrachat_251919",+ "sharegpt_GYqnAhC_190",+ "ultrachat_484591",+ "sharegpt_xY3p1o7_127",+ "sharegpt_NTHvrGV_0",+ "ef71a35e_1",+ "4f3b36d4_3"+ ],+ "gold_session_ids": [+ "answer_c8cc60d6_2",+ "answer_c8cc60d6_1"+ ]+ },+ {+ "question_id": "60036106",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e552e1f9_1",+ "answer_e552e1f9_2",+ "sharegpt_moARQa7_0",+ "763f028a",+ "sharegpt_ckvrgF1_99",+ "sharegpt_mtS0qOD_65",+ "e2cdd030",+ "ultrachat_262356",+ "10857212_2",+ "c575291e_2"+ ],+ "gold_session_ids": [+ "answer_e552e1f9_1",+ "answer_e552e1f9_2"+ ]+ },+ {+ "question_id": "681a1674",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3be95d43_2",+ "answer_3be95d43_1",+ "d81e853b",+ "ultrachat_64650",+ "cbdbcfbc_6",+ "9146213d_3",+ "d5b3cf54_2",+ "sharegpt_2wTEFHv_47",+ "f1cf6f00_2",+ "sharegpt_8Kzmn95_0"+ ],+ "gold_session_ids": [+ "answer_3be95d43_1",+ "answer_3be95d43_2"+ ]+ },+ {+ "question_id": "e25c3b8d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6702277b_2",+ "answer_6702277b_1",+ "099c1b6c_3",+ "824634d9_1",+ "sharegpt_gQaSoiF_13",+ "c04c626c_1",+ "90fea364_1",+ "ff201786_1",+ "sharegpt_rtgucSC_0",+ "20fa75e8_3"+ ],+ "gold_session_ids": [+ "answer_6702277b_1",+ "answer_6702277b_2"+ ]+ },+ {+ "question_id": "4adc0475",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8065735963827293,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6efce493_2",+ "sharegpt_7nYPy2P_29",+ "ultrachat_186558",+ "183d2822_1",+ "sharegpt_wcTU09Q_31",+ "f82637ed_1",+ "bb27df5e_2",+ "answer_6efce493_1",+ "29e283f4_1",+ "ultrachat_30883"+ ],+ "gold_session_ids": [+ "answer_6efce493_1",+ "answer_6efce493_2"+ ]+ },+ {+ "question_id": "4bc144e2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9ef115d4_1",+ "answer_9ef115d4_2",+ "2e2ae792_1",+ "42bcee92_2",+ "32a203b4_1",+ "60207b41_3",+ "ultrachat_380790",+ "7b13b4d6_2",+ "099c1b6c_3",+ "sharegpt_lwNINvp_0"+ ],+ "gold_session_ids": [+ "answer_9ef115d4_1",+ "answer_9ef115d4_2"+ ]+ },+ {+ "question_id": "ef66a6e5",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8065735963827293,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f7fd1029_1",+ "afb58af0",+ "ultrachat_61361",+ "41e778e4_2",+ "295645f5_1",+ "sharegpt_P8MHt59_0",+ "147ab7e9_6",+ "answer_f7fd1029_2",+ "7c08d74b_1",+ "4c08398b_1"+ ],+ "gold_session_ids": [+ "answer_f7fd1029_2",+ "answer_f7fd1029_1"+ ]+ },+ {+ "question_id": "5025383b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_688f9a3f_2",+ "064f42ee_1",+ "sharegpt_Yuw2LhQ_4",+ "cf8e352f_4",+ "answer_688f9a3f_1",+ "2d21f921",+ "bab41bb6_5",+ "f40fa3ed",+ "de293134_1",+ "d10f0307_1"+ ],+ "gold_session_ids": [+ "answer_688f9a3f_1",+ "answer_688f9a3f_2"+ ]+ },+ {+ "question_id": "a1cc6108",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8065735963827293,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_17dc2f5b_2",+ "31e254b5",+ "sharegpt_jpiaPoJ_29",+ "fb328ace_1",+ "sharegpt_wrN9uUo_43",+ "001cefa7_2",+ "sharegpt_9eEV9T3_8",+ "answer_17dc2f5b_1",+ "ac72b02a_2",+ "ultrachat_152948"+ ],+ "gold_session_ids": [+ "answer_17dc2f5b_2",+ "answer_17dc2f5b_1"+ ]+ },+ {+ "question_id": "9ee3ecd6",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_66c23110_1",+ "answer_66c23110_2",+ "fa0fa74d_2",+ "fd56c5bd_2",+ "ultrachat_282862",+ "sharegpt_V2j1zkI_4",+ "ultrachat_175931",+ "a697b169_1",+ "7a7a4bf0_4",+ "f550a835_1"+ ],+ "gold_session_ids": [+ "answer_66c23110_1",+ "answer_66c23110_2"+ ]+ },+ {+ "question_id": "3fdac837",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "c85ef24e_1",+ "answer_419d21d5_1",+ "answer_419d21d5_2",+ "d1165be3_2",+ "a98ff421_7",+ "88c8df0e_3",+ "30c92c0c_1",+ "ultrachat_526748",+ "09d021e7_1",+ "daa1fe97_1"+ ],+ "gold_session_ids": [+ "answer_419d21d5_2",+ "answer_419d21d5_1"+ ]+ },+ {+ "question_id": "91b15a6e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5404a208_1",+ "answer_5404a208_2",+ "6169ef55_1",+ "7cfeb89c_2",+ "c1e170f0_2",+ "sharegpt_7r7RS6F_0",+ "sharegpt_1xq5lFg_0",+ "4c039bd7_1",+ "409dfa26_1",+ "d7cffd42_2"+ ],+ "gold_session_ids": [+ "answer_5404a208_1",+ "answer_5404a208_2"+ ]+ },+ {+ "question_id": "27016adc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a37bdf22_1",+ "answer_a37bdf22_2",+ "1a555998",+ "fa858208_2",+ "ultrachat_566359",+ "27ce9f13_2",+ "ultrachat_284316",+ "f1597b1c",+ "dd25eeb5_8",+ "ff5e0db0_1"+ ],+ "gold_session_ids": [+ "answer_a37bdf22_2",+ "answer_a37bdf22_1"+ ]+ },+ {+ "question_id": "720133ac",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c9dfeaea_1",+ "answer_c9dfeaea_2",+ "4c128d97_2",+ "ultrachat_508711",+ "e200d96c_3",+ "ultrachat_479833",+ "sharegpt_sG53ibg_116",+ "10a598b1_1",+ "87bce563",+ "sharegpt_Km6yx9W_13"+ ],+ "gold_session_ids": [+ "answer_c9dfeaea_1",+ "answer_c9dfeaea_2"+ ]+ },+ {+ "question_id": "77eafa52",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_33c251f0_1",+ "answer_33c251f0_2",+ "sharegpt_qEukY6n_14",+ "41a87f13_1",+ "f87fea58_1",+ "fc9a62c0_1",+ "sharegpt_C9GjDsF_0",+ "7b13b4d6_2",+ "sharegpt_nCTZBKf_0",+ "cd568521_3"+ ],+ "gold_session_ids": [+ "answer_33c251f0_2",+ "answer_33c251f0_1"+ ]+ },+ {+ "question_id": "8979f9ec",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35e36341_1",+ "answer_35e36341_2",+ "bcc9382a_2",+ "1064ed80_2",+ "a6b4c4c2_1",+ "ultrachat_562471",+ "052d7def",+ "91c71874_1",+ "f18ebe36_7",+ "4c967baa_1"+ ],+ "gold_session_ids": [+ "answer_35e36341_1",+ "answer_35e36341_2"+ ]+ },+ {+ "question_id": "0100672e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35c9798c_2",+ "answer_35c9798c_1",+ "b167a4a8",+ "326e86e0_3",+ "a4b223a7_1",+ "562addbd_1",+ "091aa510_2",+ "8237de6a_2",+ "07313722_2",+ "c694fe7b_1"+ ],+ "gold_session_ids": [+ "answer_35c9798c_2",+ "answer_35c9798c_1"+ ]+ },+ {+ "question_id": "a96c20ee",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef84b994_1",+ "sharegpt_00qHEQ6_0",+ "answer_ef84b994_2",+ "c69eb8ee_4",+ "ultrachat_259222",+ "18cfdd41_1",+ "ultrachat_286736",+ "9ca3534f_1",+ "dff89d9d_1",+ "e4e64e80_6"+ ],+ "gold_session_ids": [+ "answer_ef84b994_1",+ "answer_ef84b994_2"+ ]+ },+ {+ "question_id": "92a0aa75",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "a5854a8d_2",+ "answer_6cb8f792_2",+ "sharegpt_hPax8pw_11",+ "sharegpt_9l7gjYP_17",+ "1c0921ee_3",+ "85eb2b94_2",+ "ultrachat_61473",+ "173e2e47",+ "ultrachat_178564",+ "6da0bba2_1"+ ],+ "gold_session_ids": [+ "answer_6cb8f792_1",+ "answer_6cb8f792_2"+ ]+ },+ {+ "question_id": "3fe836c9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1bb63ea5_1",+ "c81897cd_2",+ "f0d7d364",+ "answer_1bb63ea5_2",+ "0a42783a",+ "dda70510_1",+ "ff49b5a5_3",+ "ultrachat_258020",+ "ultrachat_199846",+ "ultrachat_328173"+ ],+ "gold_session_ids": [+ "answer_1bb63ea5_2",+ "answer_1bb63ea5_1"+ ]+ },+ {+ "question_id": "1c549ce4",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4cb841a8_1",+ "answer_4cb841a8_2",+ "sharegpt_miOKGt1_13",+ "f1e4daa8_2",+ "35f052f9_2",+ "f999b05c_1",+ "809cbce9_1",+ "ddcd2c65_1",+ "sharegpt_Ab5RUMY_0",+ "d743153b_2"+ ],+ "gold_session_ids": [+ "answer_4cb841a8_1",+ "answer_4cb841a8_2"+ ]+ },+ {+ "question_id": "6c49646a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cc1ced21_1",+ "9f220c66_1",+ "ultrachat_565061",+ "answer_cc1ced21_2",+ "d3065d85",+ "6155addd_2",+ "1a24ef7c_5",+ "2aeb1268",+ "ultrachat_298957",+ "8df3019b_1"+ ],+ "gold_session_ids": [+ "answer_cc1ced21_2",+ "answer_cc1ced21_1"+ ]+ },+ {+ "question_id": "1192316e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e184e4c3_2",+ "answer_e184e4c3_1",+ "c5db849c_1",+ "f6ed7fa9_3",+ "574df152_3",+ "21ef2d05_1",+ "55f7cc4f_3",+ "6cd7234e_2",+ "sharegpt_5HGh4Yy_0",+ "6f8cd952"+ ],+ "gold_session_ids": [+ "answer_e184e4c3_2",+ "answer_e184e4c3_1"+ ]+ },+ {+ "question_id": "0ea62687",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dc5e537d_1",+ "answer_dc5e537d_2",+ "d1ea55c0_1",+ "d3ee5958_1",+ "e22d6aef_1",+ "5dce60dd",+ "26eedae1_1",+ "988c6b0e",+ "50904946_2",+ "b9e47a0f_2"+ ],+ "gold_session_ids": [+ "answer_dc5e537d_1",+ "answer_dc5e537d_2"+ ]+ },+ {+ "question_id": "67e0d0f2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3a5010af_2",+ "answer_3a5010af_1",+ "2252d3dd",+ "ultrachat_217333",+ "sharegpt_uD7CBng_30",+ "6868c94f",+ "b192ae00_2",+ "sharegpt_RziX6kj_0",+ "298382a6_2",+ "ultrachat_25863"+ ],+ "gold_session_ids": [+ "answer_3a5010af_2",+ "answer_3a5010af_1"+ ]+ },+ {+ "question_id": "bb7c3b45",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_de64539a_1",+ "answer_de64539a_2",+ "3891baea_4",+ "f3d43a12_3",+ "4f2d6be4_1",+ "0c061750_1",+ "a51dc9ca",+ "13b6b7ab_1",+ "0287c177_1",+ "sharegpt_l02agjs_0"+ ],+ "gold_session_ids": [+ "answer_de64539a_1",+ "answer_de64539a_2"+ ]+ },+ {+ "question_id": "ba358f49",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cbd08e3c_1",+ "fb117eb0_2",+ "f6859b48_2",+ "2a2ea005_1",+ "385683f0_2",+ "sharegpt_c2LF3xK_12",+ "sharegpt_IqVsDp7_89",+ "fb0f2ac2",+ "31419155",+ "4f838497_2"+ ],+ "gold_session_ids": [+ "answer_cbd08e3c_1",+ "answer_cbd08e3c_2"+ ]+ },+ {+ "question_id": "61f8c8f8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_872e8da2_1",+ "answer_872e8da2_2",+ "2154c810_1",+ "1a24ef7c_5",+ "6e110a53_1",+ "sharegpt_7jGo66i_99",+ "76ea58ed_1",+ "7b88c38b_2",+ "8991014d_1",+ "58335f60_2"+ ],+ "gold_session_ids": [+ "answer_872e8da2_2",+ "answer_872e8da2_1"+ ]+ },+ {+ "question_id": "60159905",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_75eca223_2",+ "answer_75eca223_1",+ "21f8f481_3",+ "6ade84f0",+ "c351fa3b_1",+ "15998e39_4",+ "b7917b32",+ "02eae87c_5",+ "b3ef2cd4_1",+ "b4f94171_2"+ ],+ "gold_session_ids": [+ "answer_75eca223_2",+ "answer_75eca223_1"+ ]+ },+ {+ "question_id": "ef9cf60a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_87e3a1cb_1",+ "answer_87e3a1cb_2",+ "2357de36_2",+ "7b88c38b_1",+ "a89815bf",+ "d677eb06",+ "71027306_1",+ "sharegpt_Ha6kIaj_0",+ "sharegpt_Lk4UN8n_159",+ "ultrachat_247949"+ ],+ "gold_session_ids": [+ "answer_87e3a1cb_1",+ "answer_87e3a1cb_2"+ ]+ },+ {+ "question_id": "73d42213",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1881e7db_2",+ "answer_1881e7db_1",+ "3cb41ab8_1",+ "sharegpt_ab6IEma_0",+ "sharegpt_YlgTw57_486",+ "9614d065_1",+ "02e9f7e3_3",+ "sharegpt_ndynIbS_0",+ "a1251fef_2",+ "a13abca5_1"+ ],+ "gold_session_ids": [+ "answer_1881e7db_1",+ "answer_1881e7db_2"+ ]+ },+ {+ "question_id": "bc149d6b",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_92147866_1",+ "7211c69e",+ "answer_92147866_2",+ "9806001e",+ "4f435d20_2",+ "9bfd836a",+ "05be3ff0_3",+ "dcb18b9b_1",+ "93a7a1bb_1",+ "ultrachat_13563"+ ],+ "gold_session_ids": [+ "answer_92147866_1",+ "answer_92147866_2"+ ]+ },+ {+ "question_id": "099778bb",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_80d6d664_1",+ "answer_80d6d664_2",+ "b1793bba",+ "95ce0e21_1",+ "sharegpt_TMWJOGD_21",+ "5b014f9b_1",+ "sharegpt_O8djord_90",+ "4acdd888",+ "sharegpt_1dQ2mgC_0",+ "ultrachat_60492"+ ],+ "gold_session_ids": [+ "answer_80d6d664_2",+ "answer_80d6d664_1"+ ]+ },+ {+ "question_id": "09ba9854",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96c743d0_2",+ "answer_96c743d0_1",+ "f7644391_2",+ "8c63238e",+ "32e6c308",+ "5faf5962_3",+ "69a42467_2",+ "e760dc71_1",+ "ultrachat_447648",+ "2bd23659_1"+ ],+ "gold_session_ids": [+ "answer_96c743d0_2",+ "answer_96c743d0_1"+ ]+ },+ {+ "question_id": "d6062bb9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_23f3a657_2",+ "answer_23f3a657_1",+ "86298245_2",+ "5b6aedac",+ "65d28641_2",+ "sharegpt_xuhzAgG_19",+ "f7644391_2",+ "59c704ad_2",+ "5c95d0d0",+ "sharegpt_PjFSCFK_9"+ ],+ "gold_session_ids": [+ "answer_23f3a657_2",+ "answer_23f3a657_1"+ ]+ },+ {+ "question_id": "157a136e",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8de18468_2",+ "377f8fd9_4",+ "298bf7a3_1",+ "7746ba1b_1",+ "ultrachat_506827",+ "answer_8de18468_1",+ "ultrachat_255132",+ "9b3a7f2c_4",+ "6715a0c6_1",+ "7614c21f_2"+ ],+ "gold_session_ids": [+ "answer_8de18468_2",+ "answer_8de18468_1"+ ]+ },+ {+ "question_id": "c18a7dc8",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9d5a389d",+ "answer_2e2085fa_1",+ "ultrachat_353377",+ "ultrachat_418857",+ "2d74df23_1",+ "sharegpt_YPC6Oll_18",+ "ultrachat_463016",+ "f716720c_3",+ "5022812c",+ "ultrachat_415226"+ ],+ "gold_session_ids": [+ "answer_2e2085fa_2",+ "answer_2e2085fa_1"+ ]+ },+ {+ "question_id": "a3332713",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_16ece55f_2",+ "answer_16ece55f_1",+ "67388dfc_2",+ "1c8a264a_4",+ "117098d9_3",+ "62be624f",+ "sharegpt_7uSzi6w_2",+ "sharegpt_bYymtlW_0",+ "202c6329_1",+ "33994682_3"+ ],+ "gold_session_ids": [+ "answer_16ece55f_2",+ "answer_16ece55f_1"+ ]+ },+ {+ "question_id": "55241a1f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fa08bf49_1",+ "answer_fa08bf49_2",+ "9c9a22d8",+ "d117e2da",+ "46e4a8eb_2",+ "76491ee4_1",+ "2aeb1268",+ "98c83fe4",+ "ultrachat_348345",+ "60153a02_4"+ ],+ "gold_session_ids": [+ "answer_fa08bf49_1",+ "answer_fa08bf49_2"+ ]+ },+ {+ "question_id": "a08a253f",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_47152166_1",+ "answer_47152166_2",+ "771570c5_1",+ "d79173aa_3",+ "6e5130ed_2",+ "28209b6a_4",+ "b1d9eb66_4",+ "798fa5f2",+ "4524c250_2",+ "c7fc2bda_1"+ ],+ "gold_session_ids": [+ "answer_47152166_2",+ "answer_47152166_1"+ ]+ },+ {+ "question_id": "f0e564bc",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cfcf5340_2",+ "answer_cfcf5340_1",+ "798e4ba3_2",+ "6dfc99bf_2",+ "932a97c6",+ "9f35ac68",+ "bc1f1051_1",+ "sharegpt_W3YFpVK_7",+ "683fdb17",+ "658150c4"+ ],+ "gold_session_ids": [+ "answer_cfcf5340_1",+ "answer_cfcf5340_2"+ ]+ },+ {+ "question_id": "078150f1",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_254d8b09_1",+ "answer_254d8b09_2",+ "61b27d67_3",+ "b18ba8d0_1",+ "ultrachat_202549",+ "203a48bb_1",+ "ultrachat_276164",+ "e4c97ea0_1",+ "08256f84_1",+ "2655efcc_1"+ ],+ "gold_session_ids": [+ "answer_254d8b09_1",+ "answer_254d8b09_2"+ ]+ },+ {+ "question_id": "8cf4d046",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e2278b24_2",+ "sharegpt_WjpyexI_15",+ "answer_e2278b24_1",+ "ultrachat_554855",+ "sharegpt_P1Okivz_9",+ "sharegpt_XsIUcaD_89",+ "6c6c8df7",+ "410ec447",+ "sharegpt_kyCtrXT_0",+ "7a5c32b0_3"+ ],+ "gold_session_ids": [+ "answer_e2278b24_1",+ "answer_e2278b24_2"+ ]+ },+ {+ "question_id": "a346bb18",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4934b2d7_2",+ "53949711_2",+ "5726dc37_2",+ "59e96888_1",+ "771ba1d5",+ "answer_4934b2d7_1",+ "feb5f98a_1",+ "c51572bb_2",+ "fd5448f0",+ "2655efcc_1"+ ],+ "gold_session_ids": [+ "answer_4934b2d7_2",+ "answer_4934b2d7_1"+ ]+ },+ {+ "question_id": "37f165cf",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6b9b2b1e_2",+ "answer_6b9b2b1e_1",+ "20b1b65f_2",+ "37b19c82",+ "7e8e790a",+ "0f134440",+ "cdcf791b_1",+ "395a2f92",+ "sharegpt_2yaKUmC_0",+ "38de4c2a_1"+ ],+ "gold_session_ids": [+ "answer_6b9b2b1e_2",+ "answer_6b9b2b1e_1"+ ]+ },+ {+ "question_id": "8e91e7d9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_147574",+ "answer_477ae455_2",+ "sharegpt_8lpX5R7_19",+ "4e3da326_1",+ "da71721f",+ "sharegpt_PZrBCF2_0",+ "4f8caea3_2",+ "17cb8d2f_3",+ "7a389bd1_7",+ "c3023a45_4"+ ],+ "gold_session_ids": [+ "answer_477ae455_1",+ "answer_477ae455_2"+ ]+ },+ {+ "question_id": "87f22b4a",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f56e6152_1",+ "answer_f56e6152_2",+ "ea28bb75",+ "c6015a0c_3",+ "a8ac3d1d_3",+ "ultrachat_182225",+ "7c2ec9e9_2",+ "889076d6",+ "02b99ec3_1",+ "c4c504d2_6"+ ],+ "gold_session_ids": [+ "answer_f56e6152_2",+ "answer_f56e6152_1"+ ]+ },+ {+ "question_id": "e56a43b9",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_430d0a87_2",+ "7596ba6a",+ "answer_430d0a87_1",+ "57a6a404_3",+ "372544ba",+ "38e81260_7",+ "ultrachat_12201",+ "3c0ed3bf_1",+ "a59b73f6",+ "ce21f0e4"+ ],+ "gold_session_ids": [+ "answer_430d0a87_1",+ "answer_430d0a87_2"+ ]+ },+ {+ "question_id": "efc3f7c2",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fc81d1a2_1",+ "88a7faa7_1",+ "answer_fc81d1a2_2",+ "bbdc7b0a",+ "8a7df17e_1",+ "0f013a55",+ "bbca6598_1",+ "fc005760",+ "ultrachat_38938",+ "c4251d8d_1"+ ],+ "gold_session_ids": [+ "answer_fc81d1a2_1",+ "answer_fc81d1a2_2"+ ]+ },+ {+ "question_id": "21d02d0d",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2c637141_1",+ "answer_2c637141_2",+ "2e28f3db",+ "a15fa6eb",+ "42d9467c_2",+ "ultrachat_36404",+ "414b8ddb",+ "6ff8954a_2",+ "94bc18df_2",+ "ultrachat_285752"+ ],+ "gold_session_ids": [+ "answer_2c637141_2",+ "answer_2c637141_1"+ ]+ },+ {+ "question_id": "2311e44b_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bf633415_abs_1",+ "answer_bf633415_abs_2",+ "35915bf5",+ "313404c6_2",+ "dc880372_2",+ "4c8afa9c_1",+ "sharegpt_l2SM5sp_0",+ "sharegpt_3bZSvyG_0",+ "369695b4_2",+ "a7bd5b45_3"+ ],+ "gold_session_ids": [+ "answer_bf633415_abs_1",+ "answer_bf633415_abs_2"+ ]+ },+ {+ "question_id": "6456829e_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_743f03a1_abs_2",+ "answer_743f03a1_abs_1",+ "b61cc91b_2",+ "sharegpt_Hc0Zm6i_0",+ "d20bb5aa_1",+ "3d8d4828_2",+ "ultrachat_567580",+ "89cce497",+ "ultrachat_345787",+ "ultrachat_568772"+ ],+ "gold_session_ids": [+ "answer_743f03a1_abs_1",+ "answer_743f03a1_abs_2"+ ]+ },+ {+ "question_id": "e5ba910e_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e49ed9d3_abs_2",+ "answer_e49ed9d3_abs_1",+ "a239515b",+ "6d013bad_3",+ "6339aa1d_2",+ "f2df24cd_2",+ "ultrachat_282969",+ "bd8a4fcb_2",+ "cc3f0564_1",+ "7e0fa6a6"+ ],+ "gold_session_ids": [+ "answer_e49ed9d3_abs_1",+ "answer_e49ed9d3_abs_2"+ ]+ },+ {+ "question_id": "a96c20ee_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ef84b994_abs_2",+ "4163e709_3",+ "answer_ef84b994_abs_1",+ "ultrachat_59306",+ "ultrachat_194192",+ "ultrachat_293509",+ "sharegpt_jM9e92F_0",+ "ultrachat_36312",+ "ultrachat_184142",+ "ultrachat_24979"+ ],+ "gold_session_ids": [+ "answer_ef84b994_abs_1",+ "answer_ef84b994_abs_2"+ ]+ },+ {+ "question_id": "ba358f49_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cbd08e3c_abs_1",+ "07ebc271",+ "5829ab38",+ "sharegpt_7kFao5h_0",+ "1162611c",+ "40648965",+ "20ea9554_1",+ "9844f740",+ "answer_cbd08e3c_abs_2",+ "sharegpt_HRpe8C8_0"+ ],+ "gold_session_ids": [+ "answer_cbd08e3c_abs_1",+ "answer_cbd08e3c_abs_2"+ ]+ },+ {+ "question_id": "09ba9854_abs",+ "question_type": "multi-session",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_96c743d0_abs_1",+ "answer_96c743d0_abs_2",+ "ultrachat_342361",+ "a50b0d73_2",+ "66e58c90_2",+ "sharegpt_JZm4ich_15",+ "64460e9d_2",+ "85ca9420",+ "338ac116_2",+ "e4250940"+ ],+ "gold_session_ids": [+ "answer_96c743d0_abs_1",+ "answer_96c743d0_abs_2"+ ]+ },+ {+ "question_id": "gpt4_59149c77",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9e2c2a6c_2",+ "answer_d00ba6d0_2",+ "answer_d00ba6d0_1",+ "sharegpt_XlNs8Lz_199",+ "ab520c81_1",+ "a7403b3d_4",+ "148f87a0_2",+ "6e672b84_1",+ "ultrachat_318178",+ "64c2b3c8"+ ],+ "gold_session_ids": [+ "answer_d00ba6d0_1",+ "answer_d00ba6d0_2"+ ]+ },+ {+ "question_id": "gpt4_f49edff3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9469024295259745,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e9fce53_3",+ "answer_3e9fce53_2",+ "4376945f",+ "40a289b7_2",+ "answer_3e9fce53_1",+ "7d33fcb4_1",+ "cbc1588c_2",+ "8e226a86",+ "dc7c53fa_4",+ "4798296c_2"+ ],+ "gold_session_ids": [+ "answer_3e9fce53_1",+ "answer_3e9fce53_2",+ "answer_3e9fce53_3"+ ]+ },+ {+ "question_id": "71017276",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0b4a8adc_1",+ "sharegpt_CjoQMV6_0",+ "sharegpt_tCLMKZY_0",+ "sharegpt_qevDNvo_0",+ "398f23e7",+ "f78a22b6_4",+ "sharegpt_duoAhsS_0",+ "10889cf0_1",+ "683c8358_1",+ "sharegpt_ZUJIuGt_13"+ ],+ "gold_session_ids": [+ "answer_0b4a8adc_1"+ ]+ },+ {+ "question_id": "b46e15ed",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6525874238732422,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4bfcc250_4",+ "aeab3296",+ "ultrachat_265690",+ "53949711_2",+ "47311a4c_1",+ "answer_4bfcc250_3",+ "sharegpt_cXkL3cR_28",+ "answer_4bfcc250_1",+ "f598d30f_1",+ "ultrachat_396986"+ ],+ "gold_session_ids": [+ "answer_4bfcc250_4",+ "answer_4bfcc250_3",+ "answer_4bfcc250_2",+ "answer_4bfcc250_1"+ ]+ },+ {+ "question_id": "gpt4_fa19884c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ff201786_2",+ "answer_ff201786_1",+ "e8de3056_2",+ "cbf48918_1",+ "f4f4b504_2",+ "bb2180e9_3",+ "6414f676_2",+ "ultrachat_278956",+ "ultrachat_423544",+ "f3745025_1"+ ],+ "gold_session_ids": [+ "answer_ff201786_1",+ "answer_ff201786_2"+ ]+ },+ {+ "question_id": "0bc8ad92",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f4ea84fb_3",+ "answer_f4ea84fb_1",+ "d3cc5bdc_1",+ "answer_f4ea84fb_2",+ "0868982f_1",+ "717d1aea_1",+ "ultrachat_349938",+ "sharegpt_ntu1Cds_0",+ "f58c36b8_1",+ "sharegpt_vClQxDr_21"+ ],+ "gold_session_ids": [+ "answer_f4ea84fb_3",+ "answer_f4ea84fb_2",+ "answer_f4ea84fb_1"+ ]+ },+ {+ "question_id": "af082822",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.06666666666666667,+ "retrieved_session_ids": [+ "1dfef591_2",+ "6cf1848e_3",+ "e132259f",+ "bc5827b5",+ "1a920713_2",+ "facef6e0_1",+ "bf38543d_2",+ "1fcb4134_2",+ "fbe6fa2c_2",+ "1040e24b_1"+ ],+ "gold_session_ids": [+ "answer_b51b6115_1"+ ]+ },+ {+ "question_id": "gpt4_4929293a",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_add9b012_2",+ "answer_add9b012_1",+ "ultrachat_139245",+ "ultrachat_512282",+ "b8bc1464_4",+ "ultrachat_329640",+ "e2304e5e_1",+ "73353aee_1",+ "c1e598cd_1",+ "ultrachat_127833"+ ],+ "gold_session_ids": [+ "answer_add9b012_2",+ "answer_add9b012_1"+ ]+ },+ {+ "question_id": "gpt4_b5700ca9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a17423e7_1",+ "5053474c_2",+ "47382adf_1",+ "33bdd89a_1",+ "31829c1b_1",+ "sharegpt_FqOC5e9_169",+ "c81aa33c_2",+ "d743153b_1",+ "a53c7542_1",+ "ultrachat_383774"+ ],+ "gold_session_ids": [+ "answer_a17423e7_1"+ ]+ },+ {+ "question_id": "9a707b81",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dba89487_2",+ "4abbeb2a_2",+ "answer_dba89487_1",+ "e1d11615_1",+ "990f3ef9_6",+ "ultrachat_233519",+ "e75c302b_2",+ "3fcbb58c_4",+ "sharegpt_sW9QRz1_0",+ "5a13d047_1"+ ],+ "gold_session_ids": [+ "answer_dba89487_2",+ "answer_dba89487_1"+ ]+ },+ {+ "question_id": "gpt4_1d4ab0c9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_febde667_1",+ "answer_febde667_2",+ "sharegpt_Caxpnd3_12",+ "7257bd15_1",+ "sharegpt_MIYBRJz_27",+ "sharegpt_htSCqmh_171",+ "c14e3f09_1",+ "ultrachat_125013",+ "4ff83df8",+ "28d52bff"+ ],+ "gold_session_ids": [+ "answer_febde667_1",+ "answer_febde667_2"+ ]+ },+ {+ "question_id": "gpt4_e072b769",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c19bd2bf_1",+ "0dd54b7c_1",+ "4f7b5dc9_1",+ "5558a42e_1",+ "sharegpt_9Ln5Ood_0",+ "66d3fdb7_2",+ "sharegpt_zdDMlRC_0",+ "sharegpt_VbIxRWJ_33",+ "ultrachat_423979",+ "2fc9ce79_2"+ ],+ "gold_session_ids": [+ "answer_c19bd2bf_1"+ ]+ },+ {+ "question_id": "0db4c65d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b9e32ff8_2",+ "answer_b9e32ff8_1",+ "f78a22b6_3",+ "953d5a4b_1",+ "71dc2037_2",+ "sharegpt_UGg8d44_9",+ "6461402b",+ "ultrachat_324557",+ "d36d11b9_2",+ "e6c3a50a"+ ],+ "gold_session_ids": [+ "answer_b9e32ff8_1",+ "answer_b9e32ff8_2"+ ]+ },+ {+ "question_id": "gpt4_1d80365e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_661b711f_1",+ "answer_661b711f_2",+ "805e571c_1",+ "d6b17438_1",+ "554b054e_2",+ "3be19aed_1",+ "ef0341d4_2",+ "18f6e3be_2",+ "sharegpt_5uZ7IdY_0",+ "f7a61595_2"+ ],+ "gold_session_ids": [+ "answer_661b711f_1",+ "answer_661b711f_2"+ ]+ },+ {+ "question_id": "gpt4_7f6b06db",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.35853358850813305,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "4bf66c38_2",+ "6f051087_2",+ "da42526b_1",+ "answer_5d8c99d3_2",+ "252715f8_2",+ "928890e4_3",+ "answer_5d8c99d3_3",+ "42165950_8",+ "39c1bf6a",+ "a6109135_1"+ ],+ "gold_session_ids": [+ "answer_5d8c99d3_1",+ "answer_5d8c99d3_2",+ "answer_5d8c99d3_3"+ ]+ },+ {+ "question_id": "gpt4_6dc9b45b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_384387",+ "answer_c4df007f_1",+ "afa0c2e7_5",+ "9adc81d7_2",+ "ultrachat_35147",+ "27639dd8_1",+ "cfcf5340_2",+ "e7cc239c_4",+ "d8f48b0a",+ "ultrachat_453768"+ ],+ "gold_session_ids": [+ "answer_c4df007f_1"+ ]+ },+ {+ "question_id": "gpt4_8279ba02",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.08333333333333333,+ "retrieved_session_ids": [+ "b909542d_1",+ "b5b8f8f9_1",+ "01f1e64a_3",+ "1a037873_2",+ "b6018747_2",+ "sharegpt_BSRNM84_16",+ "86c5d31d",+ "85f19a30_1",+ "886b70b1_2",+ "67d1de46_2"+ ],+ "gold_session_ids": [+ "answer_56521e65_1"+ ]+ },+ {+ "question_id": "gpt4_18c2b244",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c862f65a_3",+ "answer_c862f65a_2",+ "answer_c862f65a_1",+ "ultrachat_503659",+ "7e2c6065",+ "dfbc46bd_1",+ "746ae854_2",+ "ffd66315_2",+ "7c4431e2_1",+ "3176940c_2"+ ],+ "gold_session_ids": [+ "answer_c862f65a_2",+ "answer_c862f65a_3",+ "answer_c862f65a_1"+ ]+ },+ {+ "question_id": "gpt4_a1b77f9c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e9ad5914_4",+ "answer_e9ad5914_2",+ "answer_e9ad5914_3",+ "answer_e9ad5914_5",+ "answer_e9ad5914_1",+ "answer_e9ad5914_6",+ "d52d478b",+ "47ec1674_2",+ "0dcfb511_4",+ "660c4dba_2"+ ],+ "gold_session_ids": [+ "answer_e9ad5914_1",+ "answer_e9ad5914_2",+ "answer_e9ad5914_3",+ "answer_e9ad5914_4",+ "answer_e9ad5914_5",+ "answer_e9ad5914_6"+ ]+ },+ {+ "question_id": "gpt4_1916e0ea",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_447052a5_1",+ "sharegpt_gw7Ym6y_6",+ "fa1ce406_2",+ "b9890c15_2",+ "3f3013b4_1",+ "d3fe25ff",+ "280352e9",+ "4b72fe4c_1",+ "answer_447052a5_2",+ "fe4dd702_2"+ ],+ "gold_session_ids": [+ "answer_447052a5_2",+ "answer_447052a5_1"+ ]+ },+ {+ "question_id": "gpt4_7a0daae1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4d5490f1_2",+ "answer_4d5490f1_1",+ "bf3aebdb_2",+ "2ff297a8_1",+ "819dfad7_2",+ "sharegpt_xcd9nky_8",+ "facb94a8_3",+ "ultrachat_253041",+ "ultrachat_49899",+ "ultrachat_9855"+ ],+ "gold_session_ids": [+ "answer_4d5490f1_1",+ "answer_4d5490f1_2"+ ]+ },+ {+ "question_id": "gpt4_468eb063",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.2,+ "retrieved_session_ids": [+ "sharegpt_Sf51OjE_8",+ "e60a93ff_2",+ "sharegpt_gfhUw40_17",+ "71888aff_2",+ "answer_9b09d95b_1",+ "sharegpt_ipLglky_42",+ "7ca763c3_2",+ "ce584ba0",+ "6410c263_1",+ "sharegpt_8M7HcM3_11"+ ],+ "gold_session_ids": [+ "answer_9b09d95b_1"+ ]+ },+ {+ "question_id": "gpt4_7abb270c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7976403384579975,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7093d898_5",+ "answer_7093d898_6",+ "087d2b0a_2",+ "1bfd5a8b_2",+ "5808284e",+ "answer_7093d898_2",+ "answer_7093d898_1",+ "answer_7093d898_4",+ "85056d5f_1",+ "3c9cdf01"+ ],+ "gold_session_ids": [+ "answer_7093d898_1",+ "answer_7093d898_2",+ "answer_7093d898_3",+ "answer_7093d898_4",+ "answer_7093d898_5",+ "answer_7093d898_6"+ ]+ },+ {+ "question_id": "gpt4_1e4a8aeb",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_16bd5ea5_2",+ "answer_16bd5ea5_1",+ "28621d6a_2",+ "fd18e0dd_1",+ "6be23caa",+ "ae0cd456",+ "f7838a91_1",+ "sharegpt_Nk10TjE_29",+ "6a7d7055_1",+ "e83a2abc_1"+ ],+ "gold_session_ids": [+ "answer_16bd5ea5_1",+ "answer_16bd5ea5_2"+ ]+ },+ {+ "question_id": "gpt4_4fc4f797",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_be07688f_1",+ "answer_be07688f_2",+ "39566615_1",+ "bdcee74e_3",+ "sharegpt_pRqHb1o_0",+ "08834bf2_1",+ "660cea1d",+ "59fc4d6b",+ "10d7d2b4_2",+ "e77ef495_1"+ ],+ "gold_session_ids": [+ "answer_be07688f_1",+ "answer_be07688f_2"+ ]+ },+ {+ "question_id": "4dfccbf7",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "8fa624b2_6",+ "answer_4bebc782_2",+ "answer_4bebc782_1",+ "e1b7b676",+ "sharegpt_Dcshjlv_0",+ "ultrachat_373047",+ "36f8b380_1",+ "aa71db11_2",+ "f5493bf3",+ "sharegpt_cXkL3cR_69"+ ],+ "gold_session_ids": [+ "answer_4bebc782_1",+ "answer_4bebc782_2"+ ]+ },+ {+ "question_id": "gpt4_61e13b3c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e831a29f_1",+ "answer_e831a29f_2",+ "ultrachat_441289",+ "2b5c911e_3",+ "e75828ad_3",+ "ultrachat_213012",+ "8e10bf6b_1",+ "cc380119_3",+ "d4439279_1",+ "a6f2320f_2"+ ],+ "gold_session_ids": [+ "answer_e831a29f_1",+ "answer_e831a29f_2"+ ]+ },+ {+ "question_id": "gpt4_45189cb4",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8179809735263114,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e6c20e52_3",+ "dc3ee1d1_1",+ "22452c22",+ "5c16fe0b_1",+ "answer_e6c20e52_1",+ "answer_e6c20e52_2",+ "36e72174_1",+ "93395e5f_3",+ "6593cb8b_1",+ "ebf5b3bc_2"+ ],+ "gold_session_ids": [+ "answer_e6c20e52_3",+ "answer_e6c20e52_2",+ "answer_e6c20e52_1"+ ]+ },+ {+ "question_id": "2ebe6c90",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c9d35c00_2",+ "answer_c9d35c00_1",+ "sharegpt_jyMHY1J_31",+ "ultrachat_4566",+ "dddce60a_2",+ "9ccea2f0_2",+ "0fe848c5_3",+ "c17f3491",+ "453d354e_1",+ "8fa64936_1"+ ],+ "gold_session_ids": [+ "answer_c9d35c00_1",+ "answer_c9d35c00_2"+ ]+ },+ {+ "question_id": "gpt4_e061b84f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6105456988825855,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8c64ce25_3",+ "78d28576_2",+ "0a6bf5e4_1",+ "773287dc_1",+ "539c2346_3",+ "bab3f409",+ "ultrachat_129606",+ "3275acf9",+ "answer_8c64ce25_1",+ "ultrachat_194928"+ ],+ "gold_session_ids": [+ "answer_8c64ce25_2",+ "answer_8c64ce25_1",+ "answer_8c64ce25_3"+ ]+ },+ {+ "question_id": "370a8ff4",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_61d1be50_1",+ "answer_61d1be50_2",+ "sharegpt_jGPXxpI_17",+ "2b4eaee0_1",+ "8c64ce25_1",+ "21a51dee_2",+ "df53e1b7_2",+ "ultrachat_26182",+ "11c0b1c5",+ "4e77dbbe_5"+ ],+ "gold_session_ids": [+ "answer_61d1be50_1",+ "answer_61d1be50_2"+ ]+ },+ {+ "question_id": "gpt4_d6585ce8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f999b05b_4",+ "answer_f999b05b_5",+ "answer_f999b05b_2",+ "answer_f999b05b_3",+ "answer_f999b05b_1",+ "ultrachat_531211",+ "5d4010bd",+ "23901329_2",+ "sharegpt_Asf5ayG_41",+ "dc0802a9_1"+ ],+ "gold_session_ids": [+ "answer_f999b05b_1",+ "answer_f999b05b_4",+ "answer_f999b05b_2",+ "answer_f999b05b_5",+ "answer_f999b05b_3"+ ]+ },+ {+ "question_id": "gpt4_4ef30696",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f964cea3_2",+ "sharegpt_ELYlA30_25",+ "4b29957c_1",+ "answer_f964cea3_1",+ "80b3f093",+ "d9868305_2",+ "ultrachat_356406",+ "750b437b_1",+ "1ccb08a0",+ "ultrachat_511839"+ ],+ "gold_session_ids": [+ "answer_f964cea3_1",+ "answer_f964cea3_2"+ ]+ },+ {+ "question_id": "gpt4_ec93e27f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7903864795495061,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e3aa84c4_1",+ "6b63bdc5_2",+ "sharegpt_Sg0ctDe_14",+ "85846900_2",+ "ultrachat_133145",+ "ultrachat_401609",+ "ultrachat_29858",+ "5a994ed3_2",+ "b76a71b9_2",+ "answer_e3aa84c4_2"+ ],+ "gold_session_ids": [+ "answer_e3aa84c4_2",+ "answer_e3aa84c4_1"+ ]+ },+ {+ "question_id": "6e984301",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_88841f26_1",+ "answer_88841f26_2",+ "0125f748_1",+ "31903f54",+ "b38766c1_2",+ "ultrachat_402356",+ "2bc4ae7d",+ "8f6e71cf",+ "ultrachat_111630",+ "93bed3d8_2"+ ],+ "gold_session_ids": [+ "answer_88841f26_1",+ "answer_88841f26_2"+ ]+ },+ {+ "question_id": "8077ef71",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0dd54b7c_1",+ "f7ffddbc",+ "sharegpt_Wi7Op1u_39",+ "1da90ded",+ "sharegpt_hoQsVGz_51",+ "7fa3c60f_1",+ "5365840b_1",+ "30711c1e_1",+ "56ee0a51",+ "d87a6ef8"+ ],+ "gold_session_ids": [+ "answer_0dd54b7c_1"+ ]+ },+ {+ "question_id": "gpt4_f420262c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9747429528567879,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8a1af6b_2",+ "answer_d8a1af6b_4",+ "answer_d8a1af6b_3",+ "a823dc7c",+ "answer_d8a1af6b_5",+ "answer_d8a1af6b_1",+ "3418b277_5",+ "116bae09",+ "3ea07188_2",+ "sharegpt_jFDrHZ3_0"+ ],+ "gold_session_ids": [+ "answer_d8a1af6b_1",+ "answer_d8a1af6b_2",+ "answer_d8a1af6b_3",+ "answer_d8a1af6b_4",+ "answer_d8a1af6b_5"+ ]+ },+ {+ "question_id": "gpt4_8e165409",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d97db962_1",+ "answer_d97db962_2",+ "sharegpt_JRQtEcd_115",+ "4c2bd0fe_2",+ "58ab5fc5",+ "ultrachat_77564",+ "sharegpt_B7mSlfQ_8",+ "4c967baa_1",+ "4c9f6bf5_2",+ "b49985d9_1"+ ],+ "gold_session_ids": [+ "answer_d97db962_1",+ "answer_d97db962_2"+ ]+ },+ {+ "question_id": "gpt4_74aed68e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aed8cf17_2",+ "answer_aed8cf17_1",+ "2a86d0da_1",+ "ultrachat_176592",+ "69a9211c_1",+ "318048af_2",+ "470f29c0_1",+ "ultrachat_35782",+ "25b3e36c_2",+ "da71721f"+ ],+ "gold_session_ids": [+ "answer_aed8cf17_1",+ "answer_aed8cf17_2"+ ]+ },+ {+ "question_id": "bcbe585f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_43ba3965_1",+ "5dbe11b9",+ "7fa8099b_1",+ "8953c8c8_2",+ "6bed7862",+ "896162a2",+ "sharegpt_WsDgKko_13",+ "5edfe253_1",+ "89a16e7f_1",+ "d8a1af6c_5"+ ],+ "gold_session_ids": [+ "answer_43ba3965_1"+ ]+ },+ {+ "question_id": "gpt4_21adecb5",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.44130740935663865,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "sharegpt_ipLglky_42",+ "0f0f9d80_1",+ "e1f37e24",+ "answer_1e2369c9_2",+ "19f2e2c5_2",+ "ultrachat_444790",+ "sharegpt_cUXo8S0_0",+ "b99bd2df",+ "sharegpt_MCmVQ20_0",+ "answer_1e2369c9_1"+ ],+ "gold_session_ids": [+ "answer_1e2369c9_1",+ "answer_1e2369c9_2"+ ]+ },+ {+ "question_id": "5e1b23de",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c18d480b_1",+ "1a5344e9_3",+ "c28aed33",+ "7ff028d8_2",+ "41782aae_1",+ "cdf75ca3_2",+ "1380576d_1",+ "ultrachat_190837",+ "ultrachat_524214",+ "0ba46bdc_2"+ ],+ "gold_session_ids": [+ "answer_c18d480b_1"+ ]+ },+ {+ "question_id": "gpt4_98f46fc6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5850de18_1",+ "answer_5850de18_2",+ "sharegpt_rnL5VQO_0",+ "32e292f2",+ "sharegpt_FhQzMiW_0",+ "sharegpt_Hj47IDr_0",+ "ultrachat_238480",+ "2640383e",+ "28897044",+ "sharegpt_PLN69pg_19"+ ],+ "gold_session_ids": [+ "answer_5850de18_2",+ "answer_5850de18_1"+ ]+ },+ {+ "question_id": "gpt4_af6db32f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_184c8f56_1",+ "f26846e1_3",+ "ec1b557a_1",+ "aee13015_6",+ "ultrachat_251919",+ "sharegpt_1jjEIai_173",+ "df6938a4_5",+ "sharegpt_Np41szj_13",+ "8ccbe6e0",+ "8617737e"+ ],+ "gold_session_ids": [+ "answer_184c8f56_1"+ ]+ },+ {+ "question_id": "eac54adc",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0d4d0347_2",+ "answer_0d4d0347_1",+ "sharegpt_PgmAp0p_0",+ "b40f0a7a",+ "sharegpt_Jm8wWJN_0",+ "sharegpt_z88NqeI_0",+ "ae051325_3",+ "32fc393b_2",+ "80a6c3b7",+ "dba97bb1_2"+ ],+ "gold_session_ids": [+ "answer_0d4d0347_1",+ "answer_0d4d0347_2"+ ]+ },+ {+ "question_id": "gpt4_7ddcf75f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ad8a76cb_1",+ "9bfccfdc_2",+ "8c652eb0_2",+ "9283c152",+ "sharegpt_ZUxBndS_26",+ "sharegpt_mMdyu3t_0",+ "836f5ae4",+ "9f8c432e_3",+ "ultrachat_281183",+ "a0b8b0ad_7"+ ],+ "gold_session_ids": [+ "answer_ad8a76cb_1"+ ]+ },+ {+ "question_id": "gpt4_a2d1d1f6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f6d6e33f_1",+ "a21f3697_3",+ "e40d733b_2",+ "2c31d8be_3",+ "9a584663_1",+ "sharegpt_gI1U1eK_4",+ "894dd0c2_3",+ "534be93d_1",+ "ultrachat_5344",+ "a1bfb382_1"+ ],+ "gold_session_ids": [+ "answer_f6d6e33f_1"+ ]+ },+ {+ "question_id": "gpt4_85da3956",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_581ab834_1",+ "14b000d9_1",+ "ultrachat_373754",+ "2fd445a2_4",+ "sharegpt_ewad5Sr_0",+ "sharegpt_opQbgPL_13",+ "3a4012a5_4",+ "3e6b6802_3",+ "d8b3e1c8_1",+ "e6fddf6c"+ ],+ "gold_session_ids": [+ "answer_581ab834_1"+ ]+ },+ {+ "question_id": "gpt4_b0863698",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_550bb2d1_1",+ "91e581ad_3",+ "4a79d078_2",+ "5b014f9b_1",+ "ce8b2a9e_2",+ "554b054e_2",+ "710833f3",+ "b2fa108b_1",+ "sharegpt_FPbCOuq_30",+ "sharegpt_vGZ5lH1_21"+ ],+ "gold_session_ids": [+ "answer_550bb2d1_1"+ ]+ },+ {+ "question_id": "gpt4_68e94287",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9793daa3_2",+ "50f136f2_3",+ "64460e9d_1",+ "answer_9793daa3_1",+ "74bcc8f4_1",+ "edfa358d_1",+ "sharegpt_XzAEFL4_0",+ "dadd6379_1",+ "50ca8a86",+ "43502c3c_2"+ ],+ "gold_session_ids": [+ "answer_9793daa3_2",+ "answer_9793daa3_1"+ ]+ },+ {+ "question_id": "gpt4_e414231e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e28c1f0d_2",+ "answer_e28c1f0d_1",+ "7e3fa056_2",+ "d3065d85",+ "9f8fc173_3",+ "aee32463_1",+ "0254514e_1",+ "d4439279_2",+ "sharegpt_CdnyHep_0",+ "c87e6c56"+ ],+ "gold_session_ids": [+ "answer_e28c1f0d_1",+ "answer_e28c1f0d_2"+ ]+ },+ {+ "question_id": "gpt4_7ca326fa",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf021b36_1",+ "answer_cf021b36_3",+ "answer_cf021b36_2",+ "sharegpt_LDpwZD3_0",+ "c70f9f9c",+ "1301fbd1",+ "8d0ca228",+ "ultrachat_147001",+ "sharegpt_7SCdbX2_0",+ "5429e768_1"+ ],+ "gold_session_ids": [+ "answer_cf021b36_1",+ "answer_cf021b36_2",+ "answer_cf021b36_3"+ ]+ },+ {+ "question_id": "gpt4_7bc6cf22",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e22d6aef_1",+ "3e43bf49_1",+ "70b9616e",+ "ultrachat_330542",+ "09d021e7_1",+ "ultrachat_215356",+ "68396ed3_2",+ "eca3b71c_2",+ "e6cc6157_1",+ "7a4d00b3_1"+ ],+ "gold_session_ids": [+ "answer_e22d6aef_1"+ ]+ },+ {+ "question_id": "2ebe6c92",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6052602440527057,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "1ba79ea8_1",+ "answer_c9d35c00_2",+ "ca3a4e4f_1",+ "d9e023f2_2",+ "9d4312f6_3",+ "answer_c9d35c00_1",+ "719a18dd_2",+ "acbb04c7",+ "16756728_1",+ "sharegpt_iQp7qQ2_0"+ ],+ "gold_session_ids": [+ "answer_c9d35c00_1",+ "answer_c9d35c00_2"+ ]+ },+ {+ "question_id": "gpt4_e061b84g",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5307212739772434,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9dac9e37_1",+ "answer_8c64ce26_3",+ "answer_8c64ce26_2",+ "3929e6cc_1",+ "ae15e8b6_1",+ "sharegpt_i33zh1e_15",+ "ce584ba0",+ "68705467_1",+ "ultrachat_547960",+ "51221cac"+ ],+ "gold_session_ids": [+ "answer_8c64ce26_2",+ "answer_8c64ce26_1",+ "answer_8c64ce26_3"+ ]+ },+ {+ "question_id": "71017277",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0b4a8adc_1",+ "f8de4e92_4",+ "ultrachat_557308",+ "ad175dc4_5",+ "0aacd15f_5",+ "6de70f46_2",+ "85a1be56_2",+ "fd749c9a_1",+ "451ecb23_3",+ "e64f9553_1"+ ],+ "gold_session_ids": [+ "answer_0b4a8adc_1"+ ]+ },+ {+ "question_id": "b46e15ee",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9619991470595832,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4bfcc251_4",+ "answer_4bfcc251_1",+ "answer_4bfcc251_2",+ "495cf518_2",+ "d4446e25_2",+ "e03c71c5_1",+ "answer_4bfcc251_3",+ "ultrachat_436509",+ "f3fcf4fd_1",+ "c92239bc"+ ],+ "gold_session_ids": [+ "answer_4bfcc251_4",+ "answer_4bfcc251_3",+ "answer_4bfcc251_2",+ "answer_4bfcc251_1"+ ]+ },+ {+ "question_id": "gpt4_d6585ce9",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8738100898801323,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f999b05c_5",+ "58dd3f35",+ "answer_f999b05c_4",+ "a554ed79_4",+ "answer_f999b05c_2",+ "answer_f999b05c_1",+ "answer_f999b05c_3",+ "b3763b6b_1",+ "a15fa6eb",+ "a5532feb_2"+ ],+ "gold_session_ids": [+ "answer_f999b05c_1",+ "answer_f999b05c_4",+ "answer_f999b05c_2",+ "answer_f999b05c_5",+ "answer_f999b05c_3"+ ]+ },+ {+ "question_id": "gpt4_1e4a8aec",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20438239758848611,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "9f1f3bf4",+ "b1265fd7_1",+ "e675c69a_1",+ "1c662b7b_1",+ "b34a52aa_3",+ "0b635369_2",+ "answer_16bd5ea6_1",+ "sharegpt_sRhcMiu_0",+ "4c8455cb_2",+ "743b1fc3_2"+ ],+ "gold_session_ids": [+ "answer_16bd5ea6_1",+ "answer_16bd5ea6_2"+ ]+ },+ {+ "question_id": "gpt4_f420262d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8a1af6c_2",+ "answer_d8a1af6c_5",+ "answer_d8a1af6c_4",+ "answer_d8a1af6c_1",+ "answer_d8a1af6c_3",+ "27639dd8_3",+ "dfd4e4b7_1",+ "1032cbcf_1",+ "534716de",+ "a7c62fd2_2"+ ],+ "gold_session_ids": [+ "answer_d8a1af6c_1",+ "answer_d8a1af6c_2",+ "answer_d8a1af6c_5",+ "answer_d8a1af6c_4",+ "answer_d8a1af6c_3"+ ]+ },+ {+ "question_id": "gpt4_59149c78",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8065735963827293,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d00ba6d1_1",+ "23754665",+ "ultrachat_463998",+ "765ce8a7_2",+ "a8ac3d1d_1",+ "ultrachat_131385",+ "665f8750_1",+ "answer_d00ba6d1_2",+ "78e40ca2_2",+ "9848a1e3_2"+ ],+ "gold_session_ids": [+ "answer_d00ba6d1_1",+ "answer_d00ba6d1_2"+ ]+ },+ {+ "question_id": "gpt4_e414231f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e28c1f0e_1",+ "ultrachat_150948",+ "answer_e28c1f0e_2",+ "752c438c_3",+ "deb5bb60",+ "78ef59d0_1",+ "sharegpt_p57YcDV_15",+ "3acb50ef_4",+ "135f761b_1",+ "sharegpt_mFa5dxA_11"+ ],+ "gold_session_ids": [+ "answer_e28c1f0e_1",+ "answer_e28c1f0e_2"+ ]+ },+ {+ "question_id": "gpt4_4929293b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.20438239758848611,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "7fa98614_1",+ "ultrachat_342539",+ "ultrachat_156176",+ "ultrachat_326769",+ "cae65795_1",+ "sharegpt_bq9G6bT_0",+ "answer_add9b013_2",+ "bda611f6_3",+ "e76430ea_3",+ "ea8c38f8"+ ],+ "gold_session_ids": [+ "answer_add9b013_2",+ "answer_add9b013_1"+ ]+ },+ {+ "question_id": "gpt4_468eb064",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.2890648263178879,+ "mrr": 0.1,+ "retrieved_session_ids": [+ "sharegpt_wrwD6bG_0",+ "c3023a45_1",+ "sharegpt_0PLBHdX_15",+ "0944933f_1",+ "25e5b797_1",+ "a2f0054f",+ "db1aefb6_3",+ "ultrachat_59629",+ "1e5bd28d_2",+ "answer_9b09d95b_1"+ ],+ "gold_session_ids": [+ "answer_9b09d95b_1"+ ]+ },+ {+ "question_id": "gpt4_fa19884d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8315546295836225,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ff201787_2",+ "ccc87da9_1",+ "588a2123",+ "f910dc31_4",+ "5f9dd782",+ "answer_ff201787_1",+ "b35fadaa",+ "040615bc_1",+ "ebd2a973",+ "8f8f2ced_1"+ ],+ "gold_session_ids": [+ "answer_ff201787_1",+ "answer_ff201787_2"+ ]+ },+ {+ "question_id": "9a707b82",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "990f3ef9_2",+ "fab41c07",+ "answer_dba89488_1",+ "f8e9d4fa",+ "68d35085_1",+ "8e78fa70_1",+ "ultrachat_466884",+ "answer_dba89488_2",+ "7a4d00b3_2",+ "e627b1aa_1"+ ],+ "gold_session_ids": [+ "answer_dba89488_2",+ "answer_dba89488_1"+ ]+ },+ {+ "question_id": "eac54add",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "7bb01253",+ "answer_0d4d0348_1",+ "ultrachat_91693",+ "2946bf12",+ "773aebbd_3",+ "577da76d_2",+ "ultrachat_134291",+ "sharegpt_PJmlduQ_0",+ "79987901_2",+ "e552e1f9_2"+ ],+ "gold_session_ids": [+ "answer_0d4d0348_1",+ "answer_0d4d0348_2"+ ]+ },+ {+ "question_id": "4dfccbf8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0,+ "mrr": 0.0625,+ "retrieved_session_ids": [+ "953aaad5_3",+ "7db28e68_1",+ "87317e05_1",+ "edd89480_1",+ "0ae4c9b0_2",+ "a63ad8e3_3",+ "1c177942_4",+ "sharegpt_3D3oQC0_213",+ "sharegpt_oXgiN7q_131",+ "b4f63a70_3"+ ],+ "gold_session_ids": [+ "answer_4bebc783_1",+ "answer_4bebc783_2"+ ]+ },+ {+ "question_id": "0bc8ad93",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f4ea84fc_3",+ "answer_f4ea84fc_1",+ "answer_f4ea84fc_2",+ "sharegpt_HOO59Ue_99",+ "dd25eeb5_2",+ "85736a07_1",+ "ultrachat_260376",+ "2c185a2b_1",+ "sharegpt_afCogMg_34",+ "7285299a_2"+ ],+ "gold_session_ids": [+ "answer_f4ea84fc_3",+ "answer_f4ea84fc_2",+ "answer_f4ea84fc_1"+ ]+ },+ {+ "question_id": "6e984302",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_88841f27_1",+ "41dc5d45_1",+ "c03fc56c_1",+ "7cd7c296_1",+ "answer_88841f27_2",+ "b7702ee4_1",+ "7285299a_8",+ "33a39aa7_3",+ "cf3dc165_1",+ "ultrachat_542864"+ ],+ "gold_session_ids": [+ "answer_88841f27_1",+ "answer_88841f27_2"+ ]+ },+ {+ "question_id": "gpt4_8279ba03",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "49b78a55_1",+ "bb107057_2",+ "answer_56521e66_1",+ "864a563d_5",+ "50d66391_4",+ "518d26d3_4",+ "c4661b3c_2",+ "ultrachat_526748",+ "084bb880_2",+ "d9868305_1"+ ],+ "gold_session_ids": [+ "answer_56521e66_1"+ ]+ },+ {+ "question_id": "gpt4_b5700ca0",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "109e07e8_2",+ "answer_a17423e8_1",+ "ultrachat_540666",+ "b403d6f0_4",+ "534be93d_1",+ "064f42ee_3",+ "e789afdb_2",+ "0f09447b",+ "c4251d8d_2",+ "14b000d9_4"+ ],+ "gold_session_ids": [+ "answer_a17423e8_1"+ ]+ },+ {+ "question_id": "gpt4_68e94288",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9793daa4_1",+ "128f4e4d_2",+ "ultrachat_74747",+ "ba80721c",+ "9d2fdc71",+ "d2606fe9",+ "sharegpt_RD37kJL_31",+ "d8454317_7",+ "4c49e37f",+ "ultrachat_301272"+ ],+ "gold_session_ids": [+ "answer_9793daa4_2",+ "answer_9793daa4_1"+ ]+ },+ {+ "question_id": "gpt4_2655b836",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4be1b6b4_2",+ "answer_4be1b6b4_1",+ "answer_4be1b6b4_3",+ "d8a1af6b_4",+ "sharegpt_KzDwaf1_237",+ "sharegpt_Tv9eXvf_0",+ "sharegpt_CUs8n0A_0",+ "ea8cc87b",+ "f220dd04_1",+ "e6120ff9_2"+ ],+ "gold_session_ids": [+ "answer_4be1b6b4_1",+ "answer_4be1b6b4_2",+ "answer_4be1b6b4_3"+ ]+ },+ {+ "question_id": "gpt4_2487a7cb",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1c6b85ea_2",+ "answer_1c6b85ea_1",+ "4f5880c6_6",+ "67a8ef5f",+ "4c1982d8_2",+ "f87fea58_4",+ "b7901ed7",+ "sharegpt_QN26oUg_0",+ "3afa6478",+ "sharegpt_gLb26GY_0"+ ],+ "gold_session_ids": [+ "answer_1c6b85ea_2",+ "answer_1c6b85ea_1"+ ]+ },+ {+ "question_id": "gpt4_76048e76",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b535969f_1",+ "answer_b535969f_2",+ "sharegpt_vx14orQ_11",+ "33630c9e_2",+ "6a78e959_2",+ "e2d9bcfe",+ "0125f748_1",+ "sharegpt_JprREQx_0",+ "33a39aa7_2",+ "907b6ed7_1"+ ],+ "gold_session_ids": [+ "answer_b535969f_1",+ "answer_b535969f_2"+ ]+ },+ {+ "question_id": "gpt4_2312f94c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5328c3c2_2",+ "answer_5328c3c2_1",+ "sharegpt_XYtuANl_9",+ "e831a29f_2",+ "ultrachat_437795",+ "3db86aab_1",+ "a764b677_2",+ "00ebfcf5",+ "5e39a233_1",+ "f34bf5e1"+ ],+ "gold_session_ids": [+ "answer_5328c3c2_2",+ "answer_5328c3c2_1"+ ]+ },+ {+ "question_id": "0bb5a684",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e936197f_1",+ "826d51da_3",+ "answer_e936197f_2",+ "565929a2_2",+ "3c8e7c0e_3",+ "ultrachat_62919",+ "3548cae3",+ "d794dec9_2",+ "sharegpt_AzkYB5s_0",+ "sharegpt_6dPUCcK_13"+ ],+ "gold_session_ids": [+ "answer_e936197f_1",+ "answer_e936197f_2"+ ]+ },+ {+ "question_id": "08f4fc43",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6ea1541e_2",+ "answer_6ea1541e_1",+ "58d11d0c_2",+ "sharegpt_wXqtiUe_0",+ "9216c4d3_4",+ "cdbd6862_1",+ "sharegpt_JJu53iW_0",+ "ultrachat_359609",+ "ultrachat_292416",+ "f3bd00ec_2"+ ],+ "gold_session_ids": [+ "answer_6ea1541e_2",+ "answer_6ea1541e_1"+ ]+ },+ {+ "question_id": "2c63a862",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5706417189553201,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "ad175dc4_2",+ "33baaaab_1",+ "answer_d39b7977_2",+ "answer_d39b7977_1",+ "sharegpt_lXtWJ7d_0",+ "918c4788_2",+ "1854abc7_2",+ "0fccc817_1",+ "ultrachat_179062",+ "b25f9c97_3"+ ],+ "gold_session_ids": [+ "answer_d39b7977_1",+ "answer_d39b7977_2"+ ]+ },+ {+ "question_id": "gpt4_385a5000",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7a4a93f1_2",+ "answer_7a4a93f1_1",+ "8062f4ab_2",+ "dc832f73",+ "cae65795_1",+ "28f74b09_1",+ "ultrachat_286149",+ "ultrachat_480527",+ "90019a03_2",+ "sharegpt_A4N0rHY_13"+ ],+ "gold_session_ids": [+ "answer_7a4a93f1_2",+ "answer_7a4a93f1_1"+ ]+ },+ {+ "question_id": "2a1811e2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1cc3cd0c_1",+ "answer_1cc3cd0c_2",+ "ultrachat_187883",+ "1aaf8057_3",+ "89e05490",+ "aac025d1_1",+ "b0e040c1",+ "b94ad1bc_4",+ "88ad913f_2",+ "sharegpt_v2b1TzW_35"+ ],+ "gold_session_ids": [+ "answer_1cc3cd0c_1",+ "answer_1cc3cd0c_2"+ ]+ },+ {+ "question_id": "bbf86515",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b3763b6b_1",+ "75b98d2a_1",+ "592cb8d2",+ "answer_b3763b6b_2",+ "df003c93_2",+ "e6fad20f",+ "sharegpt_GrLzv0V_432",+ "05260a4b_2",+ "6cd203f7_2",+ "e9e19917_1"+ ],+ "gold_session_ids": [+ "answer_b3763b6b_1",+ "answer_b3763b6b_2"+ ]+ },+ {+ "question_id": "gpt4_5dcc0aab",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_099c1b6c_3",+ "answer_099c1b6c_5",+ "answer_099c1b6c_4",+ "answer_099c1b6c_1",+ "answer_099c1b6c_2",+ "3891baea_3",+ "353d3c6d_1",+ "ultrachat_156812",+ "b10c041b_1",+ "3a192d16_1"+ ],+ "gold_session_ids": [+ "answer_099c1b6c_5",+ "answer_099c1b6c_2",+ "answer_099c1b6c_4",+ "answer_099c1b6c_3",+ "answer_099c1b6c_1"+ ]+ },+ {+ "question_id": "gpt4_0b2f1d21",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c4e5d969_1",+ "answer_c4e5d969_2",+ "sharegpt_hDjMGr7_30",+ "f31a0422_1",+ "1cafd864_3",+ "sharegpt_35udPn0_1",+ "sharegpt_ItYxNpw_0",+ "sharegpt_9JZCfee_42",+ "ultrachat_425364",+ "sharegpt_rhWNt54_19"+ ],+ "gold_session_ids": [+ "answer_c4e5d969_2",+ "answer_c4e5d969_1"+ ]+ },+ {+ "question_id": "f0853d11",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_932ea3bf_1",+ "answer_932ea3bf_2",+ "ultrachat_114660",+ "ultrachat_32771",+ "8c68df0d_2",+ "sharegpt_Dcshjlv_33",+ "b16a8656_2",+ "661e2cf8_3",+ "ultrachat_325202",+ "ultrachat_78285"+ ],+ "gold_session_ids": [+ "answer_932ea3bf_2",+ "answer_932ea3bf_1"+ ]+ },+ {+ "question_id": "gpt4_6ed717ea",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d50a8a33_2",+ "answer_d50a8a33_1",+ "15ed03f0_1",+ "e0585cb5_1",+ "637bc32a_2",+ "50940cb7_1",+ "24bfc009_3",+ "f849dd04",+ "d4b6ec0b",+ "sharegpt_nccc9Ke_21"+ ],+ "gold_session_ids": [+ "answer_d50a8a33_1",+ "answer_d50a8a33_2"+ ]+ },+ {+ "question_id": "gpt4_70e84552",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b3070ec4_2",+ "answer_b3070ec4_1",+ "67a8ef5f",+ "2f8a3459_2",+ "30491f6f_1",+ "sharegpt_znI61kP_0",+ "sharegpt_d7bP5fb_0",+ "f479774c",+ "ultrachat_573894",+ "ultrachat_468422"+ ],+ "gold_session_ids": [+ "answer_b3070ec4_2",+ "answer_b3070ec4_1"+ ]+ },+ {+ "question_id": "a3838d2b",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9796825595768971,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4ffa04a2_1",+ "answer_4ffa04a2_6",+ "answer_4ffa04a2_2",+ "answer_4ffa04a2_3",+ "answer_4ffa04a2_5",+ "06317e04",+ "e8de3056_1",+ "ae73b0f7_2",+ "ultrachat_154465",+ "answer_4ffa04a2_4"+ ],+ "gold_session_ids": [+ "answer_4ffa04a2_1",+ "answer_4ffa04a2_6",+ "answer_4ffa04a2_4",+ "answer_4ffa04a2_3",+ "answer_4ffa04a2_5",+ "answer_4ffa04a2_2"+ ]+ },+ {+ "question_id": "gpt4_93159ced",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e5131a1b_2",+ "answer_e5131a1b_1",+ "17d91635_1",+ "460be54a",+ "sharegpt_A83arI9_383",+ "ultrachat_387032",+ "61a989b5_3",+ "ultrachat_256765",+ "263efe64_1",+ "ultrachat_324774"+ ],+ "gold_session_ids": [+ "answer_e5131a1b_2",+ "answer_e5131a1b_1"+ ]+ },+ {+ "question_id": "gpt4_2d58bcd6",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3e11e0ae_2",+ "answer_3e11e0ae_1",+ "0bd0a01c",+ "1b205d70_1",+ "1bf8b212",+ "sharegpt_XUxvLWu_0",+ "sharegpt_4tLDGmp_0",+ "be32ad25_1",+ "sharegpt_Ntr6bOu_0",+ "5e50c81b"+ ],+ "gold_session_ids": [+ "answer_3e11e0ae_1",+ "answer_3e11e0ae_2"+ ]+ },+ {+ "question_id": "gpt4_65aabe59",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_30dfe889_1",+ "6a7e3aec",+ "answer_30dfe889_2",+ "ultrachat_190669",+ "ultrachat_509625",+ "f0158960",+ "ultrachat_386890",+ "ultrachat_216833",+ "sharegpt_F0NKEOn_2",+ "ultrachat_16520"+ ],+ "gold_session_ids": [+ "answer_30dfe889_1",+ "answer_30dfe889_2"+ ]+ },+ {+ "question_id": "982b5123",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ab603dd5_1",+ "answer_ab603dd5_2",+ "bf8ab267",+ "e36b0031",+ "1bfd5a8b_3",+ "07ba9acd_2",+ "a200b713_4",+ "04e05522_2",+ "ce8b2a9e_1",+ "359a2a0d"+ ],+ "gold_session_ids": [+ "answer_ab603dd5_1",+ "answer_ab603dd5_2"+ ]+ },+ {+ "question_id": "b9cfe692",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e3bb940_2",+ "answer_5e3bb940_1",+ "answer_5e3bb940_3",+ "sharegpt_JRQtEcd_115",+ "0c27c633_2",+ "sharegpt_oXgiN7q_53",+ "da650f3b",+ "2c7bff3a_1",+ "91d847c1_6",+ "sharegpt_ErOTMZ3_149"+ ],+ "gold_session_ids": [+ "answer_5e3bb940_3",+ "answer_5e3bb940_2",+ "answer_5e3bb940_1"+ ]+ },+ {+ "question_id": "gpt4_4edbafa2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0a00c163_2",+ "answer_0a00c163_1",+ "a6f2320f_2",+ "7a7a4bf0_3",+ "7533d546_5",+ "d36a36f2",+ "2ba7fdd0_2",+ "22be78fc_2",+ "a65d076a",+ "3f5249ee_3"+ ],+ "gold_session_ids": [+ "answer_0a00c163_1",+ "answer_0a00c163_2"+ ]+ },+ {+ "question_id": "c8090214",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_70dc7d08_2",+ "162ad3bf_1",+ "c21c9d97",+ "answer_70dc7d08_1",+ "005ec7cf_1",+ "sharegpt_yXAlFu3_0",+ "sharegpt_xOzzaof_0",+ "5b415488_2",+ "ultrachat_518687",+ "f2939479_1"+ ],+ "gold_session_ids": [+ "answer_70dc7d08_2",+ "answer_70dc7d08_1"+ ]+ },+ {+ "question_id": "gpt4_483dd43c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fb793c87_1",+ "answer_fb793c87_2",+ "73353aee_3",+ "ultrachat_375572",+ "3fd56ebf",+ "9d0f4bfa_2",+ "sharegpt_I6IzvEF_0",+ "sharegpt_ItFPuha_0",+ "de9f265f",+ "4798296c_1"+ ],+ "gold_session_ids": [+ "answer_fb793c87_1",+ "answer_fb793c87_2"+ ]+ },+ {+ "question_id": "e4e14d04",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cf425855_2",+ "answer_cf425855_1",+ "5944b36a_1",+ "sharegpt_HO1gGmm_0",+ "ffc627f2",+ "d64a0896",+ "f8de4e92_2",+ "1c1f5ccc_3",+ "ultrachat_200098",+ "a92165a5"+ ],+ "gold_session_ids": [+ "answer_cf425855_1",+ "answer_cf425855_2"+ ]+ },+ {+ "question_id": "c9f37c46",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cdba3d9f_2",+ "answer_cdba3d9f_1",+ "c3d6a282_1",+ "0cf86cbe_1",+ "b38766c1_3",+ "4ffa04a2_1",+ "9d4312f6_3",+ "sharegpt_94CjWOc_0",+ "211bb1c2_1",+ "50136b31"+ ],+ "gold_session_ids": [+ "answer_cdba3d9f_1",+ "answer_cdba3d9f_2"+ ]+ },+ {+ "question_id": "gpt4_2c50253f",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9af4e346_2",+ "answer_9af4e346_1",+ "4b9ed528_2",+ "4cef8a3c_3",+ "sharegpt_Bmic1Id_51",+ "0138fdec",+ "57c8bf62_1",+ "69a9211c_3",+ "b6f35f3a_1",+ "sharegpt_CWlW9UH_0"+ ],+ "gold_session_ids": [+ "answer_9af4e346_1",+ "answer_9af4e346_2"+ ]+ },+ {+ "question_id": "dcfa8644",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5e3eeb12_2",+ "answer_5e3eeb12_1",+ "b2243528_1",+ "sharegpt_oG3LXXo_6",+ "f36d82f7",+ "29a7a78d_1",+ "09f15689",+ "d15d2899_2",+ "30bbae12_2",+ "41f94af6"+ ],+ "gold_session_ids": [+ "answer_5e3eeb12_2",+ "answer_5e3eeb12_1"+ ]+ },+ {+ "question_id": "gpt4_b4a80587",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b9d9150e_1",+ "answer_b9d9150e_2",+ "sharegpt_inuIr9J_119",+ "f79a48dd_1",+ "sharegpt_HxOhjmq_29",+ "3cb41ab8_1",+ "55bcfb77",+ "sharegpt_zLYVdSv_0",+ "866de400",+ "ultrachat_175745"+ ],+ "gold_session_ids": [+ "answer_b9d9150e_2",+ "answer_b9d9150e_1"+ ]+ },+ {+ "question_id": "gpt4_9a159967",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8a42fedf_1",+ "answer_8a42fedf_3",+ "answer_8a42fedf_2",+ "4ea94f85",+ "75671e3f_1",+ "6c9bcb0f_1",+ "b2cae160_1",+ "ca529c5b_2",+ "ultrachat_534927",+ "0df54428_4"+ ],+ "gold_session_ids": [+ "answer_8a42fedf_1",+ "answer_8a42fedf_3",+ "answer_8a42fedf_2"+ ]+ },+ {+ "question_id": "cc6d1ec1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_be73098b_1",+ "answer_be73098b_2",+ "3cb41ab8_1",+ "22db3cc3_4",+ "8a42fedf_2",+ "sharegpt_ckqShdI_21",+ "ultrachat_508911",+ "33baaaab_3",+ "e549b6b2_2",+ "336b281b_2"+ ],+ "gold_session_ids": [+ "answer_be73098b_2",+ "answer_be73098b_1"+ ]+ },+ {+ "question_id": "gpt4_8c8961ae",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_72d9aa58_1",+ "answer_72d9aa58_2",+ "3b22d17b",+ "37f0ce6b_1",+ "05fad1cb_2",+ "10466528_3",+ "ultrachat_334521",+ "1cde7ee4_2",+ "b77d0b58",+ "ultrachat_197141"+ ],+ "gold_session_ids": [+ "answer_72d9aa58_1",+ "answer_72d9aa58_2"+ ]+ },+ {+ "question_id": "gpt4_d9af6064",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da704e79_1",+ "answer_da704e79_2",+ "ultrachat_182531",+ "sharegpt_VbIxRWJ_33",+ "03b4d5d8_2",+ "ultrachat_519452",+ "sharegpt_w9YUs4c_0",+ "a1b3dd27_1",+ "ff49b5a5_2",+ "0d9a6f01"+ ],+ "gold_session_ids": [+ "answer_da704e79_2",+ "answer_da704e79_1"+ ]+ },+ {+ "question_id": "gpt4_7de946e7",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_6a78e959_2",+ "answer_6a78e959_1",+ "caa00337_2",+ "58bec5fb_3",+ "ultrachat_105249",+ "3023b36f",+ "ultrachat_293723",+ "ultrachat_167604",+ "sharegpt_6Yfj9A3_5",+ "6155addd_2"+ ],+ "gold_session_ids": [+ "answer_6a78e959_2",+ "answer_6a78e959_1"+ ]+ },+ {+ "question_id": "d01c6aa8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_991d55e5_1",+ "fc9a62c0_1",+ "7d1f5395",+ "a68ad96e_1",+ "answer_991d55e5_2",+ "sharegpt_xd1G5uV_0",+ "64460e9d_1",+ "ultrachat_10123",+ "fbd67491_2",+ "5024758e"+ ],+ "gold_session_ids": [+ "answer_991d55e5_1",+ "answer_991d55e5_2"+ ]+ },+ {+ "question_id": "993da5e2",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_54f0d6f9_2",+ "answer_54f0d6f9_1",+ "12cd736c",+ "ultrachat_217815",+ "ultrachat_73982",+ "13f3da23_2",+ "8953c8c8_3",+ "b16a8656_2",+ "c756d93e",+ "3ac33554_2"+ ],+ "gold_session_ids": [+ "answer_54f0d6f9_1",+ "answer_54f0d6f9_2"+ ]+ },+ {+ "question_id": "a3045048",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5912352048230277,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "aaf71ce2_2",+ "answer_016f6bd4_1",+ "d12d5a98_1",+ "2def525b_1",+ "sharegpt_asoSnhq_0",+ "sharegpt_YblmreD_53",+ "answer_016f6bd4_2",+ "sharegpt_LC9Pzkr_0",+ "ultrachat_19072",+ "536a334d"+ ],+ "gold_session_ids": [+ "answer_016f6bd4_2",+ "answer_016f6bd4_1"+ ]+ },+ {+ "question_id": "gpt4_d31cdae3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8464304d_2",+ "de695a4e",+ "55033f6f",+ "ultrachat_324099",+ "sharegpt_HDF4645_0",+ "sharegpt_g6KbKeo_0",+ "41743aae_2",+ "809cabca_2",+ "f87fea58_1",+ "sharegpt_7W1Sg8f_0"+ ],+ "gold_session_ids": [+ "answer_8464304d_1",+ "answer_8464304d_2"+ ]+ },+ {+ "question_id": "gpt4_cd90e484",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aa930b56_2",+ "answer_aa930b56_1",+ "6043dcf6_1",+ "4bfcc251_1",+ "866de400",+ "08c306e2",+ "ultrachat_347068",+ "61b27d67_3",+ "018eac29",+ "cc5a6057_2"+ ],+ "gold_session_ids": [+ "answer_aa930b56_1",+ "answer_aa930b56_2"+ ]+ },+ {+ "question_id": "gpt4_88806d6e",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e60a93ff_1",+ "answer_e60a93ff_2",+ "9ac3fa82_1",+ "24bfc009_5",+ "sharegpt_WT6LnXo_0",+ "743b1fc3_2",+ "sharegpt_5RbA8hu_0",+ "sharegpt_mMdyu3t_0",+ "3c11dc79",+ "c11fb634_1"+ ],+ "gold_session_ids": [+ "answer_e60a93ff_1",+ "answer_e60a93ff_2"+ ]+ },+ {+ "question_id": "gpt4_4cd9eba1",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5fcca8bc_1",+ "answer_5fcca8bc_2",+ "1b576f6e_2",+ "d36a862c_1",+ "sharegpt_3hCiKrk_0",+ "dafc2440_2",+ "456860e7_2",+ "6b9b2b1e_1",+ "8242fa3f_1",+ "ultrachat_221041"+ ],+ "gold_session_ids": [+ "answer_5fcca8bc_2",+ "answer_5fcca8bc_1"+ ]+ },+ {+ "question_id": "gpt4_93f6379c",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9469024295259745,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_544fe66c_3",+ "answer_544fe66c_2",+ "sharegpt_cfESFSo_0",+ "aee99715",+ "answer_544fe66c_1",+ "b8a688f6",+ "sharegpt_ZY5xXZa_7",+ "sharegpt_y7eQRmk_0",+ "de695a4e",+ "c07b8034"+ ],+ "gold_session_ids": [+ "answer_544fe66c_2",+ "answer_544fe66c_1",+ "answer_544fe66c_3"+ ]+ },+ {+ "question_id": "b29f3365",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_436d4309_2",+ "answer_436d4309_1",+ "6b7605d1_1",+ "6a2f9452_1",+ "06ee3665",+ "sharegpt_WjkZjl4_28",+ "53785a15",+ "1dae02fe",+ "c279c578",+ "4ccc2061"+ ],+ "gold_session_ids": [+ "answer_436d4309_2",+ "answer_436d4309_1"+ ]+ },+ {+ "question_id": "gpt4_2f56ae70",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7a36e820_2",+ "answer_7a36e820_3",+ "answer_7a36e820_1",+ "647f5851",+ "dc3ee1d1_1",+ "c578da1f_2",+ "541ecc45_2",+ "f2835879_2",+ "2889e785_1",+ "74c8f7bb"+ ],+ "gold_session_ids": [+ "answer_7a36e820_2",+ "answer_7a36e820_3",+ "answer_7a36e820_1"+ ]+ },+ {+ "question_id": "6613b389",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9674679834891693,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_aaf71ce2_2",+ "answer_aaf71ce2_3",+ "e3d8da67",+ "answer_aaf71ce2_1",+ "86c8f348_1",+ "ultrachat_31390",+ "20d6c376_1",+ "df8a7455",+ "f1cc1ddb_1",+ "1a65880d"+ ],+ "gold_session_ids": [+ "answer_aaf71ce2_3",+ "answer_aaf71ce2_2",+ "answer_aaf71ce2_1"+ ]+ },+ {+ "question_id": "gpt4_78cf46a3",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "sharegpt_rx39ipj_0",+ "answer_5a78688d_1",+ "answer_5a78688d_2",+ "ultrachat_305042",+ "52b19cba_1",+ "ultrachat_370708",+ "42b45f6e_2",+ "sharegpt_ZWqMvoL_593",+ "b89a3ec3",+ "e8e65a1f"+ ],+ "gold_session_ids": [+ "answer_5a78688d_1",+ "answer_5a78688d_2"+ ]+ },+ {+ "question_id": "gpt4_0a05b494",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a68db5db_2",+ "answer_a68db5db_1",+ "0abe3e51",+ "ultrachat_273213",+ "sharegpt_eWQFoct_17",+ "ab603dd5_2",+ "56911dc5_6",+ "8c4995d0",+ "05b84f71_1",+ "30677a32"+ ],+ "gold_session_ids": [+ "answer_a68db5db_2",+ "answer_a68db5db_1"+ ]+ },+ {+ "question_id": "gpt4_1a1dc16d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_faad7d7a_2",+ "c12d5181_1",+ "87f1f950_2",+ "answer_faad7d7a_1",+ "ultrachat_480665",+ "8bd3814a_2",+ "ultrachat_428346",+ "sharegpt_6cz1Sq6_328",+ "sharegpt_u1aJmpT_0",+ "14940349_2"+ ],+ "gold_session_ids": [+ "answer_faad7d7a_2",+ "answer_faad7d7a_1"+ ]+ },+ {+ "question_id": "gpt4_2f584639",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_11a8f823_1",+ "answer_11a8f823_2",+ "c6eb17f0_1",+ "821242cd_2",+ "13947b64_1",+ "sharegpt_bq9G6bT_2",+ "ultrachat_464469",+ "ultrachat_430445",+ "c3d89115",+ "d441f4c5"+ ],+ "gold_session_ids": [+ "answer_11a8f823_2",+ "answer_11a8f823_1"+ ]+ },+ {+ "question_id": "gpt4_213fd887",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_53582e7e_2",+ "answer_53582e7e_1",+ "9bdbfc62_2",+ "3b0a7d5d_1",+ "368276ab_3",+ "3b38bf46",+ "d033262b_2",+ "ultrachat_165091",+ "sharegpt_PvScEsS_0",+ "fb3ec1ff"+ ],+ "gold_session_ids": [+ "answer_53582e7e_2",+ "answer_53582e7e_1"+ ]+ },+ {+ "question_id": "gpt4_5438fa52",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.46845052016107697,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "a8e24e22",+ "528f481d_6",+ "d68bab3c_1",+ "answer_b10f3828_1",+ "ultrachat_69599",+ "290fe09a",+ "answer_b10f3828_2",+ "ab8dce45",+ "ultrachat_160450",+ "828c3d75"+ ],+ "gold_session_ids": [+ "answer_b10f3828_1",+ "answer_b10f3828_2"+ ]+ },+ {+ "question_id": "gpt4_c27434e8",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8e33f5c_2",+ "answer_d8e33f5c_1",+ "4d578deb",+ "sharegpt_9XlbWvL_0",+ "sharegpt_TRck0h4_0",+ "11cd43df",+ "ultrachat_556731",+ "820c1a20",+ "ultrachat_444552",+ "ultrachat_70335"+ ],+ "gold_session_ids": [+ "answer_d8e33f5c_1",+ "answer_d8e33f5c_2"+ ]+ },+ {+ "question_id": "gpt4_fe651585",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_65600ff6_2",+ "answer_65600ff6_1",+ "sharegpt_gupA3qh_0",+ "sharegpt_mJ7MnQS_219",+ "9f220c66_1",+ "ultrachat_576354",+ "sharegpt_izuEpAx_20",+ "987b36db_3",+ "e30c4a7b_2",+ "bc1f1051_1"+ ],+ "gold_session_ids": [+ "answer_65600ff6_2",+ "answer_65600ff6_1"+ ]+ },+ {+ "question_id": "8c18457d",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_124f5dc3_1",+ "answer_124f5dc3_2",+ "98cd882c_2",+ "c8dcbe7f_1",+ "1a5369b9",+ "ultrachat_417305",+ "3c8e7c0e_2",+ "4745bb2e",+ "c3023a45_2",+ "9f803df5_1"+ ],+ "gold_session_ids": [+ "answer_124f5dc3_2",+ "answer_124f5dc3_1"+ ]+ },+ {+ "question_id": "gpt4_70e84552_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b3070ec4_abs_1",+ "answer_b3070ec4_abs_2",+ "f6ba44fe_1",+ "e2b58a46_1",+ "e2cd250e_2",+ "430d0a87_2",+ "4e0da76d",+ "sharegpt_EypOKzQ_0",+ "59264d33_2",+ "ultrachat_475184"+ ],+ "gold_session_ids": [+ "answer_b3070ec4_abs_2",+ "answer_b3070ec4_abs_1"+ ]+ },+ {+ "question_id": "gpt4_93159ced_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e5131a1b_abs_1",+ "answer_e5131a1b_abs_2",+ "sharegpt_CyJ3dal_43",+ "ultrachat_195762",+ "56bba837",+ "sharegpt_gYShON7_0",+ "156988fe",+ "2f097fba_1",+ "76385c25",+ "168feef7_2"+ ],+ "gold_session_ids": [+ "answer_e5131a1b_abs_1",+ "answer_e5131a1b_abs_2"+ ]+ },+ {+ "question_id": "982b5123_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8175295903539445,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ab603dd5_abs_1",+ "11e942da_1",+ "fe82a033_1",+ "4f3b0353_2",+ "a8179676",+ "3fc2244f",+ "answer_ab603dd5_abs_2",+ "b1793bba",+ "5080119d_1",+ "6e5130ed_1"+ ],+ "gold_session_ids": [+ "answer_ab603dd5_abs_1",+ "answer_ab603dd5_abs_2"+ ]+ },+ {+ "question_id": "c8090214_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "64c71782",+ "answer_70dc7d08_abs_1",+ "b192ae00_1",+ "ultrachat_393699",+ "d4f02577_2",+ "ca3aa07f_1",+ "ultrachat_129606",+ "65c21f34_1",+ "ultrachat_133575",+ "fda42399_3"+ ],+ "gold_session_ids": [+ "answer_70dc7d08_abs_1",+ "answer_70dc7d08_abs_2"+ ]+ },+ {+ "question_id": "gpt4_c27434e8_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d8e33f5c_abs_1",+ "answer_d8e33f5c_abs_2",+ "ultrachat_106944",+ "fccc9400",+ "ultrachat_552313",+ "ae0cd456",+ "sharegpt_hGtkNGL_12",+ "2e28f3db",+ "sharegpt_iYoVtKC_0",+ "59a6150d_3"+ ],+ "gold_session_ids": [+ "answer_d8e33f5c_abs_1",+ "answer_d8e33f5c_abs_2"+ ]+ },+ {+ "question_id": "gpt4_fe651585_abs",+ "question_type": "temporal-reasoning",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_65600ff6_abs_2",+ "sharegpt_7PLgNll_0",+ "answer_65600ff6_abs_1",+ "sharegpt_v6ZRge5_0",+ "sharegpt_TBRPvuE_0",+ "sharegpt_wrN9uUo_53",+ "caa00337_1",+ "sharegpt_jVFHJsP_0",+ "sharegpt_NMLgKqu_7",+ "85a1c17b_1"+ ],+ "gold_session_ids": [+ "answer_65600ff6_abs_1",+ "answer_65600ff6_abs_2"+ ]+ },+ {+ "question_id": "6a1eabeb",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a25d4a91_2",+ "4aaf678f_3",+ "afe04238_7",+ "answer_a25d4a91_1",+ "9b3a7f2c_1",+ "6da0bba2_1",+ "d03995c6",+ "9f80607d",+ "sharegpt_3Jpu3tr_13",+ "4c49e37f"+ ],+ "gold_session_ids": [+ "answer_a25d4a91_1",+ "answer_a25d4a91_2"+ ]+ },+ {+ "question_id": "6aeb4375",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3f9693b7_1",+ "answer_3f9693b7_2",+ "402e0082_5",+ "29621a34",+ "08256f84_2",+ "1bc87711_1",+ "sharegpt_MWlvrvZ_0",+ "2fd43bba",+ "03105808_2",+ "dc9bf721_1"+ ],+ "gold_session_ids": [+ "answer_3f9693b7_1",+ "answer_3f9693b7_2"+ ]+ },+ {+ "question_id": "830ce83f",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5706417189553201,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "a7f88bb3_2",+ "dfbda5e9",+ "answer_0b1a0942_2",+ "answer_0b1a0942_1",+ "d1e7d11c_2",+ "ultrachat_288747",+ "ultrachat_441391",+ "7ddf575b",+ "7a389bd1_7",+ "ultrachat_346816"+ ],+ "gold_session_ids": [+ "answer_0b1a0942_1",+ "answer_0b1a0942_2"+ ]+ },+ {+ "question_id": "852ce960",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3a6f1e82_2",+ "answer_3a6f1e82_1",+ "010a28ab_3",+ "sharegpt_FYhsZ0Q_0",+ "a76e7e3c_1",+ "e8bfacec_1",+ "bdfcb4d3",+ "sharegpt_eN4o9Zr_0",+ "sharegpt_V2j1zkI_1",+ "sharegpt_uiECyCD_66"+ ],+ "gold_session_ids": [+ "answer_3a6f1e82_1",+ "answer_3a6f1e82_2"+ ]+ },+ {+ "question_id": "945e3d21",+ "question_type": "knowledge-update",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.40298313359773236,+ "mrr": 0.16666666666666666,+ "retrieved_session_ids": [+ "ef0341d4_2",+ "5fcca8bc_1",+ "ultrachat_242046",+ "f5025173_2",+ "sharegpt_E7VUcfj_0",+ "answer_6a4f8626_2",+ "fc69fd44_7",+ "940629b2_1",+ "answer_6a4f8626_1",+ "4019b1a8_4"+ ],+ "gold_session_ids": [+ "answer_6a4f8626_1",+ "answer_6a4f8626_2"+ ]+ },+ {+ "question_id": "d7c942c3",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5706417189553201,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "cf8e2761",+ "ddf0f116_3",+ "answer_eecb10d9_1",+ "answer_eecb10d9_2",+ "sharegpt_zxm712D_0",+ "695bc054_1",+ "sharegpt_vYQdLPe_2",+ "8bd5b3c4_2",+ "f0b803b0_2",+ "4e70b8fd"+ ],+ "gold_session_ids": [+ "answer_eecb10d9_1",+ "answer_eecb10d9_2"+ ]+ },+ {+ "question_id": "71315a70",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c44b9df4_2",+ "answer_c44b9df4_1",+ "a08fbe88_1",+ "ultrachat_42515",+ "8cfa91c3_1",+ "b5e94516_2",+ "37d634c0_1",+ "ultrachat_399278",+ "773aebbd_1",+ "ultrachat_208238"+ ],+ "gold_session_ids": [+ "answer_c44b9df4_1",+ "answer_c44b9df4_2"+ ]+ },+ {+ "question_id": "89941a93",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e1403127_2",+ "answer_e1403127_1",+ "1e88ebca",+ "dde142b2_2",+ "25956eab_1",+ "169213ae",+ "ultrachat_214088",+ "ultrachat_453348",+ "8e72316f",+ "sharegpt_jxsIY6L_0"+ ],+ "gold_session_ids": [+ "answer_e1403127_1",+ "answer_e1403127_2"+ ]+ },+ {+ "question_id": "ce6d2d27",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_73540165_2",+ "7a4d00b3_2",+ "48ad2a93",+ "answer_73540165_1",+ "168776ed_1",+ "cd92ac85_1",+ "a1c4f65a",+ "54f0d6f9_2",+ "9f6bec4f",+ "sharegpt_WQctw3k_0"+ ],+ "gold_session_ids": [+ "answer_73540165_1",+ "answer_73540165_2"+ ]+ },+ {+ "question_id": "9ea5eabc",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_02e66dec_2",+ "6e2cca63_1",+ "answer_02e66dec_1",+ "8c8b180e",+ "9bf5da2d",+ "418c27fa",+ "d1f30ac6_5",+ "62e6593e_1",+ "7c44bc2d_1",+ "29695e1c_2"+ ],+ "gold_session_ids": [+ "answer_02e66dec_1",+ "answer_02e66dec_2"+ ]+ },+ {+ "question_id": "07741c44",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7e9ad7b4_2",+ "answer_7e9ad7b4_1",+ "8d969f3e_2",+ "a4b136d1_2",+ "01d22d4d",+ "98ae7eef",+ "sharegpt_CEWG2XW_0",+ "aa71db11_1",+ "61fc1d85",+ "f6b80ebe"+ ],+ "gold_session_ids": [+ "answer_7e9ad7b4_1",+ "answer_7e9ad7b4_2"+ ]+ },+ {+ "question_id": "a1eacc2a",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0eb23770_2",+ "answer_0eb23770_1",+ "521255f7_2",+ "1f035408",+ "93f23301_1",+ "1ccb1920",+ "20d6c376_2",+ "c910dd2c_3",+ "sharegpt_ePBzYD0_57",+ "ultrachat_187728"+ ],+ "gold_session_ids": [+ "answer_0eb23770_1",+ "answer_0eb23770_2"+ ]+ },+ {+ "question_id": "184da446",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e2f4f947_2",+ "bf633415_2",+ "answer_e2f4f947_1",+ "sharegpt_IMtsj65_0",+ "61ad6074",+ "341a737e",+ "e5f08e40",+ "30a3c930",+ "sharegpt_nsHU229_0",+ "82a89303"+ ],+ "gold_session_ids": [+ "answer_e2f4f947_1",+ "answer_e2f4f947_2"+ ]+ },+ {+ "question_id": "031748ae",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8748f791_1",+ "d0488050",+ "5ac7f9c5_3",+ "70764af1_4",+ "answer_8748f791_2",+ "ultrachat_435543",+ "sharegpt_Ukl9zsG_0",+ "920fa36f",+ "sharegpt_pPNVaqR_0",+ "sharegpt_3D7sTTH_0"+ ],+ "gold_session_ids": [+ "answer_8748f791_1",+ "answer_8748f791_2"+ ]+ },+ {+ "question_id": "4d6b87c8",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_766ab8da_2",+ "answer_766ab8da_1",+ "d87e86f6_1",+ "sharegpt_xLjgprT_0",+ "4dd737a9_1",+ "5c85023a_1",+ "60d4332d",+ "sharegpt_yPCVYhX_0",+ "a0201607_3",+ "85da7827"+ ],+ "gold_session_ids": [+ "answer_766ab8da_1",+ "answer_766ab8da_2"+ ]+ },+ {+ "question_id": "0f05491a",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6052602440527057,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "9f4bec78",+ "answer_d6d2eba8_1",+ "057cbbc4_1",+ "27aba903_2",+ "sharegpt_HO1gGmm_4",+ "answer_d6d2eba8_2",+ "28000804",+ "44678992_1",+ "sharegpt_UpYQ1kh_79",+ "d81d9846"+ ],+ "gold_session_ids": [+ "answer_d6d2eba8_1",+ "answer_d6d2eba8_2"+ ]+ },+ {+ "question_id": "08e075c7",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "25b3e36c_1",+ "answer_cdbe2250_1",+ "a2fa5ba3",+ "answer_cdbe2250_2",+ "0bd59f15_3",+ "0fea08a2",+ "5ff94163_1",+ "385683f0_1",+ "b76006cb_2",+ "b1d9d555_1"+ ],+ "gold_session_ids": [+ "answer_cdbe2250_1",+ "answer_cdbe2250_2"+ ]+ },+ {+ "question_id": "f9e8c073",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.7977228895450266,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b191df5b_1",+ "ultrachat_88473",+ "sharegpt_Fhigp7F_1",+ "50b32eed_2",+ "sharegpt_WQun2Qq_0",+ "sharegpt_GDFyxeT_0",+ "4604bc73_2",+ "669318cf_3",+ "answer_b191df5b_2",+ "d5f4d9fa_3"+ ],+ "gold_session_ids": [+ "answer_b191df5b_1",+ "answer_b191df5b_2"+ ]+ },+ {+ "question_id": "41698283",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c7ddc051_1",+ "answer_c7ddc051_2",+ "431ae25c",+ "631e4016",+ "f35e3cb7_1",+ "0e8ba03b_1",+ "8f3246db",+ "bf386a67_3",+ "4464626e_1",+ "28e948f5"+ ],+ "gold_session_ids": [+ "answer_c7ddc051_1",+ "answer_c7ddc051_2"+ ]+ },+ {+ "question_id": "2698e78f",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_9282283d_2",+ "a0fcf806",+ "sharegpt_DrLV0RL_0",+ "answer_9282283d_1",+ "0fffb38f_2",+ "ultrachat_455955",+ "7ca763c3_2",+ "d31c3ec8_4",+ "ultrachat_110771",+ "ultrachat_35438"+ ],+ "gold_session_ids": [+ "answer_9282283d_1",+ "answer_9282283d_2"+ ]+ },+ {+ "question_id": "b6019101",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_67074b4b_1",+ "answer_67074b4b_2",+ "de43030f_2",+ "sharegpt_CvVLg2J_18",+ "ultrachat_72569",+ "0d66562c_4",+ "ultrachat_338597",+ "sharegpt_aVExml0_235",+ "bf4fdf8f_1",+ "ultrachat_242137"+ ],+ "gold_session_ids": [+ "answer_67074b4b_1",+ "answer_67074b4b_2"+ ]+ },+ {+ "question_id": "45dc21b6",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_07664d43_1",+ "8acb29f9_1",+ "0ad56ba6",+ "0504a710_4",+ "eb34144a_2",+ "sharegpt_gGriMhQ_91",+ "369695b4_2",+ "sharegpt_mMdyu3t_0",+ "de459c4b",+ "ultrachat_494140"+ ],+ "gold_session_ids": [+ "answer_07664d43_1",+ "answer_07664d43_2"+ ]+ },+ {+ "question_id": "5a4f22c0",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_b0f3dfff_2",+ "answer_b0f3dfff_1",+ "195f2929",+ "007e7d81_2",+ "c3f945f3_1",+ "f188b48d_1",+ "sharegpt_I0uNTjm_22",+ "ultrachat_208478",+ "75302ce3",+ "af778ece_1"+ ],+ "gold_session_ids": [+ "answer_b0f3dfff_1",+ "answer_b0f3dfff_2"+ ]+ },+ {+ "question_id": "6071bd76",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_4dac77cb_2",+ "answer_4dac77cb_1",+ "9de53d68_1",+ "e6b6353d",+ "sharegpt_3D3oQC0_12",+ "8d0b4363",+ "ultrachat_334948",+ "4e8b74ca_1",+ "710833f3",+ "ultrachat_431164"+ ],+ "gold_session_ids": [+ "answer_4dac77cb_1",+ "answer_4dac77cb_2"+ ]+ },+ {+ "question_id": "e493bb7c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1a374afa_2",+ "ultrachat_185447",+ "answer_1a374afa_1",+ "7aaa23d7_2",+ "0aacd15f_1",+ "ultrachat_363167",+ "ultrachat_415114",+ "ultrachat_392200",+ "ultrachat_555062",+ "ultrachat_203191"+ ],+ "gold_session_ids": [+ "answer_1a374afa_1",+ "answer_1a374afa_2"+ ]+ },+ {+ "question_id": "618f13b2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_caf5b52e_2",+ "answer_caf5b52e_1",+ "a20bc58f",+ "dda70510_2",+ "3a980e89_3",+ "86b56135_2",+ "748eb152_2",+ "4632c2c6_1",+ "c51583cd_3",+ "ultrachat_558040"+ ],+ "gold_session_ids": [+ "answer_caf5b52e_1",+ "answer_caf5b52e_2"+ ]+ },+ {+ "question_id": "72e3ee87",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d7de9a6a_1",+ "answer_d7de9a6a_2",+ "ultrachat_195333",+ "080a4000_2",+ "73a2f346_3",+ "be8c3970_2",+ "ultrachat_421360",+ "sharegpt_i6wl0uh_0",+ "ultrachat_233144",+ "6f2c01fc_1"+ ],+ "gold_session_ids": [+ "answer_d7de9a6a_1",+ "answer_d7de9a6a_2"+ ]+ },+ {+ "question_id": "c4ea545c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d3bf812b_2",+ "answer_d3bf812b_1",+ "sharegpt_71QV6Zs_7",+ "5be74579",+ "ea0e2434",+ "a2a8e2c9",+ "fcddf7d1_2",+ "9f9b402a_5",+ "493e23ac",+ "508d552e_5"+ ],+ "gold_session_ids": [+ "answer_d3bf812b_1",+ "answer_d3bf812b_2"+ ]+ },+ {+ "question_id": "01493427",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a7b44747_2",+ "answer_a7b44747_1",+ "88e73f02",+ "077f7bcb_3",+ "5a840928_1",+ "780b6c7c_1",+ "616fdd27",+ "58e5adf2",+ "291d8c82",+ "087ac53a_2"+ ],+ "gold_session_ids": [+ "answer_a7b44747_1",+ "answer_a7b44747_2"+ ]+ },+ {+ "question_id": "6a27ffc2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_77f32504_2",+ "answer_77f32504_1",+ "4604bc73_3",+ "6ae451d1_2",+ "sharegpt_tco8eKE_0",+ "de26a9d4",+ "ultrachat_183936",+ "sharegpt_CW8gAFB_8",+ "1388b204_2",+ "sharegpt_kkqlRlo_13"+ ],+ "gold_session_ids": [+ "answer_77f32504_1",+ "answer_77f32504_2"+ ]+ },+ {+ "question_id": "2133c1b5",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_52382508_2",+ "a52d98bf_2",+ "8ca0085a_1",+ "ultrachat_328330",+ "sharegpt_I1DGUU4_0",+ "ultrachat_354956",+ "sharegpt_bSlIUYf_32",+ "add9b012_1",+ "sharegpt_MvUpG5V_5",+ "a63ad8e3_3"+ ],+ "gold_session_ids": [+ "answer_52382508_1",+ "answer_52382508_2"+ ]+ },+ {+ "question_id": "18bc8abd",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_fff743f5_1",+ "answer_fff743f5_2",+ "951df45e",+ "7169e342_1",+ "ultrachat_529103",+ "ba6b67af_3",+ "2714885f_3",+ "ac549598_1",+ "4ad82121_2",+ "ultrachat_225891"+ ],+ "gold_session_ids": [+ "answer_fff743f5_1",+ "answer_fff743f5_2"+ ]+ },+ {+ "question_id": "db467c8c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_611b6e83_1",+ "answer_611b6e83_2",+ "ultrachat_553961",+ "77cf4551_1",+ "fb5dd87f_4",+ "sharegpt_HBmf6q0_0",+ "sharegpt_TAL32at_0",+ "ultrachat_527913",+ "6cc835bd",+ "88d42833"+ ],+ "gold_session_ids": [+ "answer_611b6e83_1",+ "answer_611b6e83_2"+ ]+ },+ {+ "question_id": "7a87bd0c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d08a934d_1",+ "answer_d08a934d_2",+ "3d98b7e6_2",+ "93e9e82c_2",+ "81f318a2_1",+ "62369c56_1",+ "15519944_1",+ "e250791e_3",+ "sharegpt_e1Mbyy2_28",+ "2c543c87"+ ],+ "gold_session_ids": [+ "answer_d08a934d_1",+ "answer_d08a934d_2"+ ]+ },+ {+ "question_id": "e61a7584",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f25c32f5_1",+ "answer_f25c32f5_2",+ "6a4b4b7e_2",+ "82159da4_1",+ "sharegpt_o0MporM_0",+ "e900ff31_2",+ "e1416816_2",+ "ultrachat_567152",+ "f379d356_3",+ "afa96a40"+ ],+ "gold_session_ids": [+ "answer_f25c32f5_1",+ "answer_f25c32f5_2"+ ]+ },+ {+ "question_id": "1cea1afa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_79c395a9_1",+ "answer_79c395a9_2",+ "e131bcd4",+ "9c479e89_1",+ "ultrachat_82739",+ "ultrachat_165628",+ "sharegpt_inuIr9J_119",+ "97f564ef",+ "602ad002_1",+ "c8fcdf5a_2"+ ],+ "gold_session_ids": [+ "answer_79c395a9_1",+ "answer_79c395a9_2"+ ]+ },+ {+ "question_id": "ed4ddc30",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_babbaccb_2",+ "answer_babbaccb_1",+ "8b726b64_1",+ "c279c578",+ "499652a0_3",+ "5a6dc9cc",+ "e4969569_2",+ "2490f2af_3",+ "5170dcc7_2",+ "75621e39_1"+ ],+ "gold_session_ids": [+ "answer_babbaccb_1",+ "answer_babbaccb_2"+ ]+ },+ {+ "question_id": "8fb83627",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_966cecbb_1",+ "answer_966cecbb_2",+ "ultrachat_172298",+ "be050ab3_2",+ "ultrachat_44939",+ "02b81ad9",+ "4019b1a8_5",+ "ultrachat_307166",+ "0638f6f6_1",+ "9315f268_1"+ ],+ "gold_session_ids": [+ "answer_966cecbb_1",+ "answer_966cecbb_2"+ ]+ },+ {+ "question_id": "b01defab",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8c0712af_2",+ "59dc866c_1",+ "answer_8c0712af_1",+ "sharegpt_VbIxRWJ_33",+ "639cec58",+ "9aa07e25",+ "ultrachat_407511",+ "b1d9eb66_2",+ "3891baea_2",+ "dc378711_2"+ ],+ "gold_session_ids": [+ "answer_8c0712af_1",+ "answer_8c0712af_2"+ ]+ },+ {+ "question_id": "22d2cb42",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_bcce0b73_1",+ "answer_bcce0b73_2",+ "060aeced",+ "74932466_1",+ "b63954ef",+ "d1e4ee27_1",+ "e1416816_1",+ "a2defa66_2",+ "5a76fadb_2",+ "ultrachat_312304"+ ],+ "gold_session_ids": [+ "answer_bcce0b73_1",+ "answer_bcce0b73_2"+ ]+ },+ {+ "question_id": "0e4e4c46",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f2f998c7_1",+ "answer_f2f998c7_2",+ "60876def_2",+ "ultrachat_467820",+ "5d1c7bf7",+ "0773943b",+ "b7702ee4_1",+ "sharegpt_1YtwybC_15",+ "70cae729_3",+ "998616b4_2"+ ],+ "gold_session_ids": [+ "answer_f2f998c7_1",+ "answer_f2f998c7_2"+ ]+ },+ {+ "question_id": "4b24c848",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_2cec623b_2",+ "1815d1b7",+ "answer_2cec623b_1",+ "16e094b3_1",+ "d38e5b38",+ "f5cf8815_1",+ "ultrachat_421733",+ "afa0c2e7_1",+ "d0c1453b_1",+ "95ae7b23_1"+ ],+ "gold_session_ids": [+ "answer_2cec623b_1",+ "answer_2cec623b_2"+ ]+ },+ {+ "question_id": "7e974930",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8772153153380493,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c9f5693c_1",+ "898ce7a5_1",+ "ultrachat_251049",+ "answer_c9f5693c_2",+ "ultrachat_113430",+ "705b5399_3",+ "0a24e95e",+ "bf139883",+ "ultrachat_271314",+ "a1e169b1"+ ],+ "gold_session_ids": [+ "answer_c9f5693c_1",+ "answer_c9f5693c_2"+ ]+ },+ {+ "question_id": "603deb26",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6131471927654584,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8afdebac_2",+ "88611fff",+ "a55a5afc_2",+ "ede65c84_2",+ "37f66729",+ "7db28e68_1",+ "5e523d80_3",+ "ultrachat_104321",+ "5e867c2f_2",+ "sharegpt_mMTudFY_0"+ ],+ "gold_session_ids": [+ "answer_8afdebac_1",+ "answer_8afdebac_2"+ ]+ },+ {+ "question_id": "59524333",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "638e7094_1",+ "answer_b28f2c7a_2",+ "answer_b28f2c7a_1",+ "sharegpt_Clr6oyu_0",+ "2d5d1c10",+ "8953c8c8_2",+ "1286422b_1",+ "e3d4f89e_3",+ "sharegpt_NYFuCmY_0",+ "ultrachat_458180"+ ],+ "gold_session_ids": [+ "answer_b28f2c7a_1",+ "answer_b28f2c7a_2"+ ]+ },+ {+ "question_id": "5831f84d",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8d63a897_1",+ "answer_8d63a897_2",+ "41c90f4c_2",+ "sharegpt_6I57AvU_67",+ "ef84b994_abs_2",+ "0d66562c_4",+ "b818e2a1_1",+ "b70358bc_6",+ "sharegpt_NHidbra_0",+ "6a471944_1"+ ],+ "gold_session_ids": [+ "answer_8d63a897_1",+ "answer_8d63a897_2"+ ]+ },+ {+ "question_id": "eace081b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8a791264_2",+ "answer_8a791264_1",+ "4d04b866",+ "413b57cb_3",+ "9316aae3_1",+ "1dcfb9a0_2",+ "sharegpt_HOO59Ue_137",+ "02eae87c_1",+ "164bd5a4",+ "4dae77d3"+ ],+ "gold_session_ids": [+ "answer_8a791264_1",+ "answer_8a791264_2"+ ]+ },+ {+ "question_id": "affe2881",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "fb5dd87f_7",+ "answer_90de9b4d_1",+ "answer_90de9b4d_2",+ "ultrachat_443550",+ "3d895af0_3",+ "f334878c",+ "1b71c896_2",+ "84daebd7_2",+ "5b4d2438_5",+ "d53b6bfe_1"+ ],+ "gold_session_ids": [+ "answer_90de9b4d_1",+ "answer_90de9b4d_2"+ ]+ },+ {+ "question_id": "50635ada",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_dcd74827_2",+ "answer_dcd74827_1",+ "sharegpt_eSTYwMj_10",+ "b6f35f3a_1",+ "03f76b6b",+ "42bcee92_2",+ "35bc3132_2",+ "9f9b402a_5",+ "sharegpt_BVhcEnF_0",+ "dcbd175f_2"+ ],+ "gold_session_ids": [+ "answer_dcd74827_1",+ "answer_dcd74827_2"+ ]+ },+ {+ "question_id": "e66b632c",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ac0140ce_1",+ "answer_ac0140ce_2",+ "259d58da_2",+ "92f1ea4d_1",+ "ultrachat_306973",+ "47d18b61_1",+ "9b820206",+ "b3070ec4_1",+ "57c8bf62_3",+ "3304f57b_1"+ ],+ "gold_session_ids": [+ "answer_ac0140ce_1",+ "answer_ac0140ce_2"+ ]+ },+ {+ "question_id": "0ddfec37",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a22b654d_2",+ "answer_a22b654d_1",+ "147ab7e9_7",+ "e4881cbd_1",+ "3422786d_2",+ "a5959675_2",+ "92509977_1",+ "60f9246d",+ "2878574e",+ "sharegpt_h0fDljQ_12"+ ],+ "gold_session_ids": [+ "answer_a22b654d_1",+ "answer_a22b654d_2"+ ]+ },+ {+ "question_id": "f685340e",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_25df025b_1",+ "answer_25df025b_2",+ "a4815e8e_2",+ "142e3203_1",+ "e7dcd5df",+ "4a9eb139_2",+ "9e7446eb",+ "bb2180e9_2",+ "992eb1f3_2",+ "8fcfbe43_2"+ ],+ "gold_session_ids": [+ "answer_25df025b_1",+ "answer_25df025b_2"+ ]+ },+ {+ "question_id": "cc5ded98",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_a5b68517_1",+ "answer_a5b68517_2",+ "8a5fe67a",+ "f14a4532_1",+ "c63b2c8f",+ "c937aab6",+ "d973fa59_1",+ "ef785bb4_2",+ "sharegpt_IEsU45Z_23",+ "b13d5e4e_1"+ ],+ "gold_session_ids": [+ "answer_a5b68517_1",+ "answer_a5b68517_2"+ ]+ },+ {+ "question_id": "dfde3500",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_35d6c0be_1",+ "answer_35d6c0be_2",+ "ad1c7ac6",+ "5fcca8bc_2",+ "779c0465",+ "5c18cb32_2",+ "86e218e3_2",+ "5d5e80c5_1",+ "sharegpt_DiRAKDo_0",+ "7de51ffe_1"+ ],+ "gold_session_ids": [+ "answer_35d6c0be_1",+ "answer_35d6c0be_2"+ ]+ },+ {+ "question_id": "69fee5aa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_d6028d6e_2",+ "answer_d6028d6e_1",+ "5d284d7d_2",+ "991d55e5_1",+ "a0b8b0ad_7",+ "8464304d_2",+ "a554ed79_2",+ "ultrachat_152449",+ "9d11d844",+ "2af27403_2"+ ],+ "gold_session_ids": [+ "answer_d6028d6e_1",+ "answer_d6028d6e_2"+ ]+ },+ {+ "question_id": "7401057b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_94650bfa_1",+ "answer_94650bfa_2",+ "42e69993_1",+ "0f09447b",+ "c3757c5b",+ "8de5b7cf_2",+ "b7146673_1",+ "fb2ae167",+ "5e906cb0_1",+ "ultrachat_389374"+ ],+ "gold_session_ids": [+ "answer_94650bfa_1",+ "answer_94650bfa_2"+ ]+ },+ {+ "question_id": "cf22b7bf",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ae3a122b_1",+ "answer_ae3a122b_2",+ "457e26bf",+ "c8fcdf5a_1",+ "36d5bbde_1",+ "8ab0292c_1",+ "8064b6ca",+ "ultrachat_346016",+ "22fe1c93",+ "8b540c1b"+ ],+ "gold_session_ids": [+ "answer_ae3a122b_1",+ "answer_ae3a122b_2"+ ]+ },+ {+ "question_id": "a2f3aa27",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_5126c02d_1",+ "answer_5126c02d_2",+ "b65f51fb_5",+ "2fa50779_2",+ "01a467ee_2",+ "ultrachat_75332",+ "c98b3349_3",+ "39709ef6_2",+ "a8028ce7_2",+ "57bf4122_1"+ ],+ "gold_session_ids": [+ "answer_5126c02d_1",+ "answer_5126c02d_2"+ ]+ },+ {+ "question_id": "c7dc5443",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_0cdbca92_2",+ "answer_0cdbca92_1",+ "9f0f59d1_3",+ "debc34d7_2",+ "3f9f66f1_2",+ "8fd3c71b",+ "sharegpt_6cz1Sq6_264",+ "sharegpt_oSemQhs_0",+ "sharegpt_35cK50k_0",+ "e200d96c_3"+ ],+ "gold_session_ids": [+ "answer_0cdbca92_1",+ "answer_0cdbca92_2"+ ]+ },+ {+ "question_id": "06db6396",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_da72b1b4_2",+ "answer_da72b1b4_1",+ "6ebdc1fe_2",+ "5bd9f1e6_1",+ "ultrachat_456942",+ "da94a327_2",+ "sharegpt_SwdaeDm_15",+ "57144028_1",+ "84d7650f_2",+ "ultrachat_279487"+ ],+ "gold_session_ids": [+ "answer_da72b1b4_1",+ "answer_da72b1b4_2"+ ]+ },+ {+ "question_id": "3ba21379",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_cd345582_2",+ "answer_cd345582_1",+ "0f78155a_1",+ "d557e57a",+ "sharegpt_xYURy32_0",+ "a946dbf5_3",+ "ultrachat_224436",+ "5e50c81b",+ "sharegpt_QQQAWUd_19",+ "d68838d0_2"+ ],+ "gold_session_ids": [+ "answer_cd345582_1",+ "answer_cd345582_2"+ ]+ },+ {+ "question_id": "9bbe84a2",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_c6a0c6c2_1",+ "ff0bb7fd_2",+ "answer_c6a0c6c2_2",+ "6b7605d1_2",+ "ultrachat_148297",+ "ad611725_2",+ "114c1fb7_6",+ "sharegpt_hcaZN94_546",+ "cd81a69b",+ "sharegpt_flmt4T1_5"+ ],+ "gold_session_ids": [+ "answer_c6a0c6c2_1",+ "answer_c6a0c6c2_2"+ ]+ },+ {+ "question_id": "10e09553",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_67be2c38_2",+ "answer_67be2c38_1",+ "58ab5fc5",+ "3021992b_2",+ "918c4788_1",+ "7d1f5395",+ "959e6cb0_7",+ "88a3c944_3",+ "ultrachat_313128",+ "sharegpt_PAveUvy_0"+ ],+ "gold_session_ids": [+ "answer_67be2c38_1",+ "answer_67be2c38_2"+ ]+ },+ {+ "question_id": "dad224aa",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "6de8645d_1",+ "answer_4a97ae40_1",+ "answer_4a97ae40_2",+ "ultrachat_106352",+ "56b5b760",+ "3dcbca49",+ "6ff8954a_2",+ "91e581ad_1",+ "42234f98",+ "ultrachat_468289"+ ],+ "gold_session_ids": [+ "answer_4a97ae40_1",+ "answer_4a97ae40_2"+ ]+ },+ {+ "question_id": "ba61f0b9",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f377cda7_2",+ "sharegpt_s8Opwwu_0",+ "answer_f377cda7_1",+ "daca659d",+ "ultrachat_421876",+ "sharegpt_5m7gg5F_0",+ "ultrachat_245108",+ "c9213a03_1",+ "ultrachat_169691",+ "ultrachat_328271"+ ],+ "gold_session_ids": [+ "answer_f377cda7_1",+ "answer_f377cda7_2"+ ]+ },+ {+ "question_id": "42ec0761",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.8503449055347546,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e3892371_1",+ "53acd215",+ "27d0d3e7",+ "6fb15dce_1",+ "answer_e3892371_2",+ "b5b8f8f9_1",+ "sharegpt_OLc5cYf_653",+ "e3150810",+ "b37a44e4_2",+ "493bd421_2"+ ],+ "gold_session_ids": [+ "answer_e3892371_1",+ "answer_e3892371_2"+ ]+ },+ {+ "question_id": "5c40ec5b",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_1cb52d0a_2",+ "answer_1cb52d0a_1",+ "d9868305_1",+ "sharegpt_Ah0tRO5_0",+ "be050ab3_2",+ "sharegpt_XkVyVGq_0",+ "25a2002c",+ "sharegpt_RVk16qK_0",+ "5cc9e1ed_3",+ "a200b713_3"+ ],+ "gold_session_ids": [+ "answer_1cb52d0a_1",+ "answer_1cb52d0a_2"+ ]+ },+ {+ "question_id": "c6853660",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_626e93c4_2",+ "answer_626e93c4_1",+ "ultrachat_435535",+ "6ed1c85e",+ "dcbd175f_2",+ "0e931802_2",+ "ultrachat_223464",+ "48a72204_2",+ "6474cfcc",+ "ultrachat_223382"+ ],+ "gold_session_ids": [+ "answer_626e93c4_1",+ "answer_626e93c4_2"+ ]+ },+ {+ "question_id": "26bdc477",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_f762ad8d_2",+ "answer_f762ad8d_1",+ "a223e307",+ "201d188a_4",+ "27ce9f13_2",+ "sharegpt_bq9G6bT_2",+ "f684ac4c_1",+ "e4bbcd5f_2",+ "551eae69_1",+ "ultrachat_525275"+ ],+ "gold_session_ids": [+ "answer_f762ad8d_1",+ "answer_f762ad8d_2"+ ]+ },+ {+ "question_id": "0977f2af",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.38685280723454163,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "ultrachat_554962",+ "answer_3bf5b73b_2",+ "b24eddd9_1",+ "b3642bd9_3",+ "sharegpt_FqOC5e9_45",+ "9b083158_1",+ "ultrachat_143766",+ "bdf45964_3",+ "f115a3db_1",+ "e0bfe67a_2"+ ],+ "gold_session_ids": [+ "answer_3bf5b73b_1",+ "answer_3bf5b73b_2"+ ]+ },+ {+ "question_id": "6aeb4375_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_3f9693b7_abs_1",+ "a5286353_1",+ "answer_3f9693b7_abs_2",+ "bf25a889",+ "24cb335f",+ "ultrachat_181672",+ "9205e608_1",+ "fcff2f22",+ "c6c5bb8b_1",+ "710f4e21_1"+ ],+ "gold_session_ids": [+ "answer_3f9693b7_abs_1",+ "answer_3f9693b7_abs_2"+ ]+ },+ {+ "question_id": "031748ae_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.9197207891481876,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_8748f791_abs_1",+ "169cd54f",+ "answer_8748f791_abs_2",+ "ultrachat_473482",+ "sharegpt_EX0p0tj_19",+ "sharegpt_wN1Z65m_0",+ "sharegpt_7PLgNll_0",+ "7e76059f_2",+ "ultrachat_561369",+ "sharegpt_Us1zWGN_19"+ ],+ "gold_session_ids": [+ "answer_8748f791_abs_1",+ "answer_8748f791_abs_2"+ ]+ },+ {+ "question_id": "2698e78f_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6509209298071326,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "2e18c90b_1",+ "answer_9282283d_abs_1",+ "cdba3d9f_1",+ "answer_9282283d_abs_2",+ "e4e64e80_8",+ "eae760f6",+ "sharegpt_dZJOxsS_0",+ "9e21d6ab_1",+ "7ebf92f6_2",+ "f607e932_1"+ ],+ "gold_session_ids": [+ "answer_9282283d_abs_1",+ "answer_9282283d_abs_2"+ ]+ },+ {+ "question_id": "2133c1b5_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.45749452618986164,+ "mrr": 0.25,+ "retrieved_session_ids": [+ "a864e7aa_5",+ "fe6d8377_1",+ "sharegpt_asoSnhq_0",+ "answer_52382508_abs_2",+ "688246d1",+ "ultrachat_420387",+ "9998814e_1",+ "answer_52382508_abs_1",+ "sharegpt_M5JnA12_12",+ "ef0341d4_1"+ ],+ "gold_session_ids": [+ "answer_52382508_abs_1",+ "answer_52382508_abs_2"+ ]+ },+ {+ "question_id": "0ddfec37_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6934264036172708,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "6de70f46_1",+ "answer_a22b654d_abs_1",+ "answer_a22b654d_abs_2",+ "6eb55dc4_2",+ "25572732",+ "3a11533e",+ "cb24e5d5_1",+ "sharegpt_V2j1zkI_0",+ "e3e061c4",+ "ultrachat_200650"+ ],+ "gold_session_ids": [+ "answer_a22b654d_abs_1",+ "answer_a22b654d_abs_2"+ ]+ },+ {+ "question_id": "f685340e_abs",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_25df025b_abs_1",+ "answer_25df025b_abs_2",+ "6b895848_1",+ "1970789e",+ "79bb1298_5",+ "bc6c5757_2",+ "ultrachat_276410",+ "be4dc764_1",+ "5a30c8b2",+ "11fa2b8f"+ ],+ "gold_session_ids": [+ "answer_25df025b_abs_1",+ "answer_25df025b_abs_2"+ ]+ },+ {+ "question_id": "89941a94",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_e1403127_2",+ "answer_e1403127_1",+ "de275215_3",+ "f7a61595_5",+ "522dd987_1",+ "ultrachat_312851",+ "6a7d7055_2",+ "2ec60f51_3",+ "85b9848f",+ "0d758215_2"+ ],+ "gold_session_ids": [+ "answer_e1403127_1",+ "answer_e1403127_2"+ ]+ },+ {+ "question_id": "07741c45",+ "question_type": "knowledge-update",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_7e9ad7b4_2",+ "answer_7e9ad7b4_1",+ "af4a2fd1",+ "df846730",+ "5300bfba",+ "37d65d40_3",+ "523e716b_2",+ "ce21f0e4",+ "604de0a3_2",+ "443688cc_2"+ ],+ "gold_session_ids": [+ "answer_7e9ad7b4_1",+ "answer_7e9ad7b4_2"+ ]+ },+ {+ "question_id": "7161e7e2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_5Lzox6N_0",+ "3c3fee41",+ "sharegpt_bYTKGgl_18",+ "2bd9990a_3",+ "ae73b0f7_2",+ "a849dd16",+ "ultrachat_447051",+ "ultrachat_89092",+ "3070419a_1",+ "ultrachat_20447"+ ],+ "gold_session_ids": [+ "answer_sharegpt_5Lzox6N_0"+ ]+ },+ {+ "question_id": "c4f10528",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_234453",+ "51c89181_1",+ "dc1f9713_2",+ "sharegpt_yMwEvl7_557",+ "11248f50",+ "7613d5ec_1",+ "de55485d_1",+ "107d40f0",+ "8b1b7007",+ "e12cfbcd"+ ],+ "gold_session_ids": [+ "answer_ultrachat_234453"+ ]+ },+ {+ "question_id": "89527b6b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_YkWn1Ne_0",+ "07f3f0e0_2",+ "ultrachat_117600",+ "689b46e2_4",+ "ultrachat_66044",+ "ultrachat_468629",+ "ultrachat_11001",+ "644e1d00",+ "ultrachat_442154",+ "990f3ef9_7"+ ],+ "gold_session_ids": [+ "answer_sharegpt_YkWn1Ne_0"+ ]+ },+ {+ "question_id": "e9327a54",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_480665",+ "c0d099e6_2",+ "c58fdf1e_2",+ "ba944cdc",+ "c38fb85b",+ "61e1ba53_2",+ "e91e8a2e_1",+ "8221810f_3",+ "7613d5ec_2",+ "ultrachat_423650"+ ],+ "gold_session_ids": [+ "answer_ultrachat_480665"+ ]+ },+ {+ "question_id": "4c36ccef",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_448704",+ "3dc8ba5f_2",+ "89749c78_1",+ "7b1d1f43_2",+ "5fc83435_2",+ "52381716_2",+ "940629b2_1",+ "07e6a3bd_1",+ "ultrachat_149723",+ "sharegpt_m8F4NxL_0"+ ],+ "gold_session_ids": [+ "answer_ultrachat_448704"+ ]+ },+ {+ "question_id": "6ae235be",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_IUWQYGQ_0",+ "ec8fa65d_1",+ "41b6e78c",+ "99188307",+ "0a1e90d1_2",+ "ultrachat_497582",+ "cc5a6057_1",+ "ultrachat_431987",+ "43811b62",+ "sharegpt_TVK4gjw_24"+ ],+ "gold_session_ids": [+ "answer_sharegpt_IUWQYGQ_0"+ ]+ },+ {+ "question_id": "7e00a6cb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_370515",+ "24f78a31_2",+ "4dd1ba8d_2",+ "9998814e_2",+ "ultrachat_188131",+ "956ce238",+ "a9981dc6_2",+ "sharegpt_n2aGNgW_9",+ "64c2b3c8",+ "sharegpt_0dfpdJm_0"+ ],+ "gold_session_ids": [+ "answer_ultrachat_370515"+ ]+ },+ {+ "question_id": "1903aded",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 0,+ "recall_any_at_20": 0,+ "ndcg_at_10": 0,+ "mrr": 0,+ "retrieved_session_ids": [+ "9f8c432e_2",+ "sharegpt_ELyN256_27",+ "9b4c070c",+ "66369fde_1",+ "b163d176",+ "f8246a66_1",+ "4d287828_2",+ "ultrachat_138198",+ "9b3611d7_1",+ "8adb232f_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hA7AkP3_0"+ ]+ },+ {+ "question_id": "ceb54acb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_cGdjmYo_0",+ "7aa2a9af_2",+ "de43030f_1",+ "de293134_1",+ "2f23dd1a",+ "a1cbe83a",+ "0bbd7094_1",+ "92762eb7",+ "dddce60a_1",+ "f6fd00cf"+ ],+ "gold_session_ids": [+ "answer_sharegpt_cGdjmYo_0"+ ]+ },+ {+ "question_id": "f523d9fe",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_m2xJfjo_0",+ "fbfda981",+ "44165708",+ "a07f9d5c_1",+ "636e0e32_1",+ "sharegpt_4RnbgQt_0",+ "debc34d7_1",+ "193c23bd_1",+ "4376945f",+ "ultrachat_418857"+ ],+ "gold_session_ids": [+ "answer_sharegpt_m2xJfjo_0"+ ]+ },+ {+ "question_id": "0e5e2d1a",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_113156",+ "c756c34e",+ "a803f50b_2",+ "sharegpt_btv4SkQ_0",+ "ultrachat_201585",+ "ultrachat_180166",+ "a6c41826_3",+ "ultrachat_222974",+ "5d79fd65_2",+ "147ab7e9_7"+ ],+ "gold_session_ids": [+ "answer_ultrachat_113156"+ ]+ },+ {+ "question_id": "fea54f57",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_187684",+ "ultrachat_129340",+ "d81c92cb",+ "3210adc8",+ "a1e90def_1",+ "c8854b28",+ "712e923c",+ "5ac7f9c5_1",+ "19ec83c5_1",+ "sharegpt_UGg8d44_4"+ ],+ "gold_session_ids": [+ "answer_ultrachat_187684"+ ]+ },+ {+ "question_id": "cc539528",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_374124",+ "18807892_4",+ "38e81260_5",+ "faa194ea",+ "1692563c_2",+ "fb760626",+ "4f859a31_1",+ "8bd3814a_1",+ "3a1ea336_3",+ "15a8ffb7_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_374124"+ ]+ },+ {+ "question_id": "dc439ea3",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_459954",+ "1bfd5a8b_2",+ "ultrachat_186512",+ "cb5637b3",+ "49d8a382",+ "76ecc9b9_1",+ "eb7366f2_1",+ "ultrachat_480287",+ "sharegpt_aVExml0_92",+ "sharegpt_q2uAW7H_0"+ ],+ "gold_session_ids": [+ "answer_ultrachat_459954"+ ]+ },+ {+ "question_id": "18dcd5a5",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "sharegpt_d1tE544_5",+ "aceffaab",+ "answer_sharegpt_hn3IS1q_0",+ "5e906cb0_2",+ "6ea1541e_1",+ "c00077f5_2",+ "sharegpt_2yaKUmC_0",+ "d89f6204",+ "ultrachat_3370",+ "47727b14"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hn3IS1q_0"+ ]+ },+ {+ "question_id": "488d3006",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_275993",+ "ce19df54_3",+ "a394f6b5_1",+ "82b82901",+ "a68090b0_2",+ "b0f8fc7b_2",+ "990c8992_1",+ "d53b6bfe_1",+ "7e4dab66_1",+ "4646c83d_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_275993"+ ]+ },+ {+ "question_id": "58470ed2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_U4oCSfU_7",+ "ultrachat_478785",+ "66bfa1db",+ "sharegpt_6QUDIXG_104",+ "ff49b5a5_3",+ "sharegpt_ty6EeyH_0",+ "718d0e40",+ "b6018747_4",+ "69bd3d4a_1",+ "ultrachat_251269"+ ],+ "gold_session_ids": [+ "answer_sharegpt_U4oCSfU_7"+ ]+ },+ {+ "question_id": "8cf51dda",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_HFMn2ZX_0",+ "e862f726_3",+ "sharegpt_fUrzIST_0",+ "sharegpt_ftoCcOn_0",+ "ultrachat_365938",+ "9f0f59d1_1",+ "sharegpt_jpiaPoJ_738",+ "sharegpt_eTtpNeO_5",+ "507514d9_1",+ "ultrachat_479158"+ ],+ "gold_session_ids": [+ "answer_sharegpt_HFMn2ZX_0"+ ]+ },+ {+ "question_id": "1d4da289",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_348449",+ "sharegpt_AfruXDt_0",+ "ultrachat_213759",+ "517ad0f0",+ "sharegpt_eVmxjQZ_0",+ "sharegpt_NaYyUjd_0",+ "sharegpt_jltxhMe_53",+ "bf85791d",+ "4cd929f8_1",+ "33baaaab_6"+ ],+ "gold_session_ids": [+ "answer_ultrachat_348449"+ ]+ },+ {+ "question_id": "8464fc84",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_467053",+ "98d39a6d",+ "21d86ebb_2",+ "d43b63a6",+ "2d003fe4_2",+ "e6ab6a7b_1",+ "6ac43c5b_4",+ "6095245e",+ "9ac77ac9_2",+ "69a105f9_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_467053"+ ]+ },+ {+ "question_id": "8aef76bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_563222",+ "146dfabe",+ "5209e813_2",+ "05e258f4_2",+ "08e084f1_1",+ "ultrachat_368669",+ "c7f6643e_2",+ "sharegpt_5VMruqC_0",+ "a14e0137",+ "87a183c1_3"+ ],+ "gold_session_ids": [+ "answer_ultrachat_563222"+ ]+ },+ {+ "question_id": "71a3fd6b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "7f7e26d2_2",+ "answer_ultrachat_417348",+ "b7146673_1",+ "36743359_3",+ "8419b385_3",+ "cc40f3ff",+ "4bebc783_2",+ "d8f4b0f4",+ "sharegpt_69zYTa4_0",+ "ultrachat_168420"+ ],+ "gold_session_ids": [+ "answer_ultrachat_417348"+ ]+ },+ {+ "question_id": "2bf43736",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_2kpncbX_13",+ "8897a2e5",+ "1a5fb2cd",+ "16c3be68_1",+ "a93750ef_2",+ "5ca6cd28",+ "2e89bd03_1",+ "ultrachat_49111",+ "445e6a7a_1",+ "9b1d5656_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_2kpncbX_13"+ ]+ },+ {+ "question_id": "70b3e69b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_334948",+ "sharegpt_O8qJclh_0",+ "ultrachat_47234",+ "ultrachat_146029",+ "ultrachat_302829",+ "3169d075_2",+ "09e1a929_2",+ "28cd2f40",+ "ab3af6c5_2",+ "ultrachat_361790"+ ],+ "gold_session_ids": [+ "answer_ultrachat_334948"+ ]+ },+ {+ "question_id": "8752c811",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_6pWK9yx_0",+ "sharegpt_cTla0Yd_0",+ "02b63d04_1",+ "5c6a5b7c_1",+ "56745369_2",+ "sharegpt_Gb68gx3_0",+ "sharegpt_Y6MteNX_0",+ "sharegpt_7E53b59_0",+ "9db30f1c_3",+ "sharegpt_lAfTTUo_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_6pWK9yx_0"+ ]+ },+ {+ "question_id": "3249768e",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "0868982f_1",+ "answer_sharegpt_CaxTGYP_0",+ "689b46e2_2",+ "11e9d591_2",+ "c6c5bb8b_2",+ "e0bd5f14_2",+ "23450b93_3",+ "ed1d68f1_1",+ "773aebbd_3",+ "268e8532_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_CaxTGYP_0"+ ]+ },+ {+ "question_id": "1b9b7252",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_115151",+ "sharegpt_1AhFMpw_5",+ "d3cc5bdc_4",+ "53ab4734",+ "0e726047",+ "f7796d96_1",+ "7798452b_6",+ "3abaa836",+ "b2e5a019",+ "af4d4ecd_6"+ ],+ "gold_session_ids": [+ "answer_ultrachat_115151"+ ]+ },+ {+ "question_id": "1568498a",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_d6JJiqH_76",+ "sharegpt_h87NLyS_49",+ "71e0db57",+ "sharegpt_YfakX5Z_0",+ "6ad46850_2",+ "ultrachat_344985",+ "ultrachat_131295",+ "1fef8ffe_1",+ "ultrachat_35159",+ "sharegpt_69agvDU_27"+ ],+ "gold_session_ids": [+ "answer_sharegpt_d6JJiqH_76"+ ]+ },+ {+ "question_id": "6222b6eb",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "04a0b385",+ "answer_sharegpt_H9PiM5G_0",+ "d1b9993e_3",+ "723bf11f",+ "25ffc571",+ "sharegpt_MvUpG5V_5",+ "caf7480d_1",+ "2bbbe083",+ "ultrachat_418305",+ "sharegpt_zS41R74_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_H9PiM5G_0"+ ]+ },+ {+ "question_id": "e8a79c70",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.5,+ "mrr": 0.3333333333333333,+ "retrieved_session_ids": [+ "c0d099e6_2",+ "89aed484_2",+ "answer_ultrachat_13075",+ "4050ebff_9",+ "sharegpt_ha8kUEr_0",+ "2b5c911e_4",+ "c2b2b2ea_1",+ "08ca1f31_2",+ "45c29267",+ "85a49fe0_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_13075"+ ]+ },+ {+ "question_id": "d596882b",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_252214",+ "0dc2efcc_2",+ "456807b7_1",+ "5558a42e_3",+ "73ba8380_3",+ "ultrachat_299672",+ "71888aff_2",+ "5fb3b5ac",+ "692075ef",+ "c324a5bc_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_252214"+ ]+ },+ {+ "question_id": "e3fc4d6e",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_5m7gg5F_0",+ "ultrachat_442052",+ "ultrachat_161238",+ "sharegpt_sXKNzPE_0",+ "9b72b151",+ "3b38bf46",+ "d31c3ec8_1",+ "bec86aec",+ "6aaf298d",+ "ultrachat_61952"+ ],+ "gold_session_ids": [+ "answer_sharegpt_5m7gg5F_0"+ ]+ },+ {+ "question_id": "51b23612",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_427265",+ "17f444f3",+ "ultrachat_555062",+ "8672f398_2",+ "409015ef_1",+ "d3c3cd97",+ "8d30ec5b",+ "ultrachat_522170",+ "ultrachat_221034",+ "fe5894c5"+ ],+ "gold_session_ids": [+ "answer_ultrachat_427265"+ ]+ },+ {+ "question_id": "3e321797",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_94624",+ "sharegpt_EZmhz8p_0",+ "0382dbfe_2",+ "96b6e5e8_1",+ "57144028_2",+ "04b7b396_1",+ "00305dd2_1",+ "e77fc218_2",+ "385a60c8_1",+ "5558a42e_3"+ ],+ "gold_session_ids": [+ "answer_ultrachat_94624"+ ]+ },+ {+ "question_id": "e982271f",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_195444",+ "db65997e",+ "7b4bce3b_2",+ "eb0d8dc1_4",+ "333a59e5_1",+ "091aa510_5",+ "ultrachat_383774",+ "e760dc71_1",+ "3e8f07c9_1",+ "ultrachat_137024"+ ],+ "gold_session_ids": [+ "answer_ultrachat_195444"+ ]+ },+ {+ "question_id": "352ab8bd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_NoDZzot_7",+ "337cbfc8_2",+ "df08e7a0",+ "3b0f28f6_2",+ "6593cb8b_6",+ "sharegpt_UCVg3jO_0",+ "sharegpt_cDlYqSf_21",+ "c7ca6dff",+ "ultrachat_1052",+ "sharegpt_7tOPXDd_50"+ ],+ "gold_session_ids": [+ "answer_sharegpt_NoDZzot_7"+ ]+ },+ {+ "question_id": "fca762bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_39395",+ "2bc4ae7d",+ "d0cdfddb_2",+ "99eec551",+ "d5b3b034_1",+ "9beab022_1",+ "a8bb9614_2",+ "06a34d82_1",+ "f1bdf7f3_4",+ "8aee9f4c_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_39395"+ ]+ },+ {+ "question_id": "7a8d0b71",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_i0tMT9q_9",+ "ultrachat_136480",+ "147ab7e9_5",+ "f20e72e4_2",+ "f8169a6d",+ "1e26715f",+ "8969b3ca_2",+ "ultrachat_324501",+ "369695b4_1",+ "sharegpt_P0Qe2Gx_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_i0tMT9q_9"+ ]+ },+ {+ "question_id": "a40e080f",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_269020",+ "4d32ba1b_1",+ "528f481d_6",+ "1c1541c3",+ "eb18a1b7",+ "4fd6129f_1",+ "89c7a48a_2",+ "cd38b6ee",+ "ultrachat_483678",+ "12c28d3f_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_269020"+ ]+ },+ {+ "question_id": "8b9d4367",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_289157",+ "ultrachat_321805",+ "7babf05b",+ "sharegpt_WZzri9J_0",+ "07b94dbf_1",+ "ultrachat_563661",+ "sharegpt_ZSnZKMn_13",+ "c77250d2",+ "007e7d81_3",+ "5534efcb_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_289157"+ ]+ },+ {+ "question_id": "5809eb10",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_4aJsGCH_0",+ "sharegpt_GDFyxeT_0",+ "5237bb0b_1",+ "ultrachat_255793",+ "sharegpt_9M4WbfX_0",+ "922c819f_2",+ "58ae7c4f_1",+ "sharegpt_oKiXiX8_0",+ "ultrachat_549762",+ "701ea427_6"+ ],+ "gold_session_ids": [+ "answer_sharegpt_4aJsGCH_0"+ ]+ },+ {+ "question_id": "41275add",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.6309297535714575,+ "mrr": 0.5,+ "retrieved_session_ids": [+ "b3619c2c",+ "answer_sharegpt_81riySf_0",+ "acc3913b",+ "9345f7dc_1",+ "4d84cbfa",+ "a68db5db_1",+ "6d96a14d_2",+ "5fe4d2ee_1",+ "sharegpt_dUoAhvI_71",+ "5b83c26e_3"+ ],+ "gold_session_ids": [+ "answer_sharegpt_81riySf_0"+ ]+ },+ {+ "question_id": "4388e9dd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_qTi81nS_0",+ "sharegpt_Tun3KJ4_0",+ "cfd21744_2",+ "3197d998_7",+ "ultrachat_412706",+ "f508f11f_2",+ "sharegpt_2LrB18X_51",+ "234c8721",+ "sharegpt_rdaQ1VE_13",+ "cf021b36_3"+ ],+ "gold_session_ids": [+ "answer_sharegpt_qTi81nS_0"+ ]+ },+ {+ "question_id": "4baee567",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_i9adwQn_0",+ "74711358_2",+ "sharegpt_wduYbsX_0",+ "6169ef55_1",+ "sharegpt_28Mwwk9_0",+ "ultrachat_487211",+ "b762a658",+ "2c22e0e8_1",+ "ultrachat_323343",+ "d5415748"+ ],+ "gold_session_ids": [+ "answer_sharegpt_i9adwQn_0"+ ]+ },+ {+ "question_id": "561fabcd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_hChsWOp_97",+ "aee13015_3",+ "33da50d0_5",+ "83fb74bf_1",+ "63c75b13_2",+ "d75f4fb5_1",+ "9a992902",+ "d5d7eff7",+ "1c1f5ccc_6",+ "34b38398_2"+ ],+ "gold_session_ids": [+ "answer_sharegpt_hChsWOp_97"+ ]+ },+ {+ "question_id": "b759caee",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_2BSXlAr_0",+ "191f4c9b_2",+ "abecc1b6",+ "ffa51434_3",+ "f55fee5b_4",+ "d3575920",+ "0bad887e_2",+ "bab41bb6_5",+ "b41aae7f_1",+ "fcae5c39_1"+ ],+ "gold_session_ids": [+ "answer_sharegpt_2BSXlAr_0"+ ]+ },+ {+ "question_id": "ac031881",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_sharegpt_GYqnAhC_190",+ "c986f83a",+ "sharegpt_cIj2dFl_4",+ "449b918c_3",+ "e30307d8_1",+ "sharegpt_1X61wv2_2",+ "sharegpt_RWoA8C4_0",+ "19c24c11",+ "83f89c02",+ "ebb47912"+ ],+ "gold_session_ids": [+ "answer_sharegpt_GYqnAhC_190"+ ]+ },+ {+ "question_id": "28bcfaac",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_446979",+ "135a62d7_2",+ "be05680b_2",+ "96e1688e",+ "4646c83d_1",+ "ultrachat_181176",+ "sharegpt_hi2xiJT_14",+ "510b98f3",+ "a27a2811",+ "485f4604_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_446979"+ ]+ },+ {+ "question_id": "16c90bf4",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_294807",+ "b8770374_2",+ "fa2714ed_1",+ "89d49715_1",+ "b0da5097_2",+ "afa0c2e7_5",+ "d4230511_1",+ "be74993b_1",+ "cc830ab4_1",+ "8464304d_2"+ ],+ "gold_session_ids": [+ "answer_ultrachat_294807"+ ]+ },+ {+ "question_id": "c8f1aeed",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_519486",+ "sharegpt_ebYKddM_87",+ "ultrachat_220071",+ "ultrachat_39392",+ "c05dde5b_2",+ "f8a9cd01_2",+ "199a5225_1",+ "a53fdd02_2",+ "af631aa3_3",+ "ea27c6f5_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_519486"+ ]+ },+ {+ "question_id": "eaca4986",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 0,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 0.3333333333333333,+ "mrr": 0.14285714285714285,+ "retrieved_session_ids": [+ "931c521e_1",+ "1c1a2b7f_2",+ "ultrachat_311235",+ "3477c37c_1",+ "ultrachat_474091",+ "37db6f4e_1",+ "answer_sharegpt_SS141vi_0",+ "ultrachat_175444",+ "0e543e9e_2",+ "sharegpt_zhAMIY7_0"+ ],+ "gold_session_ids": [+ "answer_sharegpt_SS141vi_0"+ ]+ },+ {+ "question_id": "c7cf7dfd",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_456407",+ "e27fe923_5",+ "780b6c7c_1",+ "9db6d988_2",+ "ultrachat_328173",+ "b90109d2_1",+ "ultrachat_139629",+ "ultrachat_419371",+ "285a29e7_1",+ "4acdd888"+ ],+ "gold_session_ids": [+ "answer_ultrachat_456407"+ ]+ },+ {+ "question_id": "e48988bc",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_174360",+ "476b1cbb",+ "ultrachat_494373",+ "ultrachat_307163",+ "ultrachat_283303",+ "8fb3fe3a_3",+ "0781daa1_2",+ "ultrachat_172828",+ "ultrachat_99935",+ "bae244c4"+ ],+ "gold_session_ids": [+ "answer_ultrachat_174360"+ ]+ },+ {+ "question_id": "1de5cff2",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_440262",+ "28031c43_2",+ "8fcfbe43_3",+ "c14dbe17",+ "ultrachat_186422",+ "ultrachat_58031",+ "1a693999_4",+ "f863a648_3",+ "4cb33063",+ "848d721f_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_440262"+ ]+ },+ {+ "question_id": "65240037",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_403752",+ "f8dc1273",+ "37141183",+ "faad7d7a_2",+ "dba5a924",+ "fd6f60f0_4",+ "0984a772",+ "92466e8e_1",+ "7df4c735_1",+ "sharegpt_ME35OD8_167"+ ],+ "gold_session_ids": [+ "answer_ultrachat_403752"+ ]+ },+ {+ "question_id": "778164c6",+ "question_type": "single-session-assistant",+ "recall_any_at_5": 1,+ "recall_any_at_10": 1,+ "recall_any_at_20": 1,+ "ndcg_at_10": 1,+ "mrr": 1,+ "retrieved_session_ids": [+ "answer_ultrachat_399000",+ "b605741a_1",+ "sharegpt_WXCvFir_11",+ "c15dadce_4",+ "ddaa5a15_2",+ "sharegpt_ki9IVDq_6",+ "1340826c",+ "ultrachat_346016",+ "c6bd5277",+ "025836be_1"+ ],+ "gold_session_ids": [+ "answer_ultrachat_399000"+ ]+ }+ ]+}\ No newline at end of filediff --git a/benchmark/longmemeval-bench.ts b/benchmark/longmemeval-bench.tsnew file mode 100644index 00000000..d826e1ac--- /dev/null+++ b/benchmark/longmemeval-bench.ts@@ -0,0 +1,317 @@+import { SearchIndex } from "../src/state/search-index.js";+import { VectorIndex } from "../src/state/vector-index.js";+import { HybridSearch } from "../src/state/hybrid-search.js";+import type {+ CompressedObservation,+ EmbeddingProvider,+} from "../src/types.js";+import { readFileSync, writeFileSync, existsSync } from "node:fs";++interface LongMemEvalEntry {+ question_id: string;+ question_type: string;+ question: string;+ question_date: string;+ answer: string;+ answer_session_ids: string[];+ haystack_dates: string[];+ haystack_session_ids: string[];+ haystack_sessions: Array<Array<{ role: string; content: string; has_answer?: boolean }>>;+}++interface SessionChunk {+ sessionId: string;+ text: string;+ turnCount: number;+}++interface BenchResult {+ question_id: string;+ question_type: string;+ recall_any_at_5: number;+ recall_any_at_10: number;+ recall_any_at_20: number;+ ndcg_at_10: number;+ mrr: number;+ retrieved_session_ids: string[];+ gold_session_ids: string[];+}++function chunkSessionToText(+ turns: Array<{ role: string; content: string }>,+): string {+ return turns+ .map((t) => `${t.role}: ${t.content}`)+ .join("\n");+}++function recallAny(+ retrievedSessionIds: string[],+ goldSessionIds: string[],+ k: number,+): number {+ const topK = new Set(retrievedSessionIds.slice(0, k));+ return goldSessionIds.some((gid) => topK.has(gid)) ? 1.0 : 0.0;+}++function dcg(relevances: boolean[], k: number): number {+ let sum = 0;+ for (let i = 0; i < Math.min(k, relevances.length); i++) {+ sum += (relevances[i] ? 1 : 0) / Math.log2(i + 2);+ }+ return sum;+}++function ndcg(+ retrievedSessionIds: string[],+ goldSessionIds: Set<string>,+ k: number,+): number {+ const rels = retrievedSessionIds+ .slice(0, k)+ .map((id) => goldSessionIds.has(id));+ const idealRels = Array.from(+ { length: Math.min(k, goldSessionIds.size) },+ () => true,+ );+ const idealDCG = dcg(idealRels, k);+ if (idealDCG === 0) return 0;+ return dcg(rels, k) / idealDCG;+}++function mrr(+ retrievedSessionIds: string[],+ goldSessionIds: Set<string>,+): number {+ for (let i = 0; i < retrievedSessionIds.length; i++) {+ if (goldSessionIds.has(retrievedSessionIds[i])) return 1 / (i + 1);+ }+ return 0;+}++class MockKV {+ private store = new Map<string, Map<string, unknown>>();+ async get<T>(scope: string, key: string): Promise<T> {+ const m = this.store.get(scope);+ if (!m || !m.has(key)) throw new Error(`Not found: ${scope}/${key}`);+ return m.get(key) as T;+ }+ async set(scope: string, key: string, value: unknown): Promise<void> {+ if (!this.store.has(scope)) this.store.set(scope, new Map());+ this.store.get(scope)!.set(key, value);+ }+ async list<T>(scope: string): Promise<T[]> {+ const m = this.store.get(scope);+ if (!m) return [];+ return Array.from(m.values()) as T[];+ }+ async delete(scope: string, key: string): Promise<void> {+ this.store.get(scope)?.delete(key);+ }+}++async function runBenchmark(+ mode: "bm25" | "vector" | "hybrid",+ embeddingProvider?: EmbeddingProvider,+) {+ const dataPath = new URL("./data/longmemeval_s_cleaned.json", import.meta.url).pathname;+ if (!existsSync(dataPath)) {+ console.error(`Dataset not found at ${dataPath}`);+ console.error("Download from: https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned");+ process.exit(1);+ }++ console.log(`Loading LongMemEval-S dataset...`);+ const raw = JSON.parse(readFileSync(dataPath, "utf-8")) as LongMemEvalEntry[];++ const abstentionTypes = new Set([+ "single-session-user_abs",+ "multi-session_abs",+ "knowledge-update_abs",+ "temporal-reasoning_abs",+ ]);+ const entries = raw.filter((e) => !abstentionTypes.has(e.question_type));+ console.log(+ `Loaded ${entries.length} questions (${raw.length - entries.length} abstention excluded)`,+ );++ const results: BenchResult[] = [];+ let processed = 0;++ for (const entry of entries) {+ const sessionChunks: SessionChunk[] = [];+ for (let i = 0; i < entry.haystack_sessions.length; i++) {+ const sessionId = entry.haystack_session_ids[i];+ const turns = entry.haystack_sessions[i];+ const text = chunkSessionToText(turns);+ sessionChunks.push({ sessionId, text, turnCount: turns.length });+ }++ const bm25 = new SearchIndex();+ const vector = mode !== "bm25" ? new VectorIndex() : null;+ const kv = new MockKV();++ const observations: CompressedObservation[] = [];+ for (const chunk of sessionChunks) {+ const obs: CompressedObservation = {+ id: `obs_${chunk.sessionId}`,+ sessionId: chunk.sessionId,+ timestamp: new Date().toISOString(),+ type: "conversation",+ title: chunk.text.slice(0, 80),+ facts: [],+ narrative: chunk.text,+ concepts: [],+ files: [],+ importance: 5,+ };+ observations.push(obs);+ bm25.add(obs);++ if (vector && embeddingProvider) {+ try {+ const embedding = await embeddingProvider.embed(+ chunk.text.slice(0, 512),+ );+ vector.add(obs.id, chunk.sessionId, embedding);+ } catch {}+ }++ await kv.set(`mem:obs:${chunk.sessionId}`, obs.id, obs);+ }++ let retrievedObsIds: string[];++ if (mode === "bm25") {+ const bm25Results = bm25.search(entry.question, 20);+ retrievedObsIds = bm25Results.map((r) => r.obsId);+ } else {+ const hybridSearch = new HybridSearch(+ bm25,+ vector,+ embeddingProvider || null,+ kv as any,+ 0.4,+ 0.6,+ 0.0,+ false,+ );+ const hybridResults = await hybridSearch.search(entry.question, 20);+ retrievedObsIds = hybridResults.map((r) => r.observation.id);+ }++ const retrievedSessionIds = retrievedObsIds.map((oid) =>+ oid.replace(/^obs_/, ""),+ );+ const goldSet = new Set(entry.answer_session_ids);++ const result: BenchResult = {+ question_id: entry.question_id,+ question_type: entry.question_type,+ recall_any_at_5: recallAny(retrievedSessionIds, entry.answer_session_ids, 5),+ recall_any_at_10: recallAny(retrievedSessionIds, entry.answer_session_ids, 10),+ recall_any_at_20: recallAny(retrievedSessionIds, entry.answer_session_ids, 20),+ ndcg_at_10: ndcg(retrievedSessionIds, goldSet, 10),+ mrr: mrr(retrievedSessionIds, goldSet),+ retrieved_session_ids: retrievedSessionIds.slice(0, 10),+ gold_session_ids: entry.answer_session_ids,+ };+ results.push(result);+ processed++;++ if (processed % 50 === 0) {+ const avgRecall5 =+ results.reduce((s, r) => s + r.recall_any_at_5, 0) / results.length;+ console.log(+ ` [${processed}/${entries.length}] running recall_any@5: ${(avgRecall5 * 100).toFixed(1)}%`,+ );+ }+ }++ const avgRecallAny5 =+ results.reduce((s, r) => s + r.recall_any_at_5, 0) / results.length;+ const avgRecallAny10 =+ results.reduce((s, r) => s + r.recall_any_at_10, 0) / results.length;+ const avgRecallAny20 =+ results.reduce((s, r) => s + r.recall_any_at_20, 0) / results.length;+ const avgNdcg10 =+ results.reduce((s, r) => s + r.ndcg_at_10, 0) / results.length;+ const avgMrr =+ results.reduce((s, r) => s + r.mrr, 0) / results.length;++ const byType = new Map<string, BenchResult[]>();+ for (const r of results) {+ if (!byType.has(r.question_type)) byType.set(r.question_type, []);+ byType.get(r.question_type)!.push(r);+ }++ console.log(`\n=== LongMemEval-S Results (${mode}) ===`);+ console.log(`Questions: ${results.length} (excl. abstention)`);+ console.log(`recall_any@5: ${(avgRecallAny5 * 100).toFixed(1)}%`);+ console.log(`recall_any@10: ${(avgRecallAny10 * 100).toFixed(1)}%`);+ console.log(`recall_any@20: ${(avgRecallAny20 * 100).toFixed(1)}%`);+ console.log(`NDCG@10: ${(avgNdcg10 * 100).toFixed(1)}%`);+ console.log(`MRR: ${(avgMrr * 100).toFixed(1)}%`);++ console.log(`\nBy question type:`);+ for (const [type, typeResults] of byType) {+ const r5 =+ typeResults.reduce((s, r) => s + r.recall_any_at_5, 0) /+ typeResults.length;+ const r10 =+ typeResults.reduce((s, r) => s + r.recall_any_at_10, 0) /+ typeResults.length;+ console.log(+ ` ${type.padEnd(30)} R@5: ${(r5 * 100).toFixed(1)}% R@10: ${(r10 * 100).toFixed(1)}% (n=${typeResults.length})`,+ );+ }++ const outPath = new URL(+ `./data/longmemeval_results_${mode}.json`,+ import.meta.url,+ ).pathname;+ writeFileSync(+ outPath,+ JSON.stringify(+ {+ mode,+ questions: results.length,+ recall_any_at_5: avgRecallAny5,+ recall_any_at_10: avgRecallAny10,+ recall_any_at_20: avgRecallAny20,+ ndcg_at_10: avgNdcg10,+ mrr: avgMrr,+ per_type: Object.fromEntries(+ Array.from(byType).map(([type, tr]) => [+ type,+ {+ count: tr.length,+ recall_any_at_5:+ tr.reduce((s, r) => s + r.recall_any_at_5, 0) / tr.length,+ recall_any_at_10:+ tr.reduce((s, r) => s + r.recall_any_at_10, 0) / tr.length,+ },+ ]),+ ),+ per_question: results,+ },+ null,+ 2,+ ),+ );+ console.log(`\nResults saved to ${outPath}`);+}++const mode = (process.argv[2] || "bm25") as "bm25" | "vector" | "hybrid";+console.log(`Running LongMemEval-S benchmark in ${mode} mode...`);++if (mode === "bm25") {+ runBenchmark("bm25").catch(console.error);+} else {+ import("../src/providers/embedding/local.js")+ .then(({ LocalEmbeddingProvider }) => {+ const provider = new LocalEmbeddingProvider();+ return runBenchmark(mode, provider);+ })+ .catch(console.error);+}