Skip to content

The agent engineering platform.

community: update RankLLM integration and fix LangChain deprecation (#29931)

Brayden Zhonge4515f308fbd
diff --git a/docs/docs/integrations/document_transformers/rankllm-reranker.ipynb b/docs/docs/integrations/document_transformers/rankllm-reranker.ipynb
index 6875e86fe8..9e87053987 100644
--- a/docs/docs/integrations/document_transformers/rankllm-reranker.ipynb
+++ b/docs/docs/integrations/document_transformers/rankllm-reranker.ipynb
@@ -11,39 +11,41 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "[RankLLM](https://github.com/castorini/rank_llm) offers a suite of listwise rerankers, albeit with focus on open source LLMs finetuned for the task - RankVicuna and RankZephyr being two of them."
+ "**[RankLLM](https://github.com/castorini/rank_llm)** is a **flexible reranking framework** supporting **listwise, pairwise, and pointwise ranking models**. It includes **RankVicuna, RankZephyr, MonoT5, DuoT5, LiT5, and FirstMistral**, with integration for **FastChat, vLLM, SGLang, and TensorRT-LLM** for efficient inference. RankLLM is optimized for **retrieval and ranking tasks**, leveraging both **open-source LLMs** and proprietary rerankers like **RankGPT and RankGemini**. It supports **batched inference, first-token reranking, and retrieval via BM25 and SPLADE**.\n",
+ "\n",
+ "> **Note:** If using the built-in retriever, RankLLM requires **Pyserini, JDK 21, PyTorch, and Faiss** for retrieval functionality."
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
- "%pip install --upgrade --quiet rank_llm"
+ "%pip install --upgrade --quiet rank_llm"
]
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
- "%pip install --upgrade --quiet langchain_openai"
+ "%pip install --upgrade --quiet langchain_openai"
]
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
- "%pip install --upgrade --quiet faiss-cpu"
+ "%pip install --upgrade --quiet faiss-cpu"
]
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -56,7 +58,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -64,7 +66,7 @@
"def pretty_print_docs(docs):\n",
" print(\n",
" f\"\\n{'-' * 100}\\n\".join(\n",
- " [f\"Document {i+1}:\\n\\n\" + d.page_content for i, d in enumerate(docs)]\n",
+ " [f\"Document {i + 1}:\\n\\n\" + d.page_content for i, d in enumerate(docs)]\n",
" )\n",
" )"
]
@@ -79,9 +81,17 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": null,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "2025-02-22 15:28:58,344 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
+ ]
+ }
+ ],
"source": [
"from langchain_community.document_loaders import TextLoader\n",
"from langchain_community.vectorstores import FAISS\n",
@@ -114,14 +124,14 @@
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": null,
"metadata": {},
"outputs": [
{
- "name": "stderr",
+ "name": "stdout",
"output_type": "stream",
"text": [
- "2025-02-17 04:37:08,458 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
+ "2025-02-22 15:29:00,892 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
]
},
{
@@ -331,27 +341,41 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Retrieval + Reranking with RankZephyr"
+ "RankZephyr performs listwise reranking for improved retrieval quality but requires at least 24GB of VRAM to run efficiently."
]
},
{
"cell_type": "code",
- "execution_count": 12,
+ "execution_count": null,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Downloading shards: 100%|██████████| 3/3 [00:00<00:00, 2674.37it/s]\n",
+ "Loading checkpoint shards: 100%|██████████| 3/3 [01:49<00:00, 36.39s/it]\n"
+ ]
+ }
+ ],
"source": [
+ "import torch\n",
"from langchain.retrievers.contextual_compression import ContextualCompressionRetriever\n",
"from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank\n",
"\n",
- "compressor = RankLLMRerank(top_n=3, model=\"zephyr\")\n",
+ "torch.cuda.empty_cache()\n",
+ "\n",
+ "compressor = RankLLMRerank(top_n=3, model=\"rank_zephyr\")\n",
"compression_retriever = ContextualCompressionRetriever(\n",
" base_compressor=compressor, base_retriever=retriever\n",
- ")"
+ ")\n",
+ "\n",
+ "del compressor"
]
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": null,
"metadata": {},
"outputs": [
{
@@ -386,7 +410,7 @@
]
},
{
- "name": "stderr",
+ "name": "stdout",
"output_type": "stream",
"text": [
"\n"
@@ -407,7 +431,7 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": null,
"metadata": {},
"outputs": [
{
@@ -432,7 +456,7 @@
" llm=ChatOpenAI(temperature=0), retriever=compression_retriever\n",
")\n",
"\n",
- "chain({\"query\": query})"
+ "chain.invoke({\"query\": query})"
]
},
{
@@ -451,9 +475,16 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": null,
"metadata": {},
"outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "2025-02-22 15:01:29,469 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n"
+ ]
+ },
{
"name": "stdout",
"output_type": "stream",
@@ -683,7 +714,7 @@
},
{
"cell_type": "code",
- "execution_count": 20,
+ "execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -698,9 +729,18 @@
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": null,
"metadata": {},
"outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "2025-02-22 15:01:38,554 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
+ " 0%| | 0/1 [00:00<?, ?it/s]2025-02-22 15:01:43,704 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+ "100%|██████████| 1/1 [00:05<00:00, 5.15s/it]"
+ ]
+ },
{
"name": "stdout",
"output_type": "stream",
@@ -727,7 +767,7 @@
]
},
{
- "name": "stderr",
+ "name": "stdout",
"output_type": "stream",
"text": [
"\n"
@@ -748,15 +788,14 @@
},
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": null,
"metadata": {},
"outputs": [
{
- "name": "stderr",
+ "name": "stdout",
"output_type": "stream",
"text": [
- "/tmp/ipykernel_2153001/1437145854.py:10: LangChainDeprecationWarning: The method `Chain.__call__` was deprecated in langchain 0.1.0 and will be removed in 1.0. Use :meth:`~invoke` instead.\n",
- " chain({\"query\": query})\n",
+ " chain.invoke({\"query\": query})\n",
"2025-02-17 04:30:00,016 - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings \"HTTP/1.1 200 OK\"\n",
" 0%| | 0/1 [00:00<?, ?it/s]2025-02-17 04:30:01,649 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"100%|██████████| 1/1 [00:01<00:00, 1.63s/it]\n",
@@ -785,13 +824,13 @@
" llm=ChatOpenAI(temperature=0), retriever=compression_retriever\n",
")\n",
"\n",
- "chain({\"query\": query})"
+ "chain.invoke({\"query\": query})"
]
}
],
"metadata": {
"kernelspec": {
- "display_name": "rankllm",
+ "display_name": "Python 3",
"language": "python",
"name": "python3"
},
@@ -805,7 +844,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.10.14"
+ "version": "3.13.2"
}
},
"nbformat": 4,