The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
community[docs]: Add content for the Lora adapter in the VLLM page. (#27788)
Changyong Umd9163e7afa0e
diff --git a/docs/docs/integrations/llms/vllm.ipynb b/docs/docs/integrations/llms/vllm.ipynbindex 6d45b102dc..1e1baff963 100644--- a/docs/docs/integrations/llms/vllm.ipynb+++ b/docs/docs/integrations/llms/vllm.ipynb@@ -246,6 +246,35 @@")\n","print(llm.invoke(\"Rome is\"))"]+ },+ {+ "cell_type": "markdown",+ "id": "bd3f0f51",+ "metadata": {},+ "source": [+ "## LoRA adapter\n",+ "LoRA adapters can be used with any vLLM model that implements `SupportsLoRA`."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "2682ca6c",+ "metadata": {},+ "outputs": [],+ "source": [+ "from langchain_community.llms import VLLM\n",+ "from vllm.lora.request import LoRARequest\n",+ "\n",+ "llm = VLLM(model=\"meta-llama/Llama-2-7b-hf\", enable_lora=True)\n",+ "\n",+ "LoRA_ADAPTER_PATH = \"path/to/adapter\"\n",+ "lora_adapter = LoRARequest(\"lora_adapter\", 1, LoRA_ADAPTER_PATH)\n",+ "\n",+ "print(\n",+ " llm.invoke(\"What are some popular Korean street foods?\", lora_request=lora_adapter)\n",+ ")"+ ]}],"metadata": {