AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
update example
Gavin Li3271020355c5
diff --git a/air_llm/examples/run_llama3.1_405B.ipynb b/air_llm/examples/run_llama3.1_405B.ipynbindex f4d8c6b..4a86c77 100644--- a/air_llm/examples/run_llama3.1_405B.ipynb+++ b/air_llm/examples/run_llama3.1_405B.ipynb@@ -6,7 +6,12 @@"id": "bfd29d17-9756-464f-b692-41ff20f41148","metadata": {},"outputs": [],- "source": []+ "source": [+ "# if you see Errors like: ValueError: `rope_scaling` must be a dictionary with two fields, `type` and `factor`\n",+ "# need to upgrade transformers to >= 4.43.0\n",+ "\n",+ "# !pip install transformers==4.43.3 "+ ]},{"cell_type": "code",