AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
#174 fix mlx model load
Yu Li527effa73190
diff --git a/air_llm/setup.py b/air_llm/setup.pyindex c730648..378272a 100644--- a/air_llm/setup.py+++ b/air_llm/setup.py@@ -14,7 +14,7 @@ with open("README.md", "r", encoding="utf-8") as fh:setuptools.setup(name="airllm",- version="2.10.1",+ version="2.10.2",author="Gavin Li",author_email="[email protected]",description="AirLLM allows single 4GB GPU card to run 70B large language models without quantization, distillation or pruning. 8GB vmem to run 405B Llama3.1.",