AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
Update README.md
Gavin Li029f011660f2
diff --git a/README.md b/README.mdindex ca9bbe8..ad73682 100644--- a/README.md+++ b/README.md@@ -6,7 +6,7 @@[**Example notebooks**](#example-python-notebook) |[**FAQ**](#faq)-**AirLLM** optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card. No quantization, distillation, pruning or other model compression techniques that would result in degraded model performance are needed.+**AirLLM** optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card. No quantization, distillation, pruning or other model compression techniques that would result in degraded model performance are needed. And you can run 405B Llama3.1 on 8GB vmem.<a href="https://github.com/lyogavin/airllm/stargazers"></a>[](https://pepy.tech/project/airllm)