Skip to content

AirLLM 2.8T LLM inference with single 4GB GPU

update readme

Yu Libb62d8d62cac
diff --git a/air_llm/README.md b/air_llm/README.md
index 1fe9102..b7c1749 100644
--- a/air_llm/README.md
+++ b/air_llm/README.md
@@ -81,7 +81,7 @@ Note: During inference, the original model will first be decomposed and saved la
### 3. Compression - 3x Inference Speed!
-We just added model compression based on block-wise quantization based model compression. Which can further **speed up the inference speed** for up to **3x** , with almost ignorable accuracy loss(see more performance evaluation and why we use block-wise quantization in [this paper](https://arxiv.org/abs/2212.09720))!
+We just added model compression based on block-wise quantization based model compression. Which can further **speed up the inference speed** for up to **3x** , with **almost ignorable accuracy loss!** (see more performance evaluation and why we use block-wise quantization in [this paper](https://arxiv.org/abs/2212.09720))
![speed_improvement](https://github.com/lyogavin/Anima/blob/main/assets/airllm2_time_improvement.png?v=2&raw=true)