AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
update readme
Yu Libb62d8d62cac
diff --git a/air_llm/README.md b/air_llm/README.mdindex 1fe9102..b7c1749 100644--- a/air_llm/README.md+++ b/air_llm/README.md@@ -81,7 +81,7 @@ Note: During inference, the original model will first be decomposed and saved la### 3. Compression - 3x Inference Speed!-We just added model compression based on block-wise quantization based model compression. Which can further **speed up the inference speed** for up to **3x** , with almost ignorable accuracy loss(see more performance evaluation and why we use block-wise quantization in [this paper](https://arxiv.org/abs/2212.09720))!+We just added model compression based on block-wise quantization based model compression. Which can further **speed up the inference speed** for up to **3x** , with **almost ignorable accuracy loss!** (see more performance evaluation and why we use block-wise quantization in [this paper](https://arxiv.org/abs/2212.09720))