Skip to content

AirLLM 2.8T LLM inference with single 4GB GPU

Update README.md, increase max_new_tokens

Gavin Li04da4fcfe9bb
diff --git a/air_llm/README.md b/air_llm/README.md
index c3ff36b..f9f7b54 100644
--- a/air_llm/README.md
+++ b/air_llm/README.md
@@ -54,7 +54,7 @@ input_tokens = model.tokenizer(input_text,
generation_output = model.generate(
input_tokens['input_ids'].cuda(),
- max_new_tokens=2,
+ max_new_tokens=20,
use_cache=True,
return_dict_in_generate=True)