AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
Update README.md, increase max_new_tokens
Gavin Li04da4fcfe9bb
diff --git a/air_llm/README.md b/air_llm/README.mdindex c3ff36b..f9f7b54 100644--- a/air_llm/README.md+++ b/air_llm/README.md@@ -54,7 +54,7 @@ input_tokens = model.tokenizer(input_text,generation_output = model.generate(input_tokens['input_ids'].cuda(),- max_new_tokens=2,+ max_new_tokens=20,use_cache=True,return_dict_in_generate=True)