AirLLM 2.8T LLM inference with single 4GB GPU
Python chinese-llmchinese-nlpfinetunegenerative-aiinstruct-gptinstruction-setllamallmloraopen-modelsopen-sourceopen-source-modelsqlora
Commits on main
Serve Star History chart from a committed image + daily refresh
bf8f8a9 Yu Li committed 4 months ago
Add PyPI release workflow and make root README the single source of truth
07f4511 Yu Li committed 4 months ago
Update README: highlight FP8/latest-model support and add Supported Models section
87e8035 Yu Li committed 5 months ago
Default runtime dtype to the model's config dtype instead of forcing fp16
7a426eb Yu Li committed 5 months ago
Fix split_and_save_layers dropping tensors for layers spanning multiple shards
0a88464 Yu Li committed 5 months ago
Support pre-quantized fp8 (block fp8) checkpoints in layer streaming
de984a1 Yu Li committed 5 months ago
Prefer native transformers impl; fix shard naming and attn fallback
63053d6 Yu Li committed 5 months ago
Support tied embeddings and route all standard archs through generic model
8f62eec Yu Li committed 5 months ago