Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

Commits on master

update trl
gongjy committed 2 years ago
02adb7b
update inference
gongjy committed 2 years ago
5c4b34b
update inference
gongjy committed 2 years ago
ad6fc92
update data_process
gongjy committed 2 years ago
1354216
update data_process
gongjy committed 2 years ago
2698f6b
update lora-sft
gongjy committed 2 years ago
36fadc7
update readme
gongjy committed 2 years ago
3a034a4
update readme
gongjy committed 2 years ago
5dbd617
update readme
gongjy committed 2 years ago
4542ecf
update readme
gongjy committed 2 years ago
7728341
update readme
gongjy committed 2 years ago
000b0a4
update readme
gongjy committed 2 years ago
5929d0f
update readme
gongjy committed 2 years ago
e4b8789
update readme
gongjy committed 2 years ago
eb875da
update requirements.txt
gongjy committed 2 years ago
1b86445
update model (fix loss bug)
gongjy committed 2 years ago
a87f628
fix deepspeed local_rank bug
gongjy committed 2 years ago
4ef9c41
Update data preprocessing methods
gongjy committed 2 years ago
2981d3e
Update data preprocessing methods
gongjy committed 2 years ago
75753ea
update readme info
gongjy committed 2 years ago
1cc7383
Update data preprocessing methods
gongjy committed 2 years ago
a8ae342
update batchsize
gongjy committed 2 years ago
d570376
fix dtype bug
gongjy committed 2 years ago
89d2601
Merge pull request #44 from iomgaa-ycz/wandb
jingyaogong committed 2 years ago
13105cf
Merge branch 'master' into wandb
jingyaogong committed 2 years ago
5dd4e15
更新了ReadMe
Yu Chengzhang committed 2 years ago
d7a056a
添加了argparse,方便命令行输入参数
Yu Chengzhang committed 2 years ago
51dcf51
修复了wandb的bug,避免了多次产生项目
Yu Chengzhang committed 2 years ago
ef9a592
update wandb monitor
gongjy committed 2 years ago
7947fa1
update wandb monitor
gongjy committed 2 years ago
235b6c6