Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

Commits on master

Merge pull request #43 from iomgaa-ycz/wandb
jingyaogong committed 2 years ago
15f8242
添加了wandb
Yu Chengzhang committed 2 years ago
06a66d8
fix data_process bug
gongjy committed 2 years ago
b4359b3
Merge pull request #42 from iomgaa-ycz/wandb
jingyaogong committed 2 years ago
5f8279f
更新了忽视列表
Yu Chengzhang committed 2 years ago
0fa4d17
修复了process_seq_monkey写入错误的bug
Yu Chengzhang committed 2 years ago
6fb569a
update data_process
gongjy committed 2 years ago
bf64ffb
update readme
gongjy committed 2 years ago
8ff2cca
update readme
gongjy committed 2 years ago
d41b39c
update readme
gongjy committed 2 years ago
cbdea6b
update data_process Chunk_size Read
gongjy committed 2 years ago
7115a98
update model mask
gongjy committed 2 years ago
6759da4
02297df
update readme
gongjy committed 2 years ago
0cd7d4b
update readme
gongjy committed 2 years ago
b969382
update readme
gongjy committed 2 years ago
6c1118a
update readme
gongjy committed 2 years ago
19523a1
update readme
gongjy committed 2 years ago
1c493e8
Updated some explanations
gongjy committed 2 years ago
9093519
Updated some explanations
gongjy committed 2 years ago
39423c6
update some explain of the code
gongjy committed 2 years ago
ee21840
Merge pull request #34 from chuanzhubin/master
jingyaogong committed 2 years ago
b4170e3
Merge branch 'master' into master
jingyaogong committed 2 years ago
95b1ee8
update others
gongjy committed 2 years ago
c81c17d
update readme
gongjy committed 2 years ago
48ea6a4
update minimind-v1-moe
gongjy committed 2 years ago
c28664d
2dceaf4
update minimind-v1-moe
gongjy committed 2 years ago
61cb61a
add accumulation_grad for pretrain
gongjy committed 2 years ago
c404941
update model
gongjy committed 2 years ago
8c18b32