Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

Commits on master

[update] change default seq_len
jingyaogong committed 7 months ago
b8b3d35
[update] minimind-3
jingyaogong committed 8 months ago
101d7df
Merge pull request #698 from readlnh/master
jingyaogong committed 8 months ago
83e52f6
[fix] gradient accumulation step alignment
readlnh committed 8 months ago
d25500d
[update] empty_think_ratio
jingyaogong committed 9 months ago
349e74e
[update] empty_think_ratio
jingyaogong committed 9 months ago
288e1ac
[feat] data process
jingyaogong committed 9 months ago
ccc190d
[update] save interval
jingyaogong committed 9 months ago
11a4434
[update] safe half
jingyaogong committed 9 months ago
04616c4
[fix] data skip
jingyaogong committed 10 months ago
fea69cf
[update] shuffle data
jingyaogong committed 10 months ago
f7ffdf1
[fix] max length
jingyaogong committed 10 months ago
3a5aba8
[update] pretrain load
jingyaogong committed 10 months ago
714abcf
[update] align mask
jingyaogong committed 10 months ago
aa539a8
[update] align loss
jingyaogong committed 10 months ago
c090b69
[fix] compile unpack
jingyaogong committed 10 months ago
e119db8
[feat] add compile
jingyaogong committed 10 months ago
81d24a4
[update] prompt prefill
jingyaogong committed 10 months ago
1279a61
[update] show speed
jingyaogong committed 10 months ago
05d0b21
[update] params log
jingyaogong committed 10 months ago
df89069
[update] mask log
jingyaogong committed 10 months ago
f55d4c3
[update] readme
jingyaogong committed 10 months ago
20a43d7
[update] simplify loader
jingyaogong committed 10 months ago
7641985
[update] readme
jingyaogong committed 10 months ago
0b4a8ad
[update] rename train tokenizer
jingyaogong committed 10 months ago
07364c3
[update] readme
jingyaogong committed 10 months ago
9830915
[update] rename reason
jingyaogong committed 10 months ago
4e73f34
[fix] messages num
jingyaogong committed 10 months ago
a8455ca
[fix] dist cleanup
jingyaogong committed 10 months ago
42a4e8c