The best ChatGPT that $100 can buy.
Python
Commits on master
new leaderboard record via new auto-calculated optimal batch size. for d26 it is 1M, up from 0.5M that was default earlier
5fdd5cd Andrej Karpathy committed 9 months ago
nit: don't mutate args, create new var for total_batch_size
2c062aa Andrej Karpathy committed 9 months ago
auto-calculate optimal batch size. the original setting of 0.5M was only optimal for d12, but d26 prefers 1M and so on
f41dd3c Andrej Karpathy committed 9 months ago
bring back an assert guarding against bad param sizing
98eed6d Andrej Karpathy committed 9 months ago
fix hash commit on leaderboard and a paragraph clarification
75b302f Andrej Karpathy committed 9 months ago
correctly reference NorMuon and fix misleading terms that i may have hastily ported over from modded-nanogpt
718e5e9 Andrej Karpathy committed 9 months ago
add feb2 new leaderboard record from upgrading to fp8 training, +4.3% speedup to time to GPT-2
a67eba3 Andrej Karpathy committed 9 months ago
small touchups to the eval script, re-order items etc, cosmetic
8ebc14b Andrej Karpathy committed 9 months ago
fix bug in speedrun script, batch size that doesn't OOM on 8XH100 for d24 is 16
b19b4f3 Andrej Karpathy committed 9 months ago
tune the synthetic data generation script. delete the king andrej stuff lol. also, upgrade to gemini 3
230d6cf Andrej Karpathy committed 9 months ago
manually control the over-active garbage collector, save a small few minutes from a typical run
07c4dd4 Andrej Karpathy committed 9 months ago
slightly more efficient dataloader that reduces the number of python objects flying around and causing strain on runtime and garbage collector
e8fec97 Andrej Karpathy committed 9 months ago
fix bug in chat_sft, the attention window must be preserved sigh
8b4849d Andrej Karpathy committed 9 months ago
fix path which i think was modified during the refactor and this is a bug introduced by claude i believe
eaf49a3 Andrej Karpathy committed 9 months ago
use _PEAK_FLOPS_TABLE instead of if-else structure (#479)
4d6415b Sofie Van Landeghem committed 9 months ago
clean up original tokenizing_distributed_data_loader (#478)
43078c3 Sofie Van Landeghem committed 9 months ago
Add Blackwell (SM100) GPU support via SDPA fallback (#475)
dc291c6 Franci Penov committed 9 months ago