A straightforward method for training your LLM, from downloading data to generating text.
Commits on main
Merge pull request #42: laptop track, modern decoder, new training methods, type safety, tests and CI
bdd480a Fareed Khan committed 1 month ago
docs(changelog): the --set flag, cached rollouts and the fixes found in review
6892b7e FareedKhan-dev committed 1 month ago
docs: add the laptop track, the Modern LLM section and the new pages to the site navigation and home page
7b7f6d4 FareedKhan-dev committed 1 month ago
docs(howto): student preset timing in the hardware guide
d1d6b91 FareedKhan-dev committed 1 month ago
docs(student): train an LLM on a laptop, step by step (#38)
8cbb274 FareedKhan-dev committed 1 month ago
docs(modern): overview of the modern model, the switch, the settings and a measured comparison
117d0cc FareedKhan-dev committed 1 month ago
docs(modern): inference with min-p sampling, the KV cache, speculative decoding and int8
8efe860 FareedKhan-dev committed 1 month ago
docs(foundations): scaling, with parameters, FLOPs, memory and training time
4c81316 FareedKhan-dev committed 1 month ago
docs: a full URL for the modern docs link in POST_TRAINING.md, which the commands page embeds
f7280de FareedKhan-dev committed 1 month ago
docs: chat's --int8 and automatic device, the modern pipeline and the full test suite in POST_TRAINING.md
4e73b17 FareedKhan-dev committed 1 month ago
perf(rollout): PPO and GRPO rollouts use the modern model's KV cache
9d2d50b FareedKhan-dev committed 1 month ago
fix(sft): keep Mixture-of-Experts routers balanced during SFT
59a089a FareedKhan-dev committed 1 month ago
fix(ddp): Mixture-of-Experts models no longer crash multi-GPU training
1da4317 FareedKhan-dev committed 1 month ago
fix(grpo): skipped groups no longer shrink the per-answer loss averages
00df951 FareedKhan-dev committed 1 month ago
docs(contributing): separate the fast tests from the full suite
646c6e4 FareedKhan-dev committed 1 month ago
style(ui): drop em dashes from the control panel's text
2326c0d FareedKhan-dev committed 1 month ago
feat(train): --set KEY=VALUE overrides any config value in train_transformer.py
d7d19c6 FareedKhan-dev committed 1 month ago
docs(diagrams): list the new diagrams and explain make_plots.py
f9dbfbd FareedKhan-dev committed 1 month ago
fix(scripts): render_diagrams.sh describes the PNGs it writes and accepts a local mmdc via MMDC
160f7f8 FareedKhan-dev committed 1 month ago
docs: CHANGELOG for 0.2.0, crediting the issues and pull requests it builds on
46783c3 FareedKhan-dev committed 1 month ago
docs(readme): type safety and tests, and an updated "What's Next"
c8b0a65 FareedKhan-dev committed 1 month ago
docs(readme): the modern model, presets, Muon, generation tricks and the new losses
9f6fdbd FareedKhan-dev committed 1 month ago
docs(readme): the laptop track, uv setup and the new code layout
20c507d FareedKhan-dev committed 1 month ago
docs(pipeline): link each stage page to its modern variants (architecture, Muon, LoRA, IPO/SimPO, GRPO follow-ups)
1acb34f FareedKhan-dev committed 1 month ago
docs: point line-anchored code links at the current definitions
e691b32 FareedKhan-dev committed 1 month ago
docs(howto): repo-relative paths and an up-to-date setup in POST_TRAINING.md and the how-to pages
23ef4ed FareedKhan-dev committed 1 month ago