A straightforward method for training your LLM, from downloading data to generating text.
Commits on main
fix(scripts): shell helpers without the author's venv and disk
dec0553 FareedKhan-dev committed 1 month ago
refactor(paths): keep data, checkpoints and logs in the repo, not /ephemeral
d62b5f2 FareedKhan-dev committed 1 month ago
feat(models): temperature, top-k and a context window for generate()
598f303 FareedKhan-dev committed 1 month ago
fix(models): forward_embedding with more than one block
83f0b07 FareedKhan-dev committed 1 month ago
feat(device): pick CUDA, Apple MPS or CPU automatically
d69b5c2 FareedKhan-dev committed 1 month ago
fix(gitignore): anchor data/, models/ and logs/ to the repo root
d58d5f1 FareedKhan-dev committed 1 month ago
fix(checkpoints): load models trained with torch.compile (fixes #36)
fec1848 FareedKhan-dev committed 1 month ago
fix(data): never crash or starve answers when truncating preference pairs
65ccf76 FareedKhan-dev committed 1 month ago
chore: ignore local environments and caches, keep shell scripts LF
650e702 FareedKhan-dev committed 1 month ago
Merge pull request #41 from Iams4kura/bugfix/preserve-preference-responses-20260817t080146z
a0cded0 FareedKhan-dev committed 1 month ago
Merge pull request #40 from FaintFlower/fix/star-history-chart
b995104 Fareed Khan committed 3 months ago
docs: point the README star history chart to a working provider
6d744d7 FaintFlower committed 3 months ago
Merge pull request #33 from FareedKhan-dev/docs/readme-tutorial
98f808c Fareed Khan committed 5 months ago
docs: rewrite README as a complete end-to-end tutorial
d1b71e0 FareedKhan-dev committed 5 months ago
docs: add hand-drawn diagrams and a reproducible generator
040f478 FareedKhan-dev committed 5 months ago
Merge pull request #32 from FareedKhan-dev/fix/ddp-multigpu
0cc6125 Fareed Khan committed 5 months ago
fix(ddp): make the reward model train on multiple GPUs
9476be5 FareedKhan-dev committed 5 months ago
fix(ddp): run the SFT and DPO final eval on the unwrapped model
806659c FareedKhan-dev committed 5 months ago
Merge pull request #30 from OHUVERSE/correct-equation-format
2f995f8 Fareed Khan committed 5 months ago
Merge pull request #31 from OHUVERSE/add-link-to-pretraining
e851928 Fareed Khan committed 5 months ago
feat: opt-in AMP, gradient checkpointing & gradient accumulation; fix CPU contiguity (fixes #5) (#29)
2b9bcaa Fareed Khan committed 5 months ago
Harden checkpoint writes and restore Python 3.9 typing compatibility in legacy trainer (#28)
efb9234 Copilot committed 5 months ago
Merge pull request #26 from Abellegese/docs/foundations-tutorial
cc4d983 Fareed Khan committed 5 months ago
Merge pull request #18 from FareedKhan-dev/feat/post-training-configs-ui-docs
c739e9d Fareed Khan committed 5 months ago