A straightforward method for training your LLM, from downloading data to generating text.
docs: chat's --int8 and automatic device, the modern pipeline and the full test suite in POST_TRAINING.md
FareedKhan-dev4e73b171c74f
diff --git a/POST_TRAINING.md b/POST_TRAINING.mdindex c83b6a1..ad1a8d2 100644--- a/POST_TRAINING.md+++ b/POST_TRAINING.md@@ -183,13 +183,19 @@ any external service); pass `--use_wandb true` to also mirror to Weights & Biaseenforce `prompt + generation ≤ context_length`.- **Reward hacking / KL control.** Verifier rewards are correctness-dominant with a small,bounded format bonus; a KL-to-reference penalty anchors RL to the SFT policy.+- **Two architectures, one pipeline.** `"arch": "modern"` in `configs/base.json` switches every+ stage to the modern decoder (RoPE, RMSNorm, SwiGLU, GQA or MLA, optional MoE); see+ [docs/modern](docs/modern/README.md). Its rollouts, evaluation and chat decode with a KV cache.## Tests```bashpython tests/test_post_training_smoke.py # core math: log-probs, heads, parsing, masking+pytest # the whole suite, CPU only+pytest tests/test_pipeline_e2e.py -s # every training script, end to end, on generated data```-Each trainer also runs end-to-end on a tiny model in seconds (see the smoke commands used-during development), and the PPO/GRPO math has standalone unit checks (GAE, clipped losses,-group advantages, k3 KL).+The end-to-end test writes a small generated dataset, then runs pretraining, SFT (with LoRA),+the reward model, DPO and SimPO, PPO, GRPO and chat for a few steps each, with both+architectures, so a change that breaks a stage or the hand-over between two stages fails in CI. The PPO/GRPO math has standalone unit checks (GAE, clipped losses, group+advantages, k3 KL, and the Dr. GRPO, DAPO and GSPO variants).diff --git a/docs/09_inference.md b/docs/09_inference.mdindex a1d26fb..41b3417 100644--- a/docs/09_inference.md+++ b/docs/09_inference.md@@ -80,8 +80,10 @@ python scripts/chat.py --ckpt models/base_pretrained.pt --raw --prompt "Once upopython scripts/chat.py --ckpt models/sft.pt```-Sampling controls: `--temperature`, `--top_p`, `--top_k`, or `--greedy` for deterministic argmax. Runs-on `--device cuda` or `cpu` (both verified).+Sampling controls: `--temperature`, `--top_p`, `--top_k`, or `--greedy` for deterministic argmax.+`--device` defaults to `auto` (CUDA, then Apple MPS, then the CPU), and `--int8` stores the linear+layers' weights as int8, which takes about a quarter of their memory. Modern-architecture+checkpoints generate with a KV cache, so each new token costs one short forward pass.## Sampling knobs, briefly@@ -89,4 +91,7 @@ on `--device cuda` or `cpu` (both verified).- **temperature**: higher = more random; ~`0.7–1.0` for open-ended chat.- **top_p / top_k**: nucleus / top-k truncation to cut the long tail of unlikely tokens.+[Inference](modern/inference.md) in the Modern LLM section goes further: min-p sampling, how+the KV cache works, speculative decoding and int8 weights, with measurements.+That's the full loop: pretrain → align → reason → measure → chat. Back to the [overview](README.md).