Skip to content
Arm
Git
Search or jump to…
Ctrl K
Categories
Theme: System
Sign in
Get started
armgit
/
train-llm-from-scratch
Public
Imported from GitHub
Fork
0
Watch
0
Star
0
A straightforward method for training your LLM, from downloading data to generating text.
Python
gemini
large-language-models
llm
openai
training
transformers
Code
Commits
Issues
Pull requests
Wiki
Builds
Discussions
Projects
Insights
Search
Files
main
7 branches · 0 tags
Go to file
t
Code
train-llm-from-scratch
/
docs
/
diagrams
Name
Last commit message
Last commit date
..
src
00_overview.png
322 KB
01_data_pipeline.png
570 KB
02_pretraining.png
91.4 KB
03_sft.png
117 KB
04_reward_model.png
103 KB
05_dpo.png
151 KB
06_ppo.png
139 KB
07_grpo.png
123 KB
08_evaluation.png
138 KB
09_inference.png
106 KB
16_modern_block.png
171 KB
17_gqa.png
136 KB
18_kv_cache.png
120 KB
19_moe.png
208 KB
20_grpo_variants.png
156 KB
21_student_track.png
179 KB
22_lora.png
174 KB
23_speculative.png
211 KB
24_bpe.png
120 KB
25_typed_config.png
240 KB
attention_masks.png
41.1 KB
kv_cache_memory.png
58.8 KB
lr_schedules.png
84.0 KB
PY
make_plots.py
12.4 KB
muon_singular_values.png
71.7 KB
MD
README.md
2.3 KB
README.png
310 KB
rope.png
163 KB
tiny_loss_curves.png
68.1 KB