Build a modern LLM from scratch. Every line commented. Explained like we are five.
Python attention-mechanismdeep-learningeducationalfrom-scratchgptlanguage-modelllamallmmachine-learningnatural-language-processingpythonpytorchtokenisationtransformerstutorial
Commits on master
fix rope relative positio demo, remove duplicate staticmethod, clarify dimension pairs and add Q/K note to chapter 4
f3db85c raiyanyahya committed 25 days ago
fix broken forward in 07_gpt_model.md , restore loss and return
c5b53b9 raiyanyahya committed 2 months ago
fix rotate_half to match emb, update dataset namespace, add perplexity topic
e179d03 raiyanyahya committed 5 months ago
update content on fine tuning and added some more examples and explanations
572b916 raiyanyahya committed 5 months ago
add requirements, colab notebook, and attention visualization
af415bc raiyanyahya committed 6 months ago
rope interleaving, double shift targets and remove unnecessary embed scaling. Thank you Grace for pointing this out.
4058e56 raiyanyahya committed 6 months ago
chapter 6 where we make a resuable block to stack to build a model
dad6490 raiyanyahya committed 6 months ago