A LLM trained only on data from certain time periods to reduce modern bias
Python
Update README.md
Hayk Grigoriane8e87f6a31ab
diff --git a/README.md b/README.mdindex 7b668d5..eb0f6d7 100644--- a/README.md+++ b/README.md@@ -79,6 +79,12 @@ This should be doable on my own hardware, it's good too because I can hopefullyI will train this dataset today and it should take around 4-5 hours. Once it's done and I test it, I will give updates. Thank you again to everyone whos checking out my project, I've even had some people even giving me links to OCR resources so Thank you! I hope more people try this out and expirement with they're own datasets.+## July 28th, 2025++I've gone ahead and uploaded v0.5 to Hugging Face, [Check it out](https://huggingface.co/haykgrigorian/TimeCapsuleLLM) if youd like. You can now download my repo and run it locally. Unfortunately nanoGPT doesn't work natively with HuggingFace, so you'll have to download and run the model locally.++Also I will begin curating data for my next training run, I believe I'll need maybe 5-10x more data to achieve reasoning capabilities.+### Training UpdateI started training on a 435MB (108 M tokens) corpus, it's going pretty smooth right now. Train loss dropped from 10.9 to 4.9 in the first 2800 iterations. I expect it'll take around 8 or 9 hours to complete. I'll post another update once it's done.