A LLM trained only on data from certain time periods to reduce modern bias
Python
Update README.md
Hayk Grigorian1d3c51553785
diff --git a/README.md b/README.mdindex 0448973..d1f6a53 100644--- a/README.md+++ b/README.md@@ -37,6 +37,8 @@ Trained nanoGPT with 187MB of historial text data.I started downloading texts for the second training run. I'm getting everything from Internet Archive and I've expanded the time period to 1800-1875. To get a diverse range of texts, you can use subject and search filters for publication location, time period and subjects on Internet Archive.++## July 16th, 2025I downloaded around 500 txt files from Internet Archive and after cleaning them (just deleting whitespaces, Gutenberg headers, etc) I have around 500MB of data. It's a tiny dataset but last time I trained off of 187MB so there should be at least some kind of noticable difference in the output after I train the second model. I'm hoping this model can at least produce more coherent sentences that kind of make sense. It's not a guarantee of course since this is still a tiny tiny dataset, but it's more than what I used last time.@@ -45,7 +47,6 @@ This should be doable on my own hardware, it's good too because I can hopefullyI will train this dataset today and it should take around 4-5 hours. Once it's done and I test it, I will give updates. Thank you again to everyone whos checking out my project, I've even had some people even giving me links to OCR resources so Thank you! I hope more people try this out and expirement with they're own datasets.-# Current Model Behavior & Limitations