Skip to content

A LLM trained only on data from certain time periods to reduce modern bias

Python

Update README.md

Hayk Grigorian96979a887c7f
diff --git a/README.md b/README.md
index b9689dc..d9f3c6d 100644
--- a/README.md
+++ b/README.md
@@ -178,16 +178,25 @@ For this project I'm trying to create a language model that is unclouded from mo
I'm using books, legal documents, newspapers, and other writings from 1800–1850 London. The list I linked has like 200 but for the first training I just used 50 files about ~187 MB. You can view a list of the documents:
https://github.com/haykgrigo3/TimeCapsuleLLM/blob/main/Copy%20of%20London%20Documents%20for%20Time%20Capsule%20LLM.txt
-## How large is the Version 0 model ?
+## How large are the models ?
-This model is very small right now, I'm just doing this for fun and following a strict training rule of no modern sources. It has almost 16 million parameters but I'm gonna start gathering more old texts to begin another model training. Will give updates as I go.
+V0: 16M Parameters
-## Training Specs ?
+V0.5 123M Parameters
+V1: 700M Parameters
+
+# Training Specs ?
+
+#V0/V0.5
GPU: Geforce rtx 4060
CPU: i5-13400F
Ram: 16GB DDR5.
+#V1
+GPU: A100 rented
+
+