Skip to content

A LLM trained only on data from certain time periods to reduce modern bias

Python

Update README.md

Hayk Grigorian9aa9ae42e13c
diff --git a/README.md b/README.md
index abb7bd0..ba2ceff 100644
--- a/README.md
+++ b/README.md
@@ -3,14 +3,19 @@ An LLM trained only on data from certain time periods to reduce modern bias
Imagine if an AI model didnt just pretend to be historical but actually was.
+# Project Goals
The goal of this project is to test if we can train a small LLM from the ground up and reduce modern bias to 0.
TimeLock LLM is an expirimental project that will only be trained on texts written during certain time periods. The goal is to simulate the worldview and language of specific historical eras.
+# Why fine tuning is nt enough
+
Of course achieving zero modern bias is very difficult but I want to get as close as possible to this.
I cannot simply fine tune a model, because it already knows modern things and I would just be painting over modern bias not erasing it.
+# Expected outcomes (hopefully)
+
So my goal is to train a model from scratch using only text from a certain time period, these texts will be free of modern interpretaion, translation, modern annotations, etc.
Hopefully when finished, this model will not know modern concepts and will not be able to reason beyond what it's been trained on.