The best ChatGPT that $100 can buy.
Python
fix bug in chat_sft, the attention window must be preserved sigh
Andrej Karpathy8b4849d5480a
diff --git a/scripts/chat_sft.py b/scripts/chat_sft.pyindex cad0d81..4c81f06 100644--- a/scripts/chat_sft.py+++ b/scripts/chat_sft.py@@ -301,6 +301,7 @@ while True:"n_head": model.config.n_head,"n_kv_head": model.config.n_kv_head,"n_embd": model.config.n_embd,+ "window_pattern": model.config.window_pattern,},"user_config": user_config, # inputs to the training script}