Skip to content

A straightforward method for training your LLM, from downloading data to generating text.

chore: enable anomaly detection

Tianyi (Alex) Qiubda9ad201ca2
diff --git a/scripts/train_transformer.py b/scripts/train_transformer.py
index 93ad416..8998d01 100644
--- a/scripts/train_transformer.py
+++ b/scripts/train_transformer.py
@@ -6,6 +6,8 @@ from tqdm import tqdm
import torch
import numpy as np
+torch.autograd.set_detect_anomaly(True)
+
try:
device = torch.device(config.DEVICE)
except: