Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

[fix] Refactor get_lr function to include min_lr calculation

dyhuachibf3878ace83b
diff --git a/trainer/trainer_utils.py b/trainer/trainer_utils.py
index 93c9bfc..0ad1b00 100644
--- a/trainer/trainer_utils.py
+++ b/trainer/trainer_utils.py
@@ -23,7 +23,8 @@ def Logger(content):
def get_lr(current_step, total_steps, lr):
- return lr / 10 + 0.5 * lr * (1 + math.cos(math.pi * current_step / total_steps))
+ min_lr = lr / 10
+ return min_lr + 0.5 * (lr - min_lr) * (1 + math.cos(math.pi * current_step / total_steps))
def init_distributed_mode():