The best ChatGPT that $100 can buy.
Python
fix typos
svlandega2fb3c83a66d
diff --git a/nanochat/loss_eval.py b/nanochat/loss_eval.pyindex 6fcbea3..5a556e6 100644--- a/nanochat/loss_eval.py+++ b/nanochat/loss_eval.py@@ -9,9 +9,9 @@ import torch.distributed as distdef evaluate_bpb(model, batches, steps, token_bytes):"""Instead of the naive 'mean loss', this function returns the bits per byte (bpb),- which is a tokenization vocab size-indepedent metric, meaning you are still comparing+ which is a tokenization vocab size-independent metric, meaning you are still comparingapples:apples if you change the vocab size. The way this works is that instead of just- calculating the average loss as usual, you calculate the sum loss, and indepependently+ calculating the average loss as usual, you calculate the sum loss, and independentlyalso the sum bytes (of all the target tokens), and divide. This normalizes the loss bythe number of bytes that the target tokens represent.diff --git a/scripts/chat_eval.py b/scripts/chat_eval.pyindex 616411d..cae2f0f 100644--- a/scripts/chat_eval.py+++ b/scripts/chat_eval.py@@ -1,6 +1,6 @@"""Evaluate the Chat model.-All the generic code lives here, and all the evlauation-specific+All the generic code lives here, and all the evaluation-specificcode lives in nanochat directory and is imported from here.Example runs: