Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(train): make the update-budget reference post-calibration, drop the temp JSONL

Bruce-Yii4ef9a1ae7065
diff --git a/docs/finetune.md b/docs/finetune.md
index f63e4097..6c2402b1 100644
--- a/docs/finetune.md
+++ b/docs/finetune.md
@@ -104,12 +104,14 @@ count, with the effective batch:
optimizer budget: 1100 train item(s), effective batch up to 64 (micro_batch 8 x grad_accum 8), 72 optimizer updates over 4 epoch(s)
```
-The defaults (4 epochs, effective batch 64) are the notebook's; on typed-decisions' 6,000
-decisions they work out to about 376 updates, fewer once the calibration slice is held out. That
-number is a reference for comparing runs of different sizes, not a cutoff: the same defaults on
-1,100 items give about 72 updates, and [#963](https://github.com/NandhaKishorM/laya/issues/963)
-measured a head that collapsed there while 8 epochs still failed on the task itself. A small dataset
-therefore needs more epochs to reach a similar update count.
+The defaults (4 epochs, effective batch 64) are the notebook's. On typed-decisions' 6,000
+decisions the default 400-item calibration slice leaves 5,600 training items, so a default run
+spends **352** updates — 376 only if no slice were held out. That number is a reference for
+comparing runs of different sizes, not a cutoff: 1,100 *training* items at the same settings give
+72 updates (64 when 1,100 is the pre-split count and the 10% slice holds 110 out), and
+[#963](https://github.com/NandhaKishorM/laya/issues/963) measured a head that collapsed there while
+8 epochs still failed on the task itself. A small dataset therefore needs more epochs to reach a
+similar update count.
### What a run writes
diff --git a/tests/test_train.py b/tests/test_train.py
index 6f7933ac..1b016e2d 100644
--- a/tests/test_train.py
+++ b/tests/test_train.py
@@ -1173,13 +1173,11 @@ class OptimizerBudgetTests(unittest.TestCase):
with tempfile.TemporaryDirectory() as d:
root = Path(d)
make_checkpoint(root / "base", make_tokenizer())
- data = root / "train.jsonl"
- data.write_text("\n".join(json.dumps(r) for r in rows(2)), encoding="utf-8")
config = TrainConfig(epochs=3, micro_batch=4, grad_accum=2, calib_frac=0.25, log_every=0)
buf = io.StringIO()
- with contextlib.redirect_stdout(buf):
- summary = dry_run(str(data), str(root / "base"), config)
+ with patch("laya.train.read_data", return_value=rows(2)), contextlib.redirect_stdout(buf):
+ summary = dry_run("synthetic.jsonl", str(root / "base"), config)
# The budget must come from the post-calibration split, not the raw valid-item count.
self.assertGreater(summary["calibration_items"], 0)