Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Python cliconsumer-gpudpofine-tuningggufhuggingfacellmllmopslocal-ailocal-llmloralow-vrammachine-learningollamapeftpythonpytorchqlorasfttransformers
docs(contributors): #2145 [skip ci]
Alpamys59404c13eaa1
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.mdindex 849c5e1a..30a60ede 100644--- a/CONTRIBUTORS.md+++ b/CONTRIBUTORS.md@@ -688,6 +688,8 @@ Listed by first contribution. PR numbers link the work.- Removed the two orphaned BitNet helpers (`utils/bitnet.build_bitnet_trainer`, `utils/bitnet.validate_bitnet_compat`) that nothing imported, per the ruling on #1004, and replaced their tests with absence pins: restoring either as a stub, or re-exporting the live wrapper under the old name, fails a test. **When review showed that an extra change to the async disk reader rested on a race that did not exist, they reverted it within minutes and said so plainly**, so the squashed diff touches no streaming file ([#1004](https://github.com/MakazhanAlpamys/Soup/issues/1004), [#1034](https://github.com/MakazhanAlpamys/Soup/pull/1034))- On an untied streamed checkpoint, `embed_tokens` and `lm_head` share one device slot, so every step's first lookup waited for a whole head-sized copy that `_prime()` issued with nothing to overlap it. The issue first blamed the disk reader's `_plan_queue`. They showed the cost was the shared slot's ping-pong instead, and retracted a sentence of their own analysis that nobody had challenged. The fix reloads the embedding at the tail of the previous step's backward and keeps `_prime()`'s load as a same-owner fallback: 18.6-22.7 ms of embedding wait became 0.12-0.19 ms per step on an RTX 5070, bit-identical to `main` and to resident (CPU fp32 for SFT, ORPO, SimPO, DPO and KTO; CUDA bf16 for SFT and DPO). **The part worth copying is the guard, not the speedup**: the refill's only GPU-side protection is the `wait_stream` in `LargeLayerBufferPool.load_async`. They committed a pool-level GPU test for it that does not depend on host speed, ran it on their own RTX 3050 (5/5 passing, and 5/5 failing with the guard patched out), and said so when a first mutation attempt had run against unmodified code. A 4-layer test pins that the refill fires once per backward, at layer 0. 10 of 12 mutations fail the tests; the survivors are an equivalent flag check and the GPU-only guard ([#975](https://github.com/MakazhanAlpamys/Soup/issues/975), [#1178](https://github.com/MakazhanAlpamys/Soup/pull/1178))- Once #1178 was in, what was left of an untied streamed checkpoint's large-layer wait was the output head's load into the shared slot. It was issued at the last decoder layer, where it could hide behind only that layer's compute. They moved it to right after the embedding lookup (`StreamPrefetcher.head_prefetch_layer`, default `0`; `None` restores the old timing), at no extra VRAM: the cheap alternative to #1174's second permanent slot. On an RTX 5070 Laptop GPU the step's median total wait fell from 11.96 to 4.50 ms on the RAM tier and from 15.83 to 1.42 ms on the disk tier, with every layer-0 sample below every last-layer one. A decoder copy now queues behind the head's and gives back part of the saving; their CPU trace of the copy order had shown that queueing before the card measured it. **The part worth copying is that they built the instrument before the number existed.** The switch is a plain attribute, validated on assignment, so one process runs both timings. The interleaved A/B driver they wrote for a card they do not own has an A/A control, rotating arm order and a record of the tree it measured, and it ran on that card as one command per configuration. When review found that the provenance change dropped the NVIDIA driver version from every record and recorded the driver's checkout instead of the measured tree, they fixed both and pinned them with tests within ten minutes. They then covered the driver's own `main()` with a CPU test, so all 11 mutations of its provenance code fail. Gradients and adapters are bit-identical across SFT, SFT at GA 2, ORPO, SimPO, DPO, KTO and tied SFT; all 8 mutations of the new validation are killed; and the pool-level GPU guard fails 3 of 3 with the `wait_stream` removed ([#1174](https://github.com/MakazhanAlpamys/Soup/issues/1174), [#1258](https://github.com/MakazhanAlpamys/Soup/pull/1258))+ - Closed #1432: `training.freeze_layers` / `training.freeze_ratio` did nothing under LoRA, because the freeze ran before `get_peft_model` attached a trainable adapter to every layer. They carried @AmirF194's closed #1494 over from its branch with both commits kept, closed its last review (`lm_head` is accepted because peft keeps an exact module name, six tests, the docs paragraph, the fragment) and dropped the LLaMA Pro widening that #1326 had made unreachable ([#1432](https://github.com/MakazhanAlpamys/Soup/issues/1432), [#2145](https://github.com/MakazhanAlpamys/Soup/pull/2145))+ - Their closed #1494 is the base of the #1432 fix: the freeze plan is passed to peft as `layers_to_transform`, a `cutoff == 0` plan leaves it unset, and a list target outside a numbered layer such as `embed_tokens` is refused by name instead of silently losing its adapter; the work shipped through #2145, with their two commits kept ([#1432](https://github.com/MakazhanAlpamys/Soup/issues/1432), [#2145](https://github.com/MakazhanAlpamys/Soup/pull/2145))- **fei** ([@feiiiiii5](https://github.com/feiiiiii5))- First contribution, and it fixes the command whose whole job is to report this project's most common defect. `soup doctor --config` printed `Every setting this config writes is read on this backend.` for a `task=sft backend=mlx` config setting `use_liger: true` and `neftune_alpha: 5` — two fields MLX never reads. **The wording was the real defect, not the table.** A seven-entry table produced a sentence about the entire config, so the all-clear was a guarantee built on evidence that did not support it, and the fix scopes it: `None of the N setting(s) known to be unread on task=<task> backend=<backend> is set in this config.` The two repro fields were added to `_MLX_SFT` with matching branches in `trainer/mlx_sft.py::_check_unsupported`, so the trainer reports them too rather than only the doctor. Each entry was grep-checked for zero hits across the three MLX modules **before** being declared unread, which is the step that keeps a declared table honest. Verified here against the issue's exact repro in both directions — on `main` that config prints the all-clear, with this change it prints the two-row table and the scoped count — and confirmed no conflict with the five other open PRs touching `commands/doctor.py`. The narrowing to 2 of the issue's 11 fields was declared rather than hidden, and the follow-up for the remaining nine is filed ([#903](https://github.com/MakazhanAlpamys/Soup/issues/903), [#911](https://github.com/MakazhanAlpamys/Soup/pull/911))