Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
Python cliconsumer-gpudpofine-tuningggufhuggingfacellmllmopslocal-ailocal-llmloralow-vrammachine-learningollamapeftpythonpytorchqlorasfttransformers
docs(contributors): #2102 [skip ci]
Alpamys9e7bd2561387
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.mdindex a1f2d359..9e0a34e7 100644--- a/CONTRIBUTORS.md+++ b/CONTRIBUTORS.md@@ -920,6 +920,7 @@ Listed by first contribution. PR numbers link the work.- Closed #1835, their own report: `training.relora_steps` loaded for `dpo`, `ipo`, `kto`, `bco`, `grpo` and a preference blend naming `dpo`, `ipo` or `bco`, although a ReLoRA restart merges the adapter into the base weights trl reads its frozen reference from, so from the first restart on the reference was the policy as of that step with no error. They measured the reference on a fixed batch for five tasks first (unchanged until the restart, 0.83 at it for `dpo`), made `SoupConfig` refuse the combination with the setting, the reason and the blend remedy, kept `sft`, `orpo`, `simpo` and `grpo_beta: 0`, and took the review on `ppo`, the message and the docs in one commit. 31 new tests, 22 fail without the fix, 41 mutations with 40 killed and one equivalent survivor ([#1835](https://github.com/MakazhanAlpamys/Soup/issues/1835), [#1891](https://github.com/MakazhanAlpamys/Soup/pull/1891))- Took the runtime half of #1693 (the issue stays open for the measured table): under layer streaming the output head and the loss built the full logits, so context length was capped by vocab x seq, not by the model. They measured the chunked loss and gradient against the unchunked ones on ten shapes before proposing and corrected their own table twice, then added `training.stream_loss_chunk` (head and loss in token chunks, gradient per chunk on the hidden states, TRL's logged columns and `rewind.jsonl` kept, pre-flight charged on `min(seq, N)`), with refusals for non-sft, raft, a trainable head, soft-capping and `use_cut_ce`. They said they had no GPU and left the card-dependent table to a follow-up, and answered two review rounds (the harness out of the diff, the hook removed after training, the seven lines no test pinned, the runtime closed when the hook cannot be built). 123 tests, 106 fail and 16 error without the change, 7 of 7 earlier survivors killed ([#1693](https://github.com/MakazhanAlpamys/Soup/issues/1693), [#1880](https://github.com/MakazhanAlpamys/Soup/pull/1880))- Took item 2 of #1048 (the issue stays open for save and resume and for the both-spellings test): under `training.stream_layers` a LoRA target on the streamed `lm_head` or `embed_tokens` was refused at config load by literal name only, so a regex or dotted spelling loaded and then died at the first default save, while a tied model, which round-trips cleanly, was refused. They claimed the item on the thread with a plan, asked before writing the PR how the dotted `model.embed_tokens` spelling should go (it saves only through an accident of peft's literal-name check) and pinned the maintainer's answer as an intentional refusal, then moved the check into `build_streamed_model` after `get_peft_model()`, where it looks at the modules peft actually wrapped and fires only when the model streams them. **The part worth copying is the controls**: the tied cases train one real step, save with the defaults and reload every LoRA tensor `torch.equal` after moving the fresh build off its init values, and the untargeted untied build is kept as a control. 12 tests (11 fail on `main`), 10 of 10 mutations killed under peft 0.20.0 and 0.21.2 ([#1048](https://github.com/MakazhanAlpamys/Soup/issues/1048), [#2082](https://github.com/MakazhanAlpamys/Soup/pull/2082))+ - Took the harness half of #1693 (the issue stays open for the measured table): `benchmarks/harness/stream_loss_chunk_vram.py` builds a chunked and an unchunked arm through the real `SFTTrainerWrapper.setup`, runs one streamed step each, and prints the pre-flight's own prediction, the free VRAM it compared against and the real peak per length. **They said up front they have no GPU, so it carries a CPU `--self-test`** that runs the same protocol on a tiny local Llama and checks 13 things (the realised length, the pre-flight's inputs and arithmetic, the hook on the chunked arm only, the verdicts and exit codes), and every figure in the new sizing text is labelled as the pre-flight's arithmetic, not a measurement. Review found the teardown of each arm unpinned and one repo scanner red on a `print()` assertion; the follow-up commit added a test that counts both teardown calls and the scanner's marker the same day. ([#1693](https://github.com/MakazhanAlpamys/Soup/issues/1693), [#2102](https://github.com/MakazhanAlpamys/Soup/pull/2102))- **Nariman Abdikarim** ([@Nari-Ab](https://github.com/Nari-Ab))- Sanitized model and dataset paths across CLI display sinks and both short-name branches, with 17 portable regressions covering refusals, adapter loading, saved outputs and real Elo/leaderboard table consumers. Independent maintainer validation passed all 17 cases before and after five source undos that caused 1/1/1/3/4 failures; all eight required hosted checks passed ([#1864](https://github.com/MakazhanAlpamys/Soup/issues/1864), [#1868](https://github.com/MakazhanAlpamys/Soup/pull/1868))