Skip to content

armgit / Soup

Public Imported from GitHub
0

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

docs(contributors): #2130 [skip ci]

Alpamys65392c26139b
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 07d6db6e..6d266f67 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -259,6 +259,7 @@ Listed by first contribution. PR numbers link the work.
- Closed #2119: a KTO `label` of `null`, a float, a list or an object trained as a desirable or undesirable row instead of being refused (`bool(raw_label)` made `[]` False and `["no"]` True; floats were read as `> 0`, so `0.5` was desirable and `nan` undesirable). They made the shared converter accept a bool, the six documented strings and exactly `-1`, `0` and `1`, and refuse everything else with a `ValueError` that names the label and shows the value, so `soup data validate` and `soup train --dry-run` report it before any model loads. The tests enumerate the spellings (including `1.0`, `nan`, `inf`, `-1` as a string) through the converter, the loader reason and both CLI paths, and flip the old test that pinned `None -> False`; with the file as it is on `main` 42 of 66 fail, and six source mutations (the `bool()` branch, floats as `> 0`, integral floats, any integer, `-1` inverted, a message without the value) each fail at least one named test. The explicit bool branch is redundant and survives as an equivalent mutation ([#2119](https://github.com/MakazhanAlpamys/Soup/issues/2119), [#2126](https://github.com/MakazhanAlpamys/Soup/pull/2126))
- Added the `qwen3.5-4b-dpo` recipe from #275: the Qwen3.5-9B DPO settings on `Qwen/Qwen3.5-4B`, text only with an explicit `modality: text`, the inherited (untuned) settings named as such in the docs and fragment. The model id is pinned on `RecipeMeta.model` and the YAML `base:` and each fails its own test; the recipe is registered in the Qwen3.5 family audit, the count moved 169 -> 170 at every site and the snapshot delta is purely additive. The anonymous `model_info` check (851bf6e8, `config.json` present, Apache-2.0) reproduces, `soup recipes verify` attaches the recipe on the real public config (adapted=8), and a tiny random-weight native Qwen3.5 text decoder is built on CPU in the tests. Ten source/test mutations each fail a named test; six default-equivalent lines (`epochs`, `batch_size`, `target_modules`, `quantization`, `dpo_beta`, `output`) survive as equivalents and are listed as such ([#275](https://github.com/MakazhanAlpamys/Soup/issues/275), [#2129](https://github.com/MakazhanAlpamys/Soup/pull/2129))
- Coordinated the base-model commit across distributed Hugging Face Transformers `soup train` workers: global rank zero resolves once, one nullable result is broadcast before any rank loads the base, and ranks compare receipts after setup and training. **Two review findings shaped it**: the first version created the process group with a 120 s timeout, which Trainer's later `ddp_timeout` (1800 s) cannot override, so every collective of a multi-GPU run would have timed out after two minutes; the fix names the 1800 s default and the test compares it with `TrainingArguments`' own default, so a one-second shortfall fails. Then the real two-process Gloo cases timed out on a loaded macOS runner, and they were moved to Linux only with a harness test that pins the platform gate and a failure report that prints both ranks' logs and reaps the children. The broadcast, rank-ownership, timeout, platform-gate and diagnostics boundaries were each mutated and killed by named tests ([#1902](https://github.com/MakazhanAlpamys/Soup/issues/1902), [#2127](https://github.com/MakazhanAlpamys/Soup/pull/2127))
+ - Refused the resident Hugging Face Kimi K2 bases at config validation (#2104 option 2) with the real import traceback and source hashes as evidence: `Kimi-K2-Base`, `-Instruct`, `-Instruct-0905` and `-Thinking` stop before model loading, while `stream_layers: true`, local directories, other hubs, Unsloth and K2.5/K2.6 stay as they were. **The part worth copying is the per-id guidance**: after review the Base/Instruct text names the guarded streamed route and Thinking/Instruct-0905 say "no supported route yet" without naming `stream_layers`, with each recipe carrying its own reason in the snapshot, and when #2107 merged they rewrote the wording to the merged state with its prerequisites and evidence limits; the tests assert the right text per id and the absence of the old wording, and swapping reasons between ids fails named tests. Thirteen first-round and five second-round mutations, all killed ([#2104](https://github.com/MakazhanAlpamys/Soup/issues/2104), [#2130](https://github.com/MakazhanAlpamys/Soup/pull/2130))
- **AmixDigital** ([@Amix29](https://github.com/Amix29))
- Added native Sesame CSM fine-tuning for transcript/audio pairs, training the backbone and depth decoder while keeping the Mimi codec frozen; documented supported inputs, refusals and the scoped Apple Silicon evidence ([#265](https://github.com/MakazhanAlpamys/Soup/issues/265), [#1769](https://github.com/MakazhanAlpamys/Soup/pull/1769)).