Skip to content

armgit / Soup

Public Imported from GitHub
0

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

docs(contributors): #2063 [skip ci]

Alpamys4ecf5bd6a994
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 686248d7..7c11211c 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -813,6 +813,7 @@ Listed by first contribution. PR numbers link the work.
- `soup serve --device` was ignored by the transformers loader: both `_load_model` branches and the speculative draft loader hard-coded `device_map="auto"`, so `--device cpu` could still land on a visible GPU, and an unknown value such as `tpu` was accepted and dropped (`--device "[/bold]"` crashed with a Rich `MarkupError`). They routed all three loads through #1460's `resolve_inference_device_map_and_dtype`, validated `--device` early with exit 2 on every backend, kept an explicit `--kv-cache-type` winning over the helper's dtype, and updated the help and the fragment, including that a CPU-only host now loads float32 (twice the RAM). The new file is 34 passed (19 failed / 15 passed on `main`), 627 passed across 17 related files, and 23 of 24 mutations fail a test; the one survivor (validation gated on the transformers backend, behaviour verified on vLLM and SGLang) is recorded as a follow-up test ([#1478](https://github.com/MakazhanAlpamys/Soup/issues/1478), [#1479](https://github.com/MakazhanAlpamys/Soup/pull/1479))
- Fixed #1450: a failed replace deleted the only good copy in two places. `soup reward synth --force` overwrote the existing verifier before calibrating it and deleted the file on refusal; `merge_adapter_to_dense` (behind `soup shrink` and `soup draft distill`) deleted `out_dir` before the swap and then deleted the staged model in its error handler, so neither copy survived. They moved both to a never-destroy-the-last-copy shape: a temp sibling replaced onto `-o` only after calibration accepts it; rename-aside, restore on failure, and when the restore fails as well, keep both copies and name both paths. **The review rounds were about what a test pins:** a Windows-only test that could not fail on their own machine reddened three CI cells, and the fix replaced its either-or assertion with exact bytes at the paths the error names. 11 tests, 11 of 11 mutations killed, 9 failed / 2 passed on `main` ([#1450](https://github.com/MakazhanAlpamys/Soup/issues/1450), [#1454](https://github.com/MakazhanAlpamys/Soup/pull/1454))
- Closed #1703: with a reusable layer-shard cache, the layer-streaming disk pre-flight printed the estimate for writing a fresh cache as the `Layer-shard cache` size, so that row and `Projected total on disk` described a cache that was not there. `shard_cache_disk_bytes` now adds up the cache's own files (the index, the extras shard, the large shards and each layer file across the stripe folders) and the pre-flight reports that. **The part worth copying is that the sum names the files the cache holds instead of measuring the folder**: a stray or stale file in the folder is not counted, a file that cannot be measured keeps the estimate, a cache that has to be written is reported as before, and the write planned for a reusable cache stays 0. After the first review they added the tests for the two mutations it found surviving (the vocabulary shards of an untied checkpoint, and the materialization charge through the setup) and for a cache file lost between the check and the sizing, and they made a measured size stop reading "approximately". The PR says plainly that the full suite was not run ([#1703](https://github.com/MakazhanAlpamys/Soup/issues/1703), [#1709](https://github.com/MakazhanAlpamys/Soup/pull/1709))
+ - `soup adapters check` and `soup adapters diff` crashed on an adapter saved from a bfloat16 run, because they read the weights with `safe_open(framework="numpy")` and numpy has no bfloat16 (`data type 'bfloat16' not understood`). They wrote one shared header-based reader that widens bfloat16 to float32 bit for bit and keeps the size, tensor-count and symlink checks, then, when main added the FP8 spellings, turned the resulting `KeyError` into a refusal that names the tensor and dtype. Their parametrized test compares every mapped dtype with `safe_open(framework="numpy")`; the mutations that failed no test in the first round are all caught in the second ([#2058](https://github.com/MakazhanAlpamys/Soup/issues/2058), [#2063](https://github.com/MakazhanAlpamys/Soup/pull/2063))
- **sup3dev** ([@sup3dev](https://github.com/sup3dev))
- First contribution: `soup` reported any error whose message contained the digits `401` or `403` as "Authentication failed." / "Access denied." and hid the real message. That covered a too-long SFT row 401, a tensor size of 4013, a `checkpoint-4010` path and a hash in a Hub URL. The status is now read from the exception object along its `__cause__`/`__context__` chain, with a text fallback limited to explicit HTTP phrasing, so genuine requests, urllib and huggingface_hub 401/403 errors get the specific hint instead of "HTTP request failed.". **The detail that decided a three-way race for the issue**: `GatedRepoError` and `RepositoryNotFoundError` are declined anywhere in the chain. A missing local adapter directory that peft first looks up on the Hub, which answers an unauthenticated user with 401, therefore keeps its real message; both competing fixes reported "Authentication failed." there. 16 of the 31 new tests fail on `main`, and 21 of 22 mutations fail a test ([#1267](https://github.com/MakazhanAlpamys/Soup/issues/1267), [#1268](https://github.com/MakazhanAlpamys/Soup/pull/1268))