Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docker: model files left in the older /data/models stop the start instead of being downloaded again (#3421)
Niko122110a62b8d2f50
diff --git a/docker/entrypoint.sh b/docker/entrypoint.shindex 6b6f87cfa..f5476e3a9 100755--- a/docker/entrypoint.sh+++ b/docker/entrypoint.sh@@ -10,6 +10,7 @@# MODEL model size, e.g. IQ3_XXS, Q2_0 (empty: Strata picks one for this card and says so)# FAMILY model family when a size is offered by several (qwen, swift, coder, ...)# MODEL_PATH GGUF files you already have, a folder or one .gguf, inside the container (mount it read-only)+# ALLOW_DOWNLOAD 1: go on and download the model although model files of the older layout (/data/models) were left unread (default: stop)# CONTEXT 32k, 64k, 128k, 256k or a number of tokens# PARALLEL requests answered at the same time, 1 to 8 (empty: fit to the card)# VISION off | on | gpu | cpu (pictures; the picture reader file is downloaded with the model)@@ -38,6 +39,17 @@ warn() { printf '[!] %s\n' "$*" >&2; }die() { printf '[X] %s\n' "$*" >&2; exit "${2:-1}"; }phase() { printf '%s\n' "$1" > "$PHASE_FILE" 2>/dev/null || true; }+# Model files of the older layout that this image did not move (a folder you mounted at /data/models, or a move that failed) are not read here, and+# the start would download the model again. Say so and stop instead (#3421); ALLOW_DOWNLOAD=1 goes on, with the same text as a warning.+# $1: MODEL_PATH (when it is set the files are named and nothing is lost), $2: the older models folder.+legacy_models_guard() {+ [ -n "$1" ] && return 0+ stuck=$(find "$2" -maxdepth 3 -name '*.gguf' 2>/dev/null | head -1)+ [ -n "$stuck" ] || return 0+ msg="There are model files in ${stuck%/*} (the older image's place). This image keeps its models in /data/Strata-data/models and does not read them there, so it would download the model again (tens of GB). To use them: mount that folder read-only and set MODEL_PATH to the folder that holds the shards (docs/DOCKER.md, \"Files you already have\"), or put them in /data/Strata-data/models/<model folder>/. To download anyway: set ALLOW_DOWNLOAD=1. Nothing was downloaded."+ if [ "${ALLOW_DOWNLOAD:-0}" = "1" ]; then warn "$msg Going on, because ALLOW_DOWNLOAD=1."; else die "$msg" 2; fi+}+# ---- other commands: `docker run image strata ...`, `docker run image sh`if [ "$#" -gt 0 ] && [ "$1" != "run" ] && [ "$1" != "serve" ]; thencase "$1" in strata) shift; exec "$STRATA" "$@" ;; *) exec "$@" ;; esac@@ -174,6 +186,7 @@ if [ -n "$model_path" ]; thendie "MODEL_PATH=$model_path holds no .gguf files. Mount the folder with the model's shards (the -0000N-of-0000M.gguf files). Nothing was downloaded." 2fifi+legacy_models_guard "$model_path" "${STRATA_LEGACY_MODELS:-/data/models}"[ -n "${MODEL:-}" ] && { set -- "$@" --model "$MODEL"; have=1; }[ -n "${FAMILY:-}" ] && { set -- "$@" --family "$FAMILY"; have=1; }[ -n "$model_path" ] && { set -- "$@" --model-path "$model_path"; have=1; }diff --git a/docs/DOCKER.md b/docs/DOCKER.mdindex 3ea6f661f..d0825fd60 100644--- a/docs/DOCKER.md+++ b/docs/DOCKER.md@@ -115,6 +115,7 @@ screens; leave out what you do not care about.| `MODEL` | model size: `IQ2_XS`, `IQ3_XXS`, `IQ3_S`, `Q2_0`, `UD-Q4_K_XL`, ... ([MODELS.md](MODELS.md)) | Strata picks what suits the card, and says so in the log || `FAMILY` | `qwen`, `unsloth`, `gyro`, `coder`, `swift`, ...; use `qwen` for the original ISTA-DASLab packs | installer recommendation; may differ by hardware || `MODEL_PATH` | GGUF files you already have: a folder or one `.gguf`, **inside the container** (mount it with `-v`, `:ro` is fine) | download |+| `ALLOW_DOWNLOAD` | `1`: go on and download the model although model files of the older layout were left unread in `/data/models` (see [From the older image](#from-the-older-image)) | stop || `CONTEXT` | how much text it remembers at once: `32k`, `64k`, `128k`, `256k` or a number | the largest that fits || `PARALLEL` | requests answered at the same time, 1 to 8 | fitted to the card || `VISION` | `off`, `on` (the model reads pictures), `gpu`, `cpu`; the picture-reader file (about 0.9 GB) is downloaded with the model | `off` |@@ -276,7 +277,10 @@ package and start through `strata launch`. Mapping:A volume from the older image is picked up: the first start moves `/data/models`, `/data/packs` and `/data/mtp` into`/data/Strata-data` (a rename inside the volume: nothing is copied or downloaded again, and the prepared pack is kept). The old`/data/config` is not read; the run settings are made again from the files. A folder you mounted yourself at one of those places-is left alone: point `MODEL_PATH` at it. The container name, the volume and the port can stay as they were.+is left alone: point `MODEL_PATH` at it. If model files are still in `/data/models` when the container starts (a folder you mounted there, or+a move that failed) and `MODEL_PATH` is not set, the start **stops at once and says where the files are**: it would otherwise download the model+again, tens of GB, because this image does not read that place. Name the folder in `MODEL_PATH`, or set `ALLOW_DOWNLOAD=1` to download anyway.+The container name, the volume and the port can stay as they were.## Build it yourselfdiff --git a/tests/terminal/test_docker_files.py b/tests/terminal/test_docker_files.pyindex 3f2a55203..d146282a9 100644--- a/tests/terminal/test_docker_files.py+++ b/tests/terminal/test_docker_files.py@@ -133,6 +133,63 @@ class Compose(unittest.TestCase):self.assertNotIn("/dev/kfd:/dev/kfd", intel["devices"])[email protected](SH is None, "needs sh")+class LegacyModelsGuard(unittest.TestCase):+ """#3421: model files left in the older image's /data/models (a folder the person mounted there, or a failed move) are not read by this image,+ so the start would download the model again. It stops and says where the files are; MODEL_PATH names them, ALLOW_DOWNLOAD=1 goes on."""++ def run_guard(self, folder, model_path="", allow=None):+ import os+ import tempfile+ entry = (DOCKER / "entrypoint.sh").read_text(encoding="utf-8")+ body = entry[entry.index("legacy_models_guard() {"):]+ body = body[:body.index("\n}\n") + 3]+ with tempfile.TemporaryDirectory() as tmp:+ script = Path(tmp) / "guard.sh"+ script.write_text("warn() { printf '[!] %s\\n' \"$*\" >&2; }\ndie() { printf '[X] %s\\n' \"$*\" >&2; exit \"${2:-1}\"; }\n" + body+ + f"\nlegacy_models_guard '{model_path}' '{sh_path(folder)}'\necho CONTINUED\n", encoding="utf-8", newline="\n")+ env = dict(os.environ)+ env.pop("ALLOW_DOWNLOAD", None)+ if allow is not None:+ env["ALLOW_DOWNLOAD"] = allow+ return subprocess.run([SH, sh_path(script)], capture_output=True, text=True, env=env)++ def test_files_in_the_older_place_stop_the_start_with_the_folder_named(self):+ import tempfile+ with tempfile.TemporaryDirectory() as tmp:+ shards = Path(tmp) / "IQ3_XXS"+ shards.mkdir()+ (shards / "model-00001-of-00002.gguf").write_bytes(b"GGUF")+ done = self.run_guard(Path(tmp))+ self.assertEqual(done.returncode, 2, done.stdout + done.stderr)+ self.assertNotIn("CONTINUED", done.stdout)+ self.assertIn(shards.name, done.stderr)+ self.assertIn("MODEL_PATH", done.stderr)+ self.assertIn("ALLOW_DOWNLOAD=1", done.stderr)+ self.assertIn("Nothing was downloaded", done.stderr)++ def test_model_path_allow_download_an_empty_folder_and_a_missing_one_go_on(self):+ import tempfile+ with tempfile.TemporaryDirectory() as tmp:+ (Path(tmp) / "IQ3_XXS").mkdir()+ (Path(tmp) / "IQ3_XXS" / "model-00001-of-00002.gguf").write_bytes(b"GGUF")+ named = self.run_guard(Path(tmp), model_path="/models")+ self.assertEqual((named.returncode, "CONTINUED" in named.stdout), (0, True), named.stderr)+ allowed = self.run_guard(Path(tmp), allow="1")+ self.assertEqual((allowed.returncode, "CONTINUED" in allowed.stdout), (0, True), allowed.stderr)+ self.assertIn("[!]", allowed.stderr)+ self.assertIn("ALLOW_DOWNLOAD=1", allowed.stderr)+ with tempfile.TemporaryDirectory() as empty:+ nothing = self.run_guard(Path(empty))+ self.assertEqual((nothing.returncode, "CONTINUED" in nothing.stdout), (0, True), nothing.stderr)+ missing = self.run_guard(Path("/no/such/folder"))+ self.assertEqual((missing.returncode, "CONTINUED" in missing.stdout), (0, True), missing.stderr)++ def test_the_entry_point_runs_the_guard_before_any_flag_is_built(self):+ entry = (DOCKER / "entrypoint.sh").read_text(encoding="utf-8")+ self.assertLess(entry.index('legacy_models_guard "$model_path"'), entry.index('--model "$MODEL"'))++class Document(unittest.TestCase):def test_every_documented_setting_is_read_by_the_entry_point(self):doc = (ROOT / "docs" / "DOCKER.md").read_text(encoding="utf-8")