Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

setup: the two official IST-DAS DeepSeek-V4.1-Flash sets (2.00 / 2.25 bpw) as sizes: pinned files, shared Engram + DSpark, plain pack recipe, shipped profiles, pack-info and a 16-token text check, --dspark auto; fit words from the measured numbers

Niko122181dd215de936
diff --git a/data/dsv41/profile-2.00.bin b/data/dsv41/profile-2.00.bin
new file mode 100644
index 000000000..99f186927
Binary files /dev/null and b/data/dsv41/profile-2.00.bin differ
diff --git a/data/dsv41/profile-2.00.bin.sha256 b/data/dsv41/profile-2.00.bin.sha256
new file mode 100644
index 000000000..8e68c32dc
--- /dev/null
+++ b/data/dsv41/profile-2.00.bin.sha256
@@ -0,0 +1 @@
+8db93e483b34b15dbf63a0826601f1db2107a8cfea73774e7d21d8559e0a9568 *profile-2.00.bin
diff --git a/data/dsv41/profile-2.25.bin b/data/dsv41/profile-2.25.bin
new file mode 100644
index 000000000..5270c6ce4
Binary files /dev/null and b/data/dsv41/profile-2.25.bin differ
diff --git a/data/dsv41/profile-2.25.bin.sha256 b/data/dsv41/profile-2.25.bin.sha256
new file mode 100644
index 000000000..237f57bba
--- /dev/null
+++ b/data/dsv41/profile-2.25.bin.sha256
@@ -0,0 +1 @@
+c8de062c5f3c71725ecf5328bd4ee02c71dca88478bf7f6b0f2ec258567ac6b6 *profile-2.25.bin
diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.md
index 94cebf441..8a931c51f 100644
--- a/docs/DEEPSEEK41.md
+++ b/docs/DEEPSEEK41.md
@@ -3,9 +3,9 @@
DeepSeek-V4.1-Flash is a large mixture-of-experts model: 40 layers, 384 experts in each (6 are used for every word), and a 203 GB
n-gram table next to them. Strata runs it with its own engine, `strata-dsv41` (`src/dsv41`, ported from Project Maya's
`maya-ds4`, MIT; the notice is in `third_party/project-maya`). The file is Unsloth's UD-Q8_K_XL, pinned in `setup.py`: nine
-shards, 508.4 GB, the experts as MXFP4 and the rest in BF16.
+shards, 508.4 GB, the experts as MXFP4 and the rest in BF16. The two official 2-bit sets (below) are what a normal PC runs.
-This is for a computer with several graphics cards and a lot of RAM. It was run on four Radeon AI PRO R9700 (32 GB each) with
+The 508 GB set is for a computer with several graphics cards and a lot of RAM. It was run on four Radeon AI PRO R9700 (32 GB each) with
251 GB of RAM. Strata does not refuse a smaller computer; it tells you what to expect first.
## Where it goes
@@ -24,13 +24,33 @@ With less memory than that, part of the CPU's experts are read from the SSD whil
that cannot hold its share of the dense weights (about 4 GB each on four cards, 16 GB on one) can make the engine stop at the start
with an out-of-memory message. Both are said before anything is downloaded.
+## The official 2-bit versions (the ones for a normal PC)
+
+Setup offers two official sets (helcig/tmp-oct26-2, pinned): **IST-DAS 2.00 bpw** (138 GB) and **IST-DAS 2.25 bpw** (155 GB). Both use the
+same 111 GB n-gram table (downloaded once into `models/deepseek-v4.1-flash-shared`) and the 8 GB DSpark drafter: 257 GB of disk for
+2.00, 274 GB for 2.25, 412 GB for both. Quality against the original model: perplexity +16% on 2.00, +8% on 2.25 (code and agent work
+about +3.5% / +2.5% on 2.25).
+
+Measured: an RTX 5070 (12 GB) with 64 GB of RAM and a 3.6 GB/s NVMe drive answers at about 4.7 tokens/s and reads prompts at 121
+tokens/s (4K) to 231 tokens/s (8K, layer-major path); an RTX 3060 (12 GB) with 46 GB of RAM about 7.2 tokens/s; four R9700 about 31
+tokens/s. Under about 40 GB of RAM it is untested and slow; setup says so and still installs.
+
+Setup downloads with resume and SHA-256, packs the model with the plain recipe (`pack_gguf.py --model SET --shards ... --engram
+engram-IQ4_NL.gguf --engram-meta <UD shard 1> --dspark-gguf dspark-...-Q8_0.gguf`; do not add `--skip-tensors` or `--dense-override`:
+that packs without an error and gives garbage text), copies the shipped hot-expert profile (`data/dsv41/profile-2.00.bin` or `-2.25.bin`)
+into the pack as `expert-profile.bin`, checks `strata-dsv41 --pack-info` (every file present) and answers a 16-token test question
+(`tools/dsv41/smoke.py`, the text must name Paris). The start script passes `--dspark auto` (never `off`: `off` still loads the drafter
+and its 7 GB of experts take the graphics memory); the RAM store sizes itself.
+
## Install
```
-./setup.sh --family deepseek41 # downloads 508 GB (resumable, checked against pinned sizes and SHA-256)
-./setup.sh --family deepseek41 --gguf-dir DIR # files you already have: the nine shards, or the repository folder with UD-Q8_K_XL inside
-strata launch --model ud-q8_k_xl --plan # where it would go on this computer, nothing changed
-strata launch --model-path DIR --model ud-q8_k_xl # start from files you already have
+./setup.sh --family deepseek41 --model IST-DAS-2.00bpw # 257 GB: the official 2.00 set + Engram + drafter (resumable, SHA-256 checked)
+./setup.sh --family deepseek41 --model IST-DAS-2.25bpw # 274 GB (the Engram is shared with 2.00)
+./setup.sh --family deepseek41 --model UD-Q8_K_XL # Unsloth's 508 GB set, for hundreds of GB of RAM or several cards
+./setup.sh --family deepseek41 --gguf-dir DIR # files you already have: the shards (or the folder with 2.00bpw / 2.25bpw / UD-Q8_K_XL inside); the Engram and drafter are looked for beside them
+strata launch --model ist-das-2.00bpw --plan # where it would go on this computer, nothing changed
+strata launch --model-path DIR --model ist-das-2.00bpw # start from files you already have
```
Setup needs `strata-dsv41` next to the engine (or `STRATA_DSV41_EXE`) and says so before it downloads anything. Build it with
diff --git a/setup.py b/setup.py
index 21fe40512..678b5c676 100644
--- a/setup.py
+++ b/setup.py
@@ -96,6 +96,7 @@ HF_REVISIONS = {
"unsloth/Qwen3.6-35B-A3B-MTP-GGUF": "5bc3e238d916f48a861bac2f8a1990a0e9b7e98d", # 2026-10-12
"agentionai/Qwen3.8-Flash-Next-Gyro-GGUF": "1ada08422e404aca51ab9df665f089f34c419c6b", # 2026-10-13
"unsloth/DeepSeek-V4.1-Flash-1017-GGUF": "f924b94f19c1fea02c06a65c8475b0a45e2b88b1", # 2026-10-20 (the port's reference)
+ "helcig/tmp-oct26-2": "851709dcf0d97f091a4bdce758b2e4b0b468dd26", # 2026-10-23 (the official IST-DAS DeepSeek sets)
}
@@ -674,17 +675,69 @@ DSV41_FILES = {
"dspark-DeepSeek-V4.1-Flash-1017-Q8_0.gguf": (7976914240, "6ba6ac1b331c540691a209e7fc97dec2a24e295acb1e755c84c7d24a4373927a"),
"mmproj-BF16.gguf": (972208288, "9a1ef7c40d0d702c7ed8c5f863f60abae64091387ba3fce0ad19ce9beeda3bf9"),
}
+# The two OFFICIAL IST-DAS sets (helcig/tmp-oct26-2 at the revision above; Hub LFS sha256 = what the files hash to): 4 shards each, plus ONE
+# Engram table (110.6 GB, the same file for both sets: downloaded once into the shared folder) and the DSpark drafter and the 5.8 MB
+# metadata shard of Unsloth's set that pack_gguf --engram-meta reads (both pinned in DSV41_FILES above). This is the set a normal PC runs
+# (64 GB RAM + a 12 GB card; docs/DEEPSEEK41.md); the UD-Q8_K_XL entry stays for the big machines.
+DSV41_ISTA_REPO = "helcig/tmp-oct26-2"
+DSV41_ISTA_ENGRAM = "engram-IQ4_NL.gguf"
+DSV41_ISTA_SHARD = "model-{bpw}-{i:05d}-of-00004.gguf"
+DSV41_DSPARK = "dspark-DeepSeek-V4.1-Flash-1017-Q8_0.gguf"
+DSV41_ISTA_FILES = {
+ "model-2.00bpw-00001-of-00004.gguf": (39075713728, "78ab9efc46a26647c3e94b322c99cd7e2af261dd28b38f2712e128562db1ef47"),
+ "model-2.00bpw-00002-of-00004.gguf": (39388449504, "cec1b2c3da429dc768b8bb9bc4dc4403d81f74d34db660a6a34ee3e555bf2c70"),
+ "model-2.00bpw-00003-of-00004.gguf": (39246891616, "9d9c2140fcb134a0b9a74dedd0e5167e68077af6d08ce7f856bce112f0a35dd3"),
+ "model-2.00bpw-00004-of-00004.gguf": (20527837312, "9b790a2297ed3cdede379fc984f4a52bb95f3b23724c797af11d1fa6a69db059"),
+ "model-2.25bpw-00001-of-00004.gguf": (39535776064, "ddd13aca16faa23b08f5196b5e0daddf90f04688a271f5889eb22fa7d7d65526"),
+ "model-2.25bpw-00002-of-00004.gguf": (39494617536, "933461a974fcccee741bf2f9798201e72ab4aa3f8a952be56f849896ca6ea30e"),
+ "model-2.25bpw-00003-of-00004.gguf": (39423838496, "bbf91fa346a85ebaf32dd4fd5c456b534d780d381e54f46ac90783a481373240"),
+ "model-2.25bpw-00004-of-00004.gguf": (36771591232, "e5b2f8781fab688cd4bc3bc7a69651b5dbb90ed94965657c6bead8bb4774f3cf"),
+ "engram-IQ4_NL.gguf": (110595290720, "1f17d8ae37e9df3bb45f7ef1b4fcd1706d7b759af0413c117bf8e92aed459281"),
+}
+# the files every official set shares (name -> where it comes from); they live in models/deepseek-v4.1-flash-shared, once
+DSV41_SHARED = {
+ DSV41_ISTA_ENGRAM: hf(DSV41_ISTA_REPO) + "2.00bpw/",
+ DSV41_SHARD.format(i=1): hf(DSV41_REPO) + "UD-Q8_K_XL/", # only its metadata is read (pack_gguf --engram-meta)
+ DSV41_DSPARK: hf(DSV41_REPO),
+}
+DSV41_ALL_FILES = {**DSV41_FILES, **DSV41_ISTA_FILES}
+DSV41_PROFILES = ROOT / "data" / "dsv41" # the shipped hot-expert profiles (profile-2.00.bin, profile-2.25.bin, + .sha256)
DSV41_CONTEXT = 8192 # the context the engine was checked at (4096 and 8192; 64K is not tried yet)
MULTI_FILE_FAMILIES = {
"deepseek41": {"title": "DeepSeek-V4.1-Flash", "by": "DeepSeek; GGUFs by Unsloth", "arch": "deepseek41",
- "about": "40 layers of 384 experts (6 used for each word) and a 203 GB n-gram table: a 508 GB download that "
- "runs on several graphics cards with the experts they cannot hold computed on the CPU",
+ "about": "a big model that runs on a PC with a 12 GB card and 64 GB of RAM (the two official 2-bit versions: 257 or "
+ "274 GB of disk, about 5-7 words a second) or, in Unsloth's 508 GB version, on several graphics cards",
"hf": hf(DSV41_REPO) + "UD-Q8_K_XL/", "tag": "deepseek-v4.1-flash-", "name": "deepseek-v4.1-flash",
"license": "DeepSeek's model license: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF",
"models": {"UD-Q8_K_XL": {"about": "Unsloth Dynamic 8-bit: the experts in MXFP4 as DeepSeek made them, the rest "
"in BF16 (the checkpoint's own precision)",
"ram_gb": 306, "shards": [DSV41_SHARD.format(i=i) for i in range(1, 10)]}},
- "sha256": DSV41_FILES, "vision": False}
+ "sha256": DSV41_ALL_FILES, "vision": False}
+}
+# The official sets first (a normal PC), Unsloth's 508 GB set last. Each model: its shards, the sub-folder they are in at the repository
+# (and in a --gguf-dir folder), the files shared with the other set ("shared"), the pack recipe and the shipped expert profile.
+MULTI_FILE_FAMILIES["deepseek41"]["models"] = {
+ "IST-DAS-2.00bpw": {
+ "label": "IST-DAS 2.00 bpw",
+ "about": "the smaller official version: 138 GB of model + 111 GB n-gram table + 8 GB drafter (257 GB of disk). Runs on a PC with "
+ "a 12 GB card and 40-64 GB of RAM at about 5-7 words a second; quality about 16% above the original model's "
+ "perplexity",
+ "ram_gb": 40, "recipe": "ista", "subdir": "2.00bpw", "profile": "profile-2.00.bin", "hf": hf(DSV41_ISTA_REPO) + "2.00bpw/",
+ "shards": [DSV41_ISTA_SHARD.format(bpw="2.00bpw", i=i) for i in range(1, 5)],
+ "shared": [DSV41_ISTA_ENGRAM, DSV41_SHARD.format(i=1), DSV41_DSPARK]},
+ "IST-DAS-2.25bpw": {
+ "label": "IST-DAS 2.25 bpw",
+ "about": "the better official version: 155 GB of model + 111 GB n-gram table + 8 GB drafter (274 GB of disk, 17 GB more than "
+ "2.00). About the same speed; quality about 8% above the original model's perplexity (code and agent work 2.5-3.5%)",
+ "ram_gb": 44, "recipe": "ista", "subdir": "2.25bpw", "profile": "profile-2.25.bin", "hf": hf(DSV41_ISTA_REPO) + "2.25bpw/",
+ "shards": [DSV41_ISTA_SHARD.format(bpw="2.25bpw", i=i) for i in range(1, 5)],
+ "shared": [DSV41_ISTA_ENGRAM, DSV41_SHARD.format(i=1), DSV41_DSPARK]},
+ "UD-Q8_K_XL": {
+ "label": "Unsloth UD-Q8_K_XL",
+ "about": "Unsloth Dynamic 8-bit: the experts in MXFP4 as DeepSeek made them, the rest in BF16 (the checkpoint's own precision); "
+ "508 GB, for machines with hundreds of GB of RAM or several cards",
+ "ram_gb": 306, "recipe": "ud", "subdir": "UD-Q8_K_XL", "shards": [DSV41_SHARD.format(i=i) for i in range(1, 10)],
+ "shared": []},
}
# Every model file Strata downloads has a fixed size and SHA-256: (repository, file name) -> (bytes, sha256). download()
@@ -698,6 +751,7 @@ REPO_PINS = {
"unsloth/Qwen3.6-35B-A3B-MTP-GGUF": QWEN36_FILES,
"agentionai/Qwen3.8-Flash-Next-Gyro-GGUF": GYRO_FILES,
"unsloth/DeepSeek-V4.1-Flash-1017-GGUF": DSV41_FILES,
+ "helcig/tmp-oct26-2": DSV41_ISTA_FILES,
}
# EXPERIMENTAL, off by default (setup asks): a control vector shipped with the repository, see its README
ESP_VECTOR = ROOT / "data" / "experimental-speed-projection" / "Qwen3.8-Flash-Next-experimental-speed-projection.gguf"
@@ -788,7 +842,7 @@ def run(cmd, cwd=None, env=None, check=True, quiet=False):
if CONTROL: # the app: only this Python running a bundled tool
argv = [str(c) for c in cmd]
if len(argv) < 2 or os.path.normcase(argv[0]) != os.path.normcase(sys.executable) or argv[1] == "-m" or \
- Path(argv[1]).resolve().parent != (ROOT / "tools").resolve():
+ Path(argv[1]).resolve().parent not in ((ROOT / "tools").resolve(), (ROOT / "tools" / "dsv41").resolve()):
raise acquisition.AcquisitionRefused("the desktop app's installer only runs its bundled preparation tools; "
+ " ".join(argv[:2]))
say(" > " + " ".join(str(c) for c in cmd))
@@ -7237,15 +7291,92 @@ def dsv41_exe(eng: Path) -> Path | None:
return exe if exe.is_file() else None
-def dsv41_folder(a, fam: dict, tag: str) -> Path:
- """Where the nine shards are: --gguf-dir (a folder with the shards, or the repository's folder with UD-Q8_K_XL inside), else the
- models folder's own."""
+def dsv41_folder(a, fam: dict, tag: str, model: str = "UD-Q8_K_XL") -> Path:
+ """Where the model's shards are: --gguf-dir (a folder with the shards, or the repository's folder with the set's own folder inside:
+ UD-Q8_K_XL, 2.00bpw, 2.25bpw), else the models folder's own."""
if a.gguf_dir:
folder = Path(a.gguf_dir)
- return folder / "UD-Q8_K_XL" if (folder / "UD-Q8_K_XL").is_dir() and not (folder / DSV41_SHARD.format(i=1)).exists() else folder
+ d = fam["models"][model]
+ sub = d.get("subdir") or model
+ return folder / sub if (folder / sub).is_dir() and not (folder / d["shards"][0]).exists() else folder
return Path(a.models_dir) / tag
+def dsv41_shared_dir(a, fam: dict) -> Path:
+ """Where the files the official sets share (the Engram, the DSpark drafter, the metadata shard) are downloaded to: one folder, so
+ installing the second set does not download the 111 GB Engram again."""
+ return Path(a.models_dir) / (fam["tag"] + "shared")
+
+
+def dsv41_find_shared(a, name: str, folder: Path, shared: Path) -> Path:
+ """The file `name` (an Engram, the drafter, ...) the person already has: in the shared folder, beside the shards, or (--gguf-dir) in the
+ folder above them or one folder level down from either; else where it will be downloaded to (the shared folder). The size must be the
+ pinned one, so a half-copied file is not taken for the file."""
+ size = DSV41_ALL_FILES[name][0]
+ roots = [shared, folder]
+ if a.gguf_dir:
+ base = Path(a.gguf_dir)
+ roots += [base, base.parent]
+ seen = []
+ for r in roots:
+ if r in seen:
+ continue
+ seen.append(r)
+ try:
+ kids = [r] + ([k for k in r.iterdir() if k.is_dir()] if (a.gguf_dir and r != shared) else [])
+ except OSError:
+ continue
+ for k in kids:
+ f = k / name
+ try:
+ if f.is_file() and f.stat().st_size == size:
+ return f
+ except OSError:
+ pass
+ return shared / name
+
+
+def dsv41_pack_cmd(folder: Path, shards: list, engram: Path, meta: Path, dspark: Path, pack: Path) -> list:
+ """The pack recipe for an official IST-DAS set - this and nothing else: another one (--skip-tensors / --dense-override + engram_fold)
+ packs without an error and gives garbage text (P2003, 2026-10-23). --shards names the four files, so an Engram or another
+ set in the same folder is not read as part of the model."""
+ return [sys.executable, str(ROOT / "tools" / "dsv41" / "pack_gguf.py"), "--model", str(folder), "--shards", *[Path(n).name for n in shards],
+ "--engram", str(engram), "--engram-meta", str(meta), "--dspark-gguf", str(dspark), "--out", str(pack)]
+
+
+def dsv41_engine_env(lib_dirs) -> dict:
+ env = dict(os.environ)
+ dirs = [str(d) for d in lib_dirs or [] if d]
+ if dirs:
+ env["PATH" if WIN else "LD_LIBRARY_PATH"] = os.pathsep.join(dirs + [env.get("PATH" if WIN else "LD_LIBRARY_PATH", "")])
+ return env
+
+
+def dsv41_pack_info(exe: Path, pack: Path, lib_dirs) -> tuple[bool, str]:
+ """`strata-dsv41 --pack-info --pack P`: (True, text) when every file of the pack is there (no MISSING line), else (False, text)."""
+ try:
+ r = subprocess.run([str(exe), "--pack-info", "--pack", str(pack)], capture_output=True, text=True, timeout=300,
+ env=dsv41_engine_env(lib_dirs))
+ except (OSError, subprocess.TimeoutExpired) as e:
+ return False, f"the engine could not read the pack: {e}"
+ text = (r.stdout or "") + (r.stderr or "")
+ return r.returncode == 0 and "MISSING" not in text and "all present" in text, text
+
+
+def dsv41_smoke(exe: Path, pack: Path, lib_dirs, gpus=None, timeout: int = 5400) -> tuple[bool, str]:
+ """16 tokens of a fixed prompt through the engine (tools/dsv41/smoke.py), the text checked: (True, text) or (False, why). Takes a
+ few minutes: the engine reads what it needs from the SSD the first time."""
+ cmd = [sys.executable, str(ROOT / "tools" / "dsv41" / "smoke.py"), "--exe", str(exe), "--pack", str(pack)]
+ if gpus:
+ cmd += ["--gpus", ",".join(str(g) for g in gpus)]
+ try:
+ r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, env=dsv41_engine_env(lib_dirs))
+ except (OSError, subprocess.TimeoutExpired) as e:
+ return False, f"the test run did not finish: {e}"
+ text = ((r.stdout or "") + (r.stderr or "")).strip()
+ return r.returncode == 0, text[-1500:]
+
+
def dsv41_cards(gpu: dict, all_cards: list | None, asked: bool) -> list:
"""The cards the model runs on: the ones asked for (--gpu / --gpus), else every card setup can use (recommend: the model is
larger than any one card, so the more graphics memory the more of its experts stay off the CPU)."""
@@ -7269,22 +7400,33 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,
names = list(fam["models"])
if a.model and a.model not in names:
fail(f"{fam['title']} has no {a.model} model file", "choose: " + ", ".join(names))
- model = a.model or names[0]
+ if a.model:
+ model = a.model
+ else:
+ for i, m in enumerate(names, 1):
+ say(f" {i}) {fam['models'][m].get('label', m)}: {fam['models'][m]['about']}" + (" (recommended)" if i == 1 else ""))
+ model = names[int(ask("Which version?", [str(i) for i in range(1, len(names) + 1)], "1", a.yes)) - 1]
ok(f"size: {model}")
+ md = fam["models"][model]
+ ista = md.get("recipe") == "ista"
tag = fam["tag"] + model
cards = dsv41_cards(gpu, all_cards, bool(a.gpu is not None or a.gpus))
ctx = a.context or DSV41_CONTEXT
- folder = dsv41_folder(a, fam, tag)
- shards = [folder / n for n in fam["models"][model]["shards"]]
+ folder = dsv41_folder(a, fam, tag, model)
+ shards = [folder / n for n in md["shards"]]
+ shared_dir = dsv41_shared_dir(a, fam)
+ shared = [dsv41_find_shared(a, n, folder, shared_dir) for n in md.get("shared", [])]
+ own_files = {f: bool(a.gguf_dir) or f.parent != shared_dir for f in shared} # a file found outside setup's own folder is the person's
def partial(f: Path) -> int:
p = f.with_name(f.name + ".part")
return p.stat().st_size if p.is_file() else 0
- left = sum(0 if f.exists() and (done(f) or a.gguf_dir) else fam["sha256"][f.name][0] - partial(f) for f in shards) / 1e9
+ left = sum(0 if f.exists() and (done(f) or a.gguf_dir or own_files.get(f)) else fam["sha256"][f.name][0] - partial(f)
+ for f in [*shards, *shared]) / 1e9
probe = folder if folder.exists() else (folder.parent if folder.parent.exists() else Path(a.models_dir))
free = free_gb(probe)
- plan = dsv41_fit.plan([{**c, "vram_gb": float(c.get("vram_gb") or 0)} for c in cards], ram, free if left else None, ctx)
+ plan = dsv41_fit.plan([{**c, "vram_gb": float(c.get("vram_gb") or 0)} for c in cards], ram, free if left else None, ctx, model=model)
say()
for line in plan["lines"]:
say(" " + line)
@@ -7296,8 +7438,8 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,
continue # the disk is a hard limit, said below
warn(w)
if plan["verdict"] == "spills":
- confirm_risk(f"{model} needs about {plan['ram_needed_gb']:.0f} GB of RAM beside this PC's graphics memory and this PC has "
- f"{ram:.0f} GB: part of its experts are read from the SSD while it answers (slow)", bool(a.model), a.yes,
+ confirm_risk(f"{model} runs best with about {plan['ram_needed_gb']:.0f} GB of RAM beside this PC's graphics memory and this PC has "
+ f"{ram:.0f} GB: most of its experts are read from the SSD while it answers (slow)", bool(a.model), a.yes,
f"{model} needs about {plan['ram_needed_gb']:.0f} GB of RAM", f"more RAM or cards fix it; --model {model} --yes "
"installs it anyway", " Install it anyway?")
warn(f"installing {model} with {ram:.0f} GB of RAM, as you chose")
@@ -7335,19 +7477,60 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,
if a.gguf_dir and not f.exists():
fail(f"{f} is missing", f"put {f.name} in {folder}, or drop --gguf-dir to download it")
if not a.gguf_dir:
- download(fam["hf"] + f.name, f)
+ download((md.get("hf") or fam["hf"]) + f.name, f)
verify_sha256(f, *fam["sha256"][f.name], own=bool(a.gguf_dir))
- ok(f"model files: {folder} ({len(shards)} shards)")
+ for f in shared: # the Engram, the drafter, the metadata shard: once for every official set
+ if f.exists() and (done(f) or own_files[f]):
+ ok(f"{f.name} already here ({f.parent})")
+ else:
+ download(DSV41_SHARED[f.name] + f.name, f)
+ verify_sha256(f, *fam["sha256"][f.name], own=own_files[f])
+ ok(f"model files: {folder} ({len(shards)} shards" + (f"; shared files: {shared_dir if any(not own_files[f] for f in shared) else shared[0].parent}" if shared else "") + ")")
step(6, "preparing the model for Strata")
data, _ = data_folder_for(a)
pack = data / "packs" / tag.lower() # the engine's map of the shards, the hash tables, the tokenizer: a few MB
- if not (pack / "tokenizer" / "vocab.json").exists() or not (pack / "tensors.txt").exists():
- run([sys.executable, str(ROOT / "tools" / "dsv41" / "pack_gguf.py"), "--model", str(folder), "--out", str(pack)])
- ok(f"pack: {pack}")
+ have_pack = (pack / "tokenizer" / "vocab.json").exists() and (pack / "tensors.txt").exists()
+ if have_pack and ista: # a pack from an earlier run is kept only when the engine finds all of its files
+ good, why = dsv41_pack_info(exe, pack, lib_dirs)
+ if not good:
+ warn("the pack from an earlier run is incomplete; building it again")
+ shutil.rmtree(pack, ignore_errors=True)
+ have_pack = False
+ if not have_pack:
+ if ista:
+ run(dsv41_pack_cmd(folder, md["shards"], shared[0], shared[1], shared[2], pack))
+ else:
+ run([sys.executable, str(ROOT / "tools" / "dsv41" / "pack_gguf.py"), "--model", str(folder), "--out", str(pack)])
+ if ista:
+ prof = DSV41_PROFILES / md["profile"] # which experts are used most: the RAM store starts with them (the engine reads <pack>/expert-profile.bin)
+ if prof.is_file() and not (pack / "expert-profile.bin").exists():
+ pack.mkdir(parents=True, exist_ok=True)
+ shutil.copyfile(prof, pack / "expert-profile.bin")
+ good, why = dsv41_pack_info(exe, pack, lib_dirs)
+ if not good:
+ tail = "\n".join(l for l in why.splitlines() if "MISSING" in l)[:600]
+ fail(f"the model's pack is not complete ({tail or why[-300:]})",
+ f"delete {pack} and run setup again; if it is still incomplete, the files in {folder} may be damaged (setup checks them)")
+ ok(f"pack: {pack} (checked: every file the engine needs is there)")
+ if os.environ.get("STRATA_DSV41_NO_SMOKE") == "1":
+ warn("the 16-word test run was skipped (STRATA_DSV41_NO_SMOKE=1)")
+ else:
+ say(" Testing the model with a short question (a few minutes: the engine reads the first experts from the disk) ...")
+ good, why = dsv41_smoke(exe, pack, lib_dirs, [c["index"] for c in cards] if (a.gpu is not None or a.gpus) else None)
+ if not good:
+ fail("the model does not answer sensibly after the preparation: " + (why.splitlines()[-1] if why else "no answer"),
+ f"delete {pack} and run setup again so the pack is built fresh; if the test fails again, "
+ f"tell us (the log is in {home()})")
+ ok("test question answered sensibly")
+ else:
+ ok(f"pack: {pack}")
step(7, "writing the start script")
- args = ["--pack", str(pack), "--max-context", str(ctx), "--eos", "1", *plan["args"]]
+ # --dspark auto, never off: "off" still LOADS the drafter, and its 7 GB of experts take the graphics memory (found 2026-10-23);
+ # auto keeps it unloaded when the experts are far larger than the card. The RAM store sizes itself (--ssd-cache auto is the
+ # engine's default), and <pack>/expert-profile.bin is read from the pack folder without a flag.
+ args = ["--pack", str(pack), "--max-context", str(ctx), "--eos", "1", "--dspark", "auto", *plan["args"]]
cfg = {"exe": str(exe), "args": args, "cwd": str(ROOT), "tokenizer": str(pack / "tokenizer"),
"model_name": fam["name"], "arch": fam["arch"], "log": str(home() / f"strata-{tag.lower()}.log"),
"lib_dirs": lib_dirs, "port": port, "engine_ready_s": 3600} # 300+ GB read from the files: a slow disk needs its time
@@ -7628,7 +7811,7 @@ def main() -> int:
ap.add_argument("--family", choices=list(FAMILIES) + list(PREVIEW_FAMILIES) + list(MULTI_FILE_FAMILIES),
help="qwen = Qwen3.8-Flash-Next, swift = Swift 1.5, qwen36 = Qwen3.6-35B-A3B (pictures with --vision, "
"several requests at once with --parallel; docs/QWEN36.md), deepseek41 = DeepSeek-V4.1-Flash "
- "(a 508 GB download for several graphics cards; docs/DEEPSEEK41.md)")
+ "(the official 2.00 / 2.25 bpw versions, 257 / 274 GB; docs/DEEPSEEK41.md)")
ap.add_argument("--model", choices=list(dict.fromkeys([*MODELS, *(m for f in PREVIEW_FAMILIES.values()
for m in f["models"]),
*(m for f in MULTI_FILE_FAMILIES.values() for m in f["models"])])))
@@ -8065,13 +8248,16 @@ def main() -> int:
say(f" {m:8s} needs ~{d['ram_gb']} GB {'of VRAM + RAM' if d.get('mem_total') else 'RAM'}: {verdict}")
try: # DeepSeek-V4.1-Flash: a line, from the real numbers (strata_control/dsv41_fit.py)
from strata_control import dsv41_fit
- ds = dsv41_fit.plan(amd_ok if hip else [g for g in found if gpu_problem(g) is None], ram, None)
- tail = {"fits": f"fits ({ds['experts_in_vram']:,} of {ds['experts_total']:,} experts in graphics memory, the rest on the CPU)",
- "tight": "fits, but RAM is tight",
- "spills": f"about {ds['spill_gb']:.0f} GB of its experts would be read from the SSD (slow)",
- "no_card": "needs one or more graphics cards"}.get(ds["verdict"], "not checked")
- say(f" {'DeepSeek-V4.1-Flash':8s} needs ~{ds['weights_gb']:.0f} GB of VRAM + RAM (a {ds['download_gb']:.0f} GB download): {tail}"
- " (--family deepseek41)")
+ cards_ok = amd_ok if hip else [g for g in found if gpu_problem(g) is None]
+ for mname in ("IST-DAS-2.00bpw", "IST-DAS-2.25bpw", "UD-Q8_K_XL"):
+ ds = dsv41_fit.plan(cards_ok, ram, None, model=mname)
+ tail = {"fits": f"fits ({ds['experts_in_vram']:,} of {ds['experts_total']:,} experts in graphics memory, the rest on the CPU)",
+ "tight": ("runs, with the experts RAM cannot hold read from the SSD (about 5-7 words a second measured)"
+ if ds.get("variant") else "fits, but RAM is tight"),
+ "spills": (f"works but slow: less RAM than it was measured with (46-64 GB)" if ds.get("variant")
+ else f"about {ds['spill_gb']:.0f} GB of its experts would be read from the SSD (slow)"),
+ "no_card": "needs one or more graphics cards"}.get(ds["verdict"], "not checked")
+ say(f" DeepSeek-V4.1-Flash {mname}: a {ds['download_gb']:.0f} GB download: {tail} (--family deepseek41 --model {mname})")
except Exception: # noqa: BLE001 - a line for information: never break the check
pass
if not any_fits:
diff --git a/strata_cli/launch/catalog.py b/strata_cli/launch/catalog.py
index 69fef59ba..fdb0dc644 100644
--- a/strata_cli/launch/catalog.py
+++ b/strata_cli/launch/catalog.py
@@ -320,7 +320,7 @@ class Catalog:
for size in fam["sizes"]:
f = fit_on(size, hw.get("ram_gb"), vram)
if size.get("multi_file"): # DeepSeek-V4.1-Flash: shared over every card (strata_control/dsv41_fit.py)
- dsv41 = dsv41 or self.strata.dsv41_plan(hw)
+ dsv41 = self.strata.dsv41_plan(hw, model=size["model"])
f = {"vramUseGb": _round1(dsv41["vram_experts_gb"] + sum(c["dense_gb"] for c in dsv41["cards"])),
"ramUseGb": _round1(dsv41.get("ram_needed_gb") or dsv41["cpu_experts_gb"]), "spillGb": _round1(dsv41.get("spill_gb") or 0.0)}
cat, words = verdict(size.get("on_this_pc"))
diff --git a/strata_cli/launch/runner.py b/strata_cli/launch/runner.py
index 117f867b8..ba1f5610b 100644
--- a/strata_cli/launch/runner.py
+++ b/strata_cli/launch/runner.py
@@ -809,35 +809,44 @@ class Runner:
return path
def deepseek_files(self) -> bool:
- """--model-path names DeepSeek-V4.1-Flash's shards (a folder, the repository's folder with UD-Q8_K_XL inside, or one shard): they are
- that model, whatever the fingerprint table of the Flash-Next files says. True when it did (s.family / s.model / s.model_path set); a
- shard missing from the folder is said now, in plain words, before an engine or a download is touched."""
+ """--model-path names DeepSeek-V4.1-Flash's shards (a folder, the repository's folder with the set's own folder inside - UD-Q8_K_XL,
+ 2.00bpw, 2.25bpw - or one shard): they are that model, whatever the fingerprint table of the Flash-Next files says. True when it did
+ (s.family / s.model / s.model_path set); a shard missing from the folder is said now, in plain words, before an engine or a download
+ is touched."""
s = self.s
try:
mod = self.catalog.strata.setup_module()
- shard = getattr(mod, "DSV41_SHARD", None)
- if shard is None or not s.model_path:
+ fam = getattr(mod, "MULTI_FILE_FAMILIES", {}).get("deepseek41")
+ if fam is None or not s.model_path:
return False
p = Path(s.model_path)
- folders = [p, p / "UD-Q8_K_XL"] if p.is_dir() else [p.parent]
- first = shard.format(i=1)
- others = [shard.format(i=i) for i in range(1, 10)]
- folder = next((f for f in folders if any((f / n).exists() for n in others)), None)
+ found = None
+ order = ([s.model] if s.model in fam["models"] else []) + [m for m in fam["models"] if m != s.model]
+ for m in order:
+ d = fam["models"][m]
+ folders = [p, p / (d.get("subdir") or m)] if p.is_dir() else [p.parent]
+ folder = next((f for f in folders if any((f / n).exists() for n in d["shards"])), None)
+ if folder is not None:
+ found = (m, d, folder)
+ break
except Exception: # noqa: BLE001 - not DeepSeek's: the usual check goes on
return False
- if folder is None:
+ if found is None:
return False
+ model, d, folder = found
if s.family and s.family != "deepseek41":
raise LaunchError(f"These are DeepSeek-V4.1-Flash's files, not {s.family} {s.model}'s.",
- "Leave --model out, or use --model UD-Q8_K_XL.", 2)
+ "Leave --model out, or use --model " + model + ".", 2)
+ others = list(d["shards"])
missing = [n for n in others if not (folder / n).exists()]
if missing:
- raise LaunchError(f"DeepSeek-V4.1-Flash is in nine parts and {len(missing)} of them are not in {folder} "
+ parts = {4: "four", 9: "nine"}.get(len(others), str(len(others)))
+ raise LaunchError(f"DeepSeek-V4.1-Flash {d.get('label', model)} is in {parts} parts and {len(missing)} of them are not in {folder} "
f"(first missing: {missing[0]}).",
- "Put every part in the folder (or drop --model-path and let Strata download them: about 508 GB). "
+ "Put every part in the folder (or drop --model-path and let Strata download them). "
"Nothing was downloaded or changed.", 5)
- s.family, s.model, s.model_path = "deepseek41", s.model or "UD-Q8_K_XL", str(folder)
- self.emit("model", "These files are DeepSeek-V4.1-Flash UD-Q8_K_XL; they are used where they are.", None)
+ s.family, s.model, s.model_path = "deepseek41", model, str(folder)
+ self.emit("model", f"These files are DeepSeek-V4.1-Flash {model}; they are used where they are.", None)
return True
def check_own_files(self) -> None:
@@ -1182,7 +1191,7 @@ class Runner:
free = None if (s.model_path or self.model_state().remaining_gb < 1) else self.catalog.disk_free(data)
except Exception: # noqa: BLE001 - a plan that cannot read the disk still shows the rest
free = None
- p = self.catalog.strata.dsv41_plan(hw, free, s.gpus)
+ p = self.catalog.strata.dsv41_plan(hw, free, s.gpus, model=s.model)
warnings = list(p["warnings"])
engine = self.dsv41_engine()
if engine is False:
diff --git a/strata_control/dsv41_fit.py b/strata_control/dsv41_fit.py
index f9dcbaeb0..e8ed199a4 100644
--- a/strata_control/dsv41_fit.py
+++ b/strata_control/dsv41_fit.py
@@ -48,6 +48,82 @@ MEASURED = ("4 x Radeon AI PRO R9700 (32 GB) with 251 GB of RAM: prompts of 3,00
"at 25-31 tokens/s (--tp 2); one answer at a time")
+# The two official IST-DAS sets (helcig/tmp-oct26-2, setup.py DSV41_ISTA_FILES): 4 shards each (the experts as IQ1_M / IQ2 / IQ3 types, the
+# dense weights Q8_0 / Q5_K), plus ONE Engram (110.6 GB, IQ4_NL, shared by both sets, stays on the SSD) and the DSpark drafter (8.0 GB).
+# Measured 2026-10-23 (f-dsv41-rel): quality against the original model (ppl +16% for 2.00, +7.7-8.3% for 2.25; code / agent +3.5% / +2.5%
+# on 2.25); speed on the owner PC (RTX 5070 12 GB, 64 GB RAM, 3.6 GB/s NVMe): answers ~4.7 tokens/s, prompts 121 tokens/s at 4K and
+# 231 tokens/s at 8K; RTX 3060 12 GB with 46 GB of RAM (the engine held to 36 GB): answers ~7.2 tokens/s on 2.00; 4 x Radeon AI PRO
+# R9700: answers ~31 tokens/s, prompts 330-373 tokens/s at 8K.
+ISTA_ENGRAM_BYTES = 110_595_290_720
+ISTA_DSPARK_BYTES = 7_976_914_240
+ISTA_META_BYTES = 5_780_589 # the first shard of Unsloth's UD-Q8_K_XL: only the metadata --engram-meta reads
+ISTA_SETS = {
+ "IST-DAS-2.00bpw": {"title": "IST-DAS 2.00 bpw", "set_bytes": 138_238_892_160, "experts_gb": 132.07, "profile": "profile-2.00.bin",
+ "ppl": "16%", "ram_ok_gb": 40.0, "ram_great_gb": 160.0, "disk_total": 257.0,
+ "quality": "about 16% above the original model's perplexity (the 2-bit price)"},
+ "IST-DAS-2.25bpw": {"title": "IST-DAS 2.25 bpw", "set_bytes": 155_225_823_328, "experts_gb": 149.06, "profile": "profile-2.25.bin",
+ "ppl": "8%", "ram_ok_gb": 44.0, "ram_great_gb": 176.0, "disk_total": 274.0,
+ "quality": "about 8% above the original model's perplexity (code and agent work about 2.5-3.5% above), "
+ "the better of the two, 17 GB more disk"},
+}
+ISTA_DENSE_GB = 6.16 # the dense weights in the shards (attention, router, shared experts, the head; the embedding stays on the CPU)
+ISTA_DENSE_ON_GPU_GB = 4.5 # what a card holds of them with a 12 GB card (measured on the 5070 / 3060: the engine places the rest itself)
+ISTA_MEASURED = ("an RTX 5070 (12 GB) with 64 GB of RAM and a 3.6 GB/s NVMe drive: answers about 4.7 tokens/s, a 4,000-token prompt "
+ "read at 121 tokens/s; an RTX 3060 (12 GB) with 46 GB of RAM: about 7.2 tokens/s; four Radeon AI PRO R9700 cards: "
+ "about 31 tokens/s")
+
+
+def variant(model: str | None) -> dict | None:
+ """The official-set entry for a model name (case-insensitive), None for UD-Q8_K_XL and anything else."""
+ if not model:
+ return None
+ return next((v for k, v in ISTA_SETS.items() if k.lower() == str(model).lower()), None)
+
+
+def variant_key(model: str | None) -> str | None:
+ return next((k for k in ISTA_SETS if k.lower() == str(model or "").lower()), None)
+
+
+def ista_download_bytes(model: str, have_engram: bool = False) -> int:
+ v = variant(model)
+ return v["set_bytes"] + ISTA_DSPARK_BYTES + ISTA_META_BYTES + (0 if have_engram else ISTA_ENGRAM_BYTES)
+
+
+# The two official IST-DAS sets (helcig/tmp-oct26-2, setup.py DSV41_ISTA_FILES): 4 shards each (the experts as IQ1_M / IQ2 / IQ3 types, the
+# dense weights Q8_0 / Q5_K), plus ONE Engram (110.6 GB, IQ4_NL, shared by both sets, stays on the SSD) and the DSpark drafter (8.0 GB).
+# Measured 2026-10-23 (f-dsv41-rel): quality against the original model (ppl +16% for 2.00, +7.7-8.3% for 2.25; code / agent +3.5% / +2.5%
+# on 2.25); speed on the owner PC (RTX 5070 12 GB, 64 GB RAM, 3.6 GB/s NVMe): answers ~4.7 tokens/s, prompts 121 tokens/s at 4K and
+# 231 tokens/s at 8K; RTX 3060 12 GB with 46 GB of RAM (the engine held to 36 GB): answers ~7.2 tokens/s on 2.00; 4 x Radeon AI PRO
+# R9700: answers ~31 tokens/s, prompts 330-373 tokens/s at 8K.
+ISTA_ENGRAM_BYTES = 110_595_290_720
+ISTA_DSPARK_BYTES = 7_976_914_240
+ISTA_META_BYTES = 5_780_589 # the first shard of Unsloth's UD-Q8_K_XL: only the metadata --engram-meta reads
+ISTA_SETS = {
+ "IST-DAS-2.00bpw": {"title": "IST-DAS 2.00 bpw", "set_bytes": 138_238_892_160, "experts_gb": 132.07, "profile": "profile-2.00.bin",
+ "ram_ok_gb": 40.0, "disk_total": 257.0,
+ "quality": "about 16% above the original model's perplexity (the 2-bit price)"},
+ "IST-DAS-2.25bpw": {"title": "IST-DAS 2.25 bpw", "set_bytes": 155_225_823_328, "experts_gb": 149.06, "profile": "profile-2.25.bin",
+ "ram_ok_gb": 44.0, "disk_total": 274.0,
+ "quality": "about 8% above the original model's perplexity (code and agent work about 2.5-3.5% above), "
+ "the better of the two, 17 GB more disk"},
+}
+ISTA_DENSE_GB = 6.16 # the dense weights in the shards (attention, router, shared experts, the head; the embedding stays on the CPU)
+ISTA_DENSE_ON_GPU_GB = 4.5 # what a card holds of them with a 12 GB card (the engine places the rest itself)
+ISTA_MEASURED = ("an RTX 5070 (12 GB) with 64 GB of RAM and a 3.6 GB/s NVMe drive: answers about 4.7 tokens/s, a 4,000-token prompt "
+ "read at 121 tokens/s; an RTX 3060 (12 GB) with 46 GB of RAM: about 7.2 tokens/s; four Radeon AI PRO R9700 cards: "
+ "about 31 tokens/s")
+
+
+def variant(model: str | None) -> dict | None:
+ """The official-set entry for a model name (case-insensitive), None for UD-Q8_K_XL and anything else."""
+ return ISTA_SETS.get(variant_key(model)) if variant_key(model) else None
+
+
+def ista_download_bytes(model: str, have_engram: bool = False) -> int:
+ v = variant(model)
+ return v["set_bytes"] + ISTA_DSPARK_BYTES + ISTA_META_BYTES + (0 if have_engram else ISTA_ENGRAM_BYTES)
+
+
def _round1(x: float) -> float:
return round(float(x) * 10) / 10
@@ -87,14 +163,104 @@ def tp_for(n_cards: int) -> int:
return 2 if n_cards >= 2 else 1
+def _plan_ista(model: str, cards: list[dict] | None, ram_gb: float | None, disk_free_gb: float | None, context: int,
+ vendor: str | None) -> dict[str, Any]:
+ """plan() for an official IST-DAS set (ISTA_SETS): the same keys, the measured speeds in plain words. RAM classes: with RAM for every
+ expert the CPU lane keeps ("fits"), with 40 GB or more the engine's RAM store keeps the hot experts and reads the rest from the SSD
+ ("tight": measured 4.7 tokens/s on 64 GB, 7.2 on 46 GB), under 40 GB it is untested and slower ("spills"). Never a refusal."""
+ v = variant(model)
+ keep, ignored = gpu_fit.usable_cards(gpu_fit.for_vendor(list(cards or []), vendor)[0]) if vendor else gpu_fit.usable_cards(list(cards or []))
+ keep = [c for c in keep if float(c.get("vram_gb") or 0) >= SMALLEST_CARD_GB]
+ keep.sort(key=lambda c: c.get("index", 0))
+ n = len(keep)
+ expert_gb = v["experts_gb"] / EXPERTS
+ set_gb = v["set_bytes"] / GB
+ download = ista_download_bytes(model) / GB
+ out: dict[str, Any] = {"variant": model, "cards": [], "experts_total": EXPERTS, "experts_in_vram": 0, "vram_experts_gb": 0.0,
+ "cpu_experts_gb": v["experts_gb"], "weights_gb": _round1(set_gb), "experts_gb": _round1(v["experts_gb"]),
+ "dense_gb": _round1(ISTA_DENSE_GB), "engram_gb": _round1(ISTA_ENGRAM_BYTES / GB), "download_gb": _round1(download),
+ "tp": tp_for(n), "args": [], "measured": ISTA_MEASURED, "lines": [], "warnings": [], "ram_gb": ram_gb,
+ "context": context, "disk_total_gb": v["disk_total"]}
+ need_disk = download + DISK_HEADROOM_GB
+ out["disk_need_gb"] = _round1(need_disk)
+ out["disk_ok"] = None if disk_free_gb is None else disk_free_gb >= need_disk
+ lines, warns = out["lines"], out["warnings"]
+ lines.append(f"DeepSeek-V4.1-Flash, {v['title']}: a {download:.0f} GB download ({set_gb:.0f} GB of model, {ISTA_ENGRAM_BYTES / GB:.0f} GB "
+ f"n-gram table, {ISTA_DSPARK_BYTES / GB:.0f} GB drafter). Quality: {v['quality']}.")
+ lines.append("The n-gram table stays on the SSD and is read a few rows for each word: use an NVMe drive for the model files (a slow "
+ "disk makes every word slower). The same table serves both IST-DAS sets, so installing the second one downloads it once.")
+ if n == 0:
+ out["verdict"] = "no_card"
+ warns.append("No graphics card Strata can use was found: this model needs one or more cards (8 GB or more; 12 GB is what it was "
+ "measured with).")
+ else:
+ per_dense = ISTA_DENSE_ON_GPU_GB / n
+ cap = [max(0.0, _cap_gb(float(c.get("vram_gb") or 0)) - per_dense - RESERVE_GB) for c in keep]
+ vram_exp = min(v["experts_gb"], sum(cap))
+ ranges = split_layers([float(c.get("vram_gb") or 0) for c in keep])
+ in_vram = 0
+ for c, room, (a, b) in zip(keep, cap, ranges):
+ gb = min(room, vram_exp * (room / sum(cap))) if sum(cap) else 0.0
+ ne = min(int(gb / expert_gb), (b - a + 1) * EXPERTS_PER_LAYER)
+ in_vram += ne
+ out["cards"].append({"index": c.get("index"), "name": c.get("name"), "vram_gb": float(c.get("vram_gb") or 0), "layers": [a, b],
+ "dense_gb": _round1(per_dense), "experts_gb": _round1(ne * expert_gb), "experts": ne})
+ out["experts_in_vram"] = in_vram
+ out["vram_experts_gb"] = _round1(in_vram * expert_gb)
+ out["cpu_experts_gb"] = _round1(v["experts_gb"] - in_vram * expert_gb)
+ out["tp"] = tp_for(n)
+ out["args"] = ["--tp", str(out["tp"])] if out["tp"] > 1 else []
+ names = " + ".join(f"{c['name'] or 'card ' + str(c['index'])} ({c['vram_gb']:.0f} GB)" for c in out["cards"])
+ lines.append(f"Graphics memory: {names}. About {in_vram:,} of the {EXPERTS:,} experts ({out['vram_experts_gb']:.0f} GB) stay on "
+ f"the card{'s' if n > 1 else ''}; the other {EXPERTS - in_vram:,} ({out['cpu_experts_gb']:.0f} GB) are computed on the "
+ "CPU from RAM, and the ones RAM cannot hold are read from the SSD when a word needs them.")
+ if n >= 2:
+ lines.append("Two cards share each layer's matrices (--tp 2): measured about twice as fast at reading prompts.")
+ if ram_gb:
+ all_cached = out["cpu_experts_gb"] + REST_RAM_GB
+ out["ram_needed_gb"] = _round1(all_cached)
+ out["spill_gb"] = _round1(max(0.0, all_cached - float(ram_gb)))
+ out["spare_ram_gb"] = _round1(max(0.0, float(ram_gb) - all_cached))
+ if float(ram_gb) >= all_cached:
+ out["verdict"] = "fits"
+ lines.append(f"That fits this computer: {ram_gb:.0f} GB of RAM keeps all {out['cpu_experts_gb']:.0f} GB of the CPU's experts cached.")
+ elif float(ram_gb) >= v["ram_ok_gb"]:
+ out["verdict"] = "tight"
+ lines.append(f"This computer has {ram_gb:.0f} GB of RAM: Strata keeps the experts you use most in RAM (it sizes this itself) and "
+ "reads the rest from the SSD. Measured: about 4.7 words a second on 64 GB of RAM and a 12 GB card, about 7 on "
+ "46 GB with a faster drive. More RAM or a faster NVMe drive makes it quicker.")
+ else:
+ out["verdict"] = "spills"
+ warns.append(f"This computer has {ram_gb:.0f} GB of RAM and this model was measured with 46 GB and 64 GB. With less, most "
+ "experts come from the SSD while it answers: it works but is slow (a word can take seconds). More RAM fixes "
+ "it. It is still allowed.")
+ else:
+ out["verdict"] = "unknown"
+ short = [c for c in keep if _cap_gb(float(c.get("vram_gb") or 0)) < per_dense + RESERVE_GB + 1.0]
+ if short:
+ warns.append(f"The dense weights need ~{per_dense:.0f} GB of graphics memory and {short[0].get('name') or 'a card'} "
+ f"({float(short[0].get('vram_gb') or 0):.0f} GB) has hardly that: the engine may stop at the start. It is still allowed.")
+ if ignored:
+ lines.append("Not used: " + "; ".join(f"{c.get('name') or 'a card'} ({c.get('why_ignored')})" for c in ignored) + ".")
+ lines.append("Measured on " + ISTA_MEASURED + ". Other computers are slower or faster by what their cards, RAM and drive can do.")
+ if disk_free_gb is not None and disk_free_gb < need_disk:
+ warns.append(f"The download needs about {need_disk:.0f} GB of free disk space and {disk_free_gb:.0f} GB are free where the files go. "
+ "Use --models-dir on a bigger drive, or --gguf-dir for files you already have.")
+ out["words"] = " ".join(lines)
+ return out
+
+
def plan(cards: list[dict] | None, ram_gb: float | None, disk_free_gb: float | None = None, context: int = 8192,
- vendor: str | None = None) -> dict[str, Any]:
+ vendor: str | None = None, model: str | None = None) -> dict[str, Any]:
"""Where the model goes on this computer. `cards`: the graphics cards ({"index", "name", "vram_gb", ...}, as gpu_fit takes them),
- `ram_gb` the RAM, `disk_free_gb` the free space where the files go (None: not checked).
+ `ram_gb` the RAM, `disk_free_gb` the free space where the files go (None: not checked). `model`: an official IST-DAS set
+ (ISTA_SETS: "IST-DAS-2.00bpw" / "IST-DAS-2.25bpw") gets its own numbers; None or "UD-Q8_K_XL" is Unsloth's 508 GB set.
Keys: cards (per card: index, name, vram_gb, layers, dense_gb, experts_gb, experts), experts_in_vram, experts_total, vram_experts_gb,
cpu_experts_gb, ram_needed_gb, spill_gb, spare_ram_gb, verdict (fits / tight / spills / unknown / no_card), tp, args,
download_gb, engram_gb, disk_need_gb, disk_ok, lines (plain words, one idea each), warnings (the risks, plainly), measured."""
+ if variant(model):
+ return _plan_ista(variant_key(model), cards, ram_gb, disk_free_gb, context, vendor)
keep, ignored = gpu_fit.usable_cards(gpu_fit.for_vendor(list(cards or []), vendor)[0]) if vendor else gpu_fit.usable_cards(list(cards or []))
keep = [c for c in keep if float(c.get("vram_gb") or 0) >= SMALLEST_CARD_GB]
keep.sort(key=lambda c: c.get("index", 0)) # the engine takes the layers in card order
@@ -194,8 +360,8 @@ def plan(cards: list[dict] | None, ram_gb: float | None, disk_free_gb: float | N
return out
-def _option(key: str, cards: list[dict], ram_gb: float | None, ctx: int, vendor: str | None) -> dict[str, Any]:
- p = plan(cards, ram_gb, None, ctx, vendor)
+def _option(key: str, cards: list[dict], ram_gb: float | None, ctx: int, vendor: str | None, model: str | None = None) -> dict[str, Any]:
+ p = plan(cards, ram_gb, None, ctx, vendor, model)
names = " + ".join(gpu_fit._name(c) for c in cards)
label = f"{names} + RAM" if len(cards) == 1 else f"{names} + RAM (layers shared, --tp {p['tp']})"
words = {"fits": f"fits: {p['experts_in_vram']:,} of {p['experts_total']:,} experts in graphics memory, the rest computed on the CPU "
@@ -210,7 +376,7 @@ def _option(key: str, cards: list[dict], ram_gb: float | None, ctx: int, vendor:
"recommended": False, "plan": p}
-def options(cards: list[dict], ram_gb: float | None, vendor: str | None = None, context: int = 8192) -> dict[str, Any]:
+def options(cards: list[dict], ram_gb: float | None, vendor: str | None = None, context: int = 8192, model: str | None = None) -> dict[str, Any]:
"""gpu_fit.options for this model: every card that counts (recommended: the model is bigger than any one card, so each card adds
experts that stay off the CPU), and the largest card alone. Same keys as gpu_fit.options."""
mine, others = gpu_fit.for_vendor(list(cards or []), vendor) if vendor else (list(cards or []), [])
@@ -220,9 +386,9 @@ def options(cards: list[dict], ram_gb: float | None, vendor: str | None = None,
"other_vendor_note": gpu_fit.other_vendor_note(list(cards or []), vendor)}
if not keep:
return out
- opts = [_option("all", sorted(keep, key=lambda c: c.get("index", 0)), ram_gb, context, vendor)]
+ opts = [_option("all", sorted(keep, key=lambda c: c.get("index", 0)), ram_gb, context, vendor, model)]
if len(keep) > 1:
- opts.append(_option("single", [keep[0]], ram_gb, context, vendor))
+ opts.append(_option("single", [keep[0]], ram_gb, context, vendor, model))
opts[0]["recommended"] = True
out["options"], out["recommended"] = opts, "all"
return out
diff --git a/tests/terminal/test_launch.py b/tests/terminal/test_launch.py
index f7a83280b..c14934991 100644
--- a/tests/terminal/test_launch.py
+++ b/tests/terminal/test_launch.py
@@ -109,8 +109,8 @@ class CatalogRules(unittest.TestCase):
models, families, contexts, source = cat.tables()
self.assertEqual(source, "setup.py") # the same tables setup.py and the app read
for r in rows:
- if r.family == "deepseek41": # its one size is in MULTI_FILE_FAMILIES (nine shards), not in MODELS
- self.assertAlmostEqual(r.download_gb, 508.4)
+ if r.family == "deepseek41": # its sizes are in MULTI_FILE_FAMILIES (shards + shared files), not in MODELS
+ self.assertAlmostEqual(r.download_gb, {"UD-Q8_K_XL": 508.4, "IST-DAS-2.00bpw": 256.8, "IST-DAS-2.25bpw": 273.8}[r.model], delta=0.1)
continue
self.assertAlmostEqual(r.download_gb, models[r.model]["download_gb"])
rec = [r for r in rows if r.recommended]
diff --git a/tests/terminal/test_launch_dsv41.py b/tests/terminal/test_launch_dsv41.py
index 0aae41d97..baf0e2e1d 100644
--- a/tests/terminal/test_launch_dsv41.py
+++ b/tests/terminal/test_launch_dsv41.py
@@ -36,8 +36,10 @@ class LaunchDeepSeek(unittest.TestCase):
def test_the_model_is_in_the_list(self):
rows = [r for r in catalog(self.hw).rows() if r.family == "deepseek41"]
- self.assertEqual([r.model for r in rows], ["UD-Q8_K_XL"])
- r = rows[0]
+ self.assertEqual([r.model for r in rows], ["IST-DAS-2.00bpw", "IST-DAS-2.25bpw", "UD-Q8_K_XL"])
+ self.assertEqual([r.download_gb for r in rows[:2]], [256.8, 273.8])
+ self.assertEqual(catalog(self.hw).resolve("ist-das-2.25bpw"), ("deepseek41", "IST-DAS-2.25bpw"))
+ r = rows[2]
self.assertEqual((r.id, r.download_gb, r.verdict, r.images), ("deepseek-v4.1-flash-ud-q8_k_xl", 508.4, "fits", False))
self.assertEqual(catalog(self.hw).resolve("ud-q8_k_xl"), ("deepseek41", "UD-Q8_K_XL"))
self.assertEqual(catalog(self.hw).max_context("deepseek41", "UD-Q8_K_XL"), 8192)
@@ -80,7 +82,7 @@ class LaunchDeepSeek(unittest.TestCase):
self.assertIn("read from the SSD", out)
self.assertIn("free disk space", out)
self.assertIn("It is still allowed", out)
- row = next(r for r in catalog(self.hw).rows() if r.family == "deepseek41")
+ row = next(r for r in catalog(self.hw).rows() if r.family == "deepseek41" and r.model == "UD-Q8_K_XL")
self.assertEqual(row.verdict, "no") # "too big (can still try)": the plan above is printed and nothing is refused
def test_no_card_is_said(self):
diff --git a/tools/dsv41/smoke.py b/tools/dsv41/smoke.py
new file mode 100644
index 000000000..d39a3f9f2
--- /dev/null
+++ b/tools/dsv41/smoke.py
@@ -0,0 +1,99 @@
+"""16 tokens of a fixed prompt through strata-dsv41 and a check of the TEXT (setup runs this once after the pack is built).
+
+ python tools/dsv41/smoke.py --exe strata-dsv41 --pack <pack> [--gpus 0] [--tokens 16]
+
+A pack that loads can still give garbage (a wrong pack recipe does exactly that), so the answer is read, not just the exit code:
+the model is asked for the capital of France and the reply must be words, with "Paris" in it. Exit 0 and the text on success; exit 1
+and one plain line (and the engine's last lines) when the reply is not sensible; exit 2 when the engine did not run at all.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(ROOT / "tools"))
+
+PROMPT = "<|begin▁of▁sentence|><|User|>What is the capital of France? Answer in one short sentence.<|Assistant|></think>"
+
+
+def tokenizer(pack: Path):
+ import strata_tokenizer as ST
+ tpath = pack / "tokenizer"
+ vocab = json.loads((tpath / "vocab.json").read_text(encoding="utf-8"))
+ tokens = [None] * len(vocab)
+ for t, i in vocab.items():
+ tokens[i] = t
+ merges = (tpath / "merges.txt").read_text(encoding="utf-8").split("\n")
+ types = json.loads((tpath / "token_type.json").read_text())
+ tcfg = {}
+ try:
+ tcfg = json.loads((tpath / "tokenizer.json").read_text(encoding="utf-8"))
+ except (OSError, ValueError):
+ pass
+ return ST.Tokenizer(tokens, merges, types, tcfg.get("pre") or "qwen35", tcfg.get("special_ids") or None)
+
+
+def sensible(text: str) -> tuple[bool, str]:
+ """Words, not noise: letters in a normal share, more than two different words, no replacement characters, and Paris in it."""
+ t = text.strip()
+ words = re.findall(r"[A-Za-z]{2,}", t)
+ if not t:
+ return False, "the answer is empty"
+ if "�" in t:
+ return False, "the answer has broken characters"
+ if len(set(w.lower() for w in words)) < 3:
+ return False, "the answer is not made of words"
+ letters = sum(c.isalpha() or c.isspace() or c in ".,'-" for c in t) / len(t)
+ if letters < 0.8:
+ return False, "the answer is mostly symbols"
+ if "paris" not in t.lower():
+ return False, "the answer to a question with a known answer (the capital of France) does not contain it"
+ return True, "ok"
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--exe", required=True)
+ ap.add_argument("--pack", required=True)
+ ap.add_argument("--gpus")
+ ap.add_argument("--tokens", type=int, default=16)
+ ap.add_argument("--timeout", type=int, default=5400)
+ a = ap.parse_args()
+ pack = Path(a.pack)
+ tok = tokenizer(pack)
+ ids = tok.encode(PROMPT, parse_special=True)
+ with tempfile.TemporaryDirectory() as d:
+ f = Path(d) / "ids.txt"
+ f.write_text(",".join(str(i) for i in ids), encoding="ascii")
+ cmd = [a.exe, "--pack", str(pack), "--ids", "@" + str(f), "--n", str(a.tokens), "--max-context", "1024", "--dspark", "auto",
+ "--window", "16"]
+ if a.gpus:
+ cmd += ["--gpus", a.gpus]
+ try:
+ r = subprocess.run(cmd, capture_output=True, text=True, timeout=a.timeout, env=dict(os.environ))
+ except (OSError, subprocess.TimeoutExpired) as e:
+ print(f"the engine did not finish the test run: {e}")
+ return 2
+ err = (r.stderr or "").strip().splitlines()
+ if r.returncode != 0:
+ print(f"the engine stopped with an error (exit {r.returncode}): " + " | ".join(err[-3:]))
+ return 2
+ out = [int(x) for x in re.findall(r"-?\d+", (r.stdout or "").strip().splitlines()[-1] if (r.stdout or "").strip() else "")]
+ text = tok.decode(out)
+ good, why = sensible(text)
+ print(text.strip())
+ if not good:
+ print(f"test question failed: {why}")
+ return 1
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tools/strata_mcp.py b/tools/strata_mcp.py
index 3bb1c3a94..5d0171e41 100644
--- a/tools/strata_mcp.py
+++ b/tools/strata_mcp.py
@@ -559,7 +559,7 @@ class Strata:
big = self.multi_family(family)
if big and model in big["models"]:
# nine shards; the memory the weights need is VRAM + RAM together WITHOUT the n-gram table (it stays on the SSD)
- gb = sum(big["sha256"][n][0] for n in big["models"][model]["shards"]) / 1e9
+ gb = sum(big["sha256"][n][0] for n in [*big["models"][model]["shards"], *big["models"][model].get("shared", [])]) / 1e9
return {"about": big["models"][model]["about"], "download_gb": round(gb, 1), "ram_gb": big["models"][model]["ram_gb"],
"arena_gb": round(float(big["models"][model]["ram_gb"]) - 17.0, 1), "mem_total": True, "vision": False,
"engine": None, "families": (family,), "multi_file": True}
@@ -1188,7 +1188,7 @@ class Strata:
verdict = "fits" if have_gb >= d["ram_gb"] else "tight" if have_gb >= d["ram_gb"] - 8 else "does not fit"
try:
if own and own.get("multi_file"): # DeepSeek-V4.1-Flash: strata_control/dsv41_fit.py (cards + RAM, the n-gram table on the SSD)
- verdict = self.dsv41_words(hw)
+ verdict = self.dsv41_words(hw, m)
elif own: # its own engine path: no budget or low-RAM mode of Flash-Next's to judge by
pass
elif d.get("budget") and S and hasattr(S, "unsloth_budget") and not S.unsloth_budget(m, ram):
@@ -1220,7 +1220,7 @@ class Strata:
"on_this_pc": verdict, "installed": tag in have, "id": tag,
# which graphics cards to use for it (strata_control/gpu_fit.py, the one rule app / Terminal / setup share):
# only with two or more cards that count; "recommended" names the pre-selected option
- "gpu_options": (self.dsv41_options(hw) if own and own.get("multi_file")
+ "gpu_options": (self.dsv41_options(hw, m) if own and own.get("multi_file")
else None if own else self.gpu_options(d, ram, hw))}) # Qwen3.6 runs on one card so far
out.append({"family": f, "title": fd["title"], "about": fd.get("about"),
"experimental": bool(fd.get("experimental")),
@@ -1228,20 +1228,31 @@ class Strata:
"sizes": sizes})
return out
- def dsv41_plan(self, hw, disk_free=None, only=None) -> dict:
+ def dsv41_plan(self, hw, disk_free=None, only=None, model=None) -> dict:
"""strata_control/dsv41_fit.plan for this computer; `only`: the card numbers a launch names (else every card that counts)."""
from strata_control import dsv41_fit
cards = list((hw or {}).get("gpus", []))
if only:
cards = [c for c in cards if int(c.get("index", -1)) in {int(i) for i in only}]
- return dsv41_fit.plan(cards, (hw or {}).get("ram_gb"), disk_free, vendor=(hw or {}).get("engine_vendor"))
+ return dsv41_fit.plan(cards, (hw or {}).get("ram_gb"), disk_free, vendor=(hw or {}).get("engine_vendor"), model=model)
- def dsv41_words(self, hw) -> str:
+ def dsv41_words(self, hw, model=None) -> str:
"""The catalog's `on_this_pc` text for DeepSeek-V4.1-Flash (strata_cli/launch/catalog.verdict reads its first words)."""
- pl = self.dsv41_plan(hw)
+ pl = self.dsv41_plan(hw, model=model)
v = pl["verdict"]
if v == "no_card":
return "does not fit: it needs one or more graphics cards"
+ if pl.get("variant"): # an official IST-DAS set (strata_control/dsv41_fit.py): measured on 46-64 GB, never a refusal
+ if v == "spills":
+ return (f"slow here: this computer has {pl['ram_gb']:.0f} GB of RAM and the set was measured with 46-64 GB, so most of its "
+ "experts are read from the SSD; it can still be installed")
+ if v == "tight":
+ return ("tight: runs, with the experts RAM cannot hold read from the SSD (about 5-7 words a second measured on 46-64 GB "
+ "of RAM and a 12 GB card)")
+ if v == "fits":
+ return (f"fits: {pl['experts_in_vram']:,} of {pl['experts_total']:,} experts in graphics memory, the rest cached in RAM "
+ f"({pl['ram_needed_gb']:.0f} GB)")
+ return "not checked (this computer's memory could not be read)"
if v == "spills" and pl["spill_gb"] >= 100: # most of the CPU's experts would come from the disk every word
return (f"does not fit: about {pl['spill_gb']:.0f} GB of its experts would be read from the SSD while it answers "
f"(slow: it needs ~{pl['ram_needed_gb']:.0f} GB of RAM beside {pl['vram_experts_gb']:.0f} GB of graphics memory; "
@@ -1256,12 +1267,12 @@ class Strata:
f"({pl['ram_needed_gb']:.0f} GB of RAM)")
return "not checked (this computer's memory could not be read)"
- def dsv41_options(self, hw) -> dict | None:
+ def dsv41_options(self, hw, model=None) -> dict | None:
"""gpu_fit-shaped card choices for DeepSeek-V4.1-Flash: every card (recommended: the model is bigger than any one card), or the
largest alone; each with the placement words."""
from strata_control import dsv41_fit
try:
- return dsv41_fit.options(list((hw or {}).get("gpus", [])), (hw or {}).get("ram_gb"), (hw or {}).get("engine_vendor"))
+ return dsv41_fit.options(list((hw or {}).get("gpus", [])), (hw or {}).get("ram_gb"), (hw or {}).get("engine_vendor"), model=model)
except Exception: # noqa: BLE001 - the choice is a convenience: never break the model list
return None
@@ -1419,7 +1430,9 @@ def expected_files(S, family: str, model: str, models_dir: Path, vision: bool) -
big = (getattr(S, "MULTI_FILE_FAMILIES", None) or {}).get(family)
if big: # DeepSeek-V4.1-Flash: its nine shards in the model's folder
folder = Path(models_dir) / (big["tag"] + model)
- return [{"path": str(folder / n), "bytes": big["sha256"][n][0], "role": "model"} for n in big["models"][model]["shards"]]
+ shared = Path(models_dir) / (big["tag"] + "shared") # the Engram, the drafter and the metadata shard of the official sets
+ return ([{"path": str(folder / n), "bytes": big["sha256"][n][0], "role": "model"} for n in big["models"][model]["shards"]]
+ + [{"path": str(shared / n), "bytes": big["sha256"][n][0], "role": "model"} for n in big["models"][model].get("shared", [])])
own = (getattr(S, "OWN_PATH_FAMILIES", None) or {}).get(family)
if own: # Qwen3.6: the GGUF and the picture reader in the model's folder
folder = Path(models_dir) / (own["tag"] + model)
@@ -2127,7 +2140,7 @@ class Tools:
fit = None
ram = hw.get("ram_gb")
if ram is not None and md.get("multi_file"):
- fit = s.dsv41_words(hw)
+ fit = s.dsv41_words(hw, model)
elif ram is not None:
need_ram = md["ram_gb"]
fit = "fits" if ram >= need_ram else f"needs ~{need_ram} GB of RAM, this PC has {ram:.0f} GB"
diff --git a/tools/test_setup_dsv41.py b/tools/test_setup_dsv41.py
index fd9e86865..70939b5a2 100644
--- a/tools/test_setup_dsv41.py
+++ b/tools/test_setup_dsv41.py
@@ -8,6 +8,7 @@ from __future__ import annotations
import argparse
import contextlib
+import os
import io
import json
import re
@@ -177,6 +178,7 @@ class Install(unittest.TestCase):
self.exe.parent.mkdir(parents=True, exist_ok=True)
self.exe.write_text("x")
cards = cards or FOUR
+ kw.setdefault("model", "UD-Q8_K_XL")
gpu = {"index": 0, "count": len(cards), "vram_gb": 32.0, "name": "R9700", "cards": cards}
return run(setup.install_dsv41, args(models_dir=str(self.root / "models"), **kw), "deepseek41", gpu, True, [], 13, [], 8080, ram,
None, cards)
@@ -247,7 +249,7 @@ class Install(unittest.TestCase):
self.assertIn(str(own / "UD-Q8_K_XL"), pack)
def test_little_ram_is_asked_and_kept_with_an_explicit_model(self):
- rc, out = self.install(ram=64.0, cards=FOUR[:1], model="UD-Q8_K_XL")
+ rc, out = self.install(ram=64.0, cards=FOUR[:1])
self.assertEqual(rc, 0, out)
self.assertIn("read from the SSD", out)
self.assertIn("installing UD-Q8_K_XL with 64 GB of RAM, as you chose", out)
@@ -255,7 +257,7 @@ class Install(unittest.TestCase):
self.assertNotIn("--tp", self.cfg["args"])
def test_little_ram_with_yes_alone_stops_where_the_rule_says(self):
- rc, out = self.install(ram=64.0, cards=FOUR[:1])
+ rc, out = self.install(ram=24.0, cards=FOUR[:1], model=None) # the version was not chosen on the command line
self.assertEqual(rc, 1, out)
self.assertIn("more RAM or cards fix it", out)
self.assertFalse([c for c in self.calls if c[0] == "get"])
@@ -283,13 +285,17 @@ class Catalog(unittest.TestCase):
s = M.Strata(ROOT)
fam = next(f for f in s.model_table({"ram_gb": 251.0, "gpus": [{**c, "usable": True} for c in FOUR], "engine_vendor": "amd"})
if f["family"] == "deepseek41")
- (size,) = fam["sizes"]
+ self.assertEqual([x["model"] for x in fam["sizes"]], ["IST-DAS-2.00bpw", "IST-DAS-2.25bpw", "UD-Q8_K_XL"])
+ small_sets = {x["model"]: x for x in fam["sizes"]}
+ self.assertEqual((small_sets["IST-DAS-2.00bpw"]["download_gb"], small_sets["IST-DAS-2.25bpw"]["download_gb"]), (256.8, 273.8))
+ self.assertTrue(small_sets["IST-DAS-2.00bpw"]["on_this_pc"].startswith("fits"))
+ size = small_sets["UD-Q8_K_XL"]
self.assertEqual((size["model"], size["id"], size["download_gb"], size["images"]), ("UD-Q8_K_XL", "deepseek-v4.1-flash-ud-q8_k_xl", 508.4, False))
self.assertTrue(size["on_this_pc"].startswith("fits"))
self.assertTrue(size["mem_total"] and size["multi_file"])
self.assertEqual(size["gpu_options"]["recommended"], "all")
small = next(f for f in s.model_table({"ram_gb": 32.0, "gpus": []}) if f["family"] == "deepseek41")
- self.assertTrue(small["sizes"][0]["on_this_pc"].startswith("does not fit"))
+ self.assertTrue(all(x["on_this_pc"].startswith("does not fit") for x in small["sizes"])) # no card
def test_the_install_plan(self):
import strata_mcp as M
@@ -308,5 +314,158 @@ class Catalog(unittest.TestCase):
self.assertEqual(sum(f["bytes"] for f in files), 508_376_763_197)
+ISTA = "IST-DAS-2.00bpw"
+RTX5070 = {"index": 0, "name": "NVIDIA GeForce RTX 5070", "vram_gb": 12.0, "vendor": "nvidia"}
+
+
+class OfficialSets(unittest.TestCase):
+ """The two official IST-DAS sets (helcig/tmp-oct26-2): the catalog, the pins, the pack recipe, the fit words."""
+
+ def test_the_pinned_files(self):
+ fam = setup.MULTI_FILE_FAMILIES["deepseek41"]
+ self.assertEqual(list(fam["models"]), ["IST-DAS-2.00bpw", "IST-DAS-2.25bpw", "UD-Q8_K_XL"]) # the official sets first
+ self.assertEqual(setup.HF_REVISIONS["helcig/tmp-oct26-2"], "851709dcf0d97f091a4bdce758b2e4b0b468dd26")
+ self.assertIs(setup.REPO_PINS["helcig/tmp-oct26-2"], setup.DSV41_ISTA_FILES)
+ self.assertEqual(len(setup.DSV41_ISTA_FILES), 9) # 2 x 4 shards + the Engram
+ for n, (size, sha) in setup.DSV41_ISTA_FILES.items():
+ self.assertRegex(sha, r"^[0-9a-f]{64}$", n)
+ eng = setup.DSV41_ISTA_FILES["engram-IQ4_NL.gguf"]
+ self.assertEqual(eng, (110_595_290_720, "1f17d8ae37e9df3bb45f7ef1b4fcd1706d7b759af0413c117bf8e92aed459281"))
+ self.assertEqual(setup.DSV41_ALL_FILES["dspark-DeepSeek-V4.1-Flash-1017-Q8_0.gguf"],
+ (7_976_914_240, "6ba6ac1b331c540691a209e7fc97dec2a24e295acb1e755c84c7d24a4373927a"))
+ sums = {m: sum(setup.DSV41_ALL_FILES[n][0] for n in fam["models"][m]["shards"]) for m in list(fam["models"])[:2]}
+ self.assertEqual(sums, {"IST-DAS-2.00bpw": 138_238_892_160, "IST-DAS-2.25bpw": 155_225_823_328})
+ self.assertEqual(sums["IST-DAS-2.00bpw"], dsv41_fit.ISTA_SETS["IST-DAS-2.00bpw"]["set_bytes"])
+ self.assertEqual(sums["IST-DAS-2.25bpw"], dsv41_fit.ISTA_SETS["IST-DAS-2.25bpw"]["set_bytes"])
+ self.assertEqual(setup.DSV41_ISTA_FILES["engram-IQ4_NL.gguf"][0], dsv41_fit.ISTA_ENGRAM_BYTES)
+ for m in ("IST-DAS-2.00bpw", "IST-DAS-2.25bpw"): # both sets name the same shared files: the Engram is downloaded once
+ self.assertEqual(fam["models"][m]["shared"], [setup.DSV41_ISTA_ENGRAM, setup.DSV41_SHARD.format(i=1), setup.DSV41_DSPARK])
+ for n in fam["models"][m]["shared"]:
+ self.assertIn(n, setup.DSV41_SHARED)
+ self.assertIn(n, setup.DSV41_ALL_FILES)
+ self.assertRegex(setup.DSV41_SHARED[setup.DSV41_ISTA_ENGRAM], r"helcig/tmp-oct26-2/resolve/851709dc[0-9a-f]{32}/2\.00bpw/$")
+ self.assertRegex(fam["models"]["IST-DAS-2.25bpw"]["hf"], r"helcig/tmp-oct26-2/resolve/851709dc[0-9a-f]{32}/2\.25bpw/$")
+
+ def test_shipped_profiles_match_their_hashes(self):
+ import hashlib
+ for n in ("profile-2.00.bin", "profile-2.25.bin"):
+ data = (setup.DSV41_PROFILES / n).read_bytes()
+ self.assertEqual(len(data), 66048) # uint32 [43][384]
+ want = (setup.DSV41_PROFILES / (n + ".sha256")).read_text().split()[0]
+ self.assertEqual(hashlib.sha256(data).hexdigest(), want)
+
+ def test_the_pack_recipe_is_the_plain_one(self):
+ folder, pack = Path("/m/set"), Path("/d/pack")
+ names = [f"model-2.00bpw-{i:05d}-of-00004.gguf" for i in range(1, 5)]
+ cmd = setup.dsv41_pack_cmd(folder, names, Path("/s/engram-IQ4_NL.gguf"), Path("/s/meta.gguf"), Path("/s/dspark.gguf"), pack)
+ self.assertTrue(cmd[1].endswith("pack_gguf.py"))
+ self.assertEqual(cmd[2:], ["--model", str(folder), "--shards", *names, "--engram", str(Path("/s/engram-IQ4_NL.gguf")),
+ "--engram-meta", str(Path("/s/meta.gguf")), "--dspark-gguf", str(Path("/s/dspark.gguf")), "--out", str(pack)])
+ for bad in ("--skip-tensors", "--dense-override", "--override-experts", "--tq-experts"): # these give garbage text on the official sets
+ self.assertNotIn(bad, cmd)
+
+ def test_fit_words_use_the_measured_numbers(self):
+ for ram, verdict in ((64.0, "tight"), (46.0, "tight"), (24.0, "spills"), (256.0, "fits")):
+ p = dsv41_fit.plan([RTX5070], ram, None, model=ISTA)
+ self.assertEqual(p["verdict"], verdict, ram)
+ self.assertAlmostEqual(p["download_gb"], 256.8, delta=0.1)
+ self.assertEqual(p["args"], [])
+ p = dsv41_fit.plan([RTX5070], 64.0, 100.0, model=ISTA)
+ text = " ".join(p["lines"]) + " ".join(p["warnings"])
+ for needle in ("4.7", "7.2", "31 tokens/s", "free disk space", "only once" if False else "downloads it once"):
+ self.assertIn(needle, text)
+ self.assertFalse(p["disk_ok"])
+ p225 = dsv41_fit.plan([RTX5070], 64.0, None, model="IST-DAS-2.25bpw")
+ self.assertAlmostEqual(p225["download_gb"], 273.8, delta=0.1)
+ self.assertEqual(dsv41_fit.plan([], 64.0, None, model=ISTA)["verdict"], "no_card")
+ self.assertEqual(dsv41_fit.plan([RTX5070], 64.0, None, model="UD-Q8_K_XL")["verdict"], "spills") # the big set is unchanged
+
+
+class InstallOfficial(Install):
+ """install_dsv41 for an official set, the engine and the tools mocked."""
+
+ def install_ista(self, model=ISTA, ram=64.0, **kw):
+ self.packinfo_ok = True
+ self.smoke_ok = True
+ patches = [mock.patch.object(setup, "dsv41_pack_info", lambda exe, pack, libs: (self.packinfo_ok, "files: 6, all present and long enough")),
+ mock.patch.object(setup, "dsv41_smoke", lambda *a, **k: (self.smoke_ok, "The capital of France is Paris.")
+ if self.smoke_ok else (False, "test question failed: the answer is empty"))]
+ for p in patches:
+ p.start()
+ self.addCleanup(p.stop)
+ return self.install(ram=ram, cards=[RTX5070], model=model, **kw)
+
+ def test_downloads_the_set_the_engram_once_and_the_drafter(self):
+ rc, out = self.install_ista()
+ self.assertEqual(rc, 0, out)
+ get = [c for c in self.calls if c[0] == "get"]
+ folder = self.root / "models" / "deepseek-v4.1-flash-IST-DAS-2.00bpw"
+ shared = self.root / "models" / "deepseek-v4.1-flash-shared"
+ self.assertEqual([c[2] for c in get][:4], [folder / f"model-2.00bpw-{i:05d}-of-00004.gguf" for i in range(1, 5)])
+ self.assertEqual([c[2] for c in get][4:], [shared / setup.DSV41_ISTA_ENGRAM, shared / setup.DSV41_SHARD.format(i=1),
+ shared / setup.DSV41_DSPARK])
+ self.assertTrue(get[0][1].endswith("/2.00bpw/model-2.00bpw-00001-of-00004.gguf"))
+ self.assertEqual(sum(1 for c in self.calls if c[0] == "sha"), 7)
+ pack = [c for c in self.calls if c[0] == "run"][-1][1]
+ self.assertIn("--engram-meta", pack)
+ self.assertNotIn("--skip-tensors", pack)
+
+ def test_the_second_set_does_not_download_the_engram_again(self):
+ shared = self.root / "models" / "deepseek-v4.1-flash-shared"
+ shared.mkdir(parents=True)
+ shared_names = (setup.DSV41_ISTA_ENGRAM, setup.DSV41_SHARD.format(i=1), setup.DSV41_DSPARK)
+ for n in shared_names:
+ (shared / n).write_bytes(b"x")
+ (shared / (n + ".done")).write_text("ok")
+ tiny = {n: (1, setup.DSV41_ALL_FILES[n][1]) for n in shared_names} # the stand-in files are one byte long
+ with mock.patch.dict(setup.DSV41_ALL_FILES, tiny):
+ rc, out = self.install_ista(model="IST-DAS-2.25bpw")
+ self.assertEqual(rc, 0, out)
+ got = [c[2].name for c in self.calls if c[0] == "get"]
+ self.assertEqual(got, [f"model-2.25bpw-{i:05d}-of-00004.gguf" for i in range(1, 5)]) # only the four shards
+
+ def test_pack_info_profile_smoke_and_the_launch_flags(self):
+ rc, out = self.install_ista()
+ self.assertEqual(rc, 0, out)
+ a = self.cfg["args"]
+ self.assertEqual(a[a.index("--dspark") + 1], "auto") # never "off": that still loads the drafter and takes the VRAM
+ self.assertNotIn("off", a)
+ self.assertNotIn("--ssd-cache", a) # the store sizes itself
+ self.assertEqual(a[a.index("--max-context") + 1], "8192")
+ self.assertIn("checked: every file the engine needs is there", out)
+ self.assertIn("test question answered sensibly", out)
+ self.assertTrue(Path(a[a.index("--pack") + 1]).match("*packs/deepseek-v4.1-flash-ist-das-2.00bpw"))
+
+ def test_an_incomplete_pack_stops_with_the_missing_file(self):
+ self.packinfo_ok = False
+ patches = [mock.patch.object(setup, "dsv41_pack_info", lambda exe, pack, libs: (False, " MISSING x.gguf (1.00 GB wanted)")),
+ mock.patch.object(setup, "dsv41_smoke", lambda *a, **k: (True, "Paris"))]
+ for p in patches:
+ p.start()
+ self.addCleanup(p.stop)
+ rc, out = self.install(ram=64.0, cards=[RTX5070], model=ISTA)
+ self.assertEqual(rc, 1, out)
+ self.assertIn("pack is not complete", out)
+ self.assertIn("MISSING", out)
+
+ def test_a_failed_text_check_is_said_plainly_and_stops(self):
+ patches = [mock.patch.object(setup, "dsv41_pack_info", lambda exe, pack, libs: (True, "all present")),
+ mock.patch.object(setup, "dsv41_smoke", lambda *a, **k: (False, "!!!!\ntest question failed: the answer is empty"))]
+ for p in patches:
+ p.start()
+ self.addCleanup(p.stop)
+ rc, out = self.install(ram=64.0, cards=[RTX5070], model=ISTA)
+ self.assertEqual(rc, 1, out)
+ self.assertIn("does not answer sensibly", out)
+ self.assertIn("run setup again", out)
+ self.assertFalse(self.cfg) # no start script for a model that failed its check
+
+ def test_a_disk_that_cannot_hold_the_set_stops(self):
+ with mock.patch.object(setup, "free_gb", lambda p: 100.0):
+ rc, out = self.install_ista()
+ self.assertEqual(rc, 1, out)
+ self.assertIn("not enough free disk space", out)
+
+
if __name__ == "__main__":
unittest.main()