Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
launch/setup for DeepSeek: setup installs the screens packages (textual) and launch goes on in plain lines without them; the Tuning screen shows the DeepSeek tuner's real length and what it tries; the Advanced list knows strata-dsv41's own flags; the model menu counts the disk (download + preparation) in 'fits'; docs: tuning and the separate engine package; tests
Niko12212dbb263b02d0
diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.mdindex 32bd2a22c..12021fd45 100644--- a/docs/DEEPSEEK41.md+++ b/docs/DEEPSEEK41.md@@ -71,8 +71,23 @@ The server speaks to the engine as it does for the other models, so `/v1/chat/coA value marked `string="true"` stays a string.- Defaults when a request names none: temperature 1.0, top_p 1.0.-Pictures and the DSpark drafter are in the engine (strata-dsv41 reads them from the unsloth mmproj and dspark files beside the model; the server side is not wired yet). Not in this version: several requests at once (`--parallel` is ignored with a note), tuning-(`--tune` is skipped with a note).+Not in this version: several requests at once (`--parallel` is ignored with a warning) and pictures (`--vision` is ignored with a warning).++## Tuning++`strata launch --tune quick` (or `thorough`, or the Tuning screen) works for DeepSeek with its own short tuner, `tools/dsv41/tune.py`. It starts+the engine a few times on a fixed short question and measures the answer speed: the standard settings first, then the DSpark drafter forced on,+then a shorter draft block (3), then one CPU thread for each physical core (thorough also tries every logical thread). A setting is kept only for a+gain of more than 5%, and the result is written into the launch's settings. Expect about 5 to 10 minutes (each run loads the model, which takes+1 to 3 minutes); close other heavy programs first. If a run fails or the computer is too busy to tell, the standard settings stay and one plain line+says so. The RAM store (`--ssd-cache`) sizes itself when the model starts, so the tuner does not touch it, and the context stays what you chose.++## The engine package++DeepSeek's engine, `strata-dsv41`, is a separate download from the Strata engine: when you choose DeepSeek, setup (or `strata launch`) fetches the+package for your computer from the signed update feed (resumable, signature and every file checked) and installs it in `engine-dsv41/` beside+`engine/`. The Qwen engine's files are never changed. If the feed has no package for your graphics card, setup says so in plain words and stops before+anything big is downloaded.## Compressed experts (trellis), for testingdiff --git a/setup.py b/setup.pyindex aaa732c0e..eb9cb8251 100644--- a/setup.py+++ b/setup.py@@ -3231,6 +3231,18 @@ def requirement_lines(path: Path | None = None) -> list[str]:return lines+MONITOR_REQUIREMENTS = ROOT / "requirements-monitor.txt"+++def runtime_packages() -> list[str]:+ """The packages setup installs for a run: the pinned runtime ones, plus the Terminal's screens (textual, which brings rich: `strata launch`+ and `strata monitor` need them; the packaged Terminal bundles them, a source install gets them here)."""+ base = requirement_lines() if REQUIREMENTS.exists() else list(PY_PACKAGES)+ if MONITOR_REQUIREMENTS.exists():+ base += [r for r in requirement_lines(MONITOR_REQUIREMENTS) if r not in base]+ return base++def _installed(name: str) -> bool:try:import importlib.metadata as md@@ -6645,8 +6657,8 @@ def update_install(have: list, a) -> int:say(" No model is installed in this Strata folder yet: run START-HERE.bat (Linux: ./setup.sh) to set it up -")say(" it finds an earlier install's model files next to it and reuses them.")return 0- pip_install(requirement_lines() if REQUIREMENTS.exists() else PY_PACKAGES,- "numpy, jinja2, regex, pyyaml, tqdm, requests, pillow, psutil")+ pip_install(runtime_packages(),+ "numpy, jinja2, regex, pyyaml, tqdm, requests, pillow, psutil, textual (the screens)")if not a.build:update_installed_engine(a.prebuilt)mtp_done: set = set()@@ -7576,8 +7588,8 @@ def install_own_path(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adoptewarn(f"little free disk space in {folder}: {free_gb(folder):.1f} GB free for about {left:.1f} GB. Setup goes on and should fit")step(3, "Python packages")- pip_install(requirement_lines() if REQUIREMENTS.exists() else PY_PACKAGES,- "numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil")+ pip_install(runtime_packages(),+ "numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil, textual (the screens)")step(4, "the Strata engine")llama, eng, meta, lib_dirs, gpu, vision = engine_step(a, gpu, hip, amd, cuda_tk, vision)@@ -7913,8 +7925,8 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,warn(f"--rope-scaling / --kv / --resident-budget-gib / --low-ram are Flash-Next settings: {fam['title']} does not use them")step(3, "Python packages")- pip_install(requirement_lines() if REQUIREMENTS.exists() else PY_PACKAGES,- "numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil")+ pip_install(runtime_packages(),+ "numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil, textual (the screens)")step(4, "the Strata engine")llama, eng, meta, lib_dirs, gpu, _vision = engine_step(a, gpu, hip, amd, cuda_tk, "none")@@ -9127,8 +9139,8 @@ def main() -> int:# ---- 3. python packagesstep(3, "Python packages")- pip_install(requirement_lines() if REQUIREMENTS.exists() else PY_PACKAGES,- "numpy, jinja2, regex, pyyaml, tqdm, requests, pillow, psutil")+ pip_install(runtime_packages(),+ "numpy, jinja2, regex, pyyaml, tqdm, requests, pillow, psutil, textual (the screens)")# ---- 4. the enginestep(4, "the Strata engine")diff --git a/strata_cli/launch/app.py b/strata_cli/launch/app.pyindex 4f1a565f4..058239edf 100644--- a/strata_cli/launch/app.py+++ b/strata_cli/launch/app.py@@ -148,8 +148,14 @@ class Ctx:tuned_args, tuned_env = list(merged.get("args") or []), dict(merged.get("env") or {})except Exception: # noqa: BLE001pass- exe = self.engine_exe()- text = P.engine_help_text(exe) if exe else None+ if (cfg or {}).get("arch") == "deepseek41" or self.settings.family == "deepseek41": # DeepSeek runs strata-dsv41: its settings, not the Qwen engine's+ own = (cfg or {}).get("exe")+ exe = own if own and Path(str(own)).is_file() else None+ text = P.dsv41_help_text(exe)+ exe = exe or "static"+ else:+ exe = self.engine_exe()+ text = P.engine_help_text(exe) if exe else Noneif exe and text:self.engine_flags = sorted({p.name for p in P.parse_help(text)})self.engine_note = ""diff --git a/strata_cli/launch/catalog.py b/strata_cli/launch/catalog.pyindex bde400e40..9f8e9359c 100644--- a/strata_cli/launch/catalog.py+++ b/strata_cli/launch/catalog.py@@ -72,6 +72,8 @@ def verdict(on_this_pc: str | None) -> tuple[str, str]:return TIGHT, "not checked (this computer's memory could not be read)"if low.startswith("does not fit"):return NO, "too big (can still try)"+ if "free disk space" in low: # DeepSeek: fits in memory, not on the disk (still allowed)+ return TIGHT, "tight: needs more disk space"if "low-ram mode" in low:return FITS, "fits, a bit slower"if "part of its experts" in low or "read from the ssd" in low:diff --git a/strata_cli/launch/cmd.py b/strata_cli/launch/cmd.pyindex 267d92ba4..09ffde9b7 100644--- a/strata_cli/launch/cmd.py+++ b/strata_cli/launch/cmd.py@@ -531,6 +531,13 @@ def open_page(url: str) -> bool:return False+def _screens_missing() -> str | None:+ """The Python package the screens need that this install lacks ("rich" / "textual"), None when both are there. A source install made before+ setup installed them has neither: `strata launch` then goes on in plain lines instead of stopping with a traceback."""+ import importlib.util+ return next((m for m in ("rich", "textual") if importlib.util.find_spec(m) is None), None)++def _tui(run, *args, **kw) -> int:"""Open the screens. An error of the screens themselves ends with a plain sentence and a non-zero exit code, never a traceback and exit 0(tester report T02)."""@@ -668,14 +675,21 @@ def run_launch(a: argparse.Namespace) -> int:have_choices = bool(a.preset or a.model or a.model_path or a.context or a.vision or a.parallel or a.access or a.tune or a.drafting)interactive = sys.stdin.isatty() and sys.stdout.isatty() and not a.no_tui and not machine and not a.planplain_questions = False+ no_screens_package = None+ if interactive and _screens_missing():+ no_screens_package = _screens_missing()+ interactive = False+ _say(f"The screens need a Python package this install does not have yet ({no_screens_package}), so Strata goes on in plain lines. "+ "Run setup again (START-HERE / ./setup.sh) to get the screens.")if not have_choices and not interactive and not a.plan:if machine or a.no_tui or catalog is None or not _can_ask(): # --no-tui never asks (round 8, M)return fail("`strata launch` with no choices opens screens, and this is not a terminal (or --no-tui was given). Name what to start: --preset NAME, or --model NAME ""(strata launch --help has examples; --plan shows what would happen without changing anything).", 2)# Git Bash, mintty and some editor panels cannot show the screens although a person is typing: go on in plain lines, and ask firstplain_questions = True- _say("This window cannot show Strata's screens (Git Bash and similar windows do not pass the keyboard through). "- "For the screens, open Windows Terminal, PowerShell or cmd, or run `winpty strata launch`. Going on with plain questions.")+ if no_screens_package is None:+ _say("This window cannot show Strata's screens (Git Bash and similar windows do not pass the keyboard through). "+ "For the screens, open Windows Terminal, PowerShell or cmd, or run `winpty strata launch`. Going on with plain questions.")if interactive and not have_choices:if catalog is None:return fail("This Terminal does not carry the model list, so the screens cannot open. Install it again with the one-line installer.", 5)diff --git a/strata_cli/launch/params.py b/strata_cli/launch/params.pyindex f6af060bc..9a3a98d81 100644--- a/strata_cli/launch/params.py+++ b/strata_cli/launch/params.py@@ -212,6 +212,45 @@ def flag_list(engine: str | Path | None) -> list[str] | None:return None+#: strata-dsv41's settings in plain words (its own --help is one bracketed usage line, which the Qwen engine's list parser cannot read).+DSV41_FLAGS = {+ "--pack": ("DIR", "The folder with the model's prepared data."),+ "--max-context": ("N", "How much text the model remembers at once."),+ "--eos": ("ID", "The token that ends an answer (1 for DeepSeek)."),+ "--dspark": ("on|off|none|auto", "The DSpark drafter that guesses ahead to speed up answers (auto loads it only where it helps)."),+ "--dspark-conf": ("F", "How sure a DSpark guess must be to be used (0 to 1)."),+ "--dspark-block": ("N", "The most words DSpark guesses in one round."),+ "--expert-profile": ("FILE|none", "A table of which experts are used most, to decide what is kept in memory first."),+ "--ssd-cache": ("GB", "How much RAM keeps the most used experts so they are not read from the SSD again (sizes itself when absent)."),+ "--ssd-policy": ("lru|lfu|lfulru", "Which experts the RAM store drops first."),+ "--prefill-chunk": ("N", "How many prompt words are read at once (the fast prompt path)."),+ "--threads": ("on|off", "Read long prompts layer by layer on all CPU threads."),+ "--cpu-threads": ("N", "How many CPU threads compute the experts the graphics card does not hold."),+ "--tp": ("N", "Share each layer between this many graphics cards."),+ "--gpus": ("LIST", "Which graphics cards to use together."),+ "--reserve-mib": ("N", "Graphics memory (in MiB) kept free for other programs."),+ "--vram-experts": ("F", "How many experts to keep in graphics memory."),+ "--mmap-experts": ("on|off|auto", "Read the experts in place from the model's files."),+ "--no-expert-cache": ("", "Do not keep the most used experts in graphics memory."),+ "--vision": ("on|off", "Let the model read pictures."),+ "--verbose": ("", "Print more about what the engine does."),+}+++def dsv41_help_text(engine: str | Path | None) -> str:+ """A list of strata-dsv41's settings the Advanced screen can read: ours in plain words, plus any flag its own --help names that we do not+ describe. Works before the engine is installed too (the plain words are static)."""+ lines = [f" {flag} {meta} {desc}" if meta else f" {flag} {desc}" for flag, (meta, desc) in DSV41_FLAGS.items()]+ if engine:+ import subprocess+ try:+ proc = subprocess.run([str(engine), "--help"], capture_output=True, text=True, timeout=20)+ seen = set(re.findall(r"--[a-z][a-z0-9-]*", (proc.stdout or "") + " " + (proc.stderr or "")))+ lines += [f" {f} An engine setting (the engine's help has no sentence for it)." for f in sorted(seen) if f not in DSV41_FLAGS]+ except Exception: # noqa: BLE001 - the static list stands+ pass+ return "\n".join(lines) + "\n"+def engine_help_text(engine: str | Path | None) -> str | None:if not engine:return Nonediff --git a/strata_cli/launch/screens.py b/strata_cli/launch/screens.pyindex 2c95a6ec4..167f4748f 100644--- a/strata_cli/launch/screens.py+++ b/strata_cli/launch/screens.py@@ -1041,9 +1041,15 @@ class TuningScreen(Base):def on_mount(self) -> None:s = self.ctx.settingsfirst = self.ctx.fresh_setup and not s.is_tuned- items = [Item(["Quick", "about 20 minutes", "checks the most likely gains" + ("; suggested the first time" if first else "")], key="quick"),- Item(["Thorough", "up to 2 hours", "tries more settings, measures each one several times"], key="thorough"),- Item(["Skip", "", "use the default settings (already good on most computers)"], key="skip")]+ if s.family == "deepseek41": # DeepSeek's own short tune (tools/dsv41/tune.py): its real length and what it tries+ items = [Item(["Quick", "about 5-10 minutes", "tries the DSpark drafter on or off, a shorter draft block and the CPU threads; keeps only clear gains"+ + ("; suggested the first time" if first else "")], key="quick"),+ Item(["Thorough", "about 10-15 minutes", "also tries every CPU thread"], key="thorough"),+ Item(["Skip", "", "use the default settings (already good on most computers)"], key="skip")]+ else:+ items = [Item(["Quick", "about 20 minutes", "checks the most likely gains" + ("; suggested the first time" if first else "")], key="quick"),+ Item(["Thorough", "up to 2 hours", "tries more settings, measures each one several times"], key="thorough"),+ Item(["Skip", "", "use the default settings (already good on most computers)"], key="skip")]if not self.ctx.tune_chosen and s.tune == "skip" and not s.is_tuned and self.ctx.fresh_setup:s.tune = "quick"self.query_one(Menu).set_items(items, {"quick": 0, "thorough": 1, "skip": 2}.get(s.tune, 0))diff --git a/tests/terminal/test_dsv41_tui_items.py b/tests/terminal/test_dsv41_tui_items.pynew file mode 100644index 000000000..69f42dbbd--- /dev/null+++ b/tests/terminal/test_dsv41_tui_items.py@@ -0,0 +1,73 @@+"""H100 fresh-install items for DeepSeek in the Terminal: the screens' packages, the Advanced list's own flags, the disk in "fits"."""+from __future__ import annotations++import sys+import unittest+from pathlib import Path+from unittest import mock++ROOT = Path(__file__).resolve().parents[2]+sys.path.insert(0, str(ROOT))+sys.path.insert(0, str(ROOT / "tools"))++from strata_cli.launch import cmd, params as P # noqa: E402+from strata_cli.launch.catalog import TIGHT, verdict # noqa: E402++CARD = {"index": 0, "name": "RTX 5070", "vram_gb": 12.0, "vendor": "nvidia", "usable": True}+++class Screens(unittest.TestCase):+ def test_missing_package_is_named_not_a_traceback(self):+ import importlib.util as u+ real = u.find_spec+ with mock.patch.object(u, "find_spec", lambda n, *a, **k: None if n == "rich" else real(n, *a, **k)):+ self.assertEqual(cmd._screens_missing(), "rich")+ with mock.patch.object(u, "find_spec", lambda n, *a, **k: None if n == "textual" else real(n, *a, **k)):+ self.assertEqual(cmd._screens_missing(), "textual")++ def test_setup_installs_the_screens_packages(self):+ import setup+ names = [r.split("==")[0].split(";")[0].strip() for r in setup.runtime_packages()]+ self.assertIn("textual", names)+ self.assertIn("numpy", names)+++class Advanced(unittest.TestCase):+ ARGS = ["--pack", "P", "--max-context", "8192", "--eos", "1", "--dspark", "auto", "--dspark-conf", "0.65"]++ def test_deepseeks_own_flags_are_known_before_the_engine_is_installed(self):+ rows = P.build_rows(P.dsv41_help_text(None), self.ARGS, None, {}, {}, {})+ mine = {r.name: r for r in rows if r.name in self.ARGS}+ self.assertEqual(set(mine), {"--pack", "--max-context", "--eos", "--dspark", "--dspark-conf"})+ for r in mine.values():+ self.assertNotIn("not in this engine's list", r.description)+ self.assertTrue(r.known)++ def test_a_real_typing_mistake_is_still_flagged(self):+ rows = P.build_rows(P.dsv41_help_text(None), self.ARGS + ["--dsparkk", "1"], None, {}, {}, {})+ self.assertIn("not in this engine's list", next(r for r in rows if r.name == "--dsparkk").description)+++class Disk(unittest.TestCase):+ def setUp(self):+ import strata_mcp as M+ self.s = M.Strata(ROOT)++ def hw(self, disk):+ return {"ram_gb": 64.0, "gpus": [CARD], "engine_vendor": "nvidia", "disk_free_gb": disk}++ def test_fits_in_memory_but_not_on_the_disk_is_tight_with_the_numbers(self):+ w = self.s.dsv41_words(self.hw(37.0), "IST-DAS-2.25bpw")+ self.assertTrue(w.startswith("tight"), w)+ self.assertIn("free disk space", w)+ self.assertIn("37 GB", w)+ self.assertEqual(verdict(w), (TIGHT, "tight: needs more disk space"))++ def test_enough_disk_or_already_installed_is_unchanged(self):+ self.assertNotIn("free disk space", self.s.dsv41_words(self.hw(900.0), "IST-DAS-2.25bpw"))+ self.assertNotIn("free disk space", self.s.dsv41_words(self.hw(37.0), "IST-DAS-2.25bpw", installed=True))+ self.assertNotIn("free disk space", self.s.dsv41_words({**self.hw(None)}, "IST-DAS-2.25bpw"))+++if __name__ == "__main__":+ unittest.main()diff --git a/tools/strata_mcp.py b/tools/strata_mcp.pyindex b0314f0f9..05d4931f2 100644--- a/tools/strata_mcp.py+++ b/tools/strata_mcp.py@@ -1221,7 +1221,7 @@ class Strata:verdict = "fits" if have_gb >= d["ram_gb"] else "tight" if have_gb >= d["ram_gb"] - 8 else "does not fit"try:if own and own.get("multi_file"): # DeepSeek-V4.1-Flash: strata_control/dsv41_fit.py (cards + RAM, the n-gram table on the SSD)- verdict = self.dsv41_words(hw, m)+ verdict = self.dsv41_words(hw, m, installed=tag in have)elif own: # its own engine path: no budget or low-RAM mode of Flash-Next's to judge bypasselif d.get("budget") and S and hasattr(S, "unsloth_budget") and not S.unsloth_budget(m, ram):@@ -1269,8 +1269,20 @@ class Strata:cards = [c for c in cards if int(c.get("index", -1)) in {int(i) for i in only}]return dsv41_fit.plan(cards, (hw or {}).get("ram_gb"), disk_free, vendor=(hw or {}).get("engine_vendor"), model=model)- def dsv41_words(self, hw, model=None) -> str:- """The catalog's `on_this_pc` text for DeepSeek-V4.1-Flash (strata_cli/launch/catalog.verdict reads its first words)."""+ def dsv41_words(self, hw, model=None, installed: bool = False) -> str:+ """The catalog's `on_this_pc` text for DeepSeek-V4.1-Flash, with the disk it needs (download and preparation) beside the memory: a set that+ fits in memory but not on the disk is "tight", never refused (strata_cli/launch/catalog.verdict reads its first words)."""+ words = self._dsv41_memory_words(hw, model)+ free = (hw or {}).get("disk_free_gb")+ if installed or free is None or words.startswith("does not fit"):+ return words+ pl = self.dsv41_plan(hw, free, model=model)+ if pl.get("disk_ok") is False:+ return (f"tight: it needs about {pl['disk_need_gb']:.0f} GB of free disk space (the download and its preparation) and this computer has "+ f"{free:.0f} GB free; free some space or choose another drive when you install it ({words})")+ return words++ def _dsv41_memory_words(self, hw, model=None) -> str:pl = self.dsv41_plan(hw, model=model)v = pl["verdict"]if v == "no_card":