Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Merge f-dsv41-rel (0be62ccad) into v0.2.3.1: DeepSeek V4.1-Flash on top of the released 0.2.3 line

Niko122128ce98bf167b
diff --cc packaging/ENGINE-COMPONENT.md
index 642c2f0b5,9d1944533..0d3157e85
--- a/packaging/ENGINE-COMPONENT.md
+++ b/packaging/ENGINE-COMPONENT.md
@@@ -96,11 -96,10 +96,11 @@@ the manifest file (which holds every fi
| Package | Built on | Contents | Notes |
|---|---|---|---|
- | `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (built for this variant's archs by `packaging/build_vision_windows.bat cuda13 <stage>`, which writes `vision_archs` into BUILD.json), `BUILD.json` | `min_driver` nvidia 580.65 |
-| `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (reused, unchanged since 0.1.44), `strata-dsv41.exe` (DeepSeek-V4.1, `packaging/build_dsv41_windows.bat`, sm_86 + sm_120; `--require-dsv41`), `BUILD.json` | `min_driver` nvidia 580.65 |
-| `windows-x86_64` cuda, variant `cuda12` | the owner PC (CUDA 12.9, `60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same four files | experimental, `min_driver` nvidia 575.51; the CUDA 13 build wins when both fit |
++| `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (built for this variant's archs by `packaging/build_vision_windows.bat cuda13 <stage>`, which writes `vision_archs` into BUILD.json), `strata-dsv41.exe` (DeepSeek-V4.1, `packaging/build_dsv41_windows.bat`, sm_86 + sm_120; `--require-dsv41`), `BUILD.json` | `min_driver` nvidia 580.65 |
+| `windows-x86_64` cuda, variant `cuda12` | the owner PC (CUDA 12.9, `60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same four files, with its own `strata-vision.exe` (`packaging/build_vision_windows.bat cuda12 <stage>`: sm_60 .. sm_89; the CUDA 13 reader has no Pascal code, #3669) | experimental, `min_driver` nvidia 575.51; the CUDA 13 build wins when both fit |
| `windows-x86_64` hip | the owner PC, `tools/hip/build_windows.bat` (ROCm 10.2 nightly wheels) then `tools/hip/package_windows.py` | `strata.exe`, `strata-device.exe`, the ROCm DLLs and the rocBLAS/hipBLASLt kernels of the zip (`--tree`) | no Windows AMD test machine yet; with `STRATA_NO_VISION` unset it also builds the two picture readers (`strata-vision.exe` on the CPU, `strata-vision-hip.exe` on the GPU) |
-| `linux-x86_64` cuda | a Linux box with the CUDA toolkit | `strata`, `libstrata_kq.so`, `libcudart/libcublas/libcublasLt` (13.x) | the GPU driver's `libcuda` stays the PC's |
+| `linux-x86_64` cuda, variant `cuda13` | a Linux box with the CUDA 13 toolkit, `VARIANT=cuda13 packaging/build_cuda_linux.sh` | `strata`, `libstrata_kq.so`, `strata-vision` (built by the same script for the same archs, `BUILD_VISION=0` for a text-only package), `libcudart/libcublas/libcublasLt` (13.x) | the GPU driver's `libcuda` stays the PC's |
+| `linux-x86_64` cuda, variant `cuda12` | a Linux box with CUDA 12.9, `VARIANT=cuda12 packaging/build_cuda_linux.sh` (`60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same files with the 12.x libraries (the reader has sm_60 / 61 / 70 too) | experimental, `min_driver` nvidia 575.51; the Pascal and Volta package (the 0.2.1 one lacked sm_60 / 61 / 70, #3503); `package_engine.py` refuses a cuda12 package without them |
| `linux-x86_64` hip | a box with ROCm (the Python wheels are enough) | `strata`, the HIP runtime, hipBLAS/hipBLASLt/rocBLAS, comgr + libLLVM, `rocblas/library` and `hipblaslt/library` for the listed gfx archs; `packaging/build_hip_linux.sh` builds it with both picture readers |
| `linux-x86_64` hip, variant `rx6000` (RX 6000, gfx1030 / gfx1031) | `VARIANT=rx6000 packaging/build_hip_linux.sh` in the jammy chroot, ROCm 7.13.0a20260515 `gfx103X-all` wheels | the same set as the other HIP packages, with `rocblas/library` for gfx1030 / gfx1031; no `hipblaslt/library` kernels exist for them and no `tuning/` table | built, not tested on a real RX 6000 card (the manifest notes say so). `windows-x86_64` hip, variant `rdna2`: the same archs from `tools/hip/build_windows.bat` with `STRATA_HIP_ARCHS=gfx1030;gfx1031`, packaged with `--variant rdna2`. A PC with an RX 7000 / 9000 card beside an RX 6000 one never takes it (`engine_component.hip_preference`) |
| `linux-x86_64` hip, gfx1103 (Radeon 780M / Phoenix iGPU) | `packaging/build_hip_gfx1103.sh` on the mini PC (TheRock gfx110X-all SDK) | the same set as the gfx1200/1201 package, with `rocblas/library` and `hipblaslt/library` for gfx1103 (600 MiB) | experimental, unvalidated: the iGPU resets now and then (MES REMOVE_QUEUE); never kill the engine mid-request; run configs set `STRATA_NO_ARENA_THP=1`. `engine_component.resolve` picks a HIP package only when its `archs` list a gfx target the Linux kfd topology reports |
diff --cc packaging/build_hip_linux.sh
index bed5a9d56,72ab2bb01..20f44cd4e
--- a/packaging/build_hip_linux.sh
+++ b/packaging/build_hip_linux.sh
@@@ -9,8 -9,10 +9,10 @@@
# SRC=<this checkout> the Strata source (default: the checkout this script is in)
# LLAMA=<llama.cpp checkout> at the pinned commit (setup.py LLAMA_CPP_COMMIT); mtmd and ggml come from it
# HP=<ROCm devel root> CORE=<ROCm core root> LIBW=<ROCm libraries root> (TheRock: _rocm_sdk_devel, _rocm_sdk_core, _rocm_sdk_libraries_*)
-# OUT=<work folder> PKGVER=0.1.46 COMMIT=<source commit for the build id> JOBS=16
+# OUT=<work folder> PKGVER=0.1.48 COMMIT=<source commit for the build id> JOBS=16
# VISION_ONLY=1 build only the two readers (no engine, no package)
+ # DSV41=1|0 also build strata-dsv41 (the DeepSeek-V4.1 engine, tools/dsv41/build_hip_linux.sh's flags) and ship it beside
+ # strata: default 1 for the rx9070 variant (gfx1200 gfx1201; built and run on an R9700), 0 for the others
# NO_PACKAGE=1 stop after the stage folder
# VISION_HIP_DEFS="-DGGML_CUDA_FA=OFF" extra cmake definitions for the GPU reader only
#
diff --cc packaging/package_engine.py
index f9f3e1489,702e1ead2..4d61fc5af
--- a/packaging/package_engine.py
+++ b/packaging/package_engine.py
@@@ -54,81 -57,10 +58,82 @@@ def sha256(path: Path) -> str
def wanted(path: Path) -> bool:
name = path.name.lower()
return (name.endswith((".dll", ".so", ".dylib", ".metallib")) or ".so." in name or name == "build.json"
- or name.startswith("license") or name in ("strata-vision", "strata-vision.exe", "strata-vision-hip", "strata-vision-hip.exe")) # the picture readers (CPU, AMD GPU) beside the engine
+ or name.startswith("license") or name in ("strata-vision", "strata-vision.exe", "strata-vision-hip", "strata-vision-hip.exe",
+ DSV41_EXE, DSV41_EXE + ".exe")) # the picture readers (CPU, AMD GPU) and the DeepSeek-V4.1 engine beside the engine
+READER_NAMES = ("strata-vision", "strata-vision.exe")
+
+
+def _arch_numbers(value) -> set[int]:
+ """The CUDA architectures in a BUILD.json `archs` / `vision_archs` value ("60;61;70", [60, "sm_61", "70-real"], ...), as numbers."""
+ if isinstance(value, str):
+ value = re.split(r"[;,\s]+", value)
+ found: set[int] = set()
+ for item in value or []:
+ m = re.fullmatch(r"(?:sm_|compute_)?(\d+)(?:a|f)?(?:-real|-virtual)?", str(item).strip().lower())
+ if m:
+ found.add(int(m.group(1)))
+ return found
+
+
+def check_picture_reader(src: Path, backend: str, target: str, archs: tuple[str, ...], *, variant: str = "",
+ allow_stale: bool = False, allow_none: bool = False) -> list[str]:
+ """The picture reader of a package must run on every card and Mac the package does (tools tests: I1, I2, #3669, #3697).
+
+ * CUDA package with a reader: BUILD.json (beside the engine) must list `vision_archs`, and every `sm_NN` the package carries
+ code for must be there. A reader reused from an older build (the Windows zips' strata-vision.exe, unchanged since 0.1.44, had no
+ sm_60) stops with "device kernel image is invalid" on a Pascal card while the engine runs. Refused, unless allow_stale.
+ * macOS package without a reader: refused (a Mac with this package cannot read a picture), unless allow_none.
+ * CUDA package without a reader: allowed (text-only), reported in the returned notes so the build log says it.
+ Returns notes to print."""
+ reader = next((n for n in READER_NAMES if (src / n).is_file()), None)
+ notes: list[str] = []
+ if target.startswith("macos"):
+ if reader is None and not allow_none:
+ raise SystemExit("refusing to package the macOS engine without its picture reader: a Mac with this package could not read "
+ "a picture (#3697). Build it with packaging/build_vision_mac.sh and copy strata-vision beside strata "
+ "(or give --no-picture-reader to ship a text-only package on purpose).")
+ if reader is None:
+ notes.append("no picture reader in this macOS package: pictures are off for it")
+ return notes
+ if backend != "cuda":
+ return notes
+ if reader is None:
+ notes.append("no picture reader in this CUDA package: the engine answers text only (the app says pictures need the reader)")
+ return notes
+ needed = {n for n in _arch_numbers([a for a in archs if str(a).lower().startswith("sm_")])}
+ have: set[int] = set()
+ build_json = src / "BUILD.json"
+ try:
+ have = _arch_numbers(json.loads(build_json.read_text(encoding="utf-8-sig")).get("vision_archs"))
+ except (OSError, ValueError, AttributeError):
+ pass
+ missing = sorted(needed - have)
+ if missing and not allow_stale:
+ what = ("BUILD.json has no vision_archs: the reader's build is not recorded" if not have
+ else "the reader has no code for " + ", ".join(f"sm_{n}" for n in missing))
+ raise SystemExit(f"refusing to package the {variant or backend} engine with this picture reader: {what}. A reader reused from an "
+ "older build stops with 'device kernel image is invalid' on a card it was not built for (#3669). Build it for the "
+ "package's architectures (packaging/build_vision_windows.bat, packaging/build_cuda_linux.sh) and record them as "
+ "vision_archs in BUILD.json (or give --allow-stale-vision).")
+ if missing:
+ notes.append("picture reader NOT built for " + ", ".join(f"sm_{n}" for n in missing) + " (allowed by --allow-stale-vision)")
+ return notes
+
+
+def stamp_vision_archs(stage: Path, archs: str) -> None:
+ """Record the architectures the picture reader in `stage` was built for as BUILD.json `vision_archs` (the build recipes for Windows
+ call this after building strata-vision.exe; check_picture_reader compares it with the package's archs)."""
+ path = stage / "BUILD.json"
+ try:
+ data = json.loads(path.read_text(encoding="utf-8-sig")) if path.is_file() else {}
+ except (OSError, ValueError):
+ data = {}
+ data["vision_archs"] = ";".join(str(n) for n in sorted(_arch_numbers(archs)))
+ path.write_text(json.dumps(data, indent=1) + chr(10), encoding="utf-8")
+
+
def git_build_id(repo: Path = ROOT) -> str:
"""The source commit of the engine build, `-dirty` when the tree had changes. `unknown` outside a checkout."""
try:
@@@ -164,7 -96,7 +169,8 @@@ def build(src: Path, backend: str, vers
archs: tuple[str, ...] = (), min_driver: dict | None = None, build_id: str | None = None,
min_runtime: str | None = None, notes: str = "", tree: bool = False, include: list[str] | None = None,
variant: str = "", search: list[Path] | None = None, allow_libs: list[str] | None = None,
- check_reader: bool = False, allow_stale_vision: bool = False, allow_no_reader: bool = False) -> Path:
++ check_reader: bool = False, allow_stale_vision: bool = False, allow_no_reader: bool = False,
+ require_dsv41: bool = False) -> Path:
if backend not in BACKEND_MATRIX:
raise SystemExit(f"unknown backend {backend!r}")
if backend != "cpu" and not archs:
@@@ -182,10 -109,9 +188,13 @@@
entrypoint = entry or ("strata.exe" if target.startswith("windows") else "strata")
if not (src / entrypoint).is_file():
raise SystemExit(f"{src / entrypoint} does not exist")
+ if check_reader: # the release recipes and the CLI ask for it; the library call keeps its old behaviour
+ for note in check_picture_reader(src, backend, target, archs, variant=variant, allow_stale=allow_stale_vision,
+ allow_none=allow_no_reader):
+ print("note: " + note)
+ if require_dsv41 and not any((src / n).is_file() for n in (DSV41_EXE, DSV41_EXE + ".exe")):
+ raise SystemExit(f"refusing to package: {src} has no {DSV41_EXE}[.exe] (--require-dsv41); build it with -DSTRATA_BUILD_DSV41=ON "
+ "(--target strata-dsv41) and copy it beside the engine")
folder = out / (f"{version}-{target}-{backend}" + (f"-{variant}" if variant else ""))
if folder.exists():
raise SystemExit(f"{folder} already exists; remove it first (packages are immutable)")
@@@ -295,12 -223,9 +306,14 @@@ def main(argv: list[str] | None = None
help="folder to copy a missing imported DLL from (default on Windows: CUDA_PATH/bin/x64, HIP_PATH/bin, PATH)")
ap.add_argument("--allow-lib", action="append", default=[], metavar="NAME",
help="an imported library that really is provided by the target system (rare; named so the check passes)")
+ ap.add_argument("--require-dsv41", action="store_true",
+ help="refuse the package when --src has no strata-dsv41[.exe] (the DeepSeek-V4.1 engine that setup.py looks for beside the engine)")
ap.add_argument("--include", action="append", default=[], metavar="GLOB", help="more files, relative to --src")
+ ap.add_argument("--stamp-vision", type=Path, metavar="STAGE", help="write --vision-archs into STAGE/BUILD.json and exit")
+ ap.add_argument("--vision-archs", default="", help="with --stamp-vision: the architectures the reader was built for, e.g. '60;61;70;75;80;86;89'")
+ ap.add_argument("--allow-stale-vision", action="store_true",
+ help="package a CUDA picture reader whose BUILD.json vision_archs do not cover the package's archs (it will fail on those cards)")
+ ap.add_argument("--no-picture-reader", action="store_true", help="ship a macOS package without strata-vision on purpose (pictures stay off)")
a = ap.parse_args(argv)
if a.verify:
try:
@@@ -335,8 -257,7 +351,8 @@@
folder = build(a.src, a.backend, a.version, a.platform or host_platform(), a.out, entry=a.entrypoint,
archs=tuple(a.arch), min_driver=drivers, build_id=a.build_id, min_runtime=a.min_runtime,
notes=a.notes, tree=a.tree, include=a.include, variant=a.variant,
- search=a.search, allow_libs=a.allow_lib, require_dsv41=a.require_dsv41)
+ search=a.search, allow_libs=a.allow_lib, check_reader=True, allow_stale_vision=a.allow_stale_vision,
- allow_no_reader=a.no_picture_reader)
++ allow_no_reader=a.no_picture_reader, require_dsv41=a.require_dsv41)
entries = [p for p in folder.rglob("*") if p.is_file()]
total = sum(p.stat().st_size for p in entries)
print(f"{folder} ({total / (1 << 20):.1f} MiB, {len(entries)} files, manifest sha256 {package_sha256(folder)})")
diff --cc serve/server.py
index 87f594592,948e02cd0..cd24086fa
--- a/serve/server.py
+++ b/serve/server.py
@@@ -5027,8 -4751,7 +5060,8 @@@ class Service
print(f"[strata] thinking capped at {max_new - reserve} of max_tokens {max_new} to leave room for "
f"the answer (budget {budget})", flush=True)
budget = max(1, max_new - reserve)
- parser = OutputParser(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery,
- declared_only=declared_only)
- parser = self.parser_cls(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery)
++ parser = self.parser_cls(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery,
++ declared_only=declared_only)
detok, n, finish = Detokenizer(self.tok), 0, "length"
run_tok, run_len, repeated = None, 0, False # #606: the current run of one repeated token
thinking_n = 0 # tokens written while thinking (Responses' reasoning_tokens)
diff --cc strata_cli/launch/cmd.py
index 9fec242ba,3eb04e714..97e3c1314
--- a/strata_cli/launch/cmd.py
+++ b/strata_cli/launch/cmd.py
@@@ -691,10 -638,9 +719,11 @@@ def run_launch(a: argparse.Namespace) -
_say(f" model {m.get('title') or 'your model files'}{size} (your files in {s.model_path})")
else:
_say(f" model {m.get('title') or f'{s.family} {s.model}'}{size}")
- _say(f" context {S.context_label(s.context)} pictures {PICTURE_WORDS.get(s.vision or 'off', s.vision)} parallel {s.parallel or 'fit to your card'}")
+ _say(f" context {S.context_label(s.context)} pictures {PICTURE_WORDS.get(s.vision or 'off', s.vision)} parallel {('one at a time' if plan.get('placement') else s.parallel or 'fit to your card')}")
_say(f" download {(plan.get('download') or {}).get('text', '')}")
+ if plan.get("drafting"):
+ _say(f" drafting {plan['drafting'].replace('Drafting: ', '', 1)}")
+ _placement_lines(plan.get("placement"))
_say(f" tuning {s.tune} access {S.ACCESS_WORDS[s.access]}, port {s.port}"
+ (" API key on" if s.api_key else ""))
for n in notes:
diff --cc strata_cli/launch/runner.py
index 0d2f500cc,ba1f5610b..1d1b5d7e7
--- a/strata_cli/launch/runner.py
+++ b/strata_cli/launch/runner.py
@@@ -1712,28 -1159,53 +1767,69 @@@ class Runner
title = row.title if row is not None else (f"{s.family} {s.model}" if s.model else None)
model = {"title": title, "family": s.family, "model": s.model, "model_gb": row.download_gb if row is not None else None,
"own_files": bool(s.model_path), "folder": s.model_path}
- return {"profile": self.profile, "settings": s.to_dict(with_key=False), "warnings": warns,
- "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download, "drafting": self.drafting_plan_line()}
+ out = {"profile": self.profile, "settings": s.to_dict(with_key=False), "warnings": warns,
- "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download}
++ "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download, "drafting": self.drafting_plan_line()}
+ if self.multi_file():
+ out["placement"] = self.placement()
+ return out
+ def drafting_plan_line(self) -> str:
+ """The drafting line of --plan: what drafts, or why not what was asked for. Nothing is changed or downloaded."""
+ from strata_control import drafting
+ try:
+ base, cfg = self.installed_state()
+ want = self.drafting_inputs(cfg or {})
+ return drafting.summary_line(drafting.decide(want["mode"], klass=want["klass"], model_name=want["model_name"], family=want["family"],
+ data_dir=want["data_dir"]))
+ except Exception: # noqa: BLE001
+ return ""
+
+ def dsv41_engine(self) -> bool | None:
+ """Whether the installed engine carries strata-dsv41 (True / False); None when no engine is installed yet or it cannot be told."""
+ own = (os.environ.get("STRATA_DSV41_EXE") or "").strip()
+ if own:
+ return Path(own).is_file()
+ try:
+ from strata_control import engine_component as ec
+ rep = ec.status()
+ exe = (rep.get("engine") or {}).get("exe") if rep.get("usable") else None
+ if not exe:
+ return None
+ folder = Path(str(exe)).parent
+ return any((folder / n).is_file() for n in ("strata-dsv41", "strata-dsv41.exe"))
+ except Exception: # noqa: BLE001
+ return None
+
+ def placement(self) -> dict:
+ """Where DeepSeek-V4.1-Flash would go on this computer's cards (strata_control/dsv41_fit.py): the cards this launch names, else every
+ card that counts; the real numbers in plain words, with the n-gram table's SSD note and the RAM / disk warnings (never a refusal)."""
+ s = self.s
+ hw = self.catalog.hardware()
+ try:
+ data = Path(s.data_dir) if s.data_dir else self.catalog.data_dir()
+ free = None if (s.model_path or self.model_state().remaining_gb < 1) else self.catalog.disk_free(data)
+ except Exception: # noqa: BLE001 - a plan that cannot read the disk still shows the rest
+ free = None
+ p = self.catalog.strata.dsv41_plan(hw, free, s.gpus, model=s.model)
+ warnings = list(p["warnings"])
+ engine = self.dsv41_engine()
+ if engine is False:
+ warnings.append("The engine installed here has no DeepSeek-V4.1 engine (strata-dsv41) yet: setup stops with a message before "
+ "anything is downloaded. Update Strata, or build it (cmake -DSTRATA_BUILD_DSV41=ON) and set STRATA_DSV41_EXE.")
+ return {"verdict": p["verdict"], "lines": p["lines"], "warnings": warnings, "cards": p["cards"], "args": p["args"],
+ "engine_has_dsv41": engine,
+ "experts_in_vram": p["experts_in_vram"], "experts_total": p["experts_total"], "vram_experts_gb": p["vram_experts_gb"],
+ "cpu_experts_gb": p["cpu_experts_gb"], "ram_needed_gb": p.get("ram_needed_gb"), "engram_gb": p["engram_gb"],
+ "download_gb": p["download_gb"], "measured": p["measured"]}
+
def run(self, *, start: bool = True) -> Result:
+ self.restart_after = start
try:
self.check_own_files() # a wrong --model-path is said before the engine is downloaded
+ if start:
+ reused = self.check_serving() # before anything is downloaded or stopped
+ if reused is not None:
+ return reused
self.ensure_engine()
self.check_cancel()
base = self.ensure_model()