Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Merge f-dsv41-rel (0be62ccad) into v0.2.3.1: DeepSeek V4.1-Flash on top of the released 0.2.3 line
Niko122128ce98bf167b
diff --cc packaging/ENGINE-COMPONENT.mdindex 642c2f0b5,9d1944533..0d3157e85--- a/packaging/ENGINE-COMPONENT.md+++ b/packaging/ENGINE-COMPONENT.md@@@ -96,11 -96,10 +96,11 @@@ the manifest file (which holds every fi| Package | Built on | Contents | Notes ||---|---|---|---|- | `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (built for this variant's archs by `packaging/build_vision_windows.bat cuda13 <stage>`, which writes `vision_archs` into BUILD.json), `BUILD.json` | `min_driver` nvidia 580.65 |-| `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (reused, unchanged since 0.1.44), `strata-dsv41.exe` (DeepSeek-V4.1, `packaging/build_dsv41_windows.bat`, sm_86 + sm_120; `--require-dsv41`), `BUILD.json` | `min_driver` nvidia 580.65 |-| `windows-x86_64` cuda, variant `cuda12` | the owner PC (CUDA 12.9, `60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same four files | experimental, `min_driver` nvidia 575.51; the CUDA 13 build wins when both fit |++| `windows-x86_64` cuda, variant `cuda13` | the owner PC, the same cmake line as the release zips (CUDA 13.0, `75;86;89;120` real + `120` PTX, `STRATA_PORTABLE=ON`) | `strata.exe`, `strata_kq.dll`, `strata-vision.exe` (built for this variant's archs by `packaging/build_vision_windows.bat cuda13 <stage>`, which writes `vision_archs` into BUILD.json), `strata-dsv41.exe` (DeepSeek-V4.1, `packaging/build_dsv41_windows.bat`, sm_86 + sm_120; `--require-dsv41`), `BUILD.json` | `min_driver` nvidia 580.65 |+| `windows-x86_64` cuda, variant `cuda12` | the owner PC (CUDA 12.9, `60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same four files, with its own `strata-vision.exe` (`packaging/build_vision_windows.bat cuda12 <stage>`: sm_60 .. sm_89; the CUDA 13 reader has no Pascal code, #3669) | experimental, `min_driver` nvidia 575.51; the CUDA 13 build wins when both fit || `windows-x86_64` hip | the owner PC, `tools/hip/build_windows.bat` (ROCm 10.2 nightly wheels) then `tools/hip/package_windows.py` | `strata.exe`, `strata-device.exe`, the ROCm DLLs and the rocBLAS/hipBLASLt kernels of the zip (`--tree`) | no Windows AMD test machine yet; with `STRATA_NO_VISION` unset it also builds the two picture readers (`strata-vision.exe` on the CPU, `strata-vision-hip.exe` on the GPU) |-| `linux-x86_64` cuda | a Linux box with the CUDA toolkit | `strata`, `libstrata_kq.so`, `libcudart/libcublas/libcublasLt` (13.x) | the GPU driver's `libcuda` stays the PC's |+| `linux-x86_64` cuda, variant `cuda13` | a Linux box with the CUDA 13 toolkit, `VARIANT=cuda13 packaging/build_cuda_linux.sh` | `strata`, `libstrata_kq.so`, `strata-vision` (built by the same script for the same archs, `BUILD_VISION=0` for a text-only package), `libcudart/libcublas/libcublasLt` (13.x) | the GPU driver's `libcuda` stays the PC's |+| `linux-x86_64` cuda, variant `cuda12` | a Linux box with CUDA 12.9, `VARIANT=cuda12 packaging/build_cuda_linux.sh` (`60;61;70;75;80;86;89` real + `89` PTX, `STRATA_EXPERIMENTAL_SM60=ON`) | the same files with the 12.x libraries (the reader has sm_60 / 61 / 70 too) | experimental, `min_driver` nvidia 575.51; the Pascal and Volta package (the 0.2.1 one lacked sm_60 / 61 / 70, #3503); `package_engine.py` refuses a cuda12 package without them || `linux-x86_64` hip | a box with ROCm (the Python wheels are enough) | `strata`, the HIP runtime, hipBLAS/hipBLASLt/rocBLAS, comgr + libLLVM, `rocblas/library` and `hipblaslt/library` for the listed gfx archs; `packaging/build_hip_linux.sh` builds it with both picture readers || `linux-x86_64` hip, variant `rx6000` (RX 6000, gfx1030 / gfx1031) | `VARIANT=rx6000 packaging/build_hip_linux.sh` in the jammy chroot, ROCm 7.13.0a20260515 `gfx103X-all` wheels | the same set as the other HIP packages, with `rocblas/library` for gfx1030 / gfx1031; no `hipblaslt/library` kernels exist for them and no `tuning/` table | built, not tested on a real RX 6000 card (the manifest notes say so). `windows-x86_64` hip, variant `rdna2`: the same archs from `tools/hip/build_windows.bat` with `STRATA_HIP_ARCHS=gfx1030;gfx1031`, packaged with `--variant rdna2`. A PC with an RX 7000 / 9000 card beside an RX 6000 one never takes it (`engine_component.hip_preference`) || `linux-x86_64` hip, gfx1103 (Radeon 780M / Phoenix iGPU) | `packaging/build_hip_gfx1103.sh` on the mini PC (TheRock gfx110X-all SDK) | the same set as the gfx1200/1201 package, with `rocblas/library` and `hipblaslt/library` for gfx1103 (600 MiB) | experimental, unvalidated: the iGPU resets now and then (MES REMOVE_QUEUE); never kill the engine mid-request; run configs set `STRATA_NO_ARENA_THP=1`. `engine_component.resolve` picks a HIP package only when its `archs` list a gfx target the Linux kfd topology reports |diff --cc packaging/build_hip_linux.shindex bed5a9d56,72ab2bb01..20f44cd4e--- a/packaging/build_hip_linux.sh+++ b/packaging/build_hip_linux.sh@@@ -9,8 -9,10 +9,10 @@@# SRC=<this checkout> the Strata source (default: the checkout this script is in)# LLAMA=<llama.cpp checkout> at the pinned commit (setup.py LLAMA_CPP_COMMIT); mtmd and ggml come from it# HP=<ROCm devel root> CORE=<ROCm core root> LIBW=<ROCm libraries root> (TheRock: _rocm_sdk_devel, _rocm_sdk_core, _rocm_sdk_libraries_*)-# OUT=<work folder> PKGVER=0.1.46 COMMIT=<source commit for the build id> JOBS=16+# OUT=<work folder> PKGVER=0.1.48 COMMIT=<source commit for the build id> JOBS=16# VISION_ONLY=1 build only the two readers (no engine, no package)+ # DSV41=1|0 also build strata-dsv41 (the DeepSeek-V4.1 engine, tools/dsv41/build_hip_linux.sh's flags) and ship it beside+ # strata: default 1 for the rx9070 variant (gfx1200 gfx1201; built and run on an R9700), 0 for the others# NO_PACKAGE=1 stop after the stage folder# VISION_HIP_DEFS="-DGGML_CUDA_FA=OFF" extra cmake definitions for the GPU reader only#diff --cc packaging/package_engine.pyindex f9f3e1489,702e1ead2..4d61fc5af--- a/packaging/package_engine.py+++ b/packaging/package_engine.py@@@ -54,81 -57,10 +58,82 @@@ def sha256(path: Path) -> strdef wanted(path: Path) -> bool:name = path.name.lower()return (name.endswith((".dll", ".so", ".dylib", ".metallib")) or ".so." in name or name == "build.json"- or name.startswith("license") or name in ("strata-vision", "strata-vision.exe", "strata-vision-hip", "strata-vision-hip.exe")) # the picture readers (CPU, AMD GPU) beside the engine+ or name.startswith("license") or name in ("strata-vision", "strata-vision.exe", "strata-vision-hip", "strata-vision-hip.exe",+ DSV41_EXE, DSV41_EXE + ".exe")) # the picture readers (CPU, AMD GPU) and the DeepSeek-V4.1 engine beside the engine+READER_NAMES = ("strata-vision", "strata-vision.exe")+++def _arch_numbers(value) -> set[int]:+ """The CUDA architectures in a BUILD.json `archs` / `vision_archs` value ("60;61;70", [60, "sm_61", "70-real"], ...), as numbers."""+ if isinstance(value, str):+ value = re.split(r"[;,\s]+", value)+ found: set[int] = set()+ for item in value or []:+ m = re.fullmatch(r"(?:sm_|compute_)?(\d+)(?:a|f)?(?:-real|-virtual)?", str(item).strip().lower())+ if m:+ found.add(int(m.group(1)))+ return found+++def check_picture_reader(src: Path, backend: str, target: str, archs: tuple[str, ...], *, variant: str = "",+ allow_stale: bool = False, allow_none: bool = False) -> list[str]:+ """The picture reader of a package must run on every card and Mac the package does (tools tests: I1, I2, #3669, #3697).++ * CUDA package with a reader: BUILD.json (beside the engine) must list `vision_archs`, and every `sm_NN` the package carries+ code for must be there. A reader reused from an older build (the Windows zips' strata-vision.exe, unchanged since 0.1.44, had no+ sm_60) stops with "device kernel image is invalid" on a Pascal card while the engine runs. Refused, unless allow_stale.+ * macOS package without a reader: refused (a Mac with this package cannot read a picture), unless allow_none.+ * CUDA package without a reader: allowed (text-only), reported in the returned notes so the build log says it.+ Returns notes to print."""+ reader = next((n for n in READER_NAMES if (src / n).is_file()), None)+ notes: list[str] = []+ if target.startswith("macos"):+ if reader is None and not allow_none:+ raise SystemExit("refusing to package the macOS engine without its picture reader: a Mac with this package could not read "+ "a picture (#3697). Build it with packaging/build_vision_mac.sh and copy strata-vision beside strata "+ "(or give --no-picture-reader to ship a text-only package on purpose).")+ if reader is None:+ notes.append("no picture reader in this macOS package: pictures are off for it")+ return notes+ if backend != "cuda":+ return notes+ if reader is None:+ notes.append("no picture reader in this CUDA package: the engine answers text only (the app says pictures need the reader)")+ return notes+ needed = {n for n in _arch_numbers([a for a in archs if str(a).lower().startswith("sm_")])}+ have: set[int] = set()+ build_json = src / "BUILD.json"+ try:+ have = _arch_numbers(json.loads(build_json.read_text(encoding="utf-8-sig")).get("vision_archs"))+ except (OSError, ValueError, AttributeError):+ pass+ missing = sorted(needed - have)+ if missing and not allow_stale:+ what = ("BUILD.json has no vision_archs: the reader's build is not recorded" if not have+ else "the reader has no code for " + ", ".join(f"sm_{n}" for n in missing))+ raise SystemExit(f"refusing to package the {variant or backend} engine with this picture reader: {what}. A reader reused from an "+ "older build stops with 'device kernel image is invalid' on a card it was not built for (#3669). Build it for the "+ "package's architectures (packaging/build_vision_windows.bat, packaging/build_cuda_linux.sh) and record them as "+ "vision_archs in BUILD.json (or give --allow-stale-vision).")+ if missing:+ notes.append("picture reader NOT built for " + ", ".join(f"sm_{n}" for n in missing) + " (allowed by --allow-stale-vision)")+ return notes+++def stamp_vision_archs(stage: Path, archs: str) -> None:+ """Record the architectures the picture reader in `stage` was built for as BUILD.json `vision_archs` (the build recipes for Windows+ call this after building strata-vision.exe; check_picture_reader compares it with the package's archs)."""+ path = stage / "BUILD.json"+ try:+ data = json.loads(path.read_text(encoding="utf-8-sig")) if path.is_file() else {}+ except (OSError, ValueError):+ data = {}+ data["vision_archs"] = ";".join(str(n) for n in sorted(_arch_numbers(archs)))+ path.write_text(json.dumps(data, indent=1) + chr(10), encoding="utf-8")++def git_build_id(repo: Path = ROOT) -> str:"""The source commit of the engine build, `-dirty` when the tree had changes. `unknown` outside a checkout."""try:@@@ -164,7 -96,7 +169,8 @@@ def build(src: Path, backend: str, versarchs: tuple[str, ...] = (), min_driver: dict | None = None, build_id: str | None = None,min_runtime: str | None = None, notes: str = "", tree: bool = False, include: list[str] | None = None,variant: str = "", search: list[Path] | None = None, allow_libs: list[str] | None = None,- check_reader: bool = False, allow_stale_vision: bool = False, allow_no_reader: bool = False) -> Path:++ check_reader: bool = False, allow_stale_vision: bool = False, allow_no_reader: bool = False,+ require_dsv41: bool = False) -> Path:if backend not in BACKEND_MATRIX:raise SystemExit(f"unknown backend {backend!r}")if backend != "cpu" and not archs:@@@ -182,10 -109,9 +188,13 @@@entrypoint = entry or ("strata.exe" if target.startswith("windows") else "strata")if not (src / entrypoint).is_file():raise SystemExit(f"{src / entrypoint} does not exist")+ if check_reader: # the release recipes and the CLI ask for it; the library call keeps its old behaviour+ for note in check_picture_reader(src, backend, target, archs, variant=variant, allow_stale=allow_stale_vision,+ allow_none=allow_no_reader):+ print("note: " + note)+ if require_dsv41 and not any((src / n).is_file() for n in (DSV41_EXE, DSV41_EXE + ".exe")):+ raise SystemExit(f"refusing to package: {src} has no {DSV41_EXE}[.exe] (--require-dsv41); build it with -DSTRATA_BUILD_DSV41=ON "+ "(--target strata-dsv41) and copy it beside the engine")folder = out / (f"{version}-{target}-{backend}" + (f"-{variant}" if variant else ""))if folder.exists():raise SystemExit(f"{folder} already exists; remove it first (packages are immutable)")@@@ -295,12 -223,9 +306,14 @@@ def main(argv: list[str] | None = Nonehelp="folder to copy a missing imported DLL from (default on Windows: CUDA_PATH/bin/x64, HIP_PATH/bin, PATH)")ap.add_argument("--allow-lib", action="append", default=[], metavar="NAME",help="an imported library that really is provided by the target system (rare; named so the check passes)")+ ap.add_argument("--require-dsv41", action="store_true",+ help="refuse the package when --src has no strata-dsv41[.exe] (the DeepSeek-V4.1 engine that setup.py looks for beside the engine)")ap.add_argument("--include", action="append", default=[], metavar="GLOB", help="more files, relative to --src")+ ap.add_argument("--stamp-vision", type=Path, metavar="STAGE", help="write --vision-archs into STAGE/BUILD.json and exit")+ ap.add_argument("--vision-archs", default="", help="with --stamp-vision: the architectures the reader was built for, e.g. '60;61;70;75;80;86;89'")+ ap.add_argument("--allow-stale-vision", action="store_true",+ help="package a CUDA picture reader whose BUILD.json vision_archs do not cover the package's archs (it will fail on those cards)")+ ap.add_argument("--no-picture-reader", action="store_true", help="ship a macOS package without strata-vision on purpose (pictures stay off)")a = ap.parse_args(argv)if a.verify:try:@@@ -335,8 -257,7 +351,8 @@@folder = build(a.src, a.backend, a.version, a.platform or host_platform(), a.out, entry=a.entrypoint,archs=tuple(a.arch), min_driver=drivers, build_id=a.build_id, min_runtime=a.min_runtime,notes=a.notes, tree=a.tree, include=a.include, variant=a.variant,- search=a.search, allow_libs=a.allow_lib, require_dsv41=a.require_dsv41)+ search=a.search, allow_libs=a.allow_lib, check_reader=True, allow_stale_vision=a.allow_stale_vision,- allow_no_reader=a.no_picture_reader)++ allow_no_reader=a.no_picture_reader, require_dsv41=a.require_dsv41)entries = [p for p in folder.rglob("*") if p.is_file()]total = sum(p.stat().st_size for p in entries)print(f"{folder} ({total / (1 << 20):.1f} MiB, {len(entries)} files, manifest sha256 {package_sha256(folder)})")diff --cc serve/server.pyindex 87f594592,948e02cd0..cd24086fa--- a/serve/server.py+++ b/serve/server.py@@@ -5027,8 -4751,7 +5060,8 @@@ class Serviceprint(f"[strata] thinking capped at {max_new - reserve} of max_tokens {max_new} to leave room for "f"the answer (budget {budget})", flush=True)budget = max(1, max_new - reserve)- parser = OutputParser(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery,- declared_only=declared_only)- parser = self.parser_cls(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery)++ parser = self.parser_cls(thinking=thinking, tools=tools, stream_tools=True, recover=self.tool_call_recovery,++ declared_only=declared_only)detok, n, finish = Detokenizer(self.tok), 0, "length"run_tok, run_len, repeated = None, 0, False # #606: the current run of one repeated tokenthinking_n = 0 # tokens written while thinking (Responses' reasoning_tokens)diff --cc strata_cli/launch/cmd.pyindex 9fec242ba,3eb04e714..97e3c1314--- a/strata_cli/launch/cmd.py+++ b/strata_cli/launch/cmd.py@@@ -691,10 -638,9 +719,11 @@@ def run_launch(a: argparse.Namespace) -_say(f" model {m.get('title') or 'your model files'}{size} (your files in {s.model_path})")else:_say(f" model {m.get('title') or f'{s.family} {s.model}'}{size}")- _say(f" context {S.context_label(s.context)} pictures {PICTURE_WORDS.get(s.vision or 'off', s.vision)} parallel {s.parallel or 'fit to your card'}")+ _say(f" context {S.context_label(s.context)} pictures {PICTURE_WORDS.get(s.vision or 'off', s.vision)} parallel {('one at a time' if plan.get('placement') else s.parallel or 'fit to your card')}")_say(f" download {(plan.get('download') or {}).get('text', '')}")+ if plan.get("drafting"):+ _say(f" drafting {plan['drafting'].replace('Drafting: ', '', 1)}")+ _placement_lines(plan.get("placement"))_say(f" tuning {s.tune} access {S.ACCESS_WORDS[s.access]}, port {s.port}"+ (" API key on" if s.api_key else ""))for n in notes:diff --cc strata_cli/launch/runner.pyindex 0d2f500cc,ba1f5610b..1d1b5d7e7--- a/strata_cli/launch/runner.py+++ b/strata_cli/launch/runner.py@@@ -1712,28 -1159,53 +1767,69 @@@ class Runnertitle = row.title if row is not None else (f"{s.family} {s.model}" if s.model else None)model = {"title": title, "family": s.family, "model": s.model, "model_gb": row.download_gb if row is not None else None,"own_files": bool(s.model_path), "folder": s.model_path}- return {"profile": self.profile, "settings": s.to_dict(with_key=False), "warnings": warns,- "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download, "drafting": self.drafting_plan_line()}+ out = {"profile": self.profile, "settings": s.to_dict(with_key=False), "warnings": warns,- "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download}++ "flags": S.flags_text(s), "steps": list(STAGES), "model": model, "download": download, "drafting": self.drafting_plan_line()}+ if self.multi_file():+ out["placement"] = self.placement()+ return out+ def drafting_plan_line(self) -> str:+ """The drafting line of --plan: what drafts, or why not what was asked for. Nothing is changed or downloaded."""+ from strata_control import drafting+ try:+ base, cfg = self.installed_state()+ want = self.drafting_inputs(cfg or {})+ return drafting.summary_line(drafting.decide(want["mode"], klass=want["klass"], model_name=want["model_name"], family=want["family"],+ data_dir=want["data_dir"]))+ except Exception: # noqa: BLE001+ return ""++ def dsv41_engine(self) -> bool | None:+ """Whether the installed engine carries strata-dsv41 (True / False); None when no engine is installed yet or it cannot be told."""+ own = (os.environ.get("STRATA_DSV41_EXE") or "").strip()+ if own:+ return Path(own).is_file()+ try:+ from strata_control import engine_component as ec+ rep = ec.status()+ exe = (rep.get("engine") or {}).get("exe") if rep.get("usable") else None+ if not exe:+ return None+ folder = Path(str(exe)).parent+ return any((folder / n).is_file() for n in ("strata-dsv41", "strata-dsv41.exe"))+ except Exception: # noqa: BLE001+ return None++ def placement(self) -> dict:+ """Where DeepSeek-V4.1-Flash would go on this computer's cards (strata_control/dsv41_fit.py): the cards this launch names, else every+ card that counts; the real numbers in plain words, with the n-gram table's SSD note and the RAM / disk warnings (never a refusal)."""+ s = self.s+ hw = self.catalog.hardware()+ try:+ data = Path(s.data_dir) if s.data_dir else self.catalog.data_dir()+ free = None if (s.model_path or self.model_state().remaining_gb < 1) else self.catalog.disk_free(data)+ except Exception: # noqa: BLE001 - a plan that cannot read the disk still shows the rest+ free = None+ p = self.catalog.strata.dsv41_plan(hw, free, s.gpus, model=s.model)+ warnings = list(p["warnings"])+ engine = self.dsv41_engine()+ if engine is False:+ warnings.append("The engine installed here has no DeepSeek-V4.1 engine (strata-dsv41) yet: setup stops with a message before "+ "anything is downloaded. Update Strata, or build it (cmake -DSTRATA_BUILD_DSV41=ON) and set STRATA_DSV41_EXE.")+ return {"verdict": p["verdict"], "lines": p["lines"], "warnings": warnings, "cards": p["cards"], "args": p["args"],+ "engine_has_dsv41": engine,+ "experts_in_vram": p["experts_in_vram"], "experts_total": p["experts_total"], "vram_experts_gb": p["vram_experts_gb"],+ "cpu_experts_gb": p["cpu_experts_gb"], "ram_needed_gb": p.get("ram_needed_gb"), "engram_gb": p["engram_gb"],+ "download_gb": p["download_gb"], "measured": p["measured"]}+def run(self, *, start: bool = True) -> Result:+ self.restart_after = starttry:self.check_own_files() # a wrong --model-path is said before the engine is downloaded+ if start:+ reused = self.check_serving() # before anything is downloaded or stopped+ if reused is not None:+ return reusedself.ensure_engine()self.check_cancel()base = self.ensure_model()