Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Merge f-split-int d6aa6504e into f-split-tuiqa: launch warns and asks (setup stops only for NEED_ENGINE unless launch said go on), the app warns; one picture-reader rule in strata_control/vision_reader.py (CPU reader under 24 GB, GPU reader from 24 GB, every make; pictures stay on) for setup, launch and the app catalog

Niko122195f380769985
diff --cc docs/AMD_HIP.md
index c4fb19435,9402ff22e..d282ad603
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@@ -51,7 -51,12 +51,12 @@@ the kernel's amdgpu driver (no ROCm ins
architectures above; the engine is compiled for each of them (cards of two families, e.g. gfx1100 + gfx1201, need
a system ROCm 7: AMD's wheels hold one family). A split pays only when no single card holds the model's experts
(see RDNA4 below).
- - **Limits for now:** images only through the CPU encoder (`--vision cpu`, 0.1.32). Setup does not offer the tuning
-- **Pictures:** on by default. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on the
++- **Pictures:** on by default; the reader is the CPU one under 24 GB of VRAM and the GPU one from 24 GB (the same rule for every make, `strata_control/vision_reader.py`); `--vision gpu` / `--vision cpu` switch it. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on the
+ card (llama.cpp's HIP backend; about 1.4 GiB of VRAM, measured on an R9700: 0.06 s for a 260-token picture and 0.4 s for 972
+ tokens, against 3.1 s and 14 s on the CPU), and `strata-vision`, the CPU reader. If the GPU reader cannot start (the card is
+ not one it was built for, the HIP runtime does not load, the card is busy) the server starts the CPU reader by itself and
+ prints one line; nothing is asked. `--vision cpu` picks the CPU reader on purpose.
+ - **Limits for now:** Setup does not offer the tuning
(calibration) on AMD yet: its controls are being checked on HIP one at a time (#566). Since 0.1.39 a tuning run by
hand (`./setup.sh --calibrate`) is saved for the AMD card it ran on and reused when setup runs again. The Monitor
shows the card's load, VRAM, temperature and power from Linux sysfs (0.1.32).
diff --cc docs/TERMINAL_LAUNCH.md
index b950d80df,ffb22d40c..a61214f5f
--- a/docs/TERMINAL_LAUNCH.md
+++ b/docs/TERMINAL_LAUNCH.md
@@@ -23,23 -23,19 +23,24 @@@ to your graphics card) comes with the f
2. **Model**: the catalog of the app and `setup.py` (one source), with download size, what it needs (graphics memory + RAM) and "fits / tight / does not fit this computer".
A warning, never a block. The n-gram table stays on your SSD and is not counted. `o` = your own files (a folder or a .gguf; checked with the fingerprint table).
*(Cards: only with two or more usable graphics cards.)*
-3. **Folder**: where model files are stored, free space shown.
+3. **Folder**: where model files are stored, free space shown. For a model that is already installed (also one the desktop app installed) it starts where the model is, says
+ "Already downloaded" and lists only what is really missing with its size (the draft layer, about 5.2 GB; the picture reader, 0.9 GB). It never says "download the model" when only
+ a helper file is missing.
4. **Context** (8K to 512K, with the extra memory each costs) and the memory format (`m`: 8-bit or 4-bit).
-5. **Pictures**: only for models that can read them; says what it adds (about 0.9 GB: the picture-reader file, downloaded with the model through
- setup's Control mode, the same path the app's "Read pictures" uses). On a card under 24 GB and on AMD the picture reader runs on the CPU (nothing taken
- from the graphics card); on a bigger NVIDIA card it runs on the GPU. It needs `strata-vision` beside the engine in the engine package; the screen
- warns in plain words when the installed engine has none, and the run says "Pictures could not be switched on" instead of silently answering text only.
+5. **Pictures**: only for models that can read them. They are **on by default** (Yes is preselected; `--vision off` or "No" turns them off). It says what it adds (about 0.9 GB:
- the picture-reader file, downloaded with the model through setup's Control mode, the same path the app's "Read pictures" uses). The reader runs on the graphics card
- for the NVIDIA (CUDA) and AMD (HIP) builds (it keeps about 1.4 GB free) and on the CPU for the others. It needs `strata-vision` beside the engine in the engine package; with an
++ the picture-reader file, downloaded with the model through setup's Control mode, the same path the app's "Read pictures" uses). The reader runs on the CPU
++ for a card under 24 GB (a few seconds per picture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB kept free), on every make of card; the
++ rule lives in one place, `strata_control/vision_reader.py`, for setup, `strata launch` and the app's catalog. `--vision gpu` / `--vision cpu` (or the Pictures step) switch it per model. It needs a picture reader beside the engine in the engine package; with an
+ engine that has none, pictures default to off and the screen says so in plain words. A model that cannot read pictures stays off with no question.
6. **Parallel requests**: "fit to your card" (12 GB: 1, 20: 2, 24: 3, 32+: 4) or 1 to 4; a warning if it will not fit.
-7. **Tuning**: quick (about 20 minutes) / thorough (up to 2 hours) / skip (`strata tune auto`; the "answers may change very slightly" tick is off).
+7. **Tuning**: quick (about 20 minutes) / thorough (up to 2 hours) / skip (`strata tune auto`; the "answers may change very slightly" tick is off). The screens suggest quick the first
+ time (it is done once and kept); the command line skips tuning unless you add `--tune`.
8. **Advanced** (optional): every engine setting with default, tuned value and your value; edit, reset, add a setting by name (checked against the engine's own list; a warning, not a refusal);
- the exact command and the matching `strata launch` flags.
+ the exact command (it includes the context, memory format and picture choice you made above; it has its own scroll area: Tab, then the arrows) and the matching `strata launch` flags.
9. **Access**: this computer only (127.0.0.1), your network (LAN address, key), Cloudflare quick tunnel (needs `cloudflared`), Tailscale (needs it installed and connected); port; API key
(made for you beyond this computer: the server will not start on a network without one).
-10. **Save as preset** and launch. Then downloads (with progress), engine start, and the monitor.
+10. **Save as preset** and launch. Then downloads (with progress), engine start, and the monitor. In a small window the summary shrinks so the three choices stay on screen,
+ and the key line says what Enter will do. No web page opens by itself; only `--open` opens one.
Keys: arrows choose, Enter goes on, Esc goes back, `?` explains the screen, `q` leaves. Every screen says in plain words what it means and what to do next.
diff --cc setup.py
index 4322fcfab,326663ba4..20cadecd4
--- a/setup.py
+++ b/setup.py
@@@ -2142,7 -2143,7 +2143,7 @@@ def unsloth_images_default(model, famil
return g.get("vram_gb", 0.0) >= UNSLOTH_IMAGES_MIN_VRAM_GB
--IMAGES_GPU_ENCODER_MIN_VRAM_GB = 23.5 # #2406: below this the images answer "yes" gets the CPU encoder (a card of 12-16 GB: the GPU
++IMAGES_GPU_ENCODER_MIN_VRAM_GB = 23.5 # = strata_control/vision_reader.py GPU_MIN_VRAM_GB, the one place for the rule; #2406: below this the images answer "yes" gets the CPU encoder (a card of 12-16 GB: the GPU
# encoder's ~1.1-1.4 GB is ~850 expert slots lost, 3-5% of decode and prompt on P100 / A4000
# 16 GB (Linux, IQ3_S / IQ3_XXS), and 35-60% on an RTX 4080 SUPER under Windows as reported)
@@@ -3382,17 -3384,74 +3384,74 @@@ def build_engine_hip(gpu, llama, vision
return eng
--def hip_vision(asked) -> str:
- """The image encoder with the AMD backend (--vision): the CPU one when asked for (#304); a HIP (GPU) encoder build
- is a later step, so `yes`/`gpu` leave images off, as before, and say how to get them."""
- if asked in ("yes", "gpu"):
- warn("the AMD backend has no GPU image encoder yet: images off"
- + ("" if WIN else " (--vision cpu reads them on the CPU)"))
- if asked == "cpu" and WIN:
- warn("images on the CPU with an AMD card are Linux-only for now (the ready-made Windows AMD engine has no "
- "image encoder): images off")
- """The picture reader wanted with the AMD backend (--vision): "gpu" for yes/gpu and when nothing was asked (pictures are on
- by default), "cpu" when asked for (#304), "none" for no. "gpu" is a wish: hip_reader() settles it against what the
- engine folder carries (the HIP reader, else the CPU one) once the engine is known."""
- if asked in ("no", "none"):
-- return "none"
- return "cpu" if asked == "cpu" else "none"
- return "cpu" if asked == "cpu" else "gpu"
++def hip_vision(asked, vram_gb: float = 0.0) -> str:
++ """The picture reader wanted with the AMD backend (--vision): when nothing was asked or "yes" (pictures are on by default) the
++ default of strata_control/vision_reader.py for this card - the CPU reader under 24 GB, the GPU reader from 24 GB (owner decision
++ 2026-10-18, every make of card); "gpu" / "cpu" as asked (#304), "none" for no. "gpu" is a wish: hip_reader() settles it against
++ what the engine folder carries (the HIP reader, else the CPU one) once the engine is known."""
++ from strata_control import vision_reader as VR
++ return VR.wanted(asked, vram_gb)
+
+
+ def reader_archs(eng: Path) -> list:
+ """The GPU architectures the engine folder's HIP picture reader was built for: its BUILD.json (vision_archs, else archs),
+ else the component manifest's archs; [] when neither says (the reader then checks the card itself when it starts)."""
+ for name, keys in (("BUILD.json", ("vision_archs", "archs")), ("engine-component.json", ("archs",))):
+ try:
+ data = json.loads((eng / name).read_text(encoding="utf-8-sig"))
+ except (OSError, ValueError):
+ continue
+ for key in keys:
+ if isinstance(data, dict) and isinstance(data.get(key), list) and data[key]:
+ return [str(x) for x in data[key]]
+ return []
+
+
+ def hip_reader(eng: Path, vision: str, gpu: dict):
+ """AMD: the picture reader this engine folder gets for `vision` ("gpu" or "cpu"; strata_control/vision_reader.py): the HIP
+ reader with the CPU reader to fall back on, else the CPU reader. Returns (vision, Reader); a Reader of mode "none" means
+ the folder has no reader. The one plain note (a card the HIP reader has no code for, no HIP reader in this engine) is said."""
+ from strata_control import vision_reader as VR
+ r = VR.choose(eng, vision, windows=WIN, gpu_arch=str(gpu.get("arch") or ""), archs=reader_archs(eng))
+ if r.note:
+ warn(r.note)
+ return r.mode, r
+
+
+ def build_vision_hip(eng: Path, stamp: Path, meta: dict, llama, vsrc, archs: list, vision: str) -> Path:
+ """The picture readers beside the HIP engine: the CPU reader always (the fallback), and for "gpu" the HIP reader
+ (tools/vision, STRATA_VISION_HIP) for the same architectures. The HIP reader needs the ROCm environment the engine build
+ sets; a build that fails leaves the CPU reader and says so (pictures still work, on the CPU)."""
+ if vision == "gpu" and not (shutil.which("hipconfig") or os.environ.get("ROCM_PATH")):
+ root, dirs = rocm_root(archs) # the engine was built earlier: set the same environment again
+ os.environ.update({"HIP_PLATFORM": "amd", "HIP_COMPILER": "clang", "HIP_RUNTIME": "rocclr", "ROCM_PATH": str(root),
+ "HIP_PATH": str(root)})
+ os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(dirs + [os.environ.get("LD_LIBRARY_PATH", "")]).rstrip(os.pathsep)
+ os.environ["PATH"] = os.pathsep.join([str(root / "bin"), str(root / "llvm" / "bin"), os.environ.get("PATH", "")])
+ build_vision_cpu(eng, stamp, meta, llama, vsrc)
+ meta = json.loads(stamp.read_text(encoding="utf-8"))
+ if vision != "gpu":
+ return eng
+ root, dirs = rocm_root(archs)
+ libs = [str(Path(d).parent) for d in dirs[1:]]
+ bitcode = next((p for p in (root / "lib" / "llvm" / "amdgcn" / "bitcode", root / "amdgcn" / "bitcode") if p.is_dir()),
+ root / "amdgcn" / "bitcode")
+ say(" Compiling the picture reader for your AMD GPU (a few minutes, once) ...")
+ try:
+ cmake_build(ROOT / "tools" / "vision", ROOT / "build-vision-hip", "strata-vision",
+ [f"-DLLAMA_DIR={llama}", "-DSTRATA_VISION_HIP=ON", "-DSTRATA_VISION_HIP_ARCHS=" + ";".join(archs),
+ "-DSTRATA_PORTABLE=OFF", f"-DCMAKE_HIP_COMPILER={root / 'llvm' / 'bin' / 'clang++'}",
+ f"-DCMAKE_HIP_COMPILER_ROCM_ROOT={root}", "-DCMAKE_PREFIX_PATH=" + ";".join([str(root), *libs]),
+ f"-DCMAKE_HIP_FLAGS=--rocm-path={root} --rocm-device-lib-path={bitcode}"], None, "")
+ shutil.copy2(ROOT / "build-vision-hip" / "bin" / HVEXE, eng / HVEXE)
+ except SystemExit:
+ raise
+ except Exception as e: # noqa: BLE001 - the CPU reader is there; pictures work without the GPU one
+ warn(f"the GPU picture reader did not compile ({e}): pictures are read on the CPU")
+ return eng
+ stamp.write_text(json.dumps({**meta, "vision": "gpu", "vision_archs": archs, "vision_gpu": HVEXE, "vision_cpu": VEXE}, indent=1))
+ ok(f"picture reader (GPU): {eng / HVEXE}, (CPU fallback): {eng / VEXE}")
+ return eng
def build_vision_cpu(eng: Path, stamp: Path, meta: dict, llama, vsrc) -> Path:
@@@ -5938,9 -5997,9 +5997,10 @@@ def control_engine_step(gpu, vision)
ok(f"engine component: {found.version} ({found.platform}-{found.backend}) in {eng}")
except EC.EngineMissing as e:
plain = getattr(e, "plain_notice", None) or getattr(e, "driver_notice", None)
- if (plain or os.environ.get("STRATA_NEED_ENGINE") == "1") and os.environ.get("STRATA_ALLOW_NO_ENGINE") != "1":
- # The model is tens of GB: when the engine is a dead end for this PC (a build for another card family, a driver that is too
- # old) or the caller (strata launch) needs one now, stop before step 5 instead of after the download.
- if os.environ.get("STRATA_NEED_ENGINE") == "1":
- # strata launch needs an engine now: stop before step 5 instead of after the download. The app does not stop (owner
- # decision 2026-10-18): it says so in its download card and still downloads the model.
++ if os.environ.get("STRATA_NEED_ENGINE") == "1" and os.environ.get("STRATA_ALLOW_NO_ENGINE") != "1":
++ # strata launch needs an engine now: stop before step 5 instead of after the download, unless launch itself asked to go on
++ # (STRATA_ALLOW_NO_ENGINE: the person said yes to "download the model anyway"). The app does not stop (owner decision
++ # 2026-10-18): it says so in its download card and still downloads the model.
fail(plain or "no engine is installed for this PC",
"get the engine that matches this graphics card (`strata update`, or the app's Settings), then run this again; "
"nothing was downloaded and files already downloaded are kept")
@@@ -6033,18 -6092,12 +6093,12 @@@ def own_path_vision(a, fam: dict, gpu:
" and the image encoder runs on the GPU (~1.4 GB of VRAM kept free, 0.1-0.5 s per picture)." if big else
"; the encoder runs on the CPU (a few seconds per picture), because the GPU encoder would take ~1.4 GB "
"of VRAM from the experts on this card."))
- default = "n" if (hip and WIN) else "y" if (big or hip) else "n"
+ default = "y" if (big or hip) else "n"
if ask("Do you want images?", ["y", "n"], default, a.yes) != "y":
return "none"
- asked = "cpu" if (hip or not big) else "gpu"
- asked = "gpu" if hip else "cpu" if not big else "gpu"
++ asked = "yes" if hip else "cpu" if not big else "gpu"
if hip:
- if asked == "gpu":
- say(" The AMD backend has no GPU image encoder yet: the encoder runs on the CPU.")
- if WIN:
- warn("images on the CPU with an AMD card are Linux-only for now (the ready-made Windows AMD engine has no "
- "image encoder): images off")
- return "none"
- return "cpu"
- return hip_vision(asked) # settled against the engine's readers in hip_reader() (GPU reader, else CPU)
++ return hip_vision(asked, vram) # settled against the engine's readers in hip_reader() (GPU reader, else CPU)
if asked == "yes":
asked = "gpu" if big else "cpu"
if asked == "gpu" and not big and vram > 0:
@@@ -7125,7 -7177,7 +7185,7 @@@ def main() -> int
# MODEL-UPDATE: Unsloth's files on AMD (Linux) read images with the CPU encoder by default where their experts
# all stay in RAM (it takes no VRAM; measured on an R9700 with UD-Q4_K_XL and UD-Q5_K_XL, docs/QUANTS.md)
vision = hip_vision(a.vision if a.vision is not None or not unsloth_images_default(model, family, ram, gpu)
-- else "cpu")
++ else "cpu", float((gpu or {}).get("vram_gb") or 0.0))
elif a.vision:
vision = flash_vision_encoder(a.vision, gpu) if a.vision != "no" else "none"
vision = "none" if vision in ("no", "none") else vision
diff --cc strata_cli/launch/catalog.py
index 424c8443d,2402edc9f..33041493b
--- a/strata_cli/launch/catalog.py
+++ b/strata_cli/launch/catalog.py
@@@ -386,45 -349,10 +389,49 @@@ class Catalog
pass
return rec
+ # ---- pictures
+ def engine_backend(self) -> str | None:
+ """The installed engine's backend (cuda, hip, sycl, cpu, metal), else the one this computer's graphics card needs, else None."""
+ installed = self.installed_backend()
+ if installed:
+ return installed
+ for g in self.computer().usable_gpus:
+ v = str(g.get("vendor") or "").lower()
+ if g.get("unified_gb") or v == "apple":
+ return "metal"
+ return {"nvidia": "cuda", "amd": "hip", "intel": "sycl"}.get(v)
+ return None
+
+ def reader_state(self) -> str:
+ """"present" when the installed engine carries the picture reader (strata-vision), "missing" when an engine is installed without it,
+ "unknown" before any engine is installed (the first launch fetches one that has it)."""
+ try:
+ from strata_control import engine_component as ec
+ rep = ec.status()
+ exe = (rep.get("engine") or {}).get("exe") if rep.get("usable") else None
+ except Exception: # noqa: BLE001
+ exe = None
+ if not exe:
+ return "unknown"
- folder = Path(str(exe)).parent
- return "present" if any((folder / n).is_file() for n in ("strata-vision", "strata-vision.exe")) else "missing"
++ from strata_control import vision_reader as VR
++ found = VR.find(Path(str(exe)).parent, os.name == "nt")
++ return "present" if (found["cpu"] or found["gpu"]) else "missing"
+
+ def picture_default(self, images: bool) -> str:
+ """The picture setting nobody chose: off for a model that cannot read pictures or an engine without the reader; else on, with the
- reader on the graphics card for the CUDA and HIP builds and on the CPU for the others."""
++ reader the one rule of strata_control/vision_reader.py gives: the CPU reader under 24 GB of graphics memory, the GPU reader from 24 GB,
++ on every make of card (owner decision 2026-10-18). Without a usable graphics card there is nothing to read on: the CPU reader."""
++ from strata_control import vision_reader as VR
+ if not images or self.reader_state() == "missing":
+ return "off"
- return "gpu" if self.engine_backend() in ("cuda", "hip") else "cpu"
++ vram = self.computer().main_vram_gb
++ return VR.default_mode(vram) if vram > 0 else "cpu" # a Mac (unified memory) or no card: nothing to keep free, the CPU reader
+
# ---- installed models
def installed_configs(self) -> list[dict]:
- out = []
+ out, seen = [], set()
for path in self.strata.configs():
+ seen.add(path.name)
d = self.strata.describe_config(path)
d["path"] = str(path)
out.append(d)
diff --cc strata_cli/launch/screens.py
index bba27cbaa,7b76dba70..9e762bd90
--- a/strata_cli/launch/screens.py
+++ b/strata_cli/launch/screens.py
@@@ -838,14 -819,16 +838,18 @@@ class PicturesScreen(Base)
self.paint()
def explain(self) -> Text:
- vram = self.ctx.computer.main_vram_gb
+ amd = any(str(g.get("vendor")) == "amd" for g in self.ctx.computer.usable_gpus)
- extra = ("a 0.9 GB download. On your AMD card the picture reader runs on the graphics card (about 1 GB of its memory, under a second "
- "per picture) and starts on the processor by itself if it cannot." if amd and self.reader() != "missing" else
- "a 0.9 GB download. On a card under 24 GB the picture reader runs on the CPU (a few seconds per picture, nothing "
- "taken from the graphics card). On a bigger card it runs on the GPU and keeps about 1.4 GB free, costing a few % of text speed."
- if vram < 24 else "a 0.9 GB download, and about 1.4 GB of graphics memory kept free for the picture reader (text is a few % slower).")
+ on_card = self.ctx.catalog.picture_default(True) == "gpu"
- extra = ("a 0.9 GB download (if it is not on this computer yet), and about 1.4 GB of graphics memory kept free for the picture reader "
++ extra = ("a 0.9 GB download (if it is not on this computer yet). On your AMD card the picture reader runs on the graphics card "
++ "(about 1 GB of its memory, under a second per picture) and starts on the processor by itself if it cannot."
++ if amd and on_card and self.reader() != "missing" else
++ "a 0.9 GB download (if it is not on this computer yet), and about 1.4 GB of graphics memory kept free for the picture reader "
+ "(text is a few % slower)." if on_card else
+ "a 0.9 GB download (if it is not on this computer yet). The picture reader runs on the CPU: a few seconds per picture, nothing "
+ "taken from the graphics card.")
if not self.supported:
return Text("This model cannot read pictures yet, so pictures stay off. Pick another model if you need them.", style=WARN)
- out = Text("Pictures add " + extra + " Say No if you only chat in text.", style=MUTE)
+ out = Text("Pictures are on by default. They add " + extra + " Say No if you only chat in text.", style=MUTE)
if self.reader() == "missing":
out.append("\nHeads up: the engine installed here does not carry the picture reader, so the model would answer text only. "
"Update the engine, or say No.", style=WARN)
diff --cc strata_control/vision_reader.py
index 000000000,cd0001941..143445fe9
mode 000000,100644..100644
--- a/strata_control/vision_reader.py
+++ b/strata_control/vision_reader.py
@@@ -1,0 -1,101 +1,123 @@@
+ """Which picture reader an engine folder gets: the GPU one (`strata-vision-hip`, AMD) or the CPU one (`strata-vision`).
+
+ An AMD engine package carries two programs for pictures. `strata-vision` is the CPU reader: it needs nothing from the GPU
+ runtime, so it starts on every PC. `strata-vision-hip` is the same reader on the card (llama.cpp's HIP backend), a
+ fraction of a second per picture instead of seconds. Setup writes the run config with the GPU reader first and the CPU
+ reader as its fallback (`vision.cpu_exe`); the server (serve/server.py, Vision) starts the CPU reader by itself, with one
+ log line, when the GPU reader does not start: the HIP runtime does not load, the card is not one the build has code for
+ (the reader says so before it runs a kernel), or the card is busy. Nothing is asked of the user.
+
+ `choose` is the whole decision; setup.py calls it, and tools/test_vision_reader.py checks every branch.
+ """
+ from __future__ import annotations
+
+ import re
+ from dataclasses import dataclass
+ from pathlib import Path
+
+ CPU_READER = "strata-vision"
+ GPU_READER = "strata-vision-hip"
+
++#: Owner decision 2026-10-18: pictures are ON by default, and the reader runs on the CPU for a card under 24 GB (the GPU reader keeps about
++#: 1-1.4 GB of VRAM that the experts would use: a few % of decode, more on some PCs, #2406) and on the GPU for 24 GB and above - on every make
++#: of card. A person can switch it per model (--vision gpu / --vision cpu, the Pictures step); what was asked for is kept.
++GPU_MIN_VRAM_GB = 23.5
++
++
++def default_mode(vram_gb: float | None) -> str:
++ """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu")."""
++ vram = float(vram_gb or 0.0)
++ return "cpu" if 0 < vram < GPU_MIN_VRAM_GB else "gpu"
++
++
++def wanted(asked: str | None, vram_gb: float | None) -> str:
++ """What is wanted for `asked` (None, "yes", "on", "gpu", "cpu", "no"/"none"/"off"): "gpu", "cpu" or "none". Nothing asked and "yes" mean
++ the default for this card; "gpu" and "cpu" are kept as asked."""
++ a = (asked or "").lower()
++ if a in ("no", "none", "off"):
++ return "none"
++ if a in ("gpu", "cpu"):
++ return a
++ return default_mode(vram_gb)
++
+ GFX = re.compile(r"gfx[0-9a-f]+", re.IGNORECASE)
+
+
+ def reader_name(base: str, windows: bool) -> str:
+ return base + (".exe" if windows else "")
+
+
+ def find(folder: Path | str, windows: bool) -> dict:
+ """The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}."""
+ folder = Path(folder)
+ found = {}
+ for key, base in (("cpu", CPU_READER), ("gpu", GPU_READER)):
+ path = folder / reader_name(base, windows)
+ found[key] = path if path.is_file() else None
+ return found
+
+
+ def gfx_of(text: object) -> str:
+ """"gfx1201" out of a gcnArchName ("gfx1201:sramecc-:xnack-"), a device line or a plain arch; "" when there is none."""
+ m = GFX.search(str(text or ""))
+ return m.group(0).lower() if m else ""
+
+
+ @dataclass
+ class Reader:
+ mode: str # "gpu" | "cpu" | "none"
+ exe: Path | None = None # the program the run config starts
+ cpu_exe: Path | None = None # the CPU reader to fall back on when a GPU reader does not start (mode "gpu" only)
+ note: str = "" # one plain sentence for the setup output when it is not what was asked for
+
+
+ def choose(folder: Path | str, asked: str, *, windows: bool, gpu_arch: str = "", archs: list | None = None) -> Reader:
+ """The reader for `asked` ("yes"/"on"/"gpu" = the best the folder has, "cpu", "no"/"none"/"off").
+
+ gpu_arch: the card's architecture when setup knows it ("gfx1201"); archs: what the GPU reader was built for
+ (BUILD.json `vision_archs`, else the package's `archs`). A card the build does not list gets the CPU reader here;
+ when the card is unknown the GPU reader is tried and falls back by itself (see the module doc)."""
+ asked = (asked or "").lower()
+ if asked in ("no", "none", "off", ""):
+ return Reader("none")
+ have = find(folder, windows)
+ if asked == "cpu":
+ if have["cpu"]:
+ return Reader("cpu", have["cpu"])
+ return Reader("none", note="this engine has no picture reader: pictures off")
+ # yes / on / gpu
+ gfx = gfx_of(gpu_arch)
+ built = [gfx_of(a) for a in (archs or []) if gfx_of(a)]
+ if have["gpu"] is None:
+ if have["cpu"]:
+ return Reader("cpu", have["cpu"], note="this engine has no GPU picture reader: pictures are read on the CPU "
+ "(a few seconds each)" if asked == "gpu" else "")
+ return Reader("none", note="this engine has no picture reader: pictures off")
+ if gfx and built and gfx not in built:
+ if have["cpu"]:
+ return Reader("cpu", have["cpu"], note=f"the GPU picture reader has no code for your card ({gfx}): pictures "
+ "are read on the CPU (a few seconds each)")
+ return Reader("none", note=f"the GPU picture reader has no code for your card ({gfx}): pictures off")
+ return Reader("gpu", have["gpu"], have["cpu"])
+
+
+ def vision_section(reader: Reader, base: dict) -> dict:
+ """The run config's "vision" section for `reader` on top of `base` (mmproj, model, max_tokens ...): exe, gpu, and for the
+ GPU reader the CPU reader to fall back on."""
+ out = dict(base)
+ out["exe"] = str(reader.exe)
+ out["gpu"] = reader.mode == "gpu"
+ if reader.mode == "gpu" and reader.cpu_exe:
+ out["cpu_exe"] = str(reader.cpu_exe)
+ else:
+ out.pop("cpu_exe", None)
+ return out
+
+
+ def cpu_fallback(vision_cfg: dict) -> dict | None:
+ """The same section for the CPU reader, when the GPU reader has one to fall back on; None otherwise."""
+ if not vision_cfg.get("gpu") or not vision_cfg.get("cpu_exe"):
+ return None
+ out = {k: v for k, v in vision_cfg.items() if k not in ("cpu_exe", "elastic", "elastic_mib", "elastic_idle_s")}
+ out["exe"], out["gpu"] = vision_cfg["cpu_exe"], False
+ return out
diff --cc sycl/setup_intel.py
index 2c1f8c2a8,2c1f8c2a8..07a791012
--- a/sycl/setup_intel.py
+++ b/sycl/setup_intel.py
@@@ -294,7 -294,7 +294,7 @@@ def install(argv) -> None
S.gpus = lambda *a, **k: []
S.amd_gpus = lambda *a, **k: intel
S.amd_problem = lambda g: None
-- S.hip_vision = lambda asked: "none" # images are not wired on the SYCL port yet
++ S.hip_vision = lambda asked, vram_gb=0.0: "none" # images are not wired on the SYCL port yet
S.build_engine_hip = lambda *a, **k: stub
S.hipblaslt_table = lambda *a, **k: None
S.ram_gb = lambda: fake_ram # the experts are in VRAM: setup's RAM rule does not apply
diff --cc tests/terminal/test_launch.py
index 5121be785,819f8548c..68f9c355f
--- a/tests/terminal/test_launch.py
+++ b/tests/terminal/test_launch.py
@@@ -295,44 -294,6 +295,48 @@@ class SetupArgs(unittest.TestCase)
s = S.LaunchSettings(family="qwen", model="IQ3_XXS", context=32768, kv="q4_0", **kw)
return Runner(s, catalog=cat).setup_args()
- def test_pictures_default_on_the_card_for_nvidia_and_amd_and_on_the_cpu_for_the_rest(self):
++ def test_pictures_default_on_the_cpu_reader_under_24_gb_and_the_gpu_reader_from_24_gb_on_every_make(self):
+ from tests.terminal.launch_support import HW_12 as NV
- amd = {**NV, "gpus": [{"index": 0, "name": "AMD Radeon RX 9070 XT", "vram_gb": 16.0, "usable": True, "vendor": "amd"}]}
- intel = {**NV, "gpus": [{"index": 0, "name": "Intel Arc B580", "vram_gb": 12.0, "usable": True, "vendor": "intel"}]}
++ card = lambda name, gb, vendor: {**NV, "gpus": [{"index": 0, "name": name, "vram_gb": gb, "usable": True, "vendor": vendor}]} # noqa: E731
++ amd = card("AMD Radeon RX 9070 XT", 16.0, "amd")
++ intel = card("Intel Arc B580", 12.0, "intel")
+ with Home():
- for hw, want in ((NV, "gpu"), (amd, "gpu"), (intel, "cpu")):
++ for hw, want in ((NV, "cpu"), (amd, "cpu"), (intel, "cpu"), (card("NVIDIA GeForce RTX 4090", 24.0, "nvidia"), "gpu"),
++ (card("AMD Radeon AI PRO R9700", 32.0, "amd"), "gpu")):
+ cat = catalog(hw)
+ r = Runner(S.LaunchSettings(family="qwen", model="IQ3_XXS", context=32768), catalog=cat)
+ self.assertEqual(r.setup_args()[r.setup_args().index("--vision") + 1], want, hw["gpus"][0]["vendor"])
+ self.assertEqual(r.s.vision, want)
+ off = Runner(S.LaunchSettings(family="qwen", model="IQ3_XXS", context=32768, vision="off"), catalog=catalog(NV))
+ self.assertEqual(off.setup_args()[off.setup_args().index("--vision") + 1], "no")
+
+ def test_a_model_that_cannot_read_pictures_stays_off_without_a_question(self):
+ import dataclasses
+ with Home():
+ cat = catalog(HW_12)
+ real = cat.rows()
+ rows = [dataclasses.replace(r, images=False) for r in real]
+ with mock.patch.object(type(cat), "rows", lambda self, *a, **k: rows):
+ r = Runner(S.LaunchSettings(family="qwen", model="IQ3_XXS", context=32768), catalog=cat)
+ self.assertEqual(r.resolved_vision(), "off")
+
+ def test_an_engine_without_the_reader_defaults_to_off(self):
+ with Home():
+ cat = catalog(HW_12)
+ with mock.patch.object(type(cat), "reader_state", lambda self: "missing"):
+ self.assertEqual(cat.picture_default(True), "off")
+
+ def test_the_command_defaults_pictures_on_and_off_still_turns_them_off(self):
+ from strata_cli.launch.cmd import apply_defaults
+ with Home():
+ cat = catalog(HW_12)
+ s, _ = apply_defaults(S.LaunchSettings(family="qwen", model="IQ3_XXS"), cat)
- self.assertEqual(s.vision, "gpu")
++ self.assertEqual(s.vision, "cpu") # a 12 GB card: pictures ON, read on the CPU
++ s, _ = apply_defaults(S.LaunchSettings(family="qwen", model="IQ3_XXS", vision="gpu"), cat)
++ self.assertEqual(s.vision, "gpu") # what was asked for is kept
+ s, _ = apply_defaults(S.LaunchSettings(family="qwen", model="IQ3_XXS", vision="off"), cat)
+ self.assertEqual(s.vision, "off")
+
def test_the_apps_own_installer_arguments(self):
import tempfile
own = tempfile.mkdtemp()
diff --cc tests/terminal/test_launch_engine_first.py
index 8ce209203,a2a9fcfac..371a3cb10
--- a/tests/terminal/test_launch_engine_first.py
+++ b/tests/terminal/test_launch_engine_first.py
@@@ -221,17 -147,20 +221,31 @@@ class SetupControlStep(unittest.TestCas
self.assertIn("Ask for the RX 9000 build.", out)
self.assertIn("nothing was downloaded", out)
+ def test_wrong_family_goes_on_with_a_warning_when_launch_said_go_on(self):
+ exc = ec.EngineMissing("nothing matched", Path("/c"), notice=WRONG_FAMILY)
+ res, out = self.step(exc, {"STRATA_ALLOW_NO_ENGINE": "1"})
+ self.assertNotIsInstance(res, SystemExit)
+ self.assertIn("no engine is installed for this PC yet", out)
+ self.assertIn("The model is downloaded and prepared now", out)
+
+ def test_no_engine_for_launch_still_stops_setup_when_run_on_its_own(self):
+ res, out = self.step(ec.EngineMissing("the component folder holds no engine package", Path("/c")), {"STRATA_NEED_ENGINE": "1"})
+ self.assertIsInstance(res, SystemExit)
+
+ def test_the_app_with_a_wrong_family_engine_warns_and_still_downloads(self):
+ """Owner decision 2026-10-18: the app downloads the model anyway and warns (its card says so)."""
+ env = {k: v for k, v in os.environ.items() if k != "STRATA_NEED_ENGINE"}
+ exc = ec.EngineMissing("nothing matched", Path("/c"), notice=WRONG_FAMILY)
+ with mock.patch.dict(os.environ, env, clear=True):
+ res, out = self.step(exc, {})
+ self.assertNotIsInstance(res, SystemExit)
+ self.assertIn("Ask for the RX 9000 build.", out)
+ self.assertIn("The model is downloaded and prepared now", out)
+
+ def test_no_engine_stops_before_the_download_for_launch(self):
+ res, out = self.step(ec.EngineMissing("the component folder holds no engine package", Path("/c")), {"STRATA_NEED_ENGINE": "1"})
+ self.assertIsInstance(res, SystemExit)
+
def test_the_app_without_an_engine_keeps_its_order(self):
env = {k: v for k, v in os.environ.items() if k != "STRATA_NEED_ENGINE"}
with mock.patch.dict(os.environ, env, clear=True):
diff --cc tests/terminal/test_launch_tui.py
index a9aba6f6a,51ec302ee..2ef2ba608
--- a/tests/terminal/test_launch_tui.py
+++ b/tests/terminal/test_launch_tui.py
@@@ -422,12 -422,10 +422,12 @@@ class PicturesParallelTuning(Base)
await press(pilot, *self.reach_pictures())
t = text_of(app)
self.assertIn("No pictures", t)
+ self.assertIn("Pictures are on by default", flat(app))
self.assertIn("0.9 GB download", t)
- self.assertIn("graphics memory kept free", flat(app)) # the NVIDIA reader runs on the card
- self.assertIn("runs on the CPU", t) # a 12 GB card
- await press(pilot, "down", "enter")
- self.assertEqual(ctx.settings.vision, "on")
++ self.assertIn("runs on the CPU", flat(app)) # a 12 GB card: the CPU reader (the GPU one from 24 GB)
+ self.assertEqual(app.screen.menu().index, 1) # "Yes" is preselected
+ await press(pilot, "enter")
- self.assertEqual(ctx.settings.vision, "gpu")
++ self.assertEqual(ctx.settings.vision, "cpu")
self.assertIn("How many requests at the same time?", text_of(app))
self.run_app(case)
diff --cc tools/strata_mcp.py
index 120f7c4e7,ae2dcdc27..ae35b3276
--- a/tools/strata_mcp.py
+++ b/tools/strata_mcp.py
@@@ -986,6 -1059,6 +1089,9 @@@ class Strata
sizes.append({"model": m, "about": d["about"], "download_gb": d["download_gb"],
"experimental": bool(d.get("experimental")),
"images": d.get("vision", fd.get("vision", True)) is not False,
++ # which picture reader "pictures on" gets on this PC (strata_control/vision_reader.py, the one rule: the CPU reader
++ # under 24 GB of graphics memory, the GPU reader from 24 GB; the person can switch it per model)
++ "images_reader": self.images_reader(vram),
"ram_needed_gb": d["ram_gb"], "experts_gb": d.get("arena_gb"),
# RAM + VRAM the weights need together, WITHOUT the n-gram table (that stays on the SSD): Gyro states it
# (mem_total), for the rest the experts plus ~1 GB for the dense part (Swift IQ2_XS: 35.5 + 1 = 36.5)
@@@ -1004,6 -1077,6 +1110,14 @@@
"sizes": sizes})
return out
++ @staticmethod
++ def images_reader(vram) -> str:
++ try:
++ from strata_control import vision_reader
++ return vision_reader.default_mode(vram) if vram else "cpu"
++ except Exception: # noqa: BLE001
++ return "cpu"
++
@staticmethod
def gpu_options(d: dict, ram, hw) -> dict | None:
try:
@@@ -1599,13 -1770,13 +1816,13 @@@ class Tools
gpu = gpus = kv = low_ram = None
context = context or (s.mac().recommend_context(hw.get("ram_gb") or 0) if s.mac() else 32768)
if context is None:
- usable = [g for g in hw.get("gpus", []) if g.get("usable")]
+ usable = s.engine_gpus(hw)
vram = max((g["vram_gb"] for g in usable), default=12)
context = 32768 if vram < 14 else 65536 if vram < 20 else 131072
- if models[model].get("budget") and (not hasattr(s.setup_module(), "unsloth_budget") or
+ if md.get("budget") and (not hasattr(s.setup_module(), "unsloth_budget") or
s.setup_module().unsloth_budget(model, hw.get("ram_gb") or 0.0)):
context = 8192 if vram < 14 else 32768
- if vision in ("yes", "gpu", "cpu") and models[model].get("vision", families[family].get("vision")) is False:
+ if vision in ("yes", "gpu", "cpu") and md.get("vision", families[family].get("vision")) is False:
raise ToolError(f"images are not available with {families[family]['title']} {model} yet: use vision=no")
if vision is not None and backend == "hip" and vision != "no":
# setup.py's hip_vision rules (#990): the AMD backend has no GPU image encoder, but --vision cpu reads
diff --cc tools/test_setup_amd.py
index 6c0253e45,e8a75ecdc..3e1001b8d
--- a/tools/test_setup_amd.py
+++ b/tools/test_setup_amd.py
@@@ -429,13 -429,37 +429,44 @@@ class CalibrationKey(unittest.TestCase)
self.assertIsNone(setup.setup_calibration(cfg, hip=True)) # Windows HIP: defaults for now
- class WindowsHipVision(unittest.TestCase):
- def test_no_cpu_encoder_on_windows(self):
- with mock.patch.object(setup, "WIN", True), mock.patch.object(setup, "warn", lambda *a: None):
- self.assertEqual(setup.hip_vision("cpu"), "none")
- self.assertEqual(setup.hip_vision("yes"), "none")
- with mock.patch.object(setup, "WIN", False), mock.patch.object(setup, "warn", lambda *a: None):
- self.assertEqual(setup.hip_vision("cpu"), "cpu")
+ class HipVisionWish(unittest.TestCase):
+ """AMD pictures are on by default, on Linux and Windows: the wish is "gpu" (hip_reader() settles it against the readers the
+ engine folder carries: the HIP one, else the CPU one), "cpu" when asked for, "none" for no."""
+
+ def test_wish(self):
+ for win in (True, False):
+ with mock.patch.object(setup, "WIN", win):
+ self.assertEqual(setup.hip_vision(None), "gpu")
+ self.assertEqual(setup.hip_vision("yes"), "gpu")
+ self.assertEqual(setup.hip_vision("gpu"), "gpu")
+ self.assertEqual(setup.hip_vision("cpu"), "cpu")
+ self.assertEqual(setup.hip_vision("no"), "none")
+ self.assertEqual(setup.hip_vision("none"), "none")
++ # owner decision 2026-10-18: nothing asked / yes = the CPU reader under 24 GB, the GPU reader from 24 GB; gpu / cpu are kept
++ self.assertEqual(setup.hip_vision(None, 16.0), "cpu")
++ self.assertEqual(setup.hip_vision("yes", 16.0), "cpu")
++ self.assertEqual(setup.hip_vision("yes", 23.9), "gpu")
++ self.assertEqual(setup.hip_vision(None, 32.0), "gpu")
++ self.assertEqual(setup.hip_vision("gpu", 16.0), "gpu")
++ self.assertEqual(setup.hip_vision("cpu", 32.0), "cpu")
+
+ def test_reader_settled_against_the_engine_folder(self):
+ for win, suffix in ((False, ""), (True, ".exe")):
+ with tempfile.TemporaryDirectory() as d, mock.patch.object(setup, "WIN", win), mock.patch.object(setup, "warn", lambda *a: None):
+ eng = Path(d)
+ (eng / ("strata-vision" + suffix)).write_bytes(b"x")
+ self.assertEqual(setup.hip_reader(eng, "gpu", {"arch": "gfx1201"})[0], "cpu") # no HIP reader: the CPU one
+ (eng / ("strata-vision-hip" + suffix)).write_bytes(b"x")
+ mode, r = setup.hip_reader(eng, "gpu", {"arch": "gfx1201"})
+ self.assertEqual((mode, r.exe.name, r.cpu_exe.name),
+ ("gpu", "strata-vision-hip" + suffix, "strata-vision" + suffix))
+ (eng / "BUILD.json").write_text('{"vision_archs": ["gfx1100"]}')
+ self.assertEqual(setup.hip_reader(eng, "gpu", {"arch": "gfx1201"})[0], "cpu") # the card is not in its build
+ self.assertEqual(setup.hip_reader(eng, "gpu", {"arch": "gfx1100"})[0], "gpu")
+ self.assertEqual(setup.hip_reader(eng, "cpu", {"arch": "gfx1100"})[0], "cpu")
+ (eng / ("strata-vision" + suffix)).unlink()
+ (eng / ("strata-vision-hip" + suffix)).unlink()
+ self.assertEqual(setup.hip_reader(eng, "gpu", {"arch": "gfx1100"})[0], "none")
class HipRuntimeBesideExe(unittest.TestCase):
diff --cc tools/test_vision_reader.py
index 000000000,1f77b0c91..e840aafac
mode 000000,100644..100644
--- a/tools/test_vision_reader.py
+++ b/tools/test_vision_reader.py
@@@ -1,0 -1,202 +1,213 @@@
+ """The picture reader choice for an AMD engine folder (strata_control/vision_reader.py) and the server's fallback from the GPU
+ reader to the CPU reader (serve/server.py, Vision._start): hip present, hip missing, hip start failure, then cpu.
+
+ python -m unittest tools.test_vision_reader
+ """
+ from __future__ import annotations
+
+ import io
+ import tempfile
+ import unittest
+ from pathlib import Path
+ from types import SimpleNamespace
+ from unittest import mock
+
+ from strata_control import vision_reader as VR
+ import serve.server as server
+
+
+ def folder(*names: str) -> Path:
+ d = Path(tempfile.mkdtemp(prefix="vr-"))
+ for n in names:
+ (d / n).write_bytes(b"x")
+ return d
+
+
+ class Choose(unittest.TestCase):
++ def test_the_default_reader_by_card_memory_is_one_rule_for_every_make(self):
++ self.assertEqual(VR.default_mode(12), "cpu")
++ self.assertEqual(VR.default_mode(16), "cpu")
++ self.assertEqual(VR.default_mode(23.5), "gpu")
++ self.assertEqual(VR.default_mode(24), "gpu")
++ self.assertEqual(VR.default_mode(32), "gpu")
++ self.assertEqual(VR.default_mode(None), "gpu") # memory unknown: the GPU reader falls back by itself
++ for asked, vram, want in ((None, 16, "cpu"), ("yes", 12, "cpu"), ("on", 32, "gpu"), ("gpu", 12, "gpu"), ("cpu", 32, "cpu"),
++ ("no", 32, "none"), ("off", 12, "none")):
++ self.assertEqual(VR.wanted(asked, vram), want, (asked, vram))
++
+ def test_hip_present_gives_the_gpu_reader_with_the_cpu_one_to_fall_back_on(self):
+ d = folder("strata-vision", "strata-vision-hip")
+ for ask in ("yes", "on", "gpu"):
+ r = VR.choose(d, ask, windows=False, gpu_arch="gfx1201", archs=["gfx1200", "gfx1201"])
+ self.assertEqual((r.mode, r.exe.name, r.cpu_exe.name), ("gpu", "strata-vision-hip", "strata-vision"), ask)
+ self.assertEqual(r.note, "")
+
+ def test_hip_missing_gives_the_cpu_reader(self):
+ d = folder("strata-vision")
+ r = VR.choose(d, "yes", windows=False, gpu_arch="gfx1201")
+ self.assertEqual((r.mode, r.exe.name, r.cpu_exe), ("cpu", "strata-vision", None))
+ self.assertEqual(r.note, "") # "yes" never asked for the GPU in particular
+ self.assertIn("no GPU picture reader", VR.choose(d, "gpu", windows=False).note) # --vision gpu: said once
+
+ def test_card_not_in_the_build_gives_the_cpu_reader(self):
+ d = folder("strata-vision", "strata-vision-hip")
+ r = VR.choose(d, "yes", windows=False, gpu_arch="gfx1100", archs=["gfx1200", "gfx1201"])
+ self.assertEqual((r.mode, r.exe.name), ("cpu", "strata-vision"))
+ self.assertIn("gfx1100", r.note)
+
+ def test_unknown_card_tries_the_gpu_reader(self):
+ d = folder("strata-vision", "strata-vision-hip")
+ self.assertEqual(VR.choose(d, "yes", windows=False, gpu_arch="", archs=["gfx1201"]).mode, "gpu")
+ self.assertEqual(VR.choose(d, "yes", windows=False, gpu_arch="gfx1100", archs=None).mode, "gpu")
+
+ def test_gcn_arch_names_with_features_match(self):
+ self.assertEqual(VR.gfx_of("gfx1201:sramecc-:xnack-"), "gfx1201")
+ d = folder("strata-vision", "strata-vision-hip")
+ self.assertEqual(VR.choose(d, "yes", windows=False, gpu_arch="gfx1151:sramecc+:xnack-", archs=["gfx1151"]).mode, "gpu")
+
+ def test_cpu_asked_and_off(self):
+ d = folder("strata-vision", "strata-vision-hip")
+ self.assertEqual(VR.choose(d, "cpu", windows=False).exe.name, "strata-vision")
+ for ask in ("no", "none", "off", None):
+ self.assertEqual(VR.choose(d, ask, windows=False).mode, "none")
+
+ def test_no_reader_at_all(self):
+ d = folder("strata")
+ for ask in ("yes", "cpu", "gpu"):
+ r = VR.choose(d, ask, windows=False)
+ self.assertEqual(r.mode, "none", ask)
+ self.assertIn("no picture reader", r.note)
+
+ def test_windows_names(self):
+ d = folder("strata-vision.exe", "strata-vision-hip.exe")
+ r = VR.choose(d, "yes", windows=True, gpu_arch="gfx1201", archs=["gfx1201"])
+ self.assertEqual((r.mode, r.exe.name, r.cpu_exe.name), ("gpu", "strata-vision-hip.exe", "strata-vision.exe"))
+ self.assertEqual(VR.choose(folder("strata-vision-hip"), "yes", windows=True).mode, "none") # a Linux name on Windows
+
+ def test_run_config_section(self):
+ d = folder("strata-vision", "strata-vision-hip")
+ base = {"mmproj": "m.gguf", "model": "t.gguf", "max_tokens": 1024}
+ sec = VR.vision_section(VR.choose(d, "yes", windows=False), base)
+ self.assertTrue(sec["gpu"])
+ self.assertEqual(Path(sec["exe"]).name, "strata-vision-hip")
+ self.assertEqual(Path(sec["cpu_exe"]).name, "strata-vision")
+ cpu = VR.cpu_fallback(sec)
+ self.assertFalse(cpu["gpu"])
+ self.assertEqual(Path(cpu["exe"]).name, "strata-vision")
+ self.assertNotIn("cpu_exe", cpu)
+ self.assertEqual(cpu["mmproj"], "m.gguf")
+ cpu_only = VR.vision_section(VR.choose(folder("strata-vision"), "yes", windows=False), base)
+ self.assertFalse(cpu_only["gpu"])
+ self.assertNotIn("cpu_exe", cpu_only)
+ self.assertIsNone(VR.cpu_fallback(cpu_only))
+ self.assertIsNone(VR.cpu_fallback({**sec, "cpu_exe": ""}))
+ self.assertIsNone(VR.cpu_fallback({"exe": "strata-vision", "gpu": True})) # an NVIDIA config has no fallback
+
+
+ class FakeProc:
+ def __init__(self, first_line: str):
+ self.stdin, self.stdout = io.StringIO(), io.StringIO(first_line)
+ self.killed = False
+
+ def kill(self):
+ self.killed = True
+
+ def wait(self, timeout=None):
+ return 0
+
+ def poll(self):
+ return None
+
+
+ def make_vision(cfg: dict) -> "server.Vision":
+ with mock.patch.object(server.Vision, "work_dir", staticmethod(lambda: Path(tempfile.gettempdir()))):
+ return server.Vision(cfg, lazy=True)
+
+
+ class Fallback(unittest.TestCase):
+ CFG = {"exe": "/e/strata-vision-hip", "cpu_exe": "/e/strata-vision", "gpu": True, "mmproj": "/m/mm.gguf",
+ "model": "/m/t.gguf", "max_tokens": 1024, "threads": 6}
+
+ def run_start(self, answers: dict, cfg: dict | None = None):
+ """answers: exe name -> its first stdout line, or an exception to raise from popen."""
+ started = []
+
+ def popen(what, args, **kw):
+ name = Path(args[0]).name
+ started.append(args)
+ a = answers[name]
+ if isinstance(a, Exception):
+ raise a
+ return FakeProc(a)
+
+ v = make_vision(cfg or self.CFG)
+ out = io.StringIO()
+ with mock.patch.object(server, "popen", popen), mock.patch("builtins.print", lambda *a, **k: out.write(" ".join(map(str, a)) + "\n")):
+ err = None
+ try:
+ v._start()
+ except RuntimeError as e:
+ err = e
+ return v, started, out.getvalue(), err
+
+ def test_hip_present_and_starts(self):
+ v, started, log, err = self.run_start({"strata-vision-hip": "READY 2048\n"})
+ self.assertIsNone(err)
+ self.assertEqual([Path(a[0]).name for a in started], ["strata-vision-hip"])
+ self.assertIn("--gpu", started[0])
+ self.assertEqual(log, "")
+ self.assertFalse(v.stopped)
+
+ def test_hip_start_failure_falls_back_to_cpu_with_one_line(self):
+ v, started, log, err = self.run_start({"strata-vision-hip": "ERR this GPU (gfx1100) is not in the picture reader's build (gfx1200,gfx1201)\n",
+ "strata-vision": "READY 2048\n"})
+ self.assertIsNone(err)
+ self.assertEqual([Path(a[0]).name for a in started], ["strata-vision-hip", "strata-vision"])
+ self.assertNotIn("--gpu", started[1])
+ self.assertIn("--threads", started[1])
+ self.assertEqual(log.count("\n"), 1)
+ self.assertIn("gfx1100", log)
+ self.assertIn("CPU", log)
+ self.assertFalse(v.stopped)
+ self.assertIsNone(v.fallback)
+
+ def test_hip_process_that_dies_silently_falls_back(self): # a GPU runtime that aborts: stdout closes empty
+ v, started, log, err = self.run_start({"strata-vision-hip": "", "strata-vision": "READY 2048\n"})
+ self.assertIsNone(err)
+ self.assertEqual(len(started), 2)
+ self.assertEqual(log.count("\n"), 1)
+
+ def test_hip_missing_file_falls_back(self): # popen raises (exe gone, not executable)
+ v, started, log, err = self.run_start({"strata-vision-hip": FileNotFoundError("strata-vision-hip"),
+ "strata-vision": "READY 2048\n"})
+ self.assertIsNone(err)
+ self.assertEqual(len(started), 2)
+
+ def test_restart_after_a_fallback_stays_on_the_cpu_reader(self):
+ v, started, log, err = self.run_start({"strata-vision-hip": "ERR nope\n", "strata-vision": "READY 2048\n"})
+ started.clear()
+ with mock.patch.object(server, "popen", lambda what, args, **kw: started.append(args) or FakeProc("READY 2048\n")):
+ v.restart()
+ self.assertEqual([Path(a[0]).name for a in started], ["strata-vision"])
+
+ def test_cpu_failure_after_the_fallback_is_an_error(self):
+ v, started, log, err = self.run_start({"strata-vision-hip": "ERR nope\n", "strata-vision": "ERR cannot load the vision encoder\n"})
+ self.assertIsNotNone(err)
+ self.assertIn("cannot load", str(err))
+
+ def test_a_config_without_cpu_exe_has_no_fallback(self): # NVIDIA: the error is shown as before
+ cfg = {k: v for k, v in self.CFG.items() if k != "cpu_exe"}
+ v, started, log, err = self.run_start({"strata-vision-hip": "ERR nope\n"}, cfg)
+ self.assertIsNotNone(err)
+ self.assertEqual(len(started), 1)
+ self.assertEqual(log, "")
+
+ def test_cpu_config_is_unchanged(self):
+ cfg = {"exe": "/e/strata-vision", "gpu": False, "mmproj": "/m/mm.gguf", "model": "/m/t.gguf"}
+ v, started, log, err = self.run_start({"strata-vision": "READY 2048\n"}, cfg)
+ self.assertIsNone(err)
+ self.assertNotIn("--gpu", started[0])
+
+
+ if __name__ == "__main__":
+ unittest.main()