Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Merge branch 'f-021-arcvis' into f-021-int
Niko12219f5488e3b24e
diff --cc docs/MODELS.mdindex 31a8ed6ca,bf7977cf5..c51fae258--- a/docs/MODELS.md+++ b/docs/MODELS.md@@@ -107,8 -107,10 +107,10 @@@ read them. The picture reader runs on tpicture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB of VRAM; 0.1-0.5 s perpicture). The rule is the same on NVIDIA and AMD cards. `--vision off` (or "No" in the app and in `strata launch`) turnspictures off; `--vision cpu` or `--vision gpu` picks the reader. On AMD, `strata-vision-hip` is the card's reader; if it-cannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac app and the Mac Terminal have no-picture reader yet ([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the+cannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac reads pictures on its GPU- ([MAC.md](MAC.md)); the Intel Arc engine has no picture reader yet ([INTEL.md](INTEL.md#not-done)). More: [Images](DETAILS.md#images-vision).++([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the+ CPU reader unless you ask for `--vision gpu` (the Intel GPU reader has only run on an 8 GB A750), and a Battlemage card (B580, B70)+ has pictures off until you pass `--vision yes` ([INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)). More: [Images](DETAILS.md#images-vision).### Qwen3.8-Flash-Nextdiff --cc strata_control/vision_reader.pyindex 8678d5e32,3c4a6fa01..a4a058762--- a/strata_control/vision_reader.py+++ b/strata_control/vision_reader.py@@@ -1,4 -1,5 +1,5 @@@- """Which picture reader an engine folder gets: the GPU one (`strata-vision-hip`, AMD; `strata-vision --gpu` on a Mac) or the CPU one (`strata-vision`).-"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc) or the CPU one-(`strata-vision`).++"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc, `strata-vision --gpu`++on a Mac) or the CPU one (`strata-vision`).An AMD engine package carries two programs for pictures. `strata-vision` is the CPU reader: it needs nothing from the GPUruntime, so it starts on every PC. `strata-vision-hip` is the same reader on the card (llama.cpp's HIP backend), a@@@ -7,12 -8,9 +8,15 @@@ reader as its fallback (`vision.cpu_exelog line, when the GPU reader does not start: the HIP runtime does not load, the card is not one the build has code for(the reader says so before it runs a kernel), or the card is busy. Nothing is asked of the user.+A Mac has one program for both: `strata-vision` is built with Metal, `--gpu` reads on the Mac's GPU and without it on the CPU, so the+CPU reader is the same file and the fallback needs no second program. Unified memory changes the 24 GB rule: nothing is taken from a+separate card, the reader's ~1.1 GB is the mmproj file itself on either side (measured on an M4, footprint 1,050 MB on the CPU, 1,179 MB+on the GPU), and the CPU reader takes ~50 s for a 260-token picture where the GPU one takes 0.45 s (ggml has no BF16 vector kernel on+ARM). So a Mac reads on the GPU unless the person asks for the CPU.++ An Intel Arc engine folder (engine-sycl/) carries the CPU reader; `strata-vision-sycl` (llama.cpp's SYCL backend, tools/vision with+ STRATA_VISION_SYCL, built by `setup --backend sycl --vision gpu` on a PC that has the oneAPI compiler) is the GPU reader there.+`choose` is the whole decision; setup.py calls it, and tools/test_vision_reader.py checks every branch."""from __future__ import annotations@@@ -29,17 -29,22 +35,25 @@@ GPU_READERS = (GPU_READER, SYCL_READER#: of card. A person can switch it per model (--vision gpu / --vision cpu, the Pictures step); what was asked for is kept.GPU_MIN_VRAM_GB = 23.5+ #: The one exception to that rule. An Intel Arc of 24 GB or more keeps the CPU reader by default: the SYCL GPU reader works (tested on an A750,+ #: forced with --vision gpu: 0.16 s a picture against 1.7 s on the CPU, embeddings cosine >= 0.995 to the CPU reader's) but it needs the oneAPI+ #: runtime on the PC and has not run on a 24 GB+ card (the B70 is gone). --vision gpu asks for it. Set True when a 24 GB+ Arc has run it.+ INTEL_GPU_READER_BY_DEFAULT = False+- def default_mode(vram_gb: float | None, *, unified: bool = False) -> str:- """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu").- unified: a Mac, where the chip's memory is the system's - always "gpu" (see the module doc)."""-def default_mode(vram_gb: float | None, make: str = "") -> str:++def default_mode(vram_gb: float | None, make: str = "", *, unified: bool = False) -> str:+ """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu"). `make` is "intel"- for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT."""++ for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT. unified: a Mac, where the chip's memory is the system's -++ always "gpu" (see the module doc)."""+ if unified:+ return "gpu"vram = float(vram_gb or 0.0)+ if make == "intel" and not INTEL_GPU_READER_BY_DEFAULT:+ return "cpu"return "cpu" if 0 < vram < GPU_MIN_VRAM_GB else "gpu"- def wanted(asked: str | None, vram_gb: float | None, *, unified: bool = False) -> str:-def wanted(asked: str | None, vram_gb: float | None, make: str = "") -> str:++def wanted(asked: str | None, vram_gb: float | None, make: str = "", *, unified: bool = False) -> str:"""What is wanted for `asked` (None, "yes", "on", "gpu", "cpu", "no"/"none"/"off"): "gpu", "cpu" or "none". Nothing asked and "yes" meanthe default for this card; "gpu" and "cpu" are kept as asked."""a = (asked or "").lower()@@@ -47,7 -52,7 +61,7 @@@return "none"if a in ("gpu", "cpu"):return a- return default_mode(vram_gb, unified=unified)- return default_mode(vram_gb, make)++ return default_mode(vram_gb, make, unified=unified)GFX = re.compile(r"gfx[0-9a-f]+", re.IGNORECASE)@@@ -56,16 -61,13 +70,16 @@@ def reader_name(base: str, windows: booreturn base + (".exe" if windows else "")-def find(folder: Path | str, windows: bool) -> dict:- """The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}."""+def find(folder: Path | str, windows: bool, *, mac: bool = False) -> dict:+ """The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}. On a Mac (mac=True) `strata-vision` is both: it has+ Metal built in and `--gpu` picks the GPU."""folder = Path(folder)found = {}- for key, base in (("cpu", CPU_READER), ("gpu", GPU_READER)):- path = folder / reader_name(base, windows)- found[key] = path if path.is_file() else None+ for key, bases in (("cpu", (CPU_READER,)), ("gpu", GPU_READERS)):+ paths = [folder / reader_name(base, windows) for base in bases]+ found[key] = next((p for p in paths if p.is_file()), None)+ if mac:+ found["gpu"] = found["cpu"]return founddiff --cc tools/vision/CMakeLists.txtindex e2f958d4f,5ad3772c1..3b104dc0a--- a/tools/vision/CMakeLists.txt+++ b/tools/vision/CMakeLists.txt@@@ -63,15 -67,10 +68,19 @@@ endif(set(GGML_BACKEND_DL OFF CACHE BOOL "" FORCE)set(GGML_CUDA ${STRATA_VISION_CUDA} CACHE BOOL "" FORCE)set(GGML_HIP ${STRATA_VISION_HIP} CACHE BOOL "" FORCE)+if(APPLE)+ # One program for both readers: strata-vision --gpu runs the encoder on the Metal GPU, without --gpu on the CPU (Apple Silicon always+ # has the Metal framework, the engine needs it too). The Metal shaders are embedded in the binary and compiled by macOS when the+ # reader starts, so nothing but the program has to be shipped. No -march flag: the default target (an M1) runs on every Apple+ # Silicon Mac, and ggml's CPU path has no BF16 vector kernel on ARM (measured on an M4: the same ~50 s for a 260-token picture with+ # -march=armv8.6-a+bf16+i8mm as without), so a faster target would only risk the M1.+ set(GGML_METAL ${STRATA_VISION_METAL} CACHE BOOL "" FORCE)+ set(GGML_METAL_EMBED_LIBRARY ${STRATA_VISION_METAL} CACHE BOOL "" FORCE)+endif()+ set(GGML_SYCL ${STRATA_VISION_SYCL} CACHE BOOL "" FORCE)+ if(STRATA_VISION_SYCL)+ set(GGML_SYCL_TARGET "INTEL" CACHE STRING "" FORCE)+ endif()if(STRATA_VISION_HIP)# the engine's HIP build says the same (cmake/hip_backend.cmake); ggml-hip reads AMDGPU_TARGETS / CMAKE_HIP_ARCHITECTURESset(AMDGPU_TARGETS "${STRATA_VISION_HIP_ARCHS}" CACHE STRING "" FORCE)@@@ -92,9 -91,11 +101,14 @@@ if(STRATA_VISION_HIPstring(REPLACE ";" "," _hip_archs "${STRATA_VISION_HIP_ARCHS}")target_compile_definitions(strata-vision PRIVATE STRATA_VISION_HIP STRATA_VISION_HIP_ARCHS="${_hip_archs}")endif()+if(APPLE AND STRATA_VISION_METAL)+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_METAL)+endif()+ if(STRATA_VISION_SYCL)+ # the Intel GPU reader is its own program too, beside the CPU reader "strata-vision" it falls back on+ set_target_properties(strata-vision PROPERTIES OUTPUT_NAME strata-vision-sycl)+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_SYCL)+ endif()if(WIN32 AND STRATA_VISION_HIP)# hipblas.dll (and rocblas.dll behind it) live in rocm\bin of the engine package: loaded on first use, after main() has put that folder on# the DLL search path, so the reader starts from any PATH and says "no GPU" itself where there is nonediff --cc tools/vision/strata_vision.cppindex fb892630c,c5d3ab915..9cc8d321f--- a/tools/vision/strata_vision.cpp+++ b/tools/vision/strata_vision.cpp@@@ -102,8 -102,8 +102,10 @@@ int main(int argc, char** argv)else if (a == "--version") {#ifdef STRATA_VISION_HIPstd::printf("strata-vision hip (GPU architectures: %s)\n", STRATA_VISION_HIP_ARCHS);+#elif defined(STRATA_VISION_METAL)+ std::printf("strata-vision metal\n");+ #elif defined(STRATA_VISION_SYCL)+ std::printf("strata-vision sycl (Intel GPU)\n");#elsestd::printf("strata-vision\n");#endif@@@ -162,25 -168,9 +170,26 @@@std::fflush(stdout);return 3;}+ #endif}#endif+#ifdef STRATA_VISION_METAL+ if (gpu) {+ // The Mac: the same program reads on the GPU (--gpu, ggml's Metal backend) or on the CPU. With --gpu and no Metal device (a+ // virtual machine without a GPU, say), mtmd would quietly use the CPU, which reads a picture in ~50 s: say so instead, and+ // the server starts the program again without --gpu (serve/server.py, Vision._start) with one log line.+ bool have_gpu = false;+ for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {+ const auto type = ggml_backend_dev_type(ggml_backend_dev_get(i));+ have_gpu = have_gpu || type == GGML_BACKEND_DEVICE_TYPE_GPU || type == GGML_BACKEND_DEVICE_TYPE_IGPU;+ }+ if (!have_gpu) {+ std::printf("ERR this Mac has no Metal GPU for the picture reader\n");+ std::fflush(stdout);+ return 3;+ }+ }+#endifconst auto load_t0 = std::chrono::steady_clock::now();#if !defined(_WIN32) && !defined(__APPLE__) // macOS has no posix_fadvise