Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Merge branch 'f-021-arcvis' into f-021-int

Niko12219f5488e3b24e
diff --cc docs/MODELS.md
index 31a8ed6ca,bf7977cf5..c51fae258
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@@ -107,8 -107,10 +107,10 @@@ read them. The picture reader runs on t
picture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB of VRAM; 0.1-0.5 s per
picture). The rule is the same on NVIDIA and AMD cards. `--vision off` (or "No" in the app and in `strata launch`) turns
pictures off; `--vision cpu` or `--vision gpu` picks the reader. On AMD, `strata-vision-hip` is the card's reader; if it
-cannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac app and the Mac Terminal have no
-picture reader yet ([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the
+cannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac reads pictures on its GPU
- ([MAC.md](MAC.md)); the Intel Arc engine has no picture reader yet ([INTEL.md](INTEL.md#not-done)). More: [Images](DETAILS.md#images-vision).
++([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the
+ CPU reader unless you ask for `--vision gpu` (the Intel GPU reader has only run on an 8 GB A750), and a Battlemage card (B580, B70)
+ has pictures off until you pass `--vision yes` ([INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)). More: [Images](DETAILS.md#images-vision).
### Qwen3.8-Flash-Next
diff --cc strata_control/vision_reader.py
index 8678d5e32,3c4a6fa01..a4a058762
--- a/strata_control/vision_reader.py
+++ b/strata_control/vision_reader.py
@@@ -1,4 -1,5 +1,5 @@@
- """Which picture reader an engine folder gets: the GPU one (`strata-vision-hip`, AMD; `strata-vision --gpu` on a Mac) or the CPU one (`strata-vision`).
-"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc) or the CPU one
-(`strata-vision`).
++"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc, `strata-vision --gpu`
++on a Mac) or the CPU one (`strata-vision`).
An AMD engine package carries two programs for pictures. `strata-vision` is the CPU reader: it needs nothing from the GPU
runtime, so it starts on every PC. `strata-vision-hip` is the same reader on the card (llama.cpp's HIP backend), a
@@@ -7,12 -8,9 +8,15 @@@ reader as its fallback (`vision.cpu_exe
log line, when the GPU reader does not start: the HIP runtime does not load, the card is not one the build has code for
(the reader says so before it runs a kernel), or the card is busy. Nothing is asked of the user.
+A Mac has one program for both: `strata-vision` is built with Metal, `--gpu` reads on the Mac's GPU and without it on the CPU, so the
+CPU reader is the same file and the fallback needs no second program. Unified memory changes the 24 GB rule: nothing is taken from a
+separate card, the reader's ~1.1 GB is the mmproj file itself on either side (measured on an M4, footprint 1,050 MB on the CPU, 1,179 MB
+on the GPU), and the CPU reader takes ~50 s for a 260-token picture where the GPU one takes 0.45 s (ggml has no BF16 vector kernel on
+ARM). So a Mac reads on the GPU unless the person asks for the CPU.
+
+ An Intel Arc engine folder (engine-sycl/) carries the CPU reader; `strata-vision-sycl` (llama.cpp's SYCL backend, tools/vision with
+ STRATA_VISION_SYCL, built by `setup --backend sycl --vision gpu` on a PC that has the oneAPI compiler) is the GPU reader there.
+
`choose` is the whole decision; setup.py calls it, and tools/test_vision_reader.py checks every branch.
"""
from __future__ import annotations
@@@ -29,17 -29,22 +35,25 @@@ GPU_READERS = (GPU_READER, SYCL_READER
#: of card. A person can switch it per model (--vision gpu / --vision cpu, the Pictures step); what was asked for is kept.
GPU_MIN_VRAM_GB = 23.5
+ #: The one exception to that rule. An Intel Arc of 24 GB or more keeps the CPU reader by default: the SYCL GPU reader works (tested on an A750,
+ #: forced with --vision gpu: 0.16 s a picture against 1.7 s on the CPU, embeddings cosine >= 0.995 to the CPU reader's) but it needs the oneAPI
+ #: runtime on the PC and has not run on a 24 GB+ card (the B70 is gone). --vision gpu asks for it. Set True when a 24 GB+ Arc has run it.
+ INTEL_GPU_READER_BY_DEFAULT = False
+
- def default_mode(vram_gb: float | None, *, unified: bool = False) -> str:
- """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu").
- unified: a Mac, where the chip's memory is the system's - always "gpu" (see the module doc)."""
-def default_mode(vram_gb: float | None, make: str = "") -> str:
++def default_mode(vram_gb: float | None, make: str = "", *, unified: bool = False) -> str:
+ """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu"). `make` is "intel"
- for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT."""
++ for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT. unified: a Mac, where the chip's memory is the system's -
++ always "gpu" (see the module doc)."""
+ if unified:
+ return "gpu"
vram = float(vram_gb or 0.0)
+ if make == "intel" and not INTEL_GPU_READER_BY_DEFAULT:
+ return "cpu"
return "cpu" if 0 < vram < GPU_MIN_VRAM_GB else "gpu"
- def wanted(asked: str | None, vram_gb: float | None, *, unified: bool = False) -> str:
-def wanted(asked: str | None, vram_gb: float | None, make: str = "") -> str:
++def wanted(asked: str | None, vram_gb: float | None, make: str = "", *, unified: bool = False) -> str:
"""What is wanted for `asked` (None, "yes", "on", "gpu", "cpu", "no"/"none"/"off"): "gpu", "cpu" or "none". Nothing asked and "yes" mean
the default for this card; "gpu" and "cpu" are kept as asked."""
a = (asked or "").lower()
@@@ -47,7 -52,7 +61,7 @@@
return "none"
if a in ("gpu", "cpu"):
return a
- return default_mode(vram_gb, unified=unified)
- return default_mode(vram_gb, make)
++ return default_mode(vram_gb, make, unified=unified)
GFX = re.compile(r"gfx[0-9a-f]+", re.IGNORECASE)
@@@ -56,16 -61,13 +70,16 @@@ def reader_name(base: str, windows: boo
return base + (".exe" if windows else "")
-def find(folder: Path | str, windows: bool) -> dict:
- """The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}."""
+def find(folder: Path | str, windows: bool, *, mac: bool = False) -> dict:
+ """The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}. On a Mac (mac=True) `strata-vision` is both: it has
+ Metal built in and `--gpu` picks the GPU."""
folder = Path(folder)
found = {}
- for key, base in (("cpu", CPU_READER), ("gpu", GPU_READER)):
- path = folder / reader_name(base, windows)
- found[key] = path if path.is_file() else None
+ for key, bases in (("cpu", (CPU_READER,)), ("gpu", GPU_READERS)):
+ paths = [folder / reader_name(base, windows) for base in bases]
+ found[key] = next((p for p in paths if p.is_file()), None)
+ if mac:
+ found["gpu"] = found["cpu"]
return found
diff --cc tools/vision/CMakeLists.txt
index e2f958d4f,5ad3772c1..3b104dc0a
--- a/tools/vision/CMakeLists.txt
+++ b/tools/vision/CMakeLists.txt
@@@ -63,15 -67,10 +68,19 @@@ endif(
set(GGML_BACKEND_DL OFF CACHE BOOL "" FORCE)
set(GGML_CUDA ${STRATA_VISION_CUDA} CACHE BOOL "" FORCE)
set(GGML_HIP ${STRATA_VISION_HIP} CACHE BOOL "" FORCE)
+if(APPLE)
+ # One program for both readers: strata-vision --gpu runs the encoder on the Metal GPU, without --gpu on the CPU (Apple Silicon always
+ # has the Metal framework, the engine needs it too). The Metal shaders are embedded in the binary and compiled by macOS when the
+ # reader starts, so nothing but the program has to be shipped. No -march flag: the default target (an M1) runs on every Apple
+ # Silicon Mac, and ggml's CPU path has no BF16 vector kernel on ARM (measured on an M4: the same ~50 s for a 260-token picture with
+ # -march=armv8.6-a+bf16+i8mm as without), so a faster target would only risk the M1.
+ set(GGML_METAL ${STRATA_VISION_METAL} CACHE BOOL "" FORCE)
+ set(GGML_METAL_EMBED_LIBRARY ${STRATA_VISION_METAL} CACHE BOOL "" FORCE)
+endif()
+ set(GGML_SYCL ${STRATA_VISION_SYCL} CACHE BOOL "" FORCE)
+ if(STRATA_VISION_SYCL)
+ set(GGML_SYCL_TARGET "INTEL" CACHE STRING "" FORCE)
+ endif()
if(STRATA_VISION_HIP)
# the engine's HIP build says the same (cmake/hip_backend.cmake); ggml-hip reads AMDGPU_TARGETS / CMAKE_HIP_ARCHITECTURES
set(AMDGPU_TARGETS "${STRATA_VISION_HIP_ARCHS}" CACHE STRING "" FORCE)
@@@ -92,9 -91,11 +101,14 @@@ if(STRATA_VISION_HIP
string(REPLACE ";" "," _hip_archs "${STRATA_VISION_HIP_ARCHS}")
target_compile_definitions(strata-vision PRIVATE STRATA_VISION_HIP STRATA_VISION_HIP_ARCHS="${_hip_archs}")
endif()
+if(APPLE AND STRATA_VISION_METAL)
+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_METAL)
+endif()
+ if(STRATA_VISION_SYCL)
+ # the Intel GPU reader is its own program too, beside the CPU reader "strata-vision" it falls back on
+ set_target_properties(strata-vision PROPERTIES OUTPUT_NAME strata-vision-sycl)
+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_SYCL)
+ endif()
if(WIN32 AND STRATA_VISION_HIP)
# hipblas.dll (and rocblas.dll behind it) live in rocm\bin of the engine package: loaded on first use, after main() has put that folder on
# the DLL search path, so the reader starts from any PATH and says "no GPU" itself where there is none
diff --cc tools/vision/strata_vision.cpp
index fb892630c,c5d3ab915..9cc8d321f
--- a/tools/vision/strata_vision.cpp
+++ b/tools/vision/strata_vision.cpp
@@@ -102,8 -102,8 +102,10 @@@ int main(int argc, char** argv)
else if (a == "--version") {
#ifdef STRATA_VISION_HIP
std::printf("strata-vision hip (GPU architectures: %s)\n", STRATA_VISION_HIP_ARCHS);
+#elif defined(STRATA_VISION_METAL)
+ std::printf("strata-vision metal\n");
+ #elif defined(STRATA_VISION_SYCL)
+ std::printf("strata-vision sycl (Intel GPU)\n");
#else
std::printf("strata-vision\n");
#endif
@@@ -162,25 -168,9 +170,26 @@@
std::fflush(stdout);
return 3;
}
+ #endif
}
#endif
+#ifdef STRATA_VISION_METAL
+ if (gpu) {
+ // The Mac: the same program reads on the GPU (--gpu, ggml's Metal backend) or on the CPU. With --gpu and no Metal device (a
+ // virtual machine without a GPU, say), mtmd would quietly use the CPU, which reads a picture in ~50 s: say so instead, and
+ // the server starts the program again without --gpu (serve/server.py, Vision._start) with one log line.
+ bool have_gpu = false;
+ for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
+ const auto type = ggml_backend_dev_type(ggml_backend_dev_get(i));
+ have_gpu = have_gpu || type == GGML_BACKEND_DEVICE_TYPE_GPU || type == GGML_BACKEND_DEVICE_TYPE_IGPU;
+ }
+ if (!have_gpu) {
+ std::printf("ERR this Mac has no Metal GPU for the picture reader\n");
+ std::fflush(stdout);
+ return 3;
+ }
+ }
+#endif
const auto load_t0 = std::chrono::steady_clock::now();
#if !defined(_WIN32) && !defined(__APPLE__) // macOS has no posix_fadvise