Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Intel Arc pictures: on by default for the Alchemist cards (CPU reader), Battlemage stays opt-in; SYCL GPU picture reader (strata-vision-sycl) for --vision gpu; reader rule in vision_reader.py; tests and docs

Niko1221cac4816afdea
diff --git a/docs/INTEL.md b/docs/INTEL.md
index eb18988d1..f77bb006a 100644
--- a/docs/INTEL.md
+++ b/docs/INTEL.md
@@ -789,17 +789,39 @@ the fix).
## Not done
-- Images on the llama.cpp config: not wired. On the SYCL port pictures are wired but opt-in (`setup --backend sycl --vision yes`):
- setup compiles the portable CPU reader `strata-vision` (the program the AMD packages carry beside their engine) into `engine-sycl/`,
- the server runs it on the host, and the engine in the container reads its embeddings from the host's `/dev/shm`, which
- `strata-sycl.sh` shares (`STRATA_SYCL_SHM=0` keeps a private one). A SYCL GPU reader does not exist yet, so every Arc reads
- pictures on the CPU (a few seconds each). They are not on by default because the first picture request on the Arc Pro B70
- (engine 0.1.45, IQ3_S, 2026-10-19) hung the GPU: the xe driver logged an engine reset and `Fault response: Unsuccessful -EINVAL`
- and the request ended 503. Text was not affected by this wiring (the reader and the shared `/dev/shm` are only used for pictures);
- the engine's vision path on SYCL needs a look first (`setup_intel.PICTURES_ON_BY_DEFAULT` flips the default).
+- Images on the llama.cpp config: not wired. On the SYCL port they are wired, see "Pictures on an Arc" below. Open there: a
+ Battlemage card (the B-series) keeps pictures opt-in because of the Arc Pro B70 fault, and nobody has run the SYCL picture reader
+ on a card of 24 GB or more.
- Speculative decoding on the llama.cpp path (the GGUF carries no draft layer llama.cpp can use). The SYCL
port has it (MTP draft layer, `--mtp`).
+## Pictures on an Arc (2026-10-19, branch f-021-arcvis)
+
+Pictures follow the rule of every other card ([MODELS.md](MODELS.md#the-versions)): a picture reader on the CPU under 24 GB of VRAM, on
+the card from 24 GB. `strata_control/vision_reader.py` is the one place for it. On an Arc:
+
+- **The CPU reader** is `strata-vision` (llama.cpp's mtmd). Setup compiles it into `engine-sycl/`, the server runs it on the host, and the
+ engine in the container reads the picture's embeddings from the host's `/dev/shm`, which `strata-sycl.sh` shares
+ (`STRATA_SYCL_SHM=0` keeps a private one). On the A750 a 448x448 picture takes 1.73 s.
+- **The SYCL reader** is `strata-vision-sycl`, the same program on llama.cpp's SYCL backend (`tools/vision`, `STRATA_VISION_SYCL=ON`).
+ `setup --backend sycl --vision gpu` compiles it on the host when the oneAPI compiler (`icx`, `icpx`, `intel-oneapi-mkl-devel`) is
+ installed; without it setup says so and keeps the CPU reader. It needs the PC's oneAPI runtime, not the container. If it does not
+ start, the server falls back to the CPU reader with one log line. On the A750: 0.16 s per picture instead of 1.73 s (after a 30 s
+ first start that compiles the kernels), and the embeddings match the CPU reader's (cosine 0.9959 to 0.9999 over 8 pictures). It takes
+ VRAM the experts would use, which is why the rule gives it to 24 GB cards only.
+- **Default.** On for an Alchemist card (A750, A770, ...) with the CPU reader. An Arc of 24 GB or more also keeps the CPU reader by
+ default: the SYCL reader has not run on such a card (the B70 is gone), so `--vision gpu` asks for it and
+ `vision_reader.INTEL_GPU_READER_BY_DEFAULT` is the switch to flip once one has. A Battlemage card (B580, B70, ...) keeps pictures
+ **off** by default: the first picture on the Arc Pro B70 (engine 0.1.45, IQ3_S, 2026-10-19, CPU reader) logged an xe engine reset
+ (ccs + bcs) and `Fault response: Unsuccessful -EINVAL`, and the request ended 503. The CPU reader was in use, so the fault is in the engine on xe, not in a GPU reader; it
+ has not been looked at since. `--vision yes` turns pictures on there; setup says so.
+
+Tested on the A750 (IQ3_XXS, 8K context, engine from f-021-int, JIT build, 2026-10-19): a picture with a red circle, a blue square and the
+words "STRATA 42", 8 variants, answered correctly by the CPU reader 10 of 10 (a first set with smaller text read "42" as "412"), and
+by the SYCL reader 10 of 10; two pictures in one request, a picture kept in the history of a second turn, and a long text prompt after the
+pictures all answered; a text request after every picture answered; 47 picture requests, outside and inside the container (inside, one
+answer was cut off by the 200-token limit of the test), no GPU fault in the kernel log, no 503.
+
## Arc Pro B70 with a model that does not fit: Flash-Next IQ3_S (2026-10-07, branch f-141-intel)
The earlier B70 rows are for models whose experts all fit in VRAM. IQ3_S has 24,576 experts of 1.97 MB (46 GB): the 32 GB
diff --git a/docs/INTEL_ARC.md b/docs/INTEL_ARC.md
index d9d1d60a1..1389ad597 100644
--- a/docs/INTEL_ARC.md
+++ b/docs/INTEL_ARC.md
@@ -53,8 +53,9 @@ Full flags, per-request timings and engine logs are in the report linked in the
The HIP build is not affected (the option is off by default).
**Not tested by anyone yet:** Windows (no native build path; see below), Arc on WSL2 for the 0.1.39 port, integrated Arc
-GPUs (the B390 / Panther Lake in #515; the shared-memory planning does not exist yet), and images (not wired on Intel: there
-is no picture reader for the Arc engine). The Alchemist A-series was tested later, on an A750 (rows above and
+GPUs (the B390 / Panther Lake in #515; the shared-memory planning does not exist yet), and pictures on a Battlemage card
+(the first picture on the Arc Pro B70 reset the GPU; they are off by default there, [INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)).
+Pictures work on the A750 (on by default, read on the CPU). The Alchemist A-series was tested later, on an A750 (rows above and
[INTEL.md](INTEL.md), "Arc A750 and the other Alchemist cards").
## What you need
diff --git a/docs/MODELS.md b/docs/MODELS.md
index fe4c0b7c3..bf7977cf5 100644
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@ -108,7 +108,9 @@ picture, nothing taken from the card) and on the graphics card from 24 GB (about
picture). The rule is the same on NVIDIA and AMD cards. `--vision off` (or "No" in the app and in `strata launch`) turns
pictures off; `--vision cpu` or `--vision gpu` picks the reader. On AMD, `strata-vision-hip` is the card's reader; if it
cannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac app and the Mac Terminal have no
-picture reader yet ([MAC.md](MAC.md)), and neither does the Intel Arc engine ([INTEL.md](INTEL.md#not-done)). More: [Images](DETAILS.md#images-vision).
+picture reader yet ([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the
+CPU reader unless you ask for `--vision gpu` (the Intel GPU reader has only run on an 8 GB A750), and a Battlemage card (B580, B70)
+has pictures off until you pass `--vision yes` ([INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)). More: [Images](DETAILS.md#images-vision).
### Qwen3.8-Flash-Next
diff --git a/strata_control/vision_reader.py b/strata_control/vision_reader.py
index 143445fe9..3c4a6fa01 100644
--- a/strata_control/vision_reader.py
+++ b/strata_control/vision_reader.py
@@ -1,4 +1,5 @@
-"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip`, AMD) or the CPU one (`strata-vision`).
+"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc) or the CPU one
+(`strata-vision`).
An AMD engine package carries two programs for pictures. `strata-vision` is the CPU reader: it needs nothing from the GPU
runtime, so it starts on every PC. `strata-vision-hip` is the same reader on the card (llama.cpp's HIP backend), a
@@ -7,6 +8,9 @@ reader as its fallback (`vision.cpu_exe`); the server (serve/server.py, Vision)
log line, when the GPU reader does not start: the HIP runtime does not load, the card is not one the build has code for
(the reader says so before it runs a kernel), or the card is busy. Nothing is asked of the user.
+An Intel Arc engine folder (engine-sycl/) carries the CPU reader; `strata-vision-sycl` (llama.cpp's SYCL backend, tools/vision with
+STRATA_VISION_SYCL, built by `setup --backend sycl --vision gpu` on a PC that has the oneAPI compiler) is the GPU reader there.
+
`choose` is the whole decision; setup.py calls it, and tools/test_vision_reader.py checks every branch.
"""
from __future__ import annotations
@@ -17,20 +21,30 @@ from pathlib import Path
CPU_READER = "strata-vision"
GPU_READER = "strata-vision-hip"
+SYCL_READER = "strata-vision-sycl"
+GPU_READERS = (GPU_READER, SYCL_READER) # the GPU reader of an engine folder: the first one it carries
#: Owner decision 2026-10-18: pictures are ON by default, and the reader runs on the CPU for a card under 24 GB (the GPU reader keeps about
#: 1-1.4 GB of VRAM that the experts would use: a few % of decode, more on some PCs, #2406) and on the GPU for 24 GB and above - on every make
#: of card. A person can switch it per model (--vision gpu / --vision cpu, the Pictures step); what was asked for is kept.
GPU_MIN_VRAM_GB = 23.5
+#: The one exception to that rule. An Intel Arc of 24 GB or more keeps the CPU reader by default: the SYCL GPU reader works (tested on an A750,
+#: forced with --vision gpu: 0.16 s a picture against 1.7 s on the CPU, embeddings cosine >= 0.995 to the CPU reader's) but it needs the oneAPI
+#: runtime on the PC and has not run on a 24 GB+ card (the B70 is gone). --vision gpu asks for it. Set True when a 24 GB+ Arc has run it.
+INTEL_GPU_READER_BY_DEFAULT = False
+
-def default_mode(vram_gb: float | None) -> str:
- """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu")."""
+def default_mode(vram_gb: float | None, make: str = "") -> str:
+ """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu"). `make` is "intel"
+ for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT."""
vram = float(vram_gb or 0.0)
+ if make == "intel" and not INTEL_GPU_READER_BY_DEFAULT:
+ return "cpu"
return "cpu" if 0 < vram < GPU_MIN_VRAM_GB else "gpu"
-def wanted(asked: str | None, vram_gb: float | None) -> str:
+def wanted(asked: str | None, vram_gb: float | None, make: str = "") -> str:
"""What is wanted for `asked` (None, "yes", "on", "gpu", "cpu", "no"/"none"/"off"): "gpu", "cpu" or "none". Nothing asked and "yes" mean
the default for this card; "gpu" and "cpu" are kept as asked."""
a = (asked or "").lower()
@@ -38,7 +52,7 @@ def wanted(asked: str | None, vram_gb: float | None) -> str:
return "none"
if a in ("gpu", "cpu"):
return a
- return default_mode(vram_gb)
+ return default_mode(vram_gb, make)
GFX = re.compile(r"gfx[0-9a-f]+", re.IGNORECASE)
@@ -51,9 +65,9 @@ def find(folder: Path | str, windows: bool) -> dict:
"""The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}."""
folder = Path(folder)
found = {}
- for key, base in (("cpu", CPU_READER), ("gpu", GPU_READER)):
- path = folder / reader_name(base, windows)
- found[key] = path if path.is_file() else None
+ for key, bases in (("cpu", (CPU_READER,)), ("gpu", GPU_READERS)):
+ paths = [folder / reader_name(base, windows) for base in bases]
+ found[key] = next((p for p in paths if p.is_file()), None)
return found
diff --git a/sycl/setup_intel.py b/sycl/setup_intel.py
index 3e327d377..2f234b74c 100644
--- a/sycl/setup_intel.py
+++ b/sycl/setup_intel.py
@@ -11,10 +11,11 @@ setup.py's, unchanged. What is replaced:
sized from sysfs;
- the engine step: the SYCL build (build-sycl-aot/strata, run in the strata-sycl-dev image by
sycl/serve/strata-sycl.sh; docs/INTEL.md) instead of a CUDA/HIP build;
- - pictures (opt-in for now, see PICTURES_ON_BY_DEFAULT): read by the portable CPU reader (`strata-vision`, the program the AMD packages carry beside
- their engine, compiled here by setup's own step) on the host; it hands the image embeddings to the engine through
- /dev/shm, which strata-sycl.sh shares with the container. A SYCL GPU reader does not exist yet, so "gpu" is read as "cpu"
- here; --vision no is the off switch;
+ - pictures (on by default on an Alchemist card, see pictures_on_by_default): read by the portable CPU reader (`strata-vision`, the program the
+ AMD packages carry beside their engine, compiled here by setup's own step) on the host; it hands the image embeddings to the engine through
+ /dev/shm, which strata-sycl.sh shares with the container. The reader choice is strata_control/vision_reader.py's one rule (CPU reader under
+ 24 GB of VRAM); an Arc of 24 GB or more also keeps the CPU reader unless --vision gpu asks for the SYCL one (`strata-vision-sycl`, compiled
+ on the host when the oneAPI compiler is there). --vision no is the off switch;
- the RAM rule: the CUDA engine keeps every expert in RAM, the SYCL port streams them from the GGUF into VRAM
(--stream-experts), so RAM only bounds the KV streaming;
- the config: the container's paths, the SYCL flags, backend "sycl"; the run script starts
@@ -122,19 +123,88 @@ def drop(args, name, value=False):
del args[i:i + 1 + int(value)]
-# Pictures on an Intel Arc work end to end up to the engine (the CPU reader encodes, the embeddings reach the container through /dev/shm)
-# but the first picture request on the Arc Pro B70 (engine 0.1.45 SYCL, IQ3_S, 2026-10-19) hung the GPU: the xe driver logged an engine
-# reset (ccs + bcs) and "Fault response: Unsuccessful -EINVAL", and the request ended 503. Until the SYCL engine's vision path is fixed
-# pictures stay opt-in here (--vision yes); flip this to True to make them the default like on the other makes of card.
-PICTURES_ON_BY_DEFAULT = False
+# Pictures on an Intel Arc: the CPU reader encodes on the host and the embeddings reach the engine through /dev/shm. Tested end to end
+# on the Arc A750 (i915, Alchemist; 2026-10-19, docs/INTEL.md): 47 picture requests (shapes, text, two pictures in one request, a picture in the
+# history, text after pictures) through the CPU reader and the same through the SYCL GPU reader, in the container and outside it, no GPU fault.
+# The first picture request on the Arc Pro B70 (Battlemage, xe, engine 0.1.45 SYCL, IQ3_S, 2026-10-19) hung the GPU with the CPU reader: an xe
+# engine reset (ccs + bcs) and "Fault response: Unsuccessful -EINVAL", the request ended 503. That fault is in the engine on xe and has not been
+# looked at again (the B70 is gone), so a Battlemage card keeps pictures opt-in (--vision yes) and is told so.
+PICTURES_ON_BY_DEFAULT_IDS = ALCHEMIST_IDS
+
+
+def pictures_on_by_default(driver: str, devid: str = "") -> bool:
+ """Pictures are on in setup's default for an Alchemist card (i915, or the A-series on xe), which has run them; a Battlemage card
+ (xe, the B-series) keeps them opt-in until the first-picture GPU fault of the B70 is understood."""
+ return driver == "i915" or devid in PICTURES_ON_BY_DEFAULT_IDS
+
READER_DIR = ROOT / "engine-sycl" # the folder setup treats as the engine's: BUILD.json and, for pictures, the CPU reader strata-vision
def intel_vision(asked, vram_gb: float = 0.0) -> str:
- """The picture reader wanted on an Intel Arc: "none" when switched off, else "cpu" (the only reader the SYCL port has; a
- SYCL GPU reader is later work). Takes setup.hip_vision's arguments."""
- return "none" if str(asked or "").lower() in ("no", "none", "off") else "cpu"
+ """The picture reader wanted on an Intel Arc, by strata_control/vision_reader.py's rule: "none" when switched off, "cpu" by default (under 24 GB
+ of VRAM the CPU reader is the rule, and an Arc of 24 GB or more keeps it too until the SYCL reader has run on such a card), "gpu" only when
+ asked for. Takes setup.hip_vision's arguments."""
+ from strata_control import vision_reader as VR
+ return VR.wanted(asked, vram_gb, "intel")
+
+
+def oneapi_compiler():
+ """The oneAPI icx / icpx on this PC (PATH, else /opt/intel/oneapi), or None."""
+ import glob
+ import shutil
+ cc, cxx = shutil.which("icx"), shutil.which("icpx")
+ if not (cc and cxx):
+ found = sorted(glob.glob("/opt/intel/oneapi/compiler/*/bin/icpx"))
+ cxx = found[-1] if found else None
+ cc = str(Path(cxx).with_name("icx")) if cxx else None
+ return (cc, cxx) if cc and cxx and Path(cc).exists() else None
+
+
+def oneapi_environment() -> dict:
+ """The environment setvars.sh makes (the SYCL reader's build needs MKL and the compiler's libraries); {} when there is no setvars.sh."""
+ setvars = Path("/opt/intel/oneapi/setvars.sh")
+ if not setvars.is_file():
+ return {}
+ out = subprocess.run(["/bin/bash", "-c", f". {setvars} >/dev/null 2>&1; env -0"], capture_output=True, check=False).stdout
+ return dict(item.split("=", 1) for item in out.decode("utf-8", "replace").split("\0") if "=" in item)
+
+
+def build_vision_sycl_reader(stub: Path, llama) -> bool:
+ """Compile tools/vision with STRATA_VISION_SYCL into `stub`/strata-vision-sycl. False (and the reason said) when the oneAPI compiler is
+ not on this PC or the build fails: the CPU reader is there, pictures still work."""
+ import shutil
+ compiler = oneapi_compiler()
+ if compiler is None:
+ S.warn("the GPU picture reader needs the oneAPI compiler (icx / icpx), which is not on this PC: pictures are read on the CPU "
+ "(a few seconds each); install intel-oneapi-compiler-dpcpp-cpp and intel-oneapi-mkl-devel to build it")
+ return False
+ S.say(" Compiling the picture reader for your Intel GPU (a few minutes, once) ...")
+ saved = dict(os.environ)
+ try:
+ os.environ.update(oneapi_environment())
+ S.cmake_build(ROOT / "tools" / "vision", ROOT / "build-vision-sycl", "strata-vision",
+ [f"-DLLAMA_DIR={llama}", "-DSTRATA_VISION_SYCL=ON", "-DSTRATA_PORTABLE=OFF",
+ f"-DCMAKE_C_COMPILER={compiler[0]}", f"-DCMAKE_CXX_COMPILER={compiler[1]}"], None, "")
+ shutil.copy2(ROOT / "build-vision-sycl" / "bin" / "strata-vision-sycl", stub / "strata-vision-sycl")
+ except SystemExit:
+ raise
+ except Exception as e: # noqa: BLE001 - the CPU reader is there; pictures work without the GPU one
+ S.warn(f"the GPU picture reader did not compile ({e}): pictures are read on the CPU")
+ return False
+ finally:
+ os.environ.clear()
+ os.environ.update(saved)
+ return True
+
+
+def build_vision_sycl(stub: Path, stamp: Path, llama, vsrc) -> None:
+ meta = json.loads(stamp.read_text(encoding="utf-8"))
+ if (stub / "strata-vision-sycl").exists() and meta.get("vision_gpu_src") == vsrc:
+ return
+ if build_vision_sycl_reader(stub, llama):
+ stamp.write_text(json.dumps({**meta, "vision": "gpu", "vision_gpu": "strata-vision-sycl", "vision_gpu_src": vsrc}, indent=1))
+ S.ok(f"picture reader (GPU): {stub / 'strata-vision-sycl'}, (CPU fallback): {stub / 'strata-vision'}")
SYCL_PROFILE = "expert-profile-sycl.bin" # data/: the xe cards' expert ranking (to_sycl)
@@ -317,10 +387,14 @@ def install(argv) -> None:
S.hip_vision = intel_vision # pictures on by default, read on the CPU (setup's hip_reader settles it)
def build_engine_hip(gpu, llama, vision="none"):
- """No engine to compile here (the SYCL build is separate); the picture reader is: the portable CPU reader, once."""
+ """No engine to compile here (the SYCL build is separate); the picture readers are: the portable CPU reader, once, and for
+ --vision gpu the SYCL GPU reader (a few minutes, once; it needs the oneAPI compiler on this PC, else the CPU reader stays)."""
if vision != "none":
stamp = stub / "BUILD.json"
- S.build_vision_cpu(stub, stamp, json.loads(stamp.read_text(encoding="utf-8")), llama, S.source_hash(S.VISION_SOURCES))
+ vsrc = S.source_hash(S.VISION_SOURCES)
+ S.build_vision_cpu(stub, stamp, json.loads(stamp.read_text(encoding="utf-8")), llama, vsrc)
+ if vision == "gpu":
+ build_vision_sycl(stub, stamp, llama, vsrc)
return stub
S.build_engine_hip = build_engine_hip
S.hipblaslt_table = lambda *a, **k: None
@@ -403,8 +477,10 @@ def install(argv) -> None:
sys.argv = [str(ROOT / "setup.py"), *argv]
if "--backend" not in argv:
sys.argv += ["--backend", "hip"]
- if "--vision" not in argv and not PICTURES_ON_BY_DEFAULT:
+ if "--vision" not in argv and not pictures_on_by_default(intel[0]["driver"], intel[0].get("devid", "")):
sys.argv += ["--vision", "none"]
+ S.say(" Pictures are off by default on this card (Arc Pro B70 test of 2026-10-19: the first picture reset the GPU, docs/INTEL.md): "
+ "--vision yes turns them on")
if "--low-ram" not in argv:
sys.argv += ["--low-ram", "off"]
sys.exit(S.main())
diff --git a/tools/test_setup_sycl.py b/tools/test_setup_sycl.py
index 25d1bca59..ff6333f7a 100644
--- a/tools/test_setup_sycl.py
+++ b/tools/test_setup_sycl.py
@@ -223,12 +223,15 @@ class ToSycl(unittest.TestCase):
self.assertTrue(m.small_card(10.0, "xe"))
self.assertFalse(m.small_card(32.0, "xe"))
- def test_pictures_asked_for_are_read_on_the_cpu_and_the_vision_section_survives(self):
- """Intel Arc: the CPU reader (strata-vision) on the host when asked for (--vision yes); "gpu" is read as "cpu" (no SYCL GPU reader)."""
+ def test_pictures_follow_the_one_reader_rule_and_the_vision_section_survives(self):
+ """Intel Arc: pictures on gives the CPU reader (strata-vision) on the host under 24 GB and, until a 24 GB+ Arc has run the SYCL reader,
+ also from 24 GB; only --vision gpu asks for the SYCL reader (strata_control/vision_reader.py is the one place for the rule)."""
m = self.mod()
- for asked in (None, "yes", "gpu", "cpu", "on"):
+ for asked in (None, "yes", "cpu", "on"):
self.assertEqual(m.intel_vision(asked, 8.0), "cpu")
self.assertEqual(m.intel_vision(asked, 32.0), "cpu")
+ self.assertEqual(m.intel_vision("gpu", 8.0), "gpu")
+ self.assertEqual(m.intel_vision("gpu", 32.0), "gpu")
for asked in ("no", "none", "off"):
self.assertEqual(m.intel_vision(asked, 32.0), "none")
section = {"exe": "/h/engine-sycl/strata-vision", "mmproj": "/h/models/mmproj.gguf", "model": "/h/models/m.gguf", "gpu": False,
@@ -239,14 +242,28 @@ class ToSycl(unittest.TestCase):
self.assertEqual(c["vision"], section) # host paths: the reader runs on the host, not in the container
self.assertIn("--vision", c["args"])
- def test_pictures_are_opt_in_on_the_arc_until_the_engine_fault_is_fixed(self):
- """The first picture on the Arc Pro B70 hung the GPU (xe engine reset, 2026-10-19): off unless asked, --vision yes turns it on."""
+ def test_pictures_are_on_by_default_on_an_alchemist_and_opt_in_on_a_battlemage(self):
+ """The A750 ran 47 picture requests without a fault; the first picture on the Arc Pro B70 reset the GPU (xe, 2026-10-19), so a
+ Battlemage card keeps them opt-in (--vision yes) and setup says so."""
m = self.mod()
- self.assertFalse(m.PICTURES_ON_BY_DEFAULT)
+ self.assertTrue(m.pictures_on_by_default("i915", "56a1")) # A750
+ self.assertTrue(m.pictures_on_by_default("i915", ""))
+ self.assertTrue(m.pictures_on_by_default("xe", "56a0")) # A770
+ self.assertFalse(m.pictures_on_by_default("xe", "e223")) # B70
+ self.assertFalse(m.pictures_on_by_default("xe", "e20b")) # B580
src = (ROOT / "sycl" / "setup_intel.py").read_text(encoding="utf-8")
- self.assertIn('if "--vision" not in argv and not PICTURES_ON_BY_DEFAULT:', src)
+ self.assertIn('if "--vision" not in argv and not pictures_on_by_default(', src)
self.assertIn("S.hip_vision = intel_vision", src)
self.assertIn("S.build_vision_cpu(", src) # the CPU reader is compiled by setup's own step
+ self.assertIn("build_vision_sycl(", src) # --vision gpu: the SYCL reader, when the oneAPI compiler is there
+
+ def test_the_sycl_reader_is_built_only_when_asked_and_the_cpu_reader_stays_without_the_compiler(self):
+ from unittest import mock
+ m = self.mod()
+ with mock.patch.object(m, "oneapi_compiler", return_value=None), mock.patch.object(m.S, "warn") as warn:
+ self.assertFalse(m.build_vision_sycl_reader(Path("/nonexistent"), "/llama"))
+ self.assertIn("oneAPI compiler", warn.call_args[0][0])
+ self.assertIn("pictures are read on the CPU", warn.call_args[0][0])
def test_the_container_shares_dev_shm_with_the_server_for_the_embeddings(self):
sh = (ROOT / "sycl" / "serve" / "strata-sycl.sh").read_text(encoding="utf-8")
diff --git a/tools/test_vision_reader.py b/tools/test_vision_reader.py
index e840aafac..fa1ff6200 100644
--- a/tools/test_vision_reader.py
+++ b/tools/test_vision_reader.py
@@ -35,6 +35,28 @@ class Choose(unittest.TestCase):
("no", 32, "none"), ("off", 12, "none")):
self.assertEqual(VR.wanted(asked, vram), want, (asked, vram))
+ def test_an_intel_arc_keeps_the_cpu_reader_by_default_and_gpu_is_asked_for(self):
+ """The one exception to the 24 GB rule: the SYCL reader has run on a 8 GB A750 only (INTEL_GPU_READER_BY_DEFAULT)."""
+ self.assertFalse(VR.INTEL_GPU_READER_BY_DEFAULT)
+ for vram in (8, 16, 24, 32):
+ self.assertEqual(VR.default_mode(vram, "intel"), "cpu")
+ self.assertEqual(VR.wanted("yes", vram, "intel"), "cpu")
+ self.assertEqual(VR.wanted("gpu", vram, "intel"), "gpu")
+ self.assertEqual(VR.wanted("yes", 32), "gpu") # other makes: unchanged
+ with mock.patch.object(VR, "INTEL_GPU_READER_BY_DEFAULT", True):
+ self.assertEqual(VR.default_mode(32, "intel"), "gpu")
+ self.assertEqual(VR.default_mode(8, "intel"), "cpu") # with the exception off, the 24 GB rule is the same as for every make
+
+ def test_the_sycl_reader_is_the_gpu_reader_of_an_intel_folder(self):
+ d = folder("strata-vision", "strata-vision-sycl")
+ r = VR.choose(d, "gpu", windows=False)
+ self.assertEqual((r.mode, r.exe.name, r.cpu_exe.name), ("gpu", "strata-vision-sycl", "strata-vision"))
+ self.assertEqual(VR.vision_section(r, {"mmproj": "m"})["gpu"], True)
+ r = VR.choose(folder("strata-vision"), "gpu", windows=False) # no SYCL reader built: the CPU one, said
+ self.assertEqual((r.mode, r.exe.name), ("cpu", "strata-vision"))
+ self.assertIn("no GPU picture reader", r.note)
+ self.assertEqual(VR.choose(d, "cpu", windows=False).exe.name, "strata-vision")
+
def test_hip_present_gives_the_gpu_reader_with_the_cpu_one_to_fall_back_on(self):
d = folder("strata-vision", "strata-vision-hip")
for ask in ("yes", "on", "gpu"):
diff --git a/tools/vision/CMakeLists.txt b/tools/vision/CMakeLists.txt
index 5f031b9ce..5ad3772c1 100644
--- a/tools/vision/CMakeLists.txt
+++ b/tools/vision/CMakeLists.txt
@@ -3,6 +3,7 @@
#
# cmake -S tools/vision -B build-vision -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=third_party/llama.cpp
# [-DSTRATA_VISION_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native] [-DSTRATA_PORTABLE=OFF]
+# [-DSTRATA_VISION_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx (oneAPI sourced; Linux)]
# [-DSTRATA_VISION_HIP=ON -DSTRATA_VISION_HIP_ARCHS="gfx1200;gfx1201" + the HIP compiler the engine build uses]
# cmake --build build-vision --target strata-vision
#
@@ -16,10 +17,14 @@ project(strata_vision C CXX)
set(LLAMA_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../third_party/llama.cpp" CACHE PATH "llama.cpp checkout")
option(STRATA_VISION_CUDA "run the vision encoder on the GPU (builds ggml-cuda)" OFF)
option(STRATA_VISION_HIP "run the vision encoder on an AMD GPU (builds ggml-hip); the program is named strata-vision-hip" OFF)
+option(STRATA_VISION_SYCL "run the vision encoder on an Intel GPU (builds ggml-sycl, needs the oneAPI icx/icpx compilers); the program is named strata-vision-sycl" OFF)
set(STRATA_VISION_HIP_ARCHS "" CACHE STRING "the gfx architectures strata-vision-hip is compiled for (';'-separated)")
if(STRATA_VISION_HIP AND STRATA_VISION_CUDA)
message(FATAL_ERROR "strata-vision: STRATA_VISION_CUDA and STRATA_VISION_HIP are two different programs")
endif()
+if(STRATA_VISION_SYCL AND (STRATA_VISION_CUDA OR STRATA_VISION_HIP))
+ message(FATAL_ERROR "strata-vision: STRATA_VISION_SYCL is its own program: not together with CUDA or HIP")
+endif()
if(STRATA_VISION_HIP AND NOT STRATA_VISION_HIP_ARCHS)
message(FATAL_ERROR "strata-vision: STRATA_VISION_HIP needs STRATA_VISION_HIP_ARCHS, for example \"gfx1200;gfx1201\"")
endif()
@@ -62,6 +67,10 @@ endif()
set(GGML_BACKEND_DL OFF CACHE BOOL "" FORCE)
set(GGML_CUDA ${STRATA_VISION_CUDA} CACHE BOOL "" FORCE)
set(GGML_HIP ${STRATA_VISION_HIP} CACHE BOOL "" FORCE)
+set(GGML_SYCL ${STRATA_VISION_SYCL} CACHE BOOL "" FORCE)
+if(STRATA_VISION_SYCL)
+ set(GGML_SYCL_TARGET "INTEL" CACHE STRING "" FORCE)
+endif()
if(STRATA_VISION_HIP)
# the engine's HIP build says the same (cmake/hip_backend.cmake); ggml-hip reads AMDGPU_TARGETS / CMAKE_HIP_ARCHITECTURES
set(AMDGPU_TARGETS "${STRATA_VISION_HIP_ARCHS}" CACHE STRING "" FORCE)
@@ -82,6 +91,11 @@ if(STRATA_VISION_HIP)
string(REPLACE ";" "," _hip_archs "${STRATA_VISION_HIP_ARCHS}")
target_compile_definitions(strata-vision PRIVATE STRATA_VISION_HIP STRATA_VISION_HIP_ARCHS="${_hip_archs}")
endif()
+if(STRATA_VISION_SYCL)
+ # the Intel GPU reader is its own program too, beside the CPU reader "strata-vision" it falls back on
+ set_target_properties(strata-vision PROPERTIES OUTPUT_NAME strata-vision-sycl)
+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_SYCL)
+endif()
if(WIN32 AND STRATA_VISION_HIP)
# hipblas.dll (and rocblas.dll behind it) live in rocm\bin of the engine package: loaded on first use, after main() has put that folder on
# the DLL search path, so the reader starts from any PATH and says "no GPU" itself where there is none
@@ -93,6 +107,14 @@ if(UNIX AND NOT APPLE)
set_target_properties(strata-vision PROPERTIES BUILD_WITH_INSTALL_RPATH ON INSTALL_RPATH "$ORIGIN")
target_link_options(strata-vision PRIVATE -Wl,--enable-new-dtags)
endif()
+if(STRATA_VISION_SYCL AND UNIX AND NOT APPLE)
+ # libsycl and the OpenMP runtime live in the oneAPI compiler's lib folder, which only a sourced setvars.sh puts on the search path:
+ # recorded in the program, it starts from a plain shell (the server starts it with the PC's own environment)
+ get_filename_component(_icpx_bin "${CMAKE_CXX_COMPILER}" REALPATH)
+ get_filename_component(_icpx_bin "${_icpx_bin}" DIRECTORY)
+ get_filename_component(_icpx_lib "${_icpx_bin}/../lib" ABSOLUTE)
+ set_target_properties(strata-vision PROPERTIES INSTALL_RPATH "$ORIGIN;${_icpx_lib}")
+endif()
# opens a GGUF that has tensor types llama.cpp does not know (a Gyro quantization's TQK6 / TQK7) for its vocabulary only
add_executable(strata-vision-gguf-test EXCLUDE_FROM_ALL gguf_meta_test.cpp)
diff --git a/tools/vision/strata_vision.cpp b/tools/vision/strata_vision.cpp
index eef3e81c9..c5d3ab915 100644
--- a/tools/vision/strata_vision.cpp
+++ b/tools/vision/strata_vision.cpp
@@ -102,6 +102,8 @@ int main(int argc, char** argv) {
else if (a == "--version") {
#ifdef STRATA_VISION_HIP
std::printf("strata-vision hip (GPU architectures: %s)\n", STRATA_VISION_HIP_ARCHS);
+#elif defined(STRATA_VISION_SYCL)
+ std::printf("strata-vision sycl (Intel GPU)\n");
#else
std::printf("strata-vision\n");
#endif
@@ -137,9 +139,14 @@ int main(int argc, char** argv) {
llama_log_set(quiet_log, nullptr);
mtmd_helper_log_set(quiet_log, nullptr);
llama_backend_init();
-#ifdef STRATA_VISION_HIP
+#if defined(STRATA_VISION_HIP) || defined(STRATA_VISION_SYCL)
+#ifdef STRATA_VISION_SYCL
+#define STRATA_VISION_BUILT_FOR "Intel GPUs"
+#else
+#define STRATA_VISION_BUILT_FOR STRATA_VISION_HIP_ARCHS
+#endif
if (gpu) {
- // The AMD GPU reader: starting the backends prints the GPU line. With --gpu and no GPU, mtmd would quietly use the CPU
+ // The AMD / Intel GPU reader: starting the backends prints the GPU line. With --gpu and no GPU, mtmd would quietly use the CPU
// (a second, slower CPU reader inside the HIP program), and a card the build has no code for would abort at its first
// kernel. Both end here with a line the server reads: it then starts the CPU reader (serve/server.py, Vision._start).
bool have_gpu = false;
@@ -148,10 +155,11 @@ int main(int argc, char** argv) {
have_gpu = have_gpu || type == GGML_BACKEND_DEVICE_TYPE_GPU || type == GGML_BACKEND_DEVICE_TYPE_IGPU;
}
if (!have_gpu) {
- std::printf("ERR this picture reader found no GPU (it was built for: %s)\n", STRATA_VISION_HIP_ARCHS);
+ std::printf("ERR this picture reader found no GPU (it was built for: %s)\n", STRATA_VISION_BUILT_FOR);
std::fflush(stdout);
return 3;
}
+#ifdef STRATA_VISION_HIP
const char* check = std::getenv("STRATA_VISION_ARCH_CHECK");
if (!g_gfx.empty() && (check == nullptr || std::atoi(check) != 0) &&
!strata_arch_listed(STRATA_VISION_HIP_ARCHS, g_gfx.front())) {
@@ -160,6 +168,7 @@ int main(int argc, char** argv) {
std::fflush(stdout);
return 3;
}
+#endif
}
#endif