Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Intel Arc pictures: on by default for the Alchemist cards (CPU reader), Battlemage stays opt-in; SYCL GPU picture reader (strata-vision-sycl) for --vision gpu; reader rule in vision_reader.py; tests and docs
Niko1221cac4816afdea
diff --git a/docs/INTEL.md b/docs/INTEL.mdindex eb18988d1..f77bb006a 100644--- a/docs/INTEL.md+++ b/docs/INTEL.md@@ -789,17 +789,39 @@ the fix).## Not done-- Images on the llama.cpp config: not wired. On the SYCL port pictures are wired but opt-in (`setup --backend sycl --vision yes`):- setup compiles the portable CPU reader `strata-vision` (the program the AMD packages carry beside their engine) into `engine-sycl/`,- the server runs it on the host, and the engine in the container reads its embeddings from the host's `/dev/shm`, which- `strata-sycl.sh` shares (`STRATA_SYCL_SHM=0` keeps a private one). A SYCL GPU reader does not exist yet, so every Arc reads- pictures on the CPU (a few seconds each). They are not on by default because the first picture request on the Arc Pro B70- (engine 0.1.45, IQ3_S, 2026-10-19) hung the GPU: the xe driver logged an engine reset and `Fault response: Unsuccessful -EINVAL`- and the request ended 503. Text was not affected by this wiring (the reader and the shared `/dev/shm` are only used for pictures);- the engine's vision path on SYCL needs a look first (`setup_intel.PICTURES_ON_BY_DEFAULT` flips the default).+- Images on the llama.cpp config: not wired. On the SYCL port they are wired, see "Pictures on an Arc" below. Open there: a+ Battlemage card (the B-series) keeps pictures opt-in because of the Arc Pro B70 fault, and nobody has run the SYCL picture reader+ on a card of 24 GB or more.- Speculative decoding on the llama.cpp path (the GGUF carries no draft layer llama.cpp can use). The SYCLport has it (MTP draft layer, `--mtp`).+## Pictures on an Arc (2026-10-19, branch f-021-arcvis)++Pictures follow the rule of every other card ([MODELS.md](MODELS.md#the-versions)): a picture reader on the CPU under 24 GB of VRAM, on+the card from 24 GB. `strata_control/vision_reader.py` is the one place for it. On an Arc:++- **The CPU reader** is `strata-vision` (llama.cpp's mtmd). Setup compiles it into `engine-sycl/`, the server runs it on the host, and the+ engine in the container reads the picture's embeddings from the host's `/dev/shm`, which `strata-sycl.sh` shares+ (`STRATA_SYCL_SHM=0` keeps a private one). On the A750 a 448x448 picture takes 1.73 s.+- **The SYCL reader** is `strata-vision-sycl`, the same program on llama.cpp's SYCL backend (`tools/vision`, `STRATA_VISION_SYCL=ON`).+ `setup --backend sycl --vision gpu` compiles it on the host when the oneAPI compiler (`icx`, `icpx`, `intel-oneapi-mkl-devel`) is+ installed; without it setup says so and keeps the CPU reader. It needs the PC's oneAPI runtime, not the container. If it does not+ start, the server falls back to the CPU reader with one log line. On the A750: 0.16 s per picture instead of 1.73 s (after a 30 s+ first start that compiles the kernels), and the embeddings match the CPU reader's (cosine 0.9959 to 0.9999 over 8 pictures). It takes+ VRAM the experts would use, which is why the rule gives it to 24 GB cards only.+- **Default.** On for an Alchemist card (A750, A770, ...) with the CPU reader. An Arc of 24 GB or more also keeps the CPU reader by+ default: the SYCL reader has not run on such a card (the B70 is gone), so `--vision gpu` asks for it and+ `vision_reader.INTEL_GPU_READER_BY_DEFAULT` is the switch to flip once one has. A Battlemage card (B580, B70, ...) keeps pictures+ **off** by default: the first picture on the Arc Pro B70 (engine 0.1.45, IQ3_S, 2026-10-19, CPU reader) logged an xe engine reset+ (ccs + bcs) and `Fault response: Unsuccessful -EINVAL`, and the request ended 503. The CPU reader was in use, so the fault is in the engine on xe, not in a GPU reader; it+ has not been looked at since. `--vision yes` turns pictures on there; setup says so.++Tested on the A750 (IQ3_XXS, 8K context, engine from f-021-int, JIT build, 2026-10-19): a picture with a red circle, a blue square and the+words "STRATA 42", 8 variants, answered correctly by the CPU reader 10 of 10 (a first set with smaller text read "42" as "412"), and+by the SYCL reader 10 of 10; two pictures in one request, a picture kept in the history of a second turn, and a long text prompt after the+pictures all answered; a text request after every picture answered; 47 picture requests, outside and inside the container (inside, one+answer was cut off by the 200-token limit of the test), no GPU fault in the kernel log, no 503.+## Arc Pro B70 with a model that does not fit: Flash-Next IQ3_S (2026-10-07, branch f-141-intel)The earlier B70 rows are for models whose experts all fit in VRAM. IQ3_S has 24,576 experts of 1.97 MB (46 GB): the 32 GBdiff --git a/docs/INTEL_ARC.md b/docs/INTEL_ARC.mdindex d9d1d60a1..1389ad597 100644--- a/docs/INTEL_ARC.md+++ b/docs/INTEL_ARC.md@@ -53,8 +53,9 @@ Full flags, per-request timings and engine logs are in the report linked in theThe HIP build is not affected (the option is off by default).**Not tested by anyone yet:** Windows (no native build path; see below), Arc on WSL2 for the 0.1.39 port, integrated Arc-GPUs (the B390 / Panther Lake in #515; the shared-memory planning does not exist yet), and images (not wired on Intel: there-is no picture reader for the Arc engine). The Alchemist A-series was tested later, on an A750 (rows above and+GPUs (the B390 / Panther Lake in #515; the shared-memory planning does not exist yet), and pictures on a Battlemage card+(the first picture on the Arc Pro B70 reset the GPU; they are off by default there, [INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)).+Pictures work on the A750 (on by default, read on the CPU). The Alchemist A-series was tested later, on an A750 (rows above and[INTEL.md](INTEL.md), "Arc A750 and the other Alchemist cards").## What you needdiff --git a/docs/MODELS.md b/docs/MODELS.mdindex fe4c0b7c3..bf7977cf5 100644--- a/docs/MODELS.md+++ b/docs/MODELS.md@@ -108,7 +108,9 @@ picture, nothing taken from the card) and on the graphics card from 24 GB (aboutpicture). The rule is the same on NVIDIA and AMD cards. `--vision off` (or "No" in the app and in `strata launch`) turnspictures off; `--vision cpu` or `--vision gpu` picks the reader. On AMD, `strata-vision-hip` is the card's reader; if itcannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). The Mac app and the Mac Terminal have no-picture reader yet ([MAC.md](MAC.md)), and neither does the Intel Arc engine ([INTEL.md](INTEL.md#not-done)). More: [Images](DETAILS.md#images-vision).+picture reader yet ([MAC.md](MAC.md)). On an Intel Arc the rule is the same, with two differences: an Arc of 24 GB or more keeps the+CPU reader unless you ask for `--vision gpu` (the Intel GPU reader has only run on an 8 GB A750), and a Battlemage card (B580, B70)+has pictures off until you pass `--vision yes` ([INTEL.md](INTEL.md#pictures-on-an-arc-2026-10-19-branch-f-021-arcvis)). More: [Images](DETAILS.md#images-vision).### Qwen3.8-Flash-Nextdiff --git a/strata_control/vision_reader.py b/strata_control/vision_reader.pyindex 143445fe9..3c4a6fa01 100644--- a/strata_control/vision_reader.py+++ b/strata_control/vision_reader.py@@ -1,4 +1,5 @@-"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip`, AMD) or the CPU one (`strata-vision`).+"""Which picture reader an engine folder gets: the GPU one (`strata-vision-hip` on AMD, `strata-vision-sycl` on Intel Arc) or the CPU one+(`strata-vision`).An AMD engine package carries two programs for pictures. `strata-vision` is the CPU reader: it needs nothing from the GPUruntime, so it starts on every PC. `strata-vision-hip` is the same reader on the card (llama.cpp's HIP backend), a@@ -7,6 +8,9 @@ reader as its fallback (`vision.cpu_exe`); the server (serve/server.py, Vision)log line, when the GPU reader does not start: the HIP runtime does not load, the card is not one the build has code for(the reader says so before it runs a kernel), or the card is busy. Nothing is asked of the user.+An Intel Arc engine folder (engine-sycl/) carries the CPU reader; `strata-vision-sycl` (llama.cpp's SYCL backend, tools/vision with+STRATA_VISION_SYCL, built by `setup --backend sycl --vision gpu` on a PC that has the oneAPI compiler) is the GPU reader there.+`choose` is the whole decision; setup.py calls it, and tools/test_vision_reader.py checks every branch."""from __future__ import annotations@@ -17,20 +21,30 @@ from pathlib import PathCPU_READER = "strata-vision"GPU_READER = "strata-vision-hip"+SYCL_READER = "strata-vision-sycl"+GPU_READERS = (GPU_READER, SYCL_READER) # the GPU reader of an engine folder: the first one it carries#: Owner decision 2026-10-18: pictures are ON by default, and the reader runs on the CPU for a card under 24 GB (the GPU reader keeps about#: 1-1.4 GB of VRAM that the experts would use: a few % of decode, more on some PCs, #2406) and on the GPU for 24 GB and above - on every make#: of card. A person can switch it per model (--vision gpu / --vision cpu, the Pictures step); what was asked for is kept.GPU_MIN_VRAM_GB = 23.5+#: The one exception to that rule. An Intel Arc of 24 GB or more keeps the CPU reader by default: the SYCL GPU reader works (tested on an A750,+#: forced with --vision gpu: 0.16 s a picture against 1.7 s on the CPU, embeddings cosine >= 0.995 to the CPU reader's) but it needs the oneAPI+#: runtime on the PC and has not run on a 24 GB+ card (the B70 is gone). --vision gpu asks for it. Set True when a 24 GB+ Arc has run it.+INTEL_GPU_READER_BY_DEFAULT = False+-def default_mode(vram_gb: float | None) -> str:- """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu")."""+def default_mode(vram_gb: float | None, make: str = "") -> str:+ """"cpu" or "gpu": the reader an unspecific "pictures on" gets on a card with this much memory (unknown memory: "gpu"). `make` is "intel"+ for an Arc, which stays on the CPU reader until INTEL_GPU_READER_BY_DEFAULT."""vram = float(vram_gb or 0.0)+ if make == "intel" and not INTEL_GPU_READER_BY_DEFAULT:+ return "cpu"return "cpu" if 0 < vram < GPU_MIN_VRAM_GB else "gpu"-def wanted(asked: str | None, vram_gb: float | None) -> str:+def wanted(asked: str | None, vram_gb: float | None, make: str = "") -> str:"""What is wanted for `asked` (None, "yes", "on", "gpu", "cpu", "no"/"none"/"off"): "gpu", "cpu" or "none". Nothing asked and "yes" meanthe default for this card; "gpu" and "cpu" are kept as asked."""a = (asked or "").lower()@@ -38,7 +52,7 @@ def wanted(asked: str | None, vram_gb: float | None) -> str:return "none"if a in ("gpu", "cpu"):return a- return default_mode(vram_gb)+ return default_mode(vram_gb, make)GFX = re.compile(r"gfx[0-9a-f]+", re.IGNORECASE)@@ -51,9 +65,9 @@ def find(folder: Path | str, windows: bool) -> dict:"""The readers in an engine folder: {"cpu": Path | None, "gpu": Path | None}."""folder = Path(folder)found = {}- for key, base in (("cpu", CPU_READER), ("gpu", GPU_READER)):- path = folder / reader_name(base, windows)- found[key] = path if path.is_file() else None+ for key, bases in (("cpu", (CPU_READER,)), ("gpu", GPU_READERS)):+ paths = [folder / reader_name(base, windows) for base in bases]+ found[key] = next((p for p in paths if p.is_file()), None)return founddiff --git a/sycl/setup_intel.py b/sycl/setup_intel.pyindex 3e327d377..2f234b74c 100644--- a/sycl/setup_intel.py+++ b/sycl/setup_intel.py@@ -11,10 +11,11 @@ setup.py's, unchanged. What is replaced:sized from sysfs;- the engine step: the SYCL build (build-sycl-aot/strata, run in the strata-sycl-dev image bysycl/serve/strata-sycl.sh; docs/INTEL.md) instead of a CUDA/HIP build;- - pictures (opt-in for now, see PICTURES_ON_BY_DEFAULT): read by the portable CPU reader (`strata-vision`, the program the AMD packages carry beside- their engine, compiled here by setup's own step) on the host; it hands the image embeddings to the engine through- /dev/shm, which strata-sycl.sh shares with the container. A SYCL GPU reader does not exist yet, so "gpu" is read as "cpu"- here; --vision no is the off switch;+ - pictures (on by default on an Alchemist card, see pictures_on_by_default): read by the portable CPU reader (`strata-vision`, the program the+ AMD packages carry beside their engine, compiled here by setup's own step) on the host; it hands the image embeddings to the engine through+ /dev/shm, which strata-sycl.sh shares with the container. The reader choice is strata_control/vision_reader.py's one rule (CPU reader under+ 24 GB of VRAM); an Arc of 24 GB or more also keeps the CPU reader unless --vision gpu asks for the SYCL one (`strata-vision-sycl`, compiled+ on the host when the oneAPI compiler is there). --vision no is the off switch;- the RAM rule: the CUDA engine keeps every expert in RAM, the SYCL port streams them from the GGUF into VRAM(--stream-experts), so RAM only bounds the KV streaming;- the config: the container's paths, the SYCL flags, backend "sycl"; the run script starts@@ -122,19 +123,88 @@ def drop(args, name, value=False):del args[i:i + 1 + int(value)]-# Pictures on an Intel Arc work end to end up to the engine (the CPU reader encodes, the embeddings reach the container through /dev/shm)-# but the first picture request on the Arc Pro B70 (engine 0.1.45 SYCL, IQ3_S, 2026-10-19) hung the GPU: the xe driver logged an engine-# reset (ccs + bcs) and "Fault response: Unsuccessful -EINVAL", and the request ended 503. Until the SYCL engine's vision path is fixed-# pictures stay opt-in here (--vision yes); flip this to True to make them the default like on the other makes of card.-PICTURES_ON_BY_DEFAULT = False+# Pictures on an Intel Arc: the CPU reader encodes on the host and the embeddings reach the engine through /dev/shm. Tested end to end+# on the Arc A750 (i915, Alchemist; 2026-10-19, docs/INTEL.md): 47 picture requests (shapes, text, two pictures in one request, a picture in the+# history, text after pictures) through the CPU reader and the same through the SYCL GPU reader, in the container and outside it, no GPU fault.+# The first picture request on the Arc Pro B70 (Battlemage, xe, engine 0.1.45 SYCL, IQ3_S, 2026-10-19) hung the GPU with the CPU reader: an xe+# engine reset (ccs + bcs) and "Fault response: Unsuccessful -EINVAL", the request ended 503. That fault is in the engine on xe and has not been+# looked at again (the B70 is gone), so a Battlemage card keeps pictures opt-in (--vision yes) and is told so.+PICTURES_ON_BY_DEFAULT_IDS = ALCHEMIST_IDS+++def pictures_on_by_default(driver: str, devid: str = "") -> bool:+ """Pictures are on in setup's default for an Alchemist card (i915, or the A-series on xe), which has run them; a Battlemage card+ (xe, the B-series) keeps them opt-in until the first-picture GPU fault of the B70 is understood."""+ return driver == "i915" or devid in PICTURES_ON_BY_DEFAULT_IDS+READER_DIR = ROOT / "engine-sycl" # the folder setup treats as the engine's: BUILD.json and, for pictures, the CPU reader strata-visiondef intel_vision(asked, vram_gb: float = 0.0) -> str:- """The picture reader wanted on an Intel Arc: "none" when switched off, else "cpu" (the only reader the SYCL port has; a- SYCL GPU reader is later work). Takes setup.hip_vision's arguments."""- return "none" if str(asked or "").lower() in ("no", "none", "off") else "cpu"+ """The picture reader wanted on an Intel Arc, by strata_control/vision_reader.py's rule: "none" when switched off, "cpu" by default (under 24 GB+ of VRAM the CPU reader is the rule, and an Arc of 24 GB or more keeps it too until the SYCL reader has run on such a card), "gpu" only when+ asked for. Takes setup.hip_vision's arguments."""+ from strata_control import vision_reader as VR+ return VR.wanted(asked, vram_gb, "intel")+++def oneapi_compiler():+ """The oneAPI icx / icpx on this PC (PATH, else /opt/intel/oneapi), or None."""+ import glob+ import shutil+ cc, cxx = shutil.which("icx"), shutil.which("icpx")+ if not (cc and cxx):+ found = sorted(glob.glob("/opt/intel/oneapi/compiler/*/bin/icpx"))+ cxx = found[-1] if found else None+ cc = str(Path(cxx).with_name("icx")) if cxx else None+ return (cc, cxx) if cc and cxx and Path(cc).exists() else None+++def oneapi_environment() -> dict:+ """The environment setvars.sh makes (the SYCL reader's build needs MKL and the compiler's libraries); {} when there is no setvars.sh."""+ setvars = Path("/opt/intel/oneapi/setvars.sh")+ if not setvars.is_file():+ return {}+ out = subprocess.run(["/bin/bash", "-c", f". {setvars} >/dev/null 2>&1; env -0"], capture_output=True, check=False).stdout+ return dict(item.split("=", 1) for item in out.decode("utf-8", "replace").split("\0") if "=" in item)+++def build_vision_sycl_reader(stub: Path, llama) -> bool:+ """Compile tools/vision with STRATA_VISION_SYCL into `stub`/strata-vision-sycl. False (and the reason said) when the oneAPI compiler is+ not on this PC or the build fails: the CPU reader is there, pictures still work."""+ import shutil+ compiler = oneapi_compiler()+ if compiler is None:+ S.warn("the GPU picture reader needs the oneAPI compiler (icx / icpx), which is not on this PC: pictures are read on the CPU "+ "(a few seconds each); install intel-oneapi-compiler-dpcpp-cpp and intel-oneapi-mkl-devel to build it")+ return False+ S.say(" Compiling the picture reader for your Intel GPU (a few minutes, once) ...")+ saved = dict(os.environ)+ try:+ os.environ.update(oneapi_environment())+ S.cmake_build(ROOT / "tools" / "vision", ROOT / "build-vision-sycl", "strata-vision",+ [f"-DLLAMA_DIR={llama}", "-DSTRATA_VISION_SYCL=ON", "-DSTRATA_PORTABLE=OFF",+ f"-DCMAKE_C_COMPILER={compiler[0]}", f"-DCMAKE_CXX_COMPILER={compiler[1]}"], None, "")+ shutil.copy2(ROOT / "build-vision-sycl" / "bin" / "strata-vision-sycl", stub / "strata-vision-sycl")+ except SystemExit:+ raise+ except Exception as e: # noqa: BLE001 - the CPU reader is there; pictures work without the GPU one+ S.warn(f"the GPU picture reader did not compile ({e}): pictures are read on the CPU")+ return False+ finally:+ os.environ.clear()+ os.environ.update(saved)+ return True+++def build_vision_sycl(stub: Path, stamp: Path, llama, vsrc) -> None:+ meta = json.loads(stamp.read_text(encoding="utf-8"))+ if (stub / "strata-vision-sycl").exists() and meta.get("vision_gpu_src") == vsrc:+ return+ if build_vision_sycl_reader(stub, llama):+ stamp.write_text(json.dumps({**meta, "vision": "gpu", "vision_gpu": "strata-vision-sycl", "vision_gpu_src": vsrc}, indent=1))+ S.ok(f"picture reader (GPU): {stub / 'strata-vision-sycl'}, (CPU fallback): {stub / 'strata-vision'}")SYCL_PROFILE = "expert-profile-sycl.bin" # data/: the xe cards' expert ranking (to_sycl)@@ -317,10 +387,14 @@ def install(argv) -> None:S.hip_vision = intel_vision # pictures on by default, read on the CPU (setup's hip_reader settles it)def build_engine_hip(gpu, llama, vision="none"):- """No engine to compile here (the SYCL build is separate); the picture reader is: the portable CPU reader, once."""+ """No engine to compile here (the SYCL build is separate); the picture readers are: the portable CPU reader, once, and for+ --vision gpu the SYCL GPU reader (a few minutes, once; it needs the oneAPI compiler on this PC, else the CPU reader stays)."""if vision != "none":stamp = stub / "BUILD.json"- S.build_vision_cpu(stub, stamp, json.loads(stamp.read_text(encoding="utf-8")), llama, S.source_hash(S.VISION_SOURCES))+ vsrc = S.source_hash(S.VISION_SOURCES)+ S.build_vision_cpu(stub, stamp, json.loads(stamp.read_text(encoding="utf-8")), llama, vsrc)+ if vision == "gpu":+ build_vision_sycl(stub, stamp, llama, vsrc)return stubS.build_engine_hip = build_engine_hipS.hipblaslt_table = lambda *a, **k: None@@ -403,8 +477,10 @@ def install(argv) -> None:sys.argv = [str(ROOT / "setup.py"), *argv]if "--backend" not in argv:sys.argv += ["--backend", "hip"]- if "--vision" not in argv and not PICTURES_ON_BY_DEFAULT:+ if "--vision" not in argv and not pictures_on_by_default(intel[0]["driver"], intel[0].get("devid", "")):sys.argv += ["--vision", "none"]+ S.say(" Pictures are off by default on this card (Arc Pro B70 test of 2026-10-19: the first picture reset the GPU, docs/INTEL.md): "+ "--vision yes turns them on")if "--low-ram" not in argv:sys.argv += ["--low-ram", "off"]sys.exit(S.main())diff --git a/tools/test_setup_sycl.py b/tools/test_setup_sycl.pyindex 25d1bca59..ff6333f7a 100644--- a/tools/test_setup_sycl.py+++ b/tools/test_setup_sycl.py@@ -223,12 +223,15 @@ class ToSycl(unittest.TestCase):self.assertTrue(m.small_card(10.0, "xe"))self.assertFalse(m.small_card(32.0, "xe"))- def test_pictures_asked_for_are_read_on_the_cpu_and_the_vision_section_survives(self):- """Intel Arc: the CPU reader (strata-vision) on the host when asked for (--vision yes); "gpu" is read as "cpu" (no SYCL GPU reader)."""+ def test_pictures_follow_the_one_reader_rule_and_the_vision_section_survives(self):+ """Intel Arc: pictures on gives the CPU reader (strata-vision) on the host under 24 GB and, until a 24 GB+ Arc has run the SYCL reader,+ also from 24 GB; only --vision gpu asks for the SYCL reader (strata_control/vision_reader.py is the one place for the rule)."""m = self.mod()- for asked in (None, "yes", "gpu", "cpu", "on"):+ for asked in (None, "yes", "cpu", "on"):self.assertEqual(m.intel_vision(asked, 8.0), "cpu")self.assertEqual(m.intel_vision(asked, 32.0), "cpu")+ self.assertEqual(m.intel_vision("gpu", 8.0), "gpu")+ self.assertEqual(m.intel_vision("gpu", 32.0), "gpu")for asked in ("no", "none", "off"):self.assertEqual(m.intel_vision(asked, 32.0), "none")section = {"exe": "/h/engine-sycl/strata-vision", "mmproj": "/h/models/mmproj.gguf", "model": "/h/models/m.gguf", "gpu": False,@@ -239,14 +242,28 @@ class ToSycl(unittest.TestCase):self.assertEqual(c["vision"], section) # host paths: the reader runs on the host, not in the containerself.assertIn("--vision", c["args"])- def test_pictures_are_opt_in_on_the_arc_until_the_engine_fault_is_fixed(self):- """The first picture on the Arc Pro B70 hung the GPU (xe engine reset, 2026-10-19): off unless asked, --vision yes turns it on."""+ def test_pictures_are_on_by_default_on_an_alchemist_and_opt_in_on_a_battlemage(self):+ """The A750 ran 47 picture requests without a fault; the first picture on the Arc Pro B70 reset the GPU (xe, 2026-10-19), so a+ Battlemage card keeps them opt-in (--vision yes) and setup says so."""m = self.mod()- self.assertFalse(m.PICTURES_ON_BY_DEFAULT)+ self.assertTrue(m.pictures_on_by_default("i915", "56a1")) # A750+ self.assertTrue(m.pictures_on_by_default("i915", ""))+ self.assertTrue(m.pictures_on_by_default("xe", "56a0")) # A770+ self.assertFalse(m.pictures_on_by_default("xe", "e223")) # B70+ self.assertFalse(m.pictures_on_by_default("xe", "e20b")) # B580src = (ROOT / "sycl" / "setup_intel.py").read_text(encoding="utf-8")- self.assertIn('if "--vision" not in argv and not PICTURES_ON_BY_DEFAULT:', src)+ self.assertIn('if "--vision" not in argv and not pictures_on_by_default(', src)self.assertIn("S.hip_vision = intel_vision", src)self.assertIn("S.build_vision_cpu(", src) # the CPU reader is compiled by setup's own step+ self.assertIn("build_vision_sycl(", src) # --vision gpu: the SYCL reader, when the oneAPI compiler is there++ def test_the_sycl_reader_is_built_only_when_asked_and_the_cpu_reader_stays_without_the_compiler(self):+ from unittest import mock+ m = self.mod()+ with mock.patch.object(m, "oneapi_compiler", return_value=None), mock.patch.object(m.S, "warn") as warn:+ self.assertFalse(m.build_vision_sycl_reader(Path("/nonexistent"), "/llama"))+ self.assertIn("oneAPI compiler", warn.call_args[0][0])+ self.assertIn("pictures are read on the CPU", warn.call_args[0][0])def test_the_container_shares_dev_shm_with_the_server_for_the_embeddings(self):sh = (ROOT / "sycl" / "serve" / "strata-sycl.sh").read_text(encoding="utf-8")diff --git a/tools/test_vision_reader.py b/tools/test_vision_reader.pyindex e840aafac..fa1ff6200 100644--- a/tools/test_vision_reader.py+++ b/tools/test_vision_reader.py@@ -35,6 +35,28 @@ class Choose(unittest.TestCase):("no", 32, "none"), ("off", 12, "none")):self.assertEqual(VR.wanted(asked, vram), want, (asked, vram))+ def test_an_intel_arc_keeps_the_cpu_reader_by_default_and_gpu_is_asked_for(self):+ """The one exception to the 24 GB rule: the SYCL reader has run on a 8 GB A750 only (INTEL_GPU_READER_BY_DEFAULT)."""+ self.assertFalse(VR.INTEL_GPU_READER_BY_DEFAULT)+ for vram in (8, 16, 24, 32):+ self.assertEqual(VR.default_mode(vram, "intel"), "cpu")+ self.assertEqual(VR.wanted("yes", vram, "intel"), "cpu")+ self.assertEqual(VR.wanted("gpu", vram, "intel"), "gpu")+ self.assertEqual(VR.wanted("yes", 32), "gpu") # other makes: unchanged+ with mock.patch.object(VR, "INTEL_GPU_READER_BY_DEFAULT", True):+ self.assertEqual(VR.default_mode(32, "intel"), "gpu")+ self.assertEqual(VR.default_mode(8, "intel"), "cpu") # with the exception off, the 24 GB rule is the same as for every make++ def test_the_sycl_reader_is_the_gpu_reader_of_an_intel_folder(self):+ d = folder("strata-vision", "strata-vision-sycl")+ r = VR.choose(d, "gpu", windows=False)+ self.assertEqual((r.mode, r.exe.name, r.cpu_exe.name), ("gpu", "strata-vision-sycl", "strata-vision"))+ self.assertEqual(VR.vision_section(r, {"mmproj": "m"})["gpu"], True)+ r = VR.choose(folder("strata-vision"), "gpu", windows=False) # no SYCL reader built: the CPU one, said+ self.assertEqual((r.mode, r.exe.name), ("cpu", "strata-vision"))+ self.assertIn("no GPU picture reader", r.note)+ self.assertEqual(VR.choose(d, "cpu", windows=False).exe.name, "strata-vision")+def test_hip_present_gives_the_gpu_reader_with_the_cpu_one_to_fall_back_on(self):d = folder("strata-vision", "strata-vision-hip")for ask in ("yes", "on", "gpu"):diff --git a/tools/vision/CMakeLists.txt b/tools/vision/CMakeLists.txtindex 5f031b9ce..5ad3772c1 100644--- a/tools/vision/CMakeLists.txt+++ b/tools/vision/CMakeLists.txt@@ -3,6 +3,7 @@## cmake -S tools/vision -B build-vision -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=third_party/llama.cpp# [-DSTRATA_VISION_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=native] [-DSTRATA_PORTABLE=OFF]+# [-DSTRATA_VISION_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx (oneAPI sourced; Linux)]# [-DSTRATA_VISION_HIP=ON -DSTRATA_VISION_HIP_ARCHS="gfx1200;gfx1201" + the HIP compiler the engine build uses]# cmake --build build-vision --target strata-vision#@@ -16,10 +17,14 @@ project(strata_vision C CXX)set(LLAMA_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../third_party/llama.cpp" CACHE PATH "llama.cpp checkout")option(STRATA_VISION_CUDA "run the vision encoder on the GPU (builds ggml-cuda)" OFF)option(STRATA_VISION_HIP "run the vision encoder on an AMD GPU (builds ggml-hip); the program is named strata-vision-hip" OFF)+option(STRATA_VISION_SYCL "run the vision encoder on an Intel GPU (builds ggml-sycl, needs the oneAPI icx/icpx compilers); the program is named strata-vision-sycl" OFF)set(STRATA_VISION_HIP_ARCHS "" CACHE STRING "the gfx architectures strata-vision-hip is compiled for (';'-separated)")if(STRATA_VISION_HIP AND STRATA_VISION_CUDA)message(FATAL_ERROR "strata-vision: STRATA_VISION_CUDA and STRATA_VISION_HIP are two different programs")endif()+if(STRATA_VISION_SYCL AND (STRATA_VISION_CUDA OR STRATA_VISION_HIP))+ message(FATAL_ERROR "strata-vision: STRATA_VISION_SYCL is its own program: not together with CUDA or HIP")+endif()if(STRATA_VISION_HIP AND NOT STRATA_VISION_HIP_ARCHS)message(FATAL_ERROR "strata-vision: STRATA_VISION_HIP needs STRATA_VISION_HIP_ARCHS, for example \"gfx1200;gfx1201\"")endif()@@ -62,6 +67,10 @@ endif()set(GGML_BACKEND_DL OFF CACHE BOOL "" FORCE)set(GGML_CUDA ${STRATA_VISION_CUDA} CACHE BOOL "" FORCE)set(GGML_HIP ${STRATA_VISION_HIP} CACHE BOOL "" FORCE)+set(GGML_SYCL ${STRATA_VISION_SYCL} CACHE BOOL "" FORCE)+if(STRATA_VISION_SYCL)+ set(GGML_SYCL_TARGET "INTEL" CACHE STRING "" FORCE)+endif()if(STRATA_VISION_HIP)# the engine's HIP build says the same (cmake/hip_backend.cmake); ggml-hip reads AMDGPU_TARGETS / CMAKE_HIP_ARCHITECTURESset(AMDGPU_TARGETS "${STRATA_VISION_HIP_ARCHS}" CACHE STRING "" FORCE)@@ -82,6 +91,11 @@ if(STRATA_VISION_HIP)string(REPLACE ";" "," _hip_archs "${STRATA_VISION_HIP_ARCHS}")target_compile_definitions(strata-vision PRIVATE STRATA_VISION_HIP STRATA_VISION_HIP_ARCHS="${_hip_archs}")endif()+if(STRATA_VISION_SYCL)+ # the Intel GPU reader is its own program too, beside the CPU reader "strata-vision" it falls back on+ set_target_properties(strata-vision PROPERTIES OUTPUT_NAME strata-vision-sycl)+ target_compile_definitions(strata-vision PRIVATE STRATA_VISION_SYCL)+endif()if(WIN32 AND STRATA_VISION_HIP)# hipblas.dll (and rocblas.dll behind it) live in rocm\bin of the engine package: loaded on first use, after main() has put that folder on# the DLL search path, so the reader starts from any PATH and says "no GPU" itself where there is none@@ -93,6 +107,14 @@ if(UNIX AND NOT APPLE)set_target_properties(strata-vision PROPERTIES BUILD_WITH_INSTALL_RPATH ON INSTALL_RPATH "$ORIGIN")target_link_options(strata-vision PRIVATE -Wl,--enable-new-dtags)endif()+if(STRATA_VISION_SYCL AND UNIX AND NOT APPLE)+ # libsycl and the OpenMP runtime live in the oneAPI compiler's lib folder, which only a sourced setvars.sh puts on the search path:+ # recorded in the program, it starts from a plain shell (the server starts it with the PC's own environment)+ get_filename_component(_icpx_bin "${CMAKE_CXX_COMPILER}" REALPATH)+ get_filename_component(_icpx_bin "${_icpx_bin}" DIRECTORY)+ get_filename_component(_icpx_lib "${_icpx_bin}/../lib" ABSOLUTE)+ set_target_properties(strata-vision PROPERTIES INSTALL_RPATH "$ORIGIN;${_icpx_lib}")+endif()# opens a GGUF that has tensor types llama.cpp does not know (a Gyro quantization's TQK6 / TQK7) for its vocabulary onlyadd_executable(strata-vision-gguf-test EXCLUDE_FROM_ALL gguf_meta_test.cpp)diff --git a/tools/vision/strata_vision.cpp b/tools/vision/strata_vision.cppindex eef3e81c9..c5d3ab915 100644--- a/tools/vision/strata_vision.cpp+++ b/tools/vision/strata_vision.cpp@@ -102,6 +102,8 @@ int main(int argc, char** argv) {else if (a == "--version") {#ifdef STRATA_VISION_HIPstd::printf("strata-vision hip (GPU architectures: %s)\n", STRATA_VISION_HIP_ARCHS);+#elif defined(STRATA_VISION_SYCL)+ std::printf("strata-vision sycl (Intel GPU)\n");#elsestd::printf("strata-vision\n");#endif@@ -137,9 +139,14 @@ int main(int argc, char** argv) {llama_log_set(quiet_log, nullptr);mtmd_helper_log_set(quiet_log, nullptr);llama_backend_init();-#ifdef STRATA_VISION_HIP+#if defined(STRATA_VISION_HIP) || defined(STRATA_VISION_SYCL)+#ifdef STRATA_VISION_SYCL+#define STRATA_VISION_BUILT_FOR "Intel GPUs"+#else+#define STRATA_VISION_BUILT_FOR STRATA_VISION_HIP_ARCHS+#endifif (gpu) {- // The AMD GPU reader: starting the backends prints the GPU line. With --gpu and no GPU, mtmd would quietly use the CPU+ // The AMD / Intel GPU reader: starting the backends prints the GPU line. With --gpu and no GPU, mtmd would quietly use the CPU// (a second, slower CPU reader inside the HIP program), and a card the build has no code for would abort at its first// kernel. Both end here with a line the server reads: it then starts the CPU reader (serve/server.py, Vision._start).bool have_gpu = false;@@ -148,10 +155,11 @@ int main(int argc, char** argv) {have_gpu = have_gpu || type == GGML_BACKEND_DEVICE_TYPE_GPU || type == GGML_BACKEND_DEVICE_TYPE_IGPU;}if (!have_gpu) {- std::printf("ERR this picture reader found no GPU (it was built for: %s)\n", STRATA_VISION_HIP_ARCHS);+ std::printf("ERR this picture reader found no GPU (it was built for: %s)\n", STRATA_VISION_BUILT_FOR);std::fflush(stdout);return 3;}+#ifdef STRATA_VISION_HIPconst char* check = std::getenv("STRATA_VISION_ARCH_CHECK");if (!g_gfx.empty() && (check == nullptr || std::atoi(check) != 0) &&!strata_arch_listed(STRATA_VISION_HIP_ARCHS, g_gfx.front())) {@@ -160,6 +168,7 @@ int main(int argc, char** argv) {std::fflush(stdout);return 3;}+#endif}#endif