Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: AMD pictures run on the card by default, with the CPU reader as the automatic fallback

Niko122197cb031bf7a5
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index c4fb19435..9402ff22e 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -51,7 +51,12 @@ the kernel's amdgpu driver (no ROCm install needed):
architectures above; the engine is compiled for each of them (cards of two families, e.g. gfx1100 + gfx1201, need
a system ROCm 7: AMD's wheels hold one family). A split pays only when no single card holds the model's experts
(see RDNA4 below).
-- **Limits for now:** images only through the CPU encoder (`--vision cpu`, 0.1.32). Setup does not offer the tuning
+- **Pictures:** on by default. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on the
+ card (llama.cpp's HIP backend; about 1.4 GiB of VRAM, measured on an R9700: 0.06 s for a 260-token picture and 0.4 s for 972
+ tokens, against 3.1 s and 14 s on the CPU), and `strata-vision`, the CPU reader. If the GPU reader cannot start (the card is
+ not one it was built for, the HIP runtime does not load, the card is busy) the server starts the CPU reader by itself and
+ prints one line; nothing is asked. `--vision cpu` picks the CPU reader on purpose.
+- **Limits for now:** Setup does not offer the tuning
(calibration) on AMD yet: its controls are being checked on HIP one at a time (#566). Since 0.1.39 a tuning run by
hand (`./setup.sh --calibrate`) is saved for the AMD card it ran on and reused when setup runs again. The Monitor
shows the card's load, VRAM, temperature and power from Linux sysfs (0.1.32).
@@ -84,8 +89,7 @@ On a PC with no NVIDIA card Strata can use, the AMD card is chosen by itself; wi
the PATH): if the HIP runtime does not see the card, setup stops there and points to the driver. It also gives the
card's HIP number: with an integrated Radeon that is device 1, not 0 (#325). From then on setup lists the AMD cards
as HIP numbers them, so `--gpu N` and the config's `"gpu"` are HIP numbers.
-- **Differences from Windows-on-NVIDIA and Linux-on-AMD:** no images yet (the CPU image encoder is Linux-only for
- now), one card per model (`--gpus` is Linux-only for now), no calibration.
+- **Differences from Windows-on-NVIDIA and Linux-on-AMD:** one card per model (`--gpus` is Linux-only for now), no calibration.
- Two Windows-only engine details (#247, #325): hipBLAS can return success and still leave `hipErrorInvalidValue`
set after some BF16/FP16 GEMMs (seen on gfx1201); the engine clears that one stale error after a GEMM that
succeeded, on Windows only. `hipHostGetDevicePointer` returns the host pointer itself on Windows: kernels read
@@ -260,7 +264,7 @@ sizes itself automatically and leaves 1 GiB of VRAM headroom.
GPU memory to system RAM (GTT), the OOM killer then ends KWin/plasmashell or `systemd-oomd` ends apps, or the
compositor fails with "Failed to pin framebuffer with error -12".
-The installer supports this backend (see "Install with setup" above). Images run through the CPU encoder for now (`--vision cpu`).
+The installer supports this backend (see "Install with setup" above). Pictures are read on the card (`strata-vision-hip`) when the engine has it, else on the CPU (`strata-vision`).
Setup installs one AMD card, or several with `--gpus` (the engine's layer split; see RDNA4 below).
**Opt-in switches for RDNA2 prompts and hangs (0.1.40, off by default):** `STRATA_DENSE_MMQ=1` (#820, HIP) runs the
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
index eb8a1ee55..c9c6c888a 100644
--- a/docs/INSTALL.md
+++ b/docs/INSTALL.md
@@ -80,8 +80,7 @@ On Linux setup uses a system ROCm 7 when there is one, or installs ROCm into `.v
sudo), and compiles the engine on your PC for the card (10-20 minutes, once; it needs a C++ compiler and git:
`sudo apt install build-essential git`). Several AMD cards share the model with `--gpus`, as on NVIDIA.
-What differs from NVIDIA for now: pictures are read by the image encoder on the CPU (`--vision cpu`, about 3 s per
-picture at the default 300 image tokens on 8 cores), `--calibrate` is NVIDIA-only, and Unsloth's 4-bit model needs an NVIDIA card. Measurements per card, the
+What differs from NVIDIA for now: `--calibrate` is NVIDIA-only, and Unsloth's 4-bit model needs an NVIDIA card. Measurements per card, the
build by hand and the tuning tables: [AMD_HIP.md](AMD_HIP.md).
## Two or three cards
diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md
index 706f355d4..799ad8e5f 100644
--- a/docs/TROUBLESHOOTING.md
+++ b/docs/TROUBLESHOOTING.md
@@ -133,7 +133,8 @@ setup warns when it finds 13.2.0 or 13.2.1 for such a card, and takes an older 1
**Pictures are refused, or slow.**
"this server was started without the vision encoder": the model was set up for text only - run setup again with
`--vision gpu` (or `--vision cpu`). Pictures that take several seconds (about 3 s at 300 image tokens on 8 cores, more with more tokens) are read by the encoder on the CPU; `--vision gpu`
-(NVIDIA, ~1.4 GB of VRAM) makes it 0.1-0.5 s.
+(NVIDIA and AMD, ~1.4 GB of VRAM) makes it 0.1-0.5 s. On AMD the GPU reader is the default; if it cannot start, the server
+prints "the GPU picture reader did not start (...); pictures are read on the CPU" and goes on with the CPU reader.
## AMD cards