Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docs: AMD pictures run on the card by default, with the CPU reader as the automatic fallback
Niko122197cb031bf7a5
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.mdindex c4fb19435..9402ff22e 100644--- a/docs/AMD_HIP.md+++ b/docs/AMD_HIP.md@@ -51,7 +51,12 @@ the kernel's amdgpu driver (no ROCm install needed):architectures above; the engine is compiled for each of them (cards of two families, e.g. gfx1100 + gfx1201, needa system ROCm 7: AMD's wheels hold one family). A split pays only when no single card holds the model's experts(see RDNA4 below).-- **Limits for now:** images only through the CPU encoder (`--vision cpu`, 0.1.32). Setup does not offer the tuning+- **Pictures:** on by default. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on the+ card (llama.cpp's HIP backend; about 1.4 GiB of VRAM, measured on an R9700: 0.06 s for a 260-token picture and 0.4 s for 972+ tokens, against 3.1 s and 14 s on the CPU), and `strata-vision`, the CPU reader. If the GPU reader cannot start (the card is+ not one it was built for, the HIP runtime does not load, the card is busy) the server starts the CPU reader by itself and+ prints one line; nothing is asked. `--vision cpu` picks the CPU reader on purpose.+- **Limits for now:** Setup does not offer the tuning(calibration) on AMD yet: its controls are being checked on HIP one at a time (#566). Since 0.1.39 a tuning run byhand (`./setup.sh --calibrate`) is saved for the AMD card it ran on and reused when setup runs again. The Monitorshows the card's load, VRAM, temperature and power from Linux sysfs (0.1.32).@@ -84,8 +89,7 @@ On a PC with no NVIDIA card Strata can use, the AMD card is chosen by itself; withe PATH): if the HIP runtime does not see the card, setup stops there and points to the driver. It also gives thecard's HIP number: with an integrated Radeon that is device 1, not 0 (#325). From then on setup lists the AMD cardsas HIP numbers them, so `--gpu N` and the config's `"gpu"` are HIP numbers.-- **Differences from Windows-on-NVIDIA and Linux-on-AMD:** no images yet (the CPU image encoder is Linux-only for- now), one card per model (`--gpus` is Linux-only for now), no calibration.+- **Differences from Windows-on-NVIDIA and Linux-on-AMD:** one card per model (`--gpus` is Linux-only for now), no calibration.- Two Windows-only engine details (#247, #325): hipBLAS can return success and still leave `hipErrorInvalidValue`set after some BF16/FP16 GEMMs (seen on gfx1201); the engine clears that one stale error after a GEMM thatsucceeded, on Windows only. `hipHostGetDevicePointer` returns the host pointer itself on Windows: kernels read@@ -260,7 +264,7 @@ sizes itself automatically and leaves 1 GiB of VRAM headroom.GPU memory to system RAM (GTT), the OOM killer then ends KWin/plasmashell or `systemd-oomd` ends apps, or thecompositor fails with "Failed to pin framebuffer with error -12".-The installer supports this backend (see "Install with setup" above). Images run through the CPU encoder for now (`--vision cpu`).+The installer supports this backend (see "Install with setup" above). Pictures are read on the card (`strata-vision-hip`) when the engine has it, else on the CPU (`strata-vision`).Setup installs one AMD card, or several with `--gpus` (the engine's layer split; see RDNA4 below).**Opt-in switches for RDNA2 prompts and hangs (0.1.40, off by default):** `STRATA_DENSE_MMQ=1` (#820, HIP) runs thediff --git a/docs/INSTALL.md b/docs/INSTALL.mdindex eb8a1ee55..c9c6c888a 100644--- a/docs/INSTALL.md+++ b/docs/INSTALL.md@@ -80,8 +80,7 @@ On Linux setup uses a system ROCm 7 when there is one, or installs ROCm into `.vsudo), and compiles the engine on your PC for the card (10-20 minutes, once; it needs a C++ compiler and git:`sudo apt install build-essential git`). Several AMD cards share the model with `--gpus`, as on NVIDIA.-What differs from NVIDIA for now: pictures are read by the image encoder on the CPU (`--vision cpu`, about 3 s per-picture at the default 300 image tokens on 8 cores), `--calibrate` is NVIDIA-only, and Unsloth's 4-bit model needs an NVIDIA card. Measurements per card, the+What differs from NVIDIA for now: `--calibrate` is NVIDIA-only, and Unsloth's 4-bit model needs an NVIDIA card. Measurements per card, thebuild by hand and the tuning tables: [AMD_HIP.md](AMD_HIP.md).## Two or three cardsdiff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 706f355d4..799ad8e5f 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -133,7 +133,8 @@ setup warns when it finds 13.2.0 or 13.2.1 for such a card, and takes an older 1**Pictures are refused, or slow.**"this server was started without the vision encoder": the model was set up for text only - run setup again with`--vision gpu` (or `--vision cpu`). Pictures that take several seconds (about 3 s at 300 image tokens on 8 cores, more with more tokens) are read by the encoder on the CPU; `--vision gpu`-(NVIDIA, ~1.4 GB of VRAM) makes it 0.1-0.5 s.+(NVIDIA and AMD, ~1.4 GB of VRAM) makes it 0.1-0.5 s. On AMD the GPU reader is the default; if it cannot start, the server+prints "the GPU picture reader did not start (...); pictures are read on the CPU" and goes on with the CPU reader.## AMD cards