Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Docs consistency pass: pictures on by default on NVIDIA and AMD (Intel Arc off for now), seven monitor views, area prefix for issue and PR titles in AGENTS.md
Niko12212b209b406d14
diff --git a/AGENTS.md b/AGENTS.mdindex c32a038e1..f2696dfff 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -68,6 +68,8 @@ Terminal cannot run. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.mThe rules are in **[CONTRIBUTING.md](CONTRIBUTING.md)**. Read it before you open an issue or a pull request, andfollow it whether a person or an agent writes the text. The short version:+- Start every issue and pull request title with its area: `[Engine]`, `[GUI]`, `[TUI]`, `[Benchmark]`, `[Docs]` or+ `[Other]`, for example `[Engine] fix …`. A check on GitHub fails a pull request whose title has no area.- Search the open and closed issues and pull requests first, and add to a thread that already covers your point.- Agree the scope in an issue before a large feature, a refactor or a change to a default.- One problem per pull request. Lead with the problem and the result, then the approach, then the validationdiff --git a/docs/AI_SETUP.md b/docs/AI_SETUP.mdindex 3ce89c5c1..d9a083d4d 100644--- a/docs/AI_SETUP.md+++ b/docs/AI_SETUP.md@@ -107,7 +107,7 @@ The flags (all of them, with examples: `strata launch --help`):| `--model-path PATH` | files the user already has: a folder of shards or one `.gguf` file || `--data-dir DIR` | where the 70-120 GB of model files go || `--context SIZE` | `32k`, `64k`, `128k`, `256k` or a number of tokens |-| `--vision off\|on\|gpu\|cpu` | read pictures. On by default when the model can read them: the reader runs on the CPU for a card under 24 GB and on the GPU from 24 GB, on every make of card. `off` turns it off |+| `--vision off\|on\|gpu\|cpu` | read pictures. On by default when the model can read them: the reader runs on the CPU for a card under 24 GB and on the GPU from 24 GB, on NVIDIA and AMD cards. `off` turns it off || `--parallel N` | requests answered at the same time, 1 to 8 (default: fit to the card) || `--gpu LIST` | card numbers, e.g. `0` or `0,1` (several cards share the model) || `--tune quick\|thorough\|skip` | find the fastest settings first (default here: `skip`) |diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.mdindex 626376d25..70d480750 100644--- a/docs/AMD_HIP.md+++ b/docs/AMD_HIP.md@@ -51,7 +51,7 @@ the kernel's amdgpu driver (no ROCm install needed):architectures above; the engine is compiled for each of them (cards of two families, e.g. gfx1100 + gfx1201, needa system ROCm 7: AMD's wheels hold one family). A split pays only when no single card holds the model's experts(see RDNA4 below).-- **Pictures:** on by default; the reader is the CPU one under 24 GB of VRAM and the GPU one from 24 GB (the same rule for every make, `strata_control/vision_reader.py`); `--vision gpu` / `--vision cpu` switch it. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on the+- **Pictures:** on by default; the reader is the CPU one under 24 GB of VRAM and the GPU one from 24 GB (the same rule on NVIDIA and AMD, `strata_control/vision_reader.py`); `--vision gpu` / `--vision cpu` switch it. The engine package carries two picture readers: `strata-vision-hip`, which reads a picture on thecard (llama.cpp's HIP backend; about 1.4 GiB of VRAM, measured on an R9700: 0.06 s for a 260-token picture and 0.4 s for 972tokens, against 3.1 s and 14 s on the CPU), and `strata-vision`, the CPU reader. If the GPU reader cannot start (the card isnot one it was built for, the HIP runtime does not load, the card is busy) the server starts the CPU reader by itself and@@ -264,7 +264,7 @@ sizes itself automatically and leaves 1 GiB of VRAM headroom.GPU memory to system RAM (GTT), the OOM killer then ends KWin/plasmashell or `systemd-oomd` ends apps, or thecompositor fails with "Failed to pin framebuffer with error -12".-The installer supports this backend (see "Install with setup" above). Pictures are on by default. They are read on the CPU (`strata-vision`) under 24 GB of VRAM and on the card (`strata-vision-hip`) from 24 GB, the same rule as for every make of card; if the engine has no card reader, or it cannot start, the CPU reader is used.+The installer supports this backend (see "Install with setup" above). Pictures are on by default. They are read on the CPU (`strata-vision`) under 24 GB of VRAM and on the card (`strata-vision-hip`) from 24 GB, the same rule as on NVIDIA; if the engine has no card reader, or it cannot start, the CPU reader is used.Setup installs one AMD card, or several with `--gpus` (the engine's layer split; see RDNA4 below).**Opt-in switches for RDNA2 prompts and hangs (0.1.40, off by default):** `STRATA_DENSE_MMQ=1` (#820, HIP) runs thediff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex 4d3502b01..07c9b3e68 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -1630,8 +1630,8 @@ The config snippets for every client, the tool arguments and the safety rules arThe model has a vision encoder: [`mmproj-Qwen3.8-Flash-Next-BF16.gguf`](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF)(0.9 GB, a 27-layer ViT plus the projector into the language model). Pictures are **on by default** wherever a model can readthem. The app, `strata launch` and setup use one rule for the reader: the **CPU reader under 24 GB** of graphics memory (nothing is-taken from the card's memory, so the experts keep their room) and the **graphics card's reader from 24 GB**, on every make of-card. You can switch pictures off (`--vision no`, or "No" in the app and in `strata launch`) or pick the reader per model+taken from the card's memory, so the experts keep their room) and the **graphics card's reader from 24 GB**, on NVIDIA and AMD+cards. You can switch pictures off (`--vision no`, or "No" in the app and in `strata launch`) or pick the reader per model(`--vision gpu` or `--vision cpu`). The setup downloads the encoder and puts a small helper (`strata-vision`, from llama.cpp's`mtmd` library) beside the engine; it adds it to your start script. Nothing else changes. Where a reader is missing, picturesstay off and the screen says so: the Mac app and the Mac Terminal have no picture reader yet, and neither has the Intel Arcdiff --git a/docs/MCP_SERVER.md b/docs/MCP_SERVER.mdindex 278463f78..a86b433ab 100644--- a/docs/MCP_SERVER.md+++ b/docs/MCP_SERVER.md@@ -128,7 +128,7 @@ Every result is JSON with a one-line `summary`. It is sent both as text and as M- the size setup itself would pick: IQ3_XXS from 60 GB of RAM, Q2_0 from about 44 GB, the Coder from about 28 GB,and the Coder in the low-RAM mode below that when the GPU makes up for the RAM;- the context setup recommends for the GPU's VRAM;-- pictures on where the model can read them (the reader runs on the CPU under 24 GB of VRAM and on the GPU from 24 GB, on every make of card; a Mac has none yet).+- pictures on where the model can read them (the reader runs on the CPU under 24 GB of VRAM and on the GPU from 24 GB, on NVIDIA and AMD cards; a Mac has none yet).The first call returns the plan: the download size (58-111 GB), the free disk space, the RAM check, and the exactsetup command. The assistant should show you the plan and call again with `confirm: true` once you agree.diff --git a/docs/MODELS.md b/docs/MODELS.mdindex 5b595117b..fee706341 100644--- a/docs/MODELS.md+++ b/docs/MODELS.md@@ -97,7 +97,7 @@ Coder. With a small card most experts then come from the SSD and it is much slow**Pictures.** The versions in setup's menu read pictures, and pictures are **on by default** wherever a model canread them. The picture reader runs on the CPU when the graphics card has less than 24 GB (a few seconds perpicture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB of VRAM; 0.1-0.5 s per-picture). The rule is the same for every make of card. `--vision off` (or "No" in the app and in `strata launch`) turns+picture). The rule is the same on NVIDIA and AMD cards. `--vision off` (or "No" in the app and in `strata launch`) turnspictures off; `--vision cpu` or `--vision gpu` picks the reader. On AMD, `strata-vision-hip` is the card's reader; if itcannot start, Strata uses the CPU reader by itself ([AMD_HIP.md](AMD_HIP.md)). **The Mac app and the Mac Terminal have nopicture reader yet** ([MAC.md](MAC.md)), and neither does the Intel Arc engine ([INTEL.md](INTEL.md#not-done)). More: [Images](DETAILS.md#images-vision).diff --git a/docs/QWEN36.md b/docs/QWEN36.mdindex c39c86922..3f3ecd36d 100644--- a/docs/QWEN36.md+++ b/docs/QWEN36.md@@ -13,7 +13,7 @@ UD-Q4_K_XL in the hybrid mode: 22-37 of 256 experts per layer in VRAM, the restThe vision encoder is Unsloth's `mmproj-F16.gguf` (0.9 GB, pinned revision, SHA-256 checked by setup): a 27-layer ViT with a 2x2 merger,the same projector family as the Flash-Next models' (`qwen3vl_merger`; only its output width differs: 2048 here, 2560 there). Strata runsit with the same helper (`strata-vision`, llama.cpp's `mtmd`), so the encoder choices are the Flash-Next ones-([Images](DETAILS.md#images-vision)). **Pictures are on by default**, the same rule as for every model and every make of card:+([Images](DETAILS.md#images-vision)). **Pictures are on by default**, the same rule as for every model on NVIDIA and AMD cards:the CPU reader under 24 GB of graphics memory (a few seconds per picture, no VRAM), the card's reader from 24 GB (GPU encoder,~1.2 GB of VRAM at up to 1,024 image tokens; on AMD it is `strata-vision-hip`, with the CPU reader as its fallback).`--vision yes|no|gpu|cpu` and `--vision-tokens N` work as there; what you ask for is kept, with a note. The Mac engine doesdiff --git a/docs/TERMINAL.md b/docs/TERMINAL.mdindex 743c5103a..40c6922bc 100644--- a/docs/TERMINAL.md+++ b/docs/TERMINAL.md@@ -234,7 +234,7 @@ When `--max-queue` is set, at most **effective slots + max queue** HTTP inferenc`status --json` and `monitor --once --json` return one versioned JSON object. Remote monitor URLs require verified HTTPS; a local SSH tunnel can use `http://127.0.0.1`. For a read-only monitor key, set `STRATA_TELEMETRY_KEY` on the server and pass the same secret to the client through `--credential-env STRATA_TELEMETRY_KEY` (or another client environment variable). Use a distinct inference API key: the server rejects duplicate keys. The telemetry key authorizes `/v1/telemetry` only; it cannot call inference or management endpoints. The existing `STRATA_API_KEY` also works and has its broader legacy permissions. Keys are never put in URLs or command arguments. The client refuses redirects and responses over 2 MiB. Excessively nested JSON is reported as a malformed telemetry response with the same one-record error contract, rather than a Python traceback.-The interactive monitor has six views: Overview, Serving, Requests, Engine, Hardware and Logs. The monitor that `strata launch` opens adds a seventh, **Connect** (the address, the key and the setup for your coding apps; see [TERMINAL_LAUNCH.md](TERMINAL_LAUNCH.md#after-the-start-the-monitor)). Use keys `1`–`6` (`1`–`7` in the launch monitor) or Left/Right to switch views, Tab in Serving to cycle Devices / Requests / Groups at wide and compact widths, `/` to filter requests, Up/Down and Page Up/Down to move the selected row, Home/End to jump to the first/last row, Enter to show or hide read-only request, device or slot detail, Space to freeze the display, `r` to refresh, `t` to cycle theme, `?` or F1 for help, and `q` to detach. Filter input is limited to 80 characters and cleaned of terminal controls and bidi formatters as text arrives; `q` remains ordinary text while the filter has focus. Recent detail contains bounded token and timing summaries, never prompt bodies. A selected request, device or slot stays attached to its reported ID across polls, including when a request moves from live to recent history; detail closes and the TUI says “No longer retained” if that ID disappears. An instance restart clears selection even when a new request reuses the same ID. The 60- and 40-column layouts keep the selected request visible. It never starts or stops the server. `NO_COLOR` or `--color never` uses terminal defaults. `--glyphs ascii` avoids special glyphs. The selected terminal font is controlled by the terminal, not Strata.+The interactive monitor has seven views: Overview, Serving, Requests, Engine, Hardware, Logs and **Connect** (the address, the key and the setup for your coding apps; see [TERMINAL_LAUNCH.md](TERMINAL_LAUNCH.md#after-the-start-the-monitor)). Use keys `1`–`7` or Left/Right to switch views, Tab in Serving to cycle Devices / Requests / Groups at wide and compact widths, `/` to filter requests, Up/Down and Page Up/Down to move the selected row, Home/End to jump to the first/last row, Enter to show or hide read-only request, device or slot detail, Space to freeze the display, `r` to refresh, `t` to cycle theme, `?` or F1 for help, and `q` to detach. Filter input is limited to 80 characters and cleaned of terminal controls and bidi formatters as text arrives; `q` remains ordinary text while the filter has focus. Recent detail contains bounded token and timing summaries, never prompt bodies. A selected request, device or slot stays attached to its reported ID across polls, including when a request moves from live to recent history; detail closes and the TUI says “No longer retained” if that ID disappears. An instance restart clears selection even when a new request reuses the same ID. The 60- and 40-column layouts keep the selected request visible. It never starts or stops the server. `NO_COLOR` or `--color never` uses terminal defaults. `--glyphs ascii` avoids special glyphs. The selected terminal font is controlled by the terminal, not Strata.Remote model, device and request labels are stripped of terminal controls and bidi formatters before display. The stdlib renderer clips by terminal display cells and pads fixed columns by the same measure; local cases cover combining marks, CJK names, regional-indicator flags and joined emoji at 110, 80, 60 and 40 columns. Terminal implementations can differ on ambiguous-width characters, so the renderer uses the terminal's narrow default for them.diff --git a/docs/TERMINAL_LAUNCH.md b/docs/TERMINAL_LAUNCH.mdindex 52670c8cd..5c230737a 100644--- a/docs/TERMINAL_LAUNCH.md+++ b/docs/TERMINAL_LAUNCH.md@@ -29,7 +29,7 @@ The release folder is the latest GitHub release of Niko1221/Strata (`https://git4. **Context** (8K to 512K, with the extra memory each costs) and the memory format (`m`: 8-bit or 4-bit).5. **Pictures**: only for models that can read them. They are **on by default** (Yes is preselected; `--vision off` or "No" turns them off). It says what it adds (about 0.9 GB:the picture-reader file, downloaded with the model through setup's Control mode, the same path the app's "Read pictures" uses). The reader runs on the CPU- for a card under 24 GB (a few seconds per picture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB kept free), on every make of card; the+ for a card under 24 GB (a few seconds per picture, nothing taken from the card) and on the graphics card from 24 GB (about 1-1.4 GB kept free), on NVIDIA and AMD cards; therule lives in one place, `strata_control/vision_reader.py`, for setup, `strata launch` and the app's catalog. `--vision gpu` / `--vision cpu` (or the Pictures step) switch it per model. It needs a picture reader beside the engine in the engine package; with anengine that has none, pictures default to off and the screen says so in plain words. A model that cannot read pictures stays off with no question.6. **Parallel requests**: "fit to your card" (12 GB: 1, 20: 2, 24: 3, 32+: 4) or 1 to 4; a warning if it will not fit.diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 9e413d4e5..a9cf9e7a1 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -179,11 +179,11 @@ setup warns when it finds 13.2.0 or 13.2.1 for such a card, and takes an older 1"this server was started without the vision encoder": the model was set up for text only (someone chose "No", or theengine has no picture reader) - run setup again with `--vision yes` (or `--vision gpu` / `--vision cpu`), or switch pictureson in the app or on the Pictures step of `strata launch`. Pictures are on by default wherever a model can read them. The-reader runs on the CPU under 24 GB of graphics memory and on the card from 24 GB, on every make of card. Pictures that take+reader runs on the CPU under 24 GB of graphics memory and on the card from 24 GB, on NVIDIA and AMD cards. Pictures that takeseveral seconds (about 3 s at 300 image tokens on 8 cores, more with more tokens) are read by the encoder on the CPU;`--vision gpu` (NVIDIA and AMD, ~1.4 GB of VRAM) makes it 0.1-0.5 s. On AMD the card's reader is `strata-vision-hip`; if itcannot start, the server prints "the GPU picture reader did not start (...); pictures are read on the CPU" and goes on with-the CPU reader. The Mac app, the Mac Terminal and the Intel Arc engine have no picture reader yet.+the CPU reader. The Mac app and the Mac Terminal have no picture reader yet. On Intel Arc, pictures are off by default for now (`--vision yes` turns them on).## AMD cardsdiff --git a/docs/release-notes-0.2.0.md b/docs/release-notes-0.2.0.mdindex 7e991af1d..28c9ffedf 100644--- a/docs/release-notes-0.2.0.md+++ b/docs/release-notes-0.2.0.md@@ -1,4 +1,4 @@-**0.2.0: Strata is now an app, a Terminal and an engine; installers that carry the engine; pictures on by default on every make of card; faster defaults for NVIDIA 12-16 GB cards, AMD RX 9000 / R9700 on Linux and Intel Arc.** Each new default below says whether the answers are identical to 0.1.45 (the identity gate) or change slightly, with the measured difference. Engine package version 0.1.45.3. Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).+**0.2.0: Strata is now an app, a Terminal and an engine; installers that carry the engine; pictures on by default on NVIDIA and AMD cards; faster defaults for NVIDIA 12-16 GB cards, AMD RX 9000 / R9700 on Linux and Intel Arc.** Each new default below says whether the answers are identical to 0.1.45 (the identity gate) or change slightly, with the measured difference. Engine package version 0.1.45.3. Update with `strata update` (Terminal), Settings > Update (app), or `UPDATE.bat` / `./update.sh` (source installs).## What is new@@ -10,7 +10,7 @@- **No engine found is a warning, not a refusal.** `strata launch` and the app say in plain words what is missing and what to do next, and still download the model.- Starting a model and serving never download anything; downloads are one explicit step.-**Pictures are on by default, on every make of card.** The picture reader runs on the CPU for a card under 24 GB (a few seconds per picture, nothing taken from the card) and on the graphics card from 24 GB. NVIDIA, AMD and Intel follow the same rule; `--vision gpu` / `--vision cpu` / off still choose per model. AMD now has a GPU reader with an automatic CPU fallback (the server prints one line when the GPU reader cannot start). Measured on a Radeon AI PRO R9700: a 260-token picture 57 ms on the GPU reader against 3.13 s on the CPU, a 972-token picture 0.40 s against 14.1 s; the GPU reader keeps 1,470 MiB of VRAM. On a Radeon 780M 0.40 s against 2.42 s. Gyro and the NVFP4 models read pictures too (Gyro needs engine 0.1.45.1 or newer). The Mac app has no picture reader yet.+**Pictures are on by default on NVIDIA and AMD cards.** The picture reader runs on the CPU for a card under 24 GB (a few seconds per picture, nothing taken from the card) and on the graphics card from 24 GB. NVIDIA, AMD and Intel follow the same rule; `--vision gpu` / `--vision cpu` / off still choose per model. AMD now has a GPU reader with an automatic CPU fallback (the server prints one line when the GPU reader cannot start). Measured on a Radeon AI PRO R9700: a 260-token picture 57 ms on the GPU reader against 3.13 s on the CPU, a 972-token picture 0.40 s against 14.1 s; the GPU reader keeps 1,470 MiB of VRAM. On a Radeon 780M 0.40 s against 2.42 s. Gyro and the NVFP4 models read pictures too (Gyro needs engine 0.1.45.1 or newer). The Mac app has no picture reader yet.**Models.** Qwen3.6-35B-A3B has its three sizes in the app's catalog; the NVFP4 models have clear names.