Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
README redesign: logo header, clearer install per system, memory rule, Gyro for low memory, library analogy, support section
Niko1221193cbb1fbf15
diff --git a/README.md b/README.mdindex e9f87fa48..f608ad23a 100644--- a/README.md+++ b/README.md@@ -1,49 +1,59 @@-<h1 align="center">Strata</h1>+<p align="center">+ <picture>+ <source media="(prefers-color-scheme: dark)" srcset="apps/host/public/brand/strata-lockup-dark.svg">+ <img src="apps/host/public/brand/strata-lockup-light.svg" alt="Strata" width="380">+ </picture>+</p>-**English** · [简体中文](README.zh-CN.md) · [日本語](README.ja.md) · [Deutsch](README.de.md) · [Français](README.fr.md) · [Español](README.es.md) · [Português](README.pt-BR.md)+<p align="center"><b>English</b> · <a href="README.zh-CN.md">简体中文</a> · <a href="README.ja.md">日本語</a> · <a href="README.de.md">Deutsch</a> · <a href="README.fr.md">Français</a> · <a href="README.es.md">Español</a> · <a href="README.pt-BR.md">Português</a></p>-<p align="center"><b>Run a 125-billion-parameter AI model on your own gaming PC</b><br>-NVIDIA or AMD graphics card (12 GB or more), or an Apple Silicon Mac · Windows, Linux or Mac · free and open source<br>-<sub>Strata 0.2.0-beta · engine 0.1.45.3</sub></p>+<h3 align="center">Run a state-of-the-art model on your own gaming PC</h3>++<p align="center">Support for NVIDIA, AMD, Intel · Windows, Linux, Mac · free and open source</p>++<p align="center">+ <a href="https://github.com/Niko1221/Strata/releases"><img src="https://img.shields.io/github/v/release/Niko1221/Strata?include_prereleases&label=release" alt="Latest release"></a>+ <a href="LICENSE"><img src="https://img.shields.io/github/license/Niko1221/Strata" alt="MIT license"></a>+ <a href="https://github.com/Niko1221/Strata/stargazers"><img src="https://img.shields.io/github/stars/Niko1221/Strata?style=flat" alt="GitHub stars"></a>+</p><p align="center"><a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4"><img src="docs/media/pagoda-preview.webp" width="720" alt="A voxel pagoda garden that Strata's model wrote, running in the browser"></a><br><sub>A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) ·<a href="https://github.com/Niko1221/Strata/releases/download/v0.1.10/Pagoda.mp4">full video (49 s)</a></sub></p>-Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)** on a normal PC. This is a-large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps-and coding agents. Nothing leaves your PC.+Strata runs **[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)**, a 125-billion-parameter model that+normally needs a server, on a normal PC. It chats, writes code, reads pictures and works with your apps and coding+agents. Nothing leaves your PC.-Strata comes in three parts. Most people only need the first.+- **Runs on a gaming PC.** 32 GB of RAM and graphics memory together is enough to start.+- **Works with your tools.** It speaks the OpenAI and Anthropic APIs, so Claude Code, Codex CLI, OpenCode and other agents connect to it.+- **Reads pictures.** They are on by default.+- **Free.** MIT license, no account, no cloud.-- **The app:** a desktop program. Install it, pick a model, chat. It has the engine inside.-- **The Terminal:** the `strata` command, for people who like a terminal and for scripts and AI agents.-- **The engine:** the program that talks to your graphics card. The app and the Terminal start it for you.+Version 0.2.0-beta · engine 0.1.45.3## Install-### The app (easiest)+Pick your system. Each option starts the same engine and uses the same models.-Download the installer for your system from the [releases page](https://github.com/Niko1221/Strata/releases/latest):+### Option A: the Strata app-| System | File | Notes |-| --- | --- | --- |-| **Windows 10 / 11** | the `.exe` installer | Installs for your user. It does not need administrator rights. |-| **Linux** | the `.AppImage` | Make it runnable (`chmod +x`), then start it. It needs FUSE 2 (`libfuse2`; on Ubuntu 24.04 and newer `libfuse2t64`). |-| **Mac** (Apple Silicon, macOS 15 or newer) | the `.dmg` | Open it and drag Strata onto Applications. |+The app has the engine inside. Install it, pick a model, chat. Download from the+[releases page](https://github.com/Niko1221/Strata/releases/latest). The page says which installer fits which graphics card+(NVIDIA or AMD).-The release page says which installer is for which graphics card (NVIDIA or AMD). Take the one that matches yours.-The engine is inside, so there is nothing else to install. Then:+| System | Download | Then |+| --- | --- | --- |+| **Windows 10 / 11** | the `.exe` installer | Run it. It installs for your user and needs no administrator rights. If Windows says "Windows protected your PC", the installer is not code-signed yet: choose **More info**, then **Run anyway**. |+| **Linux** | the `.AppImage` | `chmod +x` the file and start it. It needs FUSE 2: `sudo apt install libfuse2` (Ubuntu 22.04) or `libfuse2t64` (Ubuntu 24.04 and newer). |+| **Mac** (Apple Silicon, macOS 15 or newer) | the `.dmg` | Open it and drag Strata onto Applications. If macOS says the app is "damaged", see [docs/MAC.md](docs/MAC.md). |-1. Start Strata. The first screen reads your PC and recommends a model that fits.-2. Press the install button. The model is a big download (about 40-70 GB, depending on the size). If it stops, it- continues where it stopped.-3. When the model has loaded, chat. Closing the window stops Strata and the engine.+On the first screen the app reads your PC and recommends a model that fits. Press install. The model is a big download+(about 40-70 GB) and continues where it stopped if it is interrupted. Closing the window stops Strata and the engine.-If something does not open: Windows may say "Windows protected your PC" because the installer is not code-signed yet-(choose **More info**, then **Run anyway**). If a Mac says the app is "damaged", see [docs/MAC.md](docs/MAC.md).+### Option B: the Terminal (one line)-### The Terminal (one line)+The Terminal is the `strata` command. It is the way for scripts, servers and AI agents.Linux and Mac:@@ -57,190 +67,135 @@ Windows (PowerShell):irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex```-It checks what kind of PC this is, downloads the Terminal, checks the download's SHA-256, unpacks it into your home-folder (no administrator rights) and puts `strata` on your PATH. On Windows, open a new terminal window afterwards.-Then run:+The script checks your PC, downloads the Terminal, verifies its SHA-256 and unpacks it into your home folder. It needs no+administrator rights. On Windows, open a new terminal window afterwards. Then start Strata:```shstrata launch```-It asks one question at a time: which model, how much context, pictures, who may connect. Then it downloads the-engine for your graphics card (signed, and test-started before it is used) and the model, and starts serving. A-script or an AI agent uses flags instead of questions:+It asks one question at a time: model, context, pictures, who may connect. Then it downloads the engine for your graphics+card (signed, and test-started before use) and the model, and starts serving. For scripts and AI agents, use flags:```shstrata launch --model q2_0 --yes --no-tui```-If you want to read the install script before it runs, download it first (`curl -fsSL <address> -o install.sh`),-read it, then run `sh install.sh`. More: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).+To read the install script before it runs, download it first with `curl -fsSL <address> -o install.sh`, read it, then run+`sh install.sh`. All screens and flags: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md).-### Let your AI set it up+### Option C: let your AI set it up-Do you use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it:+Paste this into Claude Code, Cursor, Codex, GitHub Copilot or another coding assistant:```textSet up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.```-It checks your graphics card, RAM and disk and picks the model that fits. Then it installs and starts it and tells-you how to connect your apps. AI tools can also install, start and stop Strata through its-[MCP server](docs/MCP_SERVER.md).+It checks your graphics card, RAM and disk, picks the model that fits, installs and starts it, and tells you how to+connect your apps. AI tools can also install, start and stop Strata through its [MCP server](docs/MCP_SERVER.md).-Do you want to install from the repository instead? See [Build from source](#build-from-source).+Prefer to build from the repository? See [Build from source](#build-from-source).## What you need-| | |-| --- | --- |-| **Graphics card** | **NVIDIA** GeForce RTX 20, 30, 40 or 50 series, or **AMD** Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. It needs **12 GB of VRAM or more**. A Mac has no separate card: see the Mac paragraph below. |-| **RAM** | 32 GB or more. Your RAM decides [which model](#which-model-should-i-pick) fits. 64 GB runs every size. |-| **Disk** | About 80 GB free. Use an SSD if you can: the first start is much faster. |-| **System** | Windows 10 / 11, Linux, or a Mac with Apple Silicon (macOS 15 or newer), and a current graphics driver from NVIDIA or AMD. |+**The main rule is at least 32 GB of RAM and graphics memory (VRAM) together.** On a Mac or a Ryzen AI Max, that is the+unified memory. More memory gives a bigger model or a faster one. A 32 GB PC with a 12-16 GB card runs the+Coder. 64 GB of RAM runs every size.-The installer sets up everything else. Two or three cards can share the model ([multi-GPU](docs/MULTI_GPU.md)).-One engine drives the cards of one make: Strata never splits a model across NVIDIA and AMD cards.+| Your computer | Details |+| --- | --- |+| **NVIDIA** GeForce RTX 20, 30, 40, 50 series (12 GB or more) | Install the app or the Terminal. |+| **AMD** Radeon RX 6800 and newer, Radeon AI PRO R9700 (12 GB or more) | [AMD cards](docs/AMD_HIP.md) |+| **Intel** Arc | [Intel Arc](docs/INTEL_ARC.md), [Intel engine](docs/INTEL.md) |+| **Apple Silicon Mac** (M1 or newer, macOS 15 or newer) | [Mac](docs/MAC.md) |+| **AMD Ryzen AI Max (Strix Halo)** and other unified-memory systems | [Strix Halo](docs/STRIX_HALO.md) |+| Older cards and processors | [Older GPUs](docs/OLDER_GPUS.md), [Older CPUs](docs/INSTALL.md#older-cpus-experimental) |-**Apple Silicon Macs (M1 to M5):** the Mac app has the Metal engine inside. A Mac needs 16 GB of unified memory or-more to try it, and the memory decides which size fits ([docs/MAC.md](docs/MAC.md)). **There is no picture reader on-the Mac yet.**+Also needed: about 80 GB of free disk (an SSD makes the first start much faster), Windows 10 / 11, Linux or macOS, and a+current graphics driver. Two or three cards of the same make can share a model ([multi-GPU](docs/MULTI_GPU.md)). Strata+never splits a model across NVIDIA and AMD cards. Full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).-Not tested by us, or experimental. Community members wrote and tested most of these on their own machines:+## Models-- **Windows with an AMD card:** the 0.2.0 installer and engine start, but we did not run a model with them on a real- AMD card on Windows. [AMD cards](docs/AMD_HIP.md).-- **Older graphics cards** (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT): [Older GPUs](docs/OLDER_GPUS.md).-- **Intel Arc** on Linux: [Intel Arc](docs/INTEL_ARC.md).-- **AMD Ryzen AI Max (Strix Halo)**, built from source on Linux. We have no Strix Halo test machine any more:- [Strix Halo](docs/STRIX_HALO.md).-- **Older processors without AVX2**: they work, but slowly. [Older CPUs](docs/INSTALL.md#older-cpus-experimental).+The app and `strata launch` recommend a model for your memory. It is one model in several sizes, compressed more or less.+Smaller sizes are faster. Larger sizes are a bit smarter.-The full list: [docs/INSTALL.md](docs/INSTALL.md#what-you-need).+### Low on memory? Try Gyro-## How fast is it?+[Gyro](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF) (by agentionai) is the smallest way to run this model.-We measured it on two ordinary gaming PCs. A token is about ¾ of a word.+| Size | Memory it needs (RAM + VRAM) | Download | What it is |+| --- | ---: | ---: | --- |+| **Gyro-S** | 27.6 GB | 58.5 GB | For PCs that fit nothing else. Quality is lower than the 2-3-bit sizes. |+| **Gyro-M** | 35 GB | 92.0 GB | About IQ3_XXS quality with about 20% less GPU memory. It writes answers 17-21% slower than Q2_0 on a 12 GB card and on an R9700. |-- **Writes answers:** how fast the reply appears in a short chat. 60 tokens per second is faster than you can read.-- **Reads your prompt:** how fast it takes in what you send (here a 32K-token document, code or chat history).+Gyro runs on NVIDIA and AMD cards (AMD is experimental), not on Intel Arc or the Mac. Details and measurements:+[docs/GYRO.md](docs/GYRO.md). Start it with `strata launch --model gyro-s`.-**These are older measurements.** They were made with older engines, not with the engine of this release (0.1.45.3).-The note under the tables says which engine measured which numbers.+### The other sizes-<table>-<tr><th>NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM</th><th>AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM</th></tr>-<tr><td>+| Your memory (RAM + VRAM) | Take | Notes |+| --- | --- | --- |+| **32 GB** | **Gyro-S**, or the **[Coder](docs/MODELS.md#coder)** for code | The Coder needs 32 GB of RAM and keeps half of the experts, so it is weaker outside code |+| **40 GB** | **Q2_0** (37.6 GB, fastest) or **IQ2_XS** (39.2 GB, recommended) | Both keep every expert |+| **48 GB** | **IQ3_XXS** (47.0 GB) | Slower, better |+| **64 GB** | **IQ3_S** (54.8 GB) | The slowest, the best: it matches the full model on the published tests |+| **96 GB or more** | Unsloth's **UD-IQ4_XS** (about 72 GB of RAM) to **Q8_0** (about 136 GB), 4 to 8 bit | Closer to the full model. With less RAM, Strata reads part of them from the SSD, which is much slower |-| Size | Writes answers | Reads your prompt |-| --- | ---: | ---: |-| **Q2_0** | 94 tokens/s | 2,650 tokens/s |-| **IQ2_XS** | 79 tokens/s | 2,090 tokens/s |-| **IQ3_XXS** | 62 tokens/s | 1,750 tokens/s |-| **IQ3_S** | 53 tokens/s | 1,620 tokens/s |-| **Coder** | 55 tokens/s | 2,180 tokens/s |+More choices:-</td><td>+- **[Swift 1.5](docs/MODELS.md#swift-15)** is a fine-tune that thinks for a shorter time before it answers, at about the same quality and RAM.+- **[NVFP4](docs/NVFP4.md)** is NVIDIA's 4-bit version for NVIDIA cards. It needs about 80 GB of RAM to keep every expert in RAM.+- **[Qwen3.6-35B-A3B](docs/QWEN36.md)** is a smaller, different model that reads pictures and can answer several requests at once.+- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs)** is set up by hand.-| Size | Writes answers | Reads your prompt |-| --- | ---: | ---: |-| **Q2_0** | 60 tokens/s | 1,160 tokens/s |-| **IQ2_XS** | 52 tokens/s | 1,110 tokens/s |-| **Coder** | 44 tokens/s | 1,420 tokens/s |+Every size, download and what fits where: [docs/MODELS.md](docs/MODELS.md).-</td></tr>-</table>+## How fast is it?-NVIDIA table: Q2_0 measured with engine 0.1.36, the other rows with engine 0.1.26 (4K answers, 32K prompts). AMD-table: measured on 2026-10-02 with an engine older than 0.1.41 (the exact build was not recorded; Linux, 4K answers,-32K prompts). The full tables are in [DETAILS.md](docs/DETAILS.md#speed-measured). Engine 0.1.45.3 changes the speed-of several cards. The measured changes, and what they were measured on, are in the-[release notes](https://github.com/Niko1221/Strata/releases). A card with more VRAM is faster: an RTX 3090 (24 GB)-should write about 100-140 tokens per second. Long chats and other cards: [speed of each model](docs/MODELS.md#how-fast-is-each-size),-[community results](docs/COMMUNITY_BENCHMARKS.md).+Based on a 12 GB graphics card (RTX 5070, Ryzen 5 7600, 64 GB RAM). See results from other cards:+[speed of each model](docs/MODELS.md#how-fast-is-each-size), [community results](docs/COMMUNITY_BENCHMARKS.md).-<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a><br>-<sub>Strata is free. If it runs well on your PC, a coffee keeps the work on it going.</sub></p>+| Model | Answers appear at | Reads your prompt at |+| --- | --- | --- |+| **Q2_0** | `████████████` 94 tokens/s | `████████████` 2,650 tokens/s |+| **IQ2_XS** | `██████████░░` 79 tokens/s | `█████████░░░` 2,090 tokens/s |+| **IQ3_XXS** | `████████░░░░` 62 tokens/s | `████████░░░░` 1,750 tokens/s |+| **IQ3_S** | `███████░░░░░` 53 tokens/s | `███████░░░░░` 1,620 tokens/s |+| **Coder** | `███████░░░░░` 55 tokens/s | `██████████░░` 2,180 tokens/s |-## Which model should I pick?+A token is about ¾ of a word, so 60 tokens per second is faster than you can read. "Reads your prompt" is how fast Strata+takes in a long document, code or chat history.-The app and `strata launch` recommend one for your RAM. The same model comes in several sizes, compressed more or-less. Smaller sizes are faster. Larger sizes are a bit smarter.+Measured on a 16 GB AMD card (RX 9070 XT, Ryzen 9 3900X, 47 GB RAM):-| Your RAM | Take | Why |+| Model | Answers appear at | Reads your prompt at || --- | --- | --- |-| **32 GB** | **Coder** | it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |-| **48 GB** | **IQ2_XS** (or Q2_0, the fastest) | the larger sizes do not fit |-| **64 GB** | **IQ2_XS** (recommended), or IQ3_XXS / IQ3_S | every size fits; IQ3_S is the best and the slowest |-| **96 GB or more** | **IQ3_S**, or Unsloth's UD-IQ4_XS (~4-bit) | room for the largest sizes with everything else open |--- **[Coder](docs/MODELS.md#coder):** a coding version with half of the experts removed. It reaches 91% of the full- model's SWE-bench Verified score (measured by its authors) and fits 32 GB of RAM. It is weaker outside code,- including Chinese and other CJK text (#438). For those, take Q2_0, IQ2_XS or IQ3_S, which keep every expert.-- **[Swift 1.5](docs/MODELS.md#swift-15):** a fine-tune that thinks for a much shorter time before it answers. You- get the answer sooner, at about the same quality.-- **[Unsloth UD-IQ4_XS](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):** Unsloth's ~4-bit version, between IQ3_S and- UD-Q4_K_XL in quality. A 94 GB download. With less than ~80 GB of RAM, Strata reads part of it from the SSD- while it answers, so it is slower there (an NVMe SSD helps).-- **[Unsloth UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL and Q8_0](docs/MODELS.md#unsloths-files-ud-iq4_xs-to-q8_0):**- 4- to 8-bit, closer to the full model (93-95% the same next token as Q8_0). Fast where every expert fits in RAM- (~88 GB for UD-Q4_K_XL up to ~136 GB for Q8_0); with less RAM Strata reads part of them from the SSD while it- answers, which is much slower (7-8.5 tokens/s for UD-Q4_K_XL on a 64 GB PC).-- **[Gyro-S and Gyro-M](docs/GYRO.md)** and **[NVFP4](docs/NVFP4.md)** are other compressions of the same model, for- special cases. **[Qwen3.6-35B-A3B](docs/QWEN36.md)** is a smaller, different model that reads pictures and can- answer several requests at once.-- **[OrcaRouter's Uncensored IQ3_XXS](docs/MODELS.md#orcarouter-uncensored-iq3_xxs):** you set it up by hand. It is- not in the installer's menu.--Sizes, downloads and what fits where: [docs/MODELS.md](docs/MODELS.md). To add another model later, open the app's-**Models** page, or run `strata launch --model NAME` in the Terminal. (From source: `SETUP.bat`, or-`./setup.sh --setup` on Linux.)+| **Q2_0** | `████████░░░░` 60 tokens/s | `█████░░░░░░░` 1,160 tokens/s |+| **IQ2_XS** | `███████░░░░░` 52 tokens/s | `█████░░░░░░░` 1,110 tokens/s |+| **Coder** | `██████░░░░░░` 44 tokens/s | `██████░░░░░░` 1,420 tokens/s |++The bars use one scale for both tables. A card with more VRAM is faster: an RTX 3090 (24 GB) should write about 100-140+tokens per second. The engine version behind each number is in [DETAILS.md](docs/DETAILS.md#speed-measured).## Using it+### In the app++The app has a **Chat**, a **Monitor** of the model and your GPU, CPU and RAM, a **Models** page and **Settings**. The chat+can work in a folder on your PC. In **Manual** mode it asks before it changes a file or runs a command. Type+`/loop 10m <what to do>` to repeat a message every 10 minutes while the chat is open.++The chat can search the web with DuckDuckGo (or Bing), with no account. You can add a search key under **Settings, Tools**.+Search is the one feature that sends something off your PC, the search words.++To run two models, turn on **Run multiple models** and give each model its own card of the same make.+<p align="center"><img src="docs/media/runpagoda.png" width="900" alt="The Strata app's Monitor tab next to a coding agent"><br><sub>The Strata app's <b>Monitor</b> (left) while a coding agent writes the pagoda garden from the video (right)</sub></p>-- **In the app:** **Chat**, a **Monitor** of the model and your GPU, CPU and RAM, a **Models** page and **Settings**.- The chat can work in a folder on your PC. In its **Manual** mode it asks before it changes a file or runs a- command. Type `/loop 10m <what to do>` to repeat a message every 10 minutes while the chat is open. The chat can- search the web: it uses DuckDuckGo (or Bing) without an account, and you can add a search key under **Settings,- Tools**. This is the one feature that sends something off your PC: the search words.-- **Your apps and coding agents:** add an "OpenAI-compatible" provider with the base URL- **`http://127.0.0.1:8080/v1`**. Any API key and any model name work.- - Apps that use Anthropic's API: use **`http://127.0.0.1:8080`** as the base URL, **without `/v1`**. Those apps add- `/v1/messages` themselves. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080`.- - Codex CLI and other apps that use the OpenAI Responses API: `/v1/responses`- ([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)).- - `strata connect` prints the exact settings for OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent and- OpenClaw. `strata connect opencode --write` writes the file for you, after asking, and keeps a backup.-- **Thinking:** choose **off, low, medium or high** in the chat menu or in your app's "reasoning effort". Off is the- fastest. High is best for hard questions.-- **Pictures:** pictures are on by default for models that can read them. Attach or paste a picture in the chat, or- send it from your app. The picture reader runs on the processor when your graphics card has less than 24 GB (a few- seconds per picture; it takes nothing from the card's memory) and on the graphics card from 24 GB. The rule is the- same for NVIDIA and AMD. You can turn pictures off per model (in the app, or `--vision off`), or choose the reader- (`--vision gpu` or `--vision cpu`). Why this rule: on a Radeon AI PRO R9700, the card's reader took 57 ms for a- 260-token picture and the processor's took 3.1 s. On 16 GB cards (Tesla P100 and RTX A4000, Linux) the card's reader- cost 3-5% of text speed, because it keeps about 1.4 GB that the model would use otherwise. **The Mac has no picture- reader yet.**-- **Two models on two cards:** in the app, turn on **Run multiple models** in Models, and give each model its own- card. Both stay loaded, and you pick one in the chat. The cards must be of the same make.-- **From your phone or another PC:** in the Terminal, `strata launch --access lan`. It makes an API key for you and- will not start on a network without one. Always use a key. (From source: `START-HERE.bat --setup --host 0.0.0.0- --api-key <secret>`.)-- **One request at a time:** by default Strata answers one request, and the others wait. To answer several at once,- set `"parallel": 2` ([BATCHING.md](docs/BATCHING.md)), or `--parallel N` in `strata launch`. On a 12 GB card this makes- each answer slower.-- **Long prompts:** Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up- messages start in seconds.-- **No browser opens by itself.** `strata launch` opens its web page only if you add `--open`.--More: [where your chats are stored](docs/INSTALL.md#where-things-are-stored), [the API](docs/DETAILS.md#using-it).--## The Terminal--The Terminal is the `strata` command. It uses the same models, engine and saved settings as the app.+### In the Terminal```shstrata launch # guided screens: model, context, pictures, access, then serve@@ -248,115 +203,147 @@ strata launch --model q2_0 --yes --no-tui # no questions: serve, print the astrata launch --plan --model q2_0 # show what would happen; nothing changesstrata launch --status # what is being served nowstrata launch --stop # stop it-strata connect claude-code # the settings for a coding app-strata monitor --url http://127.0.0.1:8080 # watch the model: speed, GPU, memory, requests+strata monitor --url http://127.0.0.1:8080 # watch speed, GPU, memory and requestsstrata update --check # is there a new engine? (strata update installs it)strata doctor # check the installed engine files```-`strata --help` lists everything. The step-by-step guide is [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md), and-the whole client is described in [docs/TERMINAL.md](docs/TERMINAL.md). Starting and serving never download anything.-Only the install, `strata launch` (when you choose a model) and `strata update` do.+`strata --help` lists everything. Starting and serving never download anything. Only the install, `strata launch` (when+you choose a model) and `strata update` do. To reach Strata from your phone or another PC, use+`strata launch --access lan`. It makes an API key and will not start on a network without one. No browser opens unless+you add `--open`. Guide: [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). Whole client: [docs/TERMINAL.md](docs/TERMINAL.md).-## Build from source+### With AI agents++An agent installs Strata with the one-line command in Option B and runs+`strata launch --model NAME --yes --no-tui`. The steps, flags and exit codes are in [docs/AI_SETUP.md](docs/AI_SETUP.md),+and the repository rules are in [AGENTS.md](AGENTS.md).++### Connect your other apps++| App type | Base URL |+| --- | --- |+| OpenAI-compatible (most apps and agents) | `http://127.0.0.1:8080/v1` (any API key, any model name) |+| Anthropic API (Claude Code and others) | `http://127.0.0.1:8080`, without `/v1`. These apps add `/v1/messages` themselves. Claude Code: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080` |+| OpenAI Responses API (Codex CLI) | `http://127.0.0.1:8080/v1/responses` ([setup](docs/DETAILS.md#the-responses-api-and-codex-cli)) |-You can also install from the repository. [Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip)-and unzip it (or `git clone` it). **Windows:** double-click **`START-HERE.bat`**. **Linux:** run **`./setup.sh`** in-the Strata folder.+`strata connect` prints the exact settings for OpenCode, Codex CLI, Claude Code, oh-my-pi, pi, Hermes Agent and OpenClaw.+`strata connect opencode --write` writes the file after asking and keeps a backup.-**Mac (Apple Silicon, M1 to M5, macOS 15 or newer):** install Apple's Command Line Tools once-(`xcode-select --install` in Terminal; Xcode itself is not needed). Then open Terminal in the Strata folder and run-**`./setup.sh`**. It builds the Mac engine on your Mac (about 7 minutes), asks the same questions, downloads the model-and starts it. Next time, `./setup.sh` starts it right away. Which size fits your Mac and how fast it is:-[docs/MAC.md](docs/MAC.md).+### Pictures-The steps are the same for NVIDIA and AMD. The installer finds your card and sets up the right engine for it. It-asks you a few questions:+Pictures are on by default for models that can read them. Attach or paste a picture in the chat, or send it from your app.+The picture reader runs on the processor when your graphics card has less than 24 GB (a few seconds per picture, nothing+taken from the card's memory) and on the graphics card from 24 GB. The rule is the same for NVIDIA and AMD. Turn pictures+off per model, or choose the reader with `--vision off`, `--vision gpu` or `--vision cpu`.-- which model and which size,-- how much context (how much text the model keeps in mind),-- whether it should read pictures.+The numbers behind the rule: on a Radeon AI PRO R9700 the card's reader took 57 ms for a 260-token picture and the+processor's took 3.1 s. On 16 GB cards (Tesla P100 and RTX A4000, Linux) the card's reader cost 3-5% of text speed,+because it holds about 1.4 GB that the model would use otherwise. The Mac has no picture reader yet.-Press Enter each time for the recommended answer. Then it downloads the model (about 70 GB) and starts it. If the-download stops, run it again: it continues where it left off. Your browser opens the Strata page at-`http://127.0.0.1:8080`.+### Good to know-> **While the model starts, your PC can be slow or stop responding for 1-3 minutes** (longest the first time).-> Strata loads 35-55 GB into your RAM and locks part of it for the graphics card. This is normal. Wait, and don't-> close the window. The window shows what Strata is doing.+- Choose the thinking level (**off, low, medium or high**) in the chat menu or in your app's "reasoning effort". Off is fastest.+- Strata answers one request at a time by default. For several at once, use `--parallel N` in `strata launch` or+ `"parallel": 2` in the config ([BATCHING.md](docs/BATCHING.md)). On a 12 GB card this makes each answer slower.+- Strata reads the first message of a chat in full, about 1 minute per 30,000 tokens. Follow-up messages start in seconds.+- Where your chats are stored and the whole API: [docs/INSTALL.md](docs/INSTALL.md#where-things-are-stored), [docs/DETAILS.md](docs/DETAILS.md#using-it).-**Next time**, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close-its window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker,-several cards, where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).+## Build from source++<details>+<summary>Windows, Linux and Mac from the repository</summary>++[Download Strata](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it, or `git clone` it.+**Windows:** double-click `START-HERE.bat`. **Linux:** run `./setup.sh` in the Strata folder.++**Mac (Apple Silicon, macOS 15 or newer):** install Apple's Command Line Tools once with `xcode-select --install`+(Xcode itself is not needed). Open Terminal in the Strata folder and run `./setup.sh`. It builds the Mac engine on your+Mac (about 7 minutes), asks the same questions, downloads the model and starts it. Next time, `./setup.sh` starts it+right away. Which size fits your Mac: [docs/MAC.md](docs/MAC.md).++The installer finds your card and sets up the right engine. It asks which model and size, how much context (how much text+the model keeps in mind) and whether it should read pictures. Press Enter for the recommended answer. Then it downloads the+model (about 70 GB) and starts it. If the download stops, run it again and it continues. Your browser opens the Strata+page at `http://127.0.0.1:8080`.-**Is it fast on your PC?** Run `BENCH.bat` (`./bench.sh`): about 5-10 minutes, it measures chat, 4K/16K prompts and an agent-style session with your own settings and writes a report to share ([how](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).+> While the model starts, your PC can be slow or stop responding for 1-3 minutes (longest the first time). Strata loads+> 35-55 GB into your RAM and locks part of it for the graphics card. Wait, and don't close the window.-### Optional: more speed on one card when the whole model fits in RAM+Next time, run `START-HERE.bat` (or `./setup.sh`) again. It starts right away and downloads nothing twice. Close its+window to stop the model. `UPDATE.bat` (`./update.sh`) updates Strata without starting it. Updating, Docker, several cards,+where the files go and every option: [docs/INSTALL.md](docs/INSTALL.md).-Two opt-in settings (by RubenTorney, #1904 and #1905). They help when you run one graphics card and your RAM holds-every expert (Strata's arena mode, for example Q2_0 or IQ3_XXS on a 64 GB PC). Open your config,-`strata-<model>.json` in the Strata data folder ([where](docs/INSTALL.md#where-things-are-stored)), and add:+To measure your PC, run `BENCH.bat` (`./bench.sh`). It takes 5-10 minutes and writes a report you can share+([how](docs/COMMUNITY_BENCHMARKS.md#run-the-benchmark)).-- **Faster decode:** `"--adapt-async", "1"` to the `"args"` list. The expert cache then updates on a helper thread- instead of pausing the answer. Measured on an RTX 5070 (Windows, Q2_0): +4-5% decode, +7-10% on code and long- documents. Answers can differ slightly from the default (they stay the same from run to run).-- **Faster prompt reading on Windows:** `"STRATA_PIN_BY_NEED": "1"` in the `"env"` block. Windows can lock only about- half the RAM for the graphics card; this locks the experts the card actually needs first. Measured on the RTX 5070:- +4.4% prompt reading.+**Optional: more speed on one card when the whole model fits in RAM.** Two opt-in settings by RubenTorney (#1904, #1905).+They help when you run one graphics card and your RAM holds every expert, for example Q2_0 or IQ3_XXS on a 64 GB PC. Open+`strata-<model>.json` in the Strata data folder ([where](docs/INSTALL.md#where-things-are-stored)) and add:++- `"--adapt-async", "1"` to the `"args"` list. The expert cache then updates on a helper thread instead of pausing the+ answer. Measured on an RTX 5070 (Windows, Q2_0): +4-5% decode, +7-10% on code and long documents. Answers can differ+ slightly from the default and stay the same from run to run.+- `"STRATA_PIN_BY_NEED": "1"` in the `"env"` block (Windows). Windows can lock only about half the RAM for the graphics+ card, and this locks the experts the card needs first. Measured on the RTX 5070: +4.4% prompt reading.```json"args": [ ..., "--adapt-async", "1" ],"env": { "STRATA_PIN_BY_NEED": "1" }```-Restart Strata (`START-HERE.bat` or `./setup.sh`) after the change. To undo, remove the two entries.--## Something went wrong?--- **My PC froze the first time Strata started.** This is normal while it loads the model. Wait, and don't close the- window. Still frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.-- **It stopped while downloading or installing.** Start the app or `strata launch` again, or run `START-HERE.bat`- (or `./setup.sh`) again. It continues where it stopped.-- **It's very slow and the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does- not have enough free RAM. Close other programs (browsers use a lot), or pick a smaller size (Q2_0 or IQ2_XS).-- **It says there is no engine for this PC.** In the app, open **Settings, Engine** and press **Update engine**. In- the Terminal, run `strata update`. `strata doctor` shows which engine files are installed.-- **It says port 8080 is already in use.** Strata is already running. Look for its window.-- **A Mac says the app is "damaged".** The file is fine. [docs/MAC.md](docs/MAC.md) has the one command.+Restart Strata after the change. To undo, remove the two entries.-More problems and their fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an-[issue](https://github.com/Niko1221/Strata/issues). In the app, attach the file from **Settings, Engine, Diagnostics,-View logs, Save logs**. In the Terminal, paste the output of `strata doctor`. From source, attach `strata-<model>.log`-from the Strata folder. Found a security problem? Report it privately: [SECURITY.md](SECURITY.md).+</details>## How does it work?Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has-12-24 GB. Strata makes the model fit by **sharing the work across your whole PC**. Think of a kitchen: the things-you use all the time stay on the counter, and the rest waits in the pantry.+12-24 GB. Strata makes the model fit by sharing the work across your whole PC. Picture a library.++- The model is a team of 24,576 small specialists ("experts"), like 24,576 reference books. Each word needs only 10 of them.+- **Your graphics card is the desk.** It is small and fast. The roughly 4,000 books used most often lie open on it.+- **Your RAM is the shelves in the reading room.** All 24,576 books are there. When a word needs a book that is not on the+ desk, your processor reads it from the shelf at once, while the card keeps working.+- **Your SSD is the archive.** It holds a 29 GB lookup table. The model reads a few rows of it per word, so it does not+ need to sit in RAM.<p align="center"><img src="docs/media/how-it-works.svg" width="860" alt="The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD"></p>-- **The model is a team of 24,576 small specialists ("experts").** Each word needs only 10 of them.-- **Your graphics card** keeps the few thousand experts that are used most often. **Your RAM** holds all of them,- and **your processor** works on the rest at the same time. **Your SSD** holds a big lookup table.+A small helper also guesses the next few words. The big model checks all of them at once and keeps the right ones. You get+the same answer, 1.6-1.8x sooner. Long texts are read in big pieces, up to 8,192 tokens at a time, at over 1,000 tokens+per second.<p align="center"><img src="docs/media/guess-and-check.svg" width="860" alt="A small helper guesses the next words; the big model checks them all at once and keeps the right ones"></p>-- **Guess, then check:** a small helper guesses the next few words. The big model checks them all at once. You get- the same answer, 1.6-1.8x sooner.-- **Long texts are read in big pieces** (up to 8,192 tokens at a time), at over 1,000 tokens per second.-The longer explanation: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md). Every part and its numbers:[the details](docs/DETAILS.md#how-it-works) and the [paper](docs/paper/Strata-Paper.pdf).+## Something went wrong?++- **My PC froze the first time Strata started.** It loads the model into RAM. Wait, and don't close the window. Still+ frozen after 10 minutes? Restart the PC, close other programs and try again, or pick a smaller size.+- **It stopped while downloading or installing.** Start the app or `strata launch` again, or run `START-HERE.bat` (or+ `./setup.sh`) again. It continues where it stopped.+- **It is very slow, the disk light keeps blinking, or it says "the engine stopped unexpectedly".** Your PC does not have+ enough free RAM. Close other programs (browsers use a lot), or pick a smaller size such as Q2_0 or IQ2_XS.+- **It says there is no engine for this PC.** In the app, open **Settings, Engine** and press **Update engine**. In the+ Terminal, run `strata update`. `strata doctor` shows which engine files are installed.+- **It says port 8080 is already in use.** Strata is already running. Look for its window.+- **A Mac says the app is "damaged".** The file is fine. [docs/MAC.md](docs/MAC.md) has the one command.++More problems and fixes: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md). Still stuck? Open an+[issue](https://github.com/Niko1221/Strata/issues). In the app, attach the file from **Settings, Engine, Diagnostics, View+logs, Save logs**. In the Terminal, paste the output of `strata doctor`. From source, attach `strata-<model>.log` from the+Strata folder. For a security problem, report it privately: [SECURITY.md](SECURITY.md).+## Credits and license-The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was-compressed by [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5)+The model is [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) by the Qwen team. It was compressed by+[ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF), UkisAI (Swift 1.5), agentionai (Gyro)and Unsloth. Strata uses parts of [llama.cpp / ggml](https://github.com/ggml-org/llama.cpp). All credits:-[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few-parts and every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).+[docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md#credits). Strata is open source under the [MIT License](LICENSE). A few parts and+every model have their own licenses ([which ones](docs/HOW_IT_WORKS.md#license)).## Contributing@@ -365,6 +352,9 @@ include so that others can check your work.## Support Strata-Strata is free and open source. If it is useful to you, you can support its development:+Strata is free and MIT-licensed. I build it and test it on whatever hardware I can get. If it runs well on your PC, a coffee helps+me keep testing new cards and fixing what you report. No pressure. Using it and reporting what you find helps just as much.++<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>-<p align="center"><a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a></p>+Interested in sponsoring Strata? Contact me via [GitHub](https://github.com/Niko1221).