Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

AGENTS, AI_SETUP, INSTALL: the app and the Terminal first, install from source as the fallback

Niko122126613a44f62a
diff --git a/AGENTS.md b/AGENTS.md
index c4726159f..c32a038e1 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,35 +1,82 @@
# AGENTS.md
-Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5 and Unsloth variants) on a
-normal PC: one NVIDIA or AMD graphics card plus system RAM, on Windows or Linux. It has a C++/CUDA/HIP engine
-(`src/`, `include/`), a Python server with an OpenAI- and Anthropic-compatible API and a web app (`serve/`), and a
-one-click installer (`setup.py`, started by `START-HERE.bat` / `setup.sh`).
+Strata runs the Qwen3.8-Flash-Next mixture-of-experts model (and its Coder, Swift 1.5, Unsloth, Gyro and NVFP4
+variants, and Qwen3.6-35B-A3B) on a normal PC: one NVIDIA or AMD graphics card plus system RAM on Windows or Linux,
+or an Apple Silicon Mac. Version 0.2.0-beta, engine 0.1.45.3. It has three parts:
+
+- **The app** (`apps/desktop`, the Tauri shell; `apps/host`, the Node host and the web UI). It is for people who do
+ not want a terminal. The installer has the engine inside.
+- **The Terminal** (`strata.py`, `strata_cli/`, `strata_control/`): the `strata` command. It sets up a model, serves
+ it and watches it. Agents use this part.
+- **The engine and the server** (`src/`, `include/`, `metal/`, `sycl/` for the C++/CUDA/HIP/SYCL/Metal engine;
+ `serve/` for the Python server with an OpenAI- and Anthropic-compatible API and a web page). `setup.py`, started by
+ `START-HERE.bat` or `setup.sh`, is the older one-click installer from source.
## Installing Strata for a user
-Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**: check the PC, pick the model by RAM, run setup non-interactively,
-start and verify the server, and connect the user's apps. Never expose the server beyond `127.0.0.1` without
-`--api-key`. As an alternative to shell commands, Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md))
-offers the same steps as tools.
+Follow **[docs/AI_SETUP.md](docs/AI_SETUP.md)**. The short version, for an agent that has a shell:
+
+1. Install the Terminal. It needs no administrator rights.
+
+ ```sh
+ curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh # Linux, Mac
+ irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex # Windows PowerShell
+ ```
+
+ On Windows, open a new terminal afterwards so that `strata` is on the PATH.
+2. Check what would happen. Nothing is changed:
+
+ ```sh
+ strata launch --plan --model q2_0
+ ```
+
+3. Set up and serve without questions. It downloads the engine for the card and the model (tens of GB), starts the
+ server, prints the addresses and returns. The server keeps running.
+
+ ```sh
+ strata launch --model q2_0 --yes --no-tui
+ ```
+
+ Useful flags (all of them: `strata launch --help`): `--family`, `--model-path` (files the user already has),
+ `--data-dir`, `--context 32k|64k|128k`, `--vision off|on|gpu|cpu`, `--parallel N`, `--gpu 0,1`,
+ `--access local|lan|cloudflare|tailscale`, `--port`, `--api-key-env NAME`, `--new-key`, `--save-preset NAME`,
+ `--output json`. `strata launch --status` shows what is serving, `strata launch --stop` stops it, and
+ `strata launch --show-key` prints the API key.
+4. Connect the user's apps: `strata connect` lists them, `strata connect <app> --check` sends one prompt and one tool
+ call the way that app does.
+
+Never expose the server beyond `127.0.0.1` without an API key. `--access lan` makes a key for you. The old route (clone
+the repository and run `START-HERE.bat --yes` or `./setup.sh --yes`) still works and is the fallback when the
+Terminal cannot run. Strata's MCP server ([docs/MCP_SERVER.md](docs/MCP_SERVER.md)) offers the same steps as tools.
## Working on the code
- How the engine works, every measured number, the API and all settings: [docs/DETAILS.md](docs/DETAILS.md) and
the [paper](docs/paper/Strata-Paper.pdf).
-- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md).
-- Setup's own tests run without a GPU or downloads: `python tools/test_setup_<name>.py` (for example
- `tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
-- Keep the docs' style: plain words, measured numbers with what they were measured on, no claims without a
- measurement.
+- The Terminal: [docs/TERMINAL.md](docs/TERMINAL.md) and [docs/TERMINAL_LAUNCH.md](docs/TERMINAL_LAUNCH.md). The app,
+ the packages and the update feed: [packaging/README.md](packaging/README.md) and [packaging/UPDATES.md](packaging/UPDATES.md).
+- AMD (HIP) build and validation: [docs/AMD_HIP.md](docs/AMD_HIP.md); multi-GPU: [docs/MULTI_GPU.md](docs/MULTI_GPU.md);
+ Mac (Metal): [docs/MAC.md](docs/MAC.md).
+- Tests: the Terminal and Control tests are in `tests/terminal` (`python -m pytest tests/terminal`). The app's UI
+ tests run with `npm test` in `apps/host`. Setup's own tests run without a GPU or downloads:
+ `python tools/test_setup_<name>.py` (for example `tools/test_setup_amd.py`, `tools/test_setup_choices.py`).
+- Keep the docs' style: plain words, short sentences, written for non-technical users first. Give measured numbers
+ with what they were measured on. Make no claim without a measurement.
## Contributing a change or report
-- Search the open issues and pull requests first, and add to a thread that already covers your point.
-- Open an issue with the form that fits (bug report, feature request or question).
-- One change per pull request. Say what it changes and what it leaves alone.
+The rules are in **[CONTRIBUTING.md](CONTRIBUTING.md)**. Read it before you open an issue or a pull request, and
+follow it whether a person or an agent writes the text. The short version:
+
+- Search the open and closed issues and pull requests first, and add to a thread that already covers your point.
+- Agree the scope in an issue before a large feature, a refactor or a change to a default.
+- One problem per pull request. Lead with the problem and the result, then the approach, then the validation
+ (passed, failed and not run, kept apart) and the limits.
- A new feature is opt-in, and the default path stays byte-identical to the last release. Say how you checked.
- Build every backend a file touches (CUDA, HIP, SYCL) before asking for review.
- Change a default only where you measured it faster, and show the numbers with what they were measured on.
+- Never submit measurements, logs or reproduction steps you did not produce. Mark a finding from reading code as
+ unconfirmed static analysis.
- A report from hardware the maintainers do not have is welcome. Follow
[docs/COMMUNITY_BENCHMARKS.md](docs/COMMUNITY_BENCHMARKS.md), compare against a same-day run of the build you are
testing, and say what you did not test.
diff --git a/docs/AI_SETUP.md b/docs/AI_SETUP.md
index 236d7fb5d..3ce89c5c1 100644
--- a/docs/AI_SETUP.md
+++ b/docs/AI_SETUP.md
@@ -2,147 +2,156 @@
This page is written for an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...) that a user asked to
set up Strata on their PC. Strata runs the Qwen3.8-Flash-Next model locally on one NVIDIA or AMD graphics card plus
-system RAM, and serves an OpenAI- and Anthropic-compatible API on `http://127.0.0.1:8080`. The human-oriented
-version of everything below is [INSTALL.md](INSTALL.md).
+system RAM (or on an Apple Silicon Mac), and serves an OpenAI- and Anthropic-compatible API on `http://127.0.0.1:8080`.
+The human-oriented version of everything below is [INSTALL.md](INSTALL.md).
+
+You use the **Terminal**: the `strata` command. Install it with one line, then `strata launch` downloads the engine
+for the card and the model, and starts the server. If the Terminal cannot run on this PC, step 9 has the older route
+from source.
Work through the steps in order. Tell the user what you are doing in plain words; they may not be technical.
## 0. Ground rules
-- **Never expose the server without an API key.** Keep the default `--host 127.0.0.1`. Only if the user asks for
- access from other devices, use `--host 0.0.0.0` **together with** `--api-key <a long random secret>`, and give the
- user the key. Never put a tunnel or port forward in front of a server without a key.
-- Do not change the user's system beyond what setup does (setup installs Python for the user account if needed,
- and everything else inside the Strata folder and its `Strata-data` folder). Ask before installing drivers.
-- The model download is ~70 GB (94 GB for Unsloth's UD-IQ4_XS, 111 GB for UD-Q4_K_XL). Confirm the user is fine with that before you
- start, especially on a metered connection.
-- Setup is long-running (an hour or more on a slow connection). Run it in the background or with a long timeout and
- poll its output; do not kill it because it is quiet for a while. It is resumable: running the same command again
- continues where it stopped.
+- **Never expose the server without an API key.** Keep the default (`--access local`, the server listens on
+ `127.0.0.1` only). Only if the user asks for access from other devices, use `--access lan`. Strata makes a long
+ random key for you then and will not start on a network without one. Give the user the key
+ (`strata launch --show-key`). Never put a tunnel or port forward in front of a server without a key.
+- Do not change the user's system beyond what Strata does. The Terminal installs into the user's home folder and needs
+ no administrator rights. It does not install drivers, Python packages or compilers. Ask before installing drivers.
+- The model download is ~70 GB (94 GB for Unsloth's UD-IQ4_XS, 111 GB for UD-Q4_K_XL). Confirm the user is fine with
+ that before you start, especially on a metered connection.
+- The first `strata launch` is long-running (an hour or more on a slow connection). Run it in the background or with a
+ long timeout and poll its output; do not kill it because it is quiet for a while. It is resumable: running the same
+ command again continues where it stopped.
+- Piping a script from the internet into a shell runs code you have not read. Tell the user what the line does (step 2).
+ If they prefer, download the script first, show it, and run it only after they agree.
## 1. Check the PC
-The quickest way is setup's own check, which prints the GPU(s), driver, RAM, CPU, free disk and whether this PC can run
-Strata, and installs nothing except Python and the `.venv` it needs to run (step 2 gets the repository first):
-
-```
-Windows: START-HERE.bat --check
-Linux: ./setup.sh --check
-```
-
-To check by hand:
-
-| What | Windows (PowerShell) | Linux |
-| --- | --- | --- |
-| OS | `[Environment]::OSVersion` | `cat /etc/os-release` |
-| NVIDIA GPU, VRAM, driver | `nvidia-smi --query-gpu=index,name,memory.total,driver_version --format=csv` | the same |
-| AMD GPU | `Get-CimInstance Win32_VideoController \| Select-Object Name, AdapterRAM` (AdapterRAM is capped at 4 GB; trust the model name) | `lspci \| grep -i -E 'vga\|display'`; VRAM: `cat /sys/class/drm/card*/device/mem_info_vram_total` (bytes); `rocm-smi` only if ROCm is installed |
-| RAM | `(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory / 1GB` | `free -g` |
-| Free disk | `Get-PSDrive -PSProvider FileSystem` | `df -h .` |
-
Requirements (details: [INSTALL.md](INSTALL.md#what-you-need)):
- **GPU:** NVIDIA RTX 20, 30, 40 or 50 series, or AMD Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT,
RX 9070 / 9070 XT, Radeon AI PRO R9700, RX 6800 / 6900 series; 12 GB of VRAM or more (an NVIDIA card with 8 GB runs,
slowly). GTX 10 series and older, and integrated GPUs, are not supported; Pascal / Volta cards (P40, V100) and some
- older AMD cards have experimental paths the user opts into ([OLDER_GPUS.md](OLDER_GPUS.md)).
+ older AMD cards have experimental paths the user opts into ([OLDER_GPUS.md](OLDER_GPUS.md)). A **Mac** needs Apple
+ Silicon (M1 or newer), macOS 15 or newer and 16 GB of memory or more ([MAC.md](MAC.md)).
- **Driver:** NVIDIA 580 or newer. AMD on Linux: the kernel's amdgpu driver; on Windows: a current AMD Adrenalin
driver. If the driver is missing or too old, tell the user to update it (NVIDIA App / nvidia.com/drivers, or AMD
Software) and restart; do not install drivers yourself unless they ask.
-- **RAM:** 32 GB or more (see step 3). **Disk:** ~80 GB free, ideally on an NVMe SSD. **CPU:** x86-64 with AVX2. Without AVX2 (Xeon E5 v1/v2 and older) setup still installs, as an experimental and slow build it compiles on the PC (10-20 minutes): tell the user that before starting ([INSTALL.md](INSTALL.md#older-cpus-experimental)).
-- **OS:** Windows 10/11 or Linux (Ubuntu 22.04/24.04 and Debian 12/13 are fully automatic; Debian 13 gets the oldest CUDA 13 its NVIDIA repo has, currently 13.1).
+- **RAM:** 32 GB or more (see step 3). **Disk:** ~80 GB free, ideally on an NVMe SSD. **CPU:** x86-64 with AVX2.
+ Without AVX2 (Xeon E5 v1/v2 and older) the ready-made engine refuses to start; the route from source builds an
+ experimental and slow engine on the PC (10-20 minutes): tell the user that before starting
+ ([INSTALL.md](INSTALL.md#older-cpus-experimental)).
+- **OS:** Windows 10/11, Linux x86-64 (Ubuntu 22.04/24.04 and Debian 12/13 are fully automatic), or macOS on Apple Silicon.
+
+To check by hand:
+
+| What | Windows (PowerShell) | Linux | Mac |
+| --- | --- | --- | --- |
+| OS | `[Environment]::OSVersion` | `cat /etc/os-release` | `sw_vers` |
+| NVIDIA GPU, VRAM, driver | `nvidia-smi --query-gpu=index,name,memory.total,driver_version --format=csv` | the same | - |
+| AMD GPU | `Get-CimInstance Win32_VideoController \| Select-Object Name, AdapterRAM` (AdapterRAM is capped at 4 GB; trust the model name) | `lspci \| grep -i -E 'vga\|display'`; VRAM: `cat /sys/class/drm/card*/device/mem_info_vram_total` (bytes); `rocm-smi` only if ROCm is installed | - |
+| RAM | `(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory / 1GB` | `free -g` | `sysctl -n hw.memsize` (bytes) |
+| Free disk | `Get-PSDrive -PSProvider FileSystem` | `df -h .` | `df -h ~` |
If the PC does not meet them, say which part is missing and stop.
-## 2. Get Strata
+## 2. Install the Terminal
+
+One line. It needs no administrator rights. It checks what kind of PC this is, downloads the Terminal archive, checks
+its SHA-256, unpacks it into the user's home folder, checks the release's signed update feed, and puts `strata` on the
+PATH.
```
-git clone https://github.com/Niko1221/Strata.git
+Linux, Mac: curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
+Windows (PowerShell): irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
-Without git: download https://github.com/Niko1221/Strata/archive/refs/heads/main.zip and unzip it. Pick a drive with
-enough free space: the model files go to a `Strata-data` folder **next to** the Strata folder (or `--data-dir <path>`
-on another drive).
+On Windows, open a new terminal window afterwards (the PATH changed). Check it: `strata --version`.
## 3. Pick the model
-By the PC's RAM (ask the user whether they mainly want it for code - then the Coder is a good pick at any RAM size):
+`strata launch --plan` shows what Strata would do and changes nothing. With no `--model` it picks the model for this
+PC's RAM and prints the exact command line (`same as strata launch --model ...`). Use it as the default, or pick
+yourself by the PC's RAM (ask the user whether they mainly want it for code - then the Coder is a good pick at any RAM
+size):
-| RAM | `--family` / `--model` | Notes |
+| RAM | `--model` (and `--family`) | Notes |
| --- | --- | --- |
-| 32 GB | `--family coder` (size IQ1_M) | for code; with a 24 GB card Q2_0 and IQ2_XS also run (setup picks the low-RAM mode) |
-| 48 GB | `--family qwen --model IQ2_XS` | or `Q2_0` (fastest) |
-| 64 GB | `--family qwen --model IQ2_XS` (recommended) | `IQ3_XXS` / `IQ3_S` are slower and a bit better |
-| 96 GB+ | `--family qwen --model IQ3_S` | `--family unsloth --model UD-IQ4_XS` (~4-bit, 94 GB, part of the experts read from the SSD under ~80 GB of RAM); `--model UD-Q4_K_XL` is experimental: NVIDIA only, NVMe SSD, 7-8.5 tokens/s on 64 GB |
-
-`--family swift` (Swift 1.5, a fine-tune that thinks shorter; sizes Q2_0, IQ2_XS, IQ3_XXS) is the alternative to
-`qwen`. With `--yes` and no `--model`, setup picks the recommended size for the RAM itself. More: [MODELS.md](MODELS.md).
+| 32 GB | `--family coder --model iq1_m` | for code; with a 24 GB card Q2_0 and IQ2_XS also run (Strata picks the low-RAM mode) |
+| 48 GB | `--model iq2_xs` | or `q2_0` (fastest) |
+| 64 GB | `--model iq2_xs` (recommended) | `iq3_xxs` / `iq3_s` are slower and a bit better |
+| 96 GB+ | `--model iq3_s` | `--family unsloth --model ud-iq4_xs` (~4-bit, 94 GB, part of the experts read from the SSD under ~80 GB of RAM); `ud-q4_k_xl` is experimental: NVIDIA only, NVMe SSD, 7-8.5 tokens/s on 64 GB |
-## 4. Run setup without questions
+`--family swift` (Swift 1.5, a fine-tune that thinks shorter; sizes IQ2_XS, IQ3_XXS, IQ3_S) is the
+alternative to `qwen`. A model name is not case sensitive. More: [MODELS.md](MODELS.md).
-From the Strata folder (Windows: in `cmd`, or `cmd /c START-HERE.bat ...` from PowerShell):
+## 4. Set up and serve, without questions
```
-Windows: START-HERE.bat --yes --family qwen --model IQ2_XS --no-start
-Linux: ./setup.sh --yes --family qwen --model IQ2_XS --no-start
+strata launch --plan --model iq2_xs # what would happen; nothing is changed
+strata launch --model iq2_xs --yes --no-tui # do it: download the engine and the model, start the server
```
-The flags (all of them: `START-HERE.bat --help`):
+With `--no-tui` and `--yes` it asks nothing, prints the addresses when the server is ready, and returns. **The server
+keeps running** in the background. `strata launch` with no `--model`, `--preset` or other choice needs a terminal for
+its screens, so an agent always names the model. Add `--output json` for output a script can read.
+
+The flags (all of them, with examples: `strata launch --help`):
| Flag | Meaning |
| --- | --- |
-| `--yes` | take the recommended answer to every question (no prompts) |
-| `--family qwen\|swift\|coder\|unsloth` | the model version |
-| `--model Q2_0\|IQ2_XS\|IQ3_XXS\|IQ3_S\|IQ1_M\|UD-IQ4_XS\|UD-Q4_K_XL` | the size (the Coder is IQ1_M, Unsloth UD-IQ4_XS or UD-Q4_K_XL) |
-| `--context N` | context in tokens; default by VRAM: 32768 under 14 GB, 65536 under 20 GB, else 131072 |
-| `--vision yes\|no\|gpu\|cpu` | read pictures; `--yes` leaves images off. AMD cards: `cpu` |
-| `--gpu N` / `--gpus 0,1` / `--gpus all` | one card, or several sharing the model (default: the card with the most VRAM) |
-| `--backend cuda\|hip` | NVIDIA or AMD engine; chosen by itself on a PC with only one kind of card |
-| `--data-dir PATH` | where the 70-120 GB of model files go |
+| `--model NAME` | the size: `q2_0`, `iq2_xs`, `iq3_xxs`, `iq3_s`, `iq1_m`, `ud-iq4_xs`, `ud-q4_k_xl`, ... (the error message for a wrong name lists them) |
+| `--family NAME` | the model version when several have that size: `qwen`, `swift`, `coder`, `unsloth` |
+| `--model-path PATH` | files the user already has: a folder of shards or one `.gguf` file |
+| `--data-dir DIR` | where the 70-120 GB of model files go |
+| `--context SIZE` | `32k`, `64k`, `128k`, `256k` or a number of tokens |
+| `--vision off\|on\|gpu\|cpu` | read pictures. On by default when the model can read them: the reader runs on the CPU for a card under 24 GB and on the GPU from 24 GB, on every make of card. `off` turns it off |
+| `--parallel N` | requests answered at the same time, 1 to 8 (default: fit to the card) |
+| `--gpu LIST` | card numbers, e.g. `0` or `0,1` (several cards share the model) |
+| `--tune quick\|thorough\|skip` | find the fastest settings first (default here: `skip`) |
+| `--access local\|lan\|cloudflare\|tailscale` | who can connect; beyond `local` a key is made for you |
| `--port N` | the server port (default 8080) |
-| `--host H --api-key K` | listen beyond this PC; **only together with a key** |
-| `--no-start` | install only, do not start the server |
-| `--setup` | install another model or change settings of an installed one |
-| `--check` | only check the PC |
-
-`--no-start` is recommended for agents: the server runs in the foreground until its window is closed, which would
-block your shell. Start it separately in step 6.
+| `--api-key KEY`, `--api-key-env NAME`, `--new-key` | the key clients must send |
+| `--save-preset NAME` | keep these choices; `strata launch --preset NAME` starts them again |
+| `--plan` | show what would happen and exit |
+| `--yes`, `--no-tui` | no questions, no screens |
+| `--status`, `--stop`, `--show-key` | what is serving, stop it, print the key |
Notes:
-- **Linux:** setup uses `sudo apt` (or dnf/pacman) to install Python with venv if it is missing, and on AMD may need
- `build-essential` and `git`. You cannot type the user's password: if a `sudo` step is needed, ask the user to run it
- (e.g. `sudo apt install python3-venv build-essential git`) and then rerun setup.
-- **AMD on Linux:** setup installs ROCm into `.venv` (~10 GB, no sudo) unless a system ROCm 7 exists, and compiles the
- engine for the card (10-20 minutes, once).
-- **AMD on Windows (new in 0.1.34):** setup downloads the ready-made AMD engine (~550 MB, ROCm included) - nothing
- is compiled and only the AMD driver is needed. Before the model download it runs `engine\strata-device.exe
- --list-devices`; if that does not list the card, the driver is the problem (tell the user to update AMD Software).
- One card per model and no images on Windows for now. It is new: ask the user to report how it runs (docs/AMD_HIP.md).
-- **NVIDIA with no ready-made engine for the card:** setup offers to install build tools and compile (20-40 minutes);
- `--yes` accepts.
+- **No engine for this PC.** `strata launch` does not refuse. It says "Strata couldn't find an engine for this computer",
+ goes on to download the model (with `--yes` it does not ask), and ends without starting anything (exit code 0; with `--output json` the answer
+ has `"serving": false` and an `engine_problem`). Tell the user which card was not recognised and stop. Updating the
+ engine later with `strata update` can fix it.
+- **Exit codes:** 0 done, 2 a flag or name is wrong, 5 the Terminal or the engine cannot do it, 6 an install or the
+ server start failed (the engine log is named in the message), 130 stopped.
+- **Linux:** nothing needs `sudo`: the Terminal does not install packages. **Windows:** the first start may ask once
+ whether to allow Strata on the network if you used `--access lan`.
+- **AMD on Windows:** only the AMD driver is needed. The AMD engine is a ready-made download. It has not been run with a
+ model on a real AMD card on Windows by us: ask the user to report how it runs ([AMD_HIP.md](AMD_HIP.md)).
+- **Mac:** the Metal engine is fetched like any other. There is no picture reader on the Mac yet.
## 5. While it downloads, tell the user
- It downloads about 70 GB from Hugging Face; how long depends on their connection (at 100 Mbit/s roughly 1.5-2
hours). It can be stopped and continues where it left off.
-- Then it prepares the model for their PC (a few minutes) and, on AMD on Linux, compiles the engine.
+- Then it prepares the model for their PC (a few minutes).
- **When the model starts, the PC can be slow or stop responding for 1-3 minutes** (longest the first time): Strata
loads 35-55 GB into RAM and locks part of it for the graphics card. That is normal; they should not close the window.
- Nothing leaves their PC: the model runs locally.
-## 6. Start the server
+## 6. Check that it is serving
-Setup prints the start script it wrote (`start script: run-<model>.bat`). The name is the family tag plus the size,
-lower case: `run-iq2_xs`, `run-swift-iq2_xs`, `run-coder-iq1_m`, `run-unsloth-ud-iq4_xs`.
+`strata launch` prints `Strata is serving.` with the addresses when it is done. Later:
```
-Windows (PowerShell): Start-Process -FilePath ".\run-iq2_xs.bat" (opens its own window)
-Linux: nohup ./run-iq2_xs.sh > strata-server.out 2>&1 &
+strata launch --status # lists what is serving, with its address
+strata launch --stop # stops it and frees the graphics memory
+strata launch --preset NAME # starts a saved preset again (after --save-preset NAME)
```
-Or `START-HERE.bat` / `./setup.sh` without flags, which starts the installed model (it asks which one when several are
-installed; add `--yes` to take the first). The server opens the user's browser on `http://127.0.0.1:8080` when it is
-ready. Closing its window (or stopping the process) stops the model.
+No browser opens by itself. Add `--open` to `strata launch` if the user wants the web page.
## 7. Verify
@@ -160,38 +169,115 @@ curl http://127.0.0.1:8080/v1/models
curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"strata\", \"messages\": [{\"role\": \"user\", \"content\": \"Say hello in five words.\"}], \"max_tokens\": 64, \"reasoning_effort\": \"none\"}"
```
-With an API key, add `-H "Authorization: Bearer <key>"`. The server window prints a `ready: http://127.0.0.1:8080/v1`
-line when it is ready; the engine log is `strata-<model>.log` in the Strata folder.
+With an API key, add `-H "Authorization: Bearer <key>"`.
## 8. Connect the user's apps
- **Browser:** `http://127.0.0.1:8080` - Chat, a live Monitor, and About (settings and addresses).
- **Any OpenAI-compatible app or agent:** base URL `http://127.0.0.1:8080/v1`, any API key (or the configured one),
any model name.
-- **Anthropic-compatible apps:** `http://127.0.0.1:8080/v1/messages`.
+- **Anthropic-compatible apps:** base URL `http://127.0.0.1:8080`, **without `/v1`**. Those apps add `/v1/messages`
+ themselves.
- **Codex CLI and other Responses API apps:** `http://127.0.0.1:8080/v1/responses` (stateless; Codex's
`config.toml`: [DETAILS.md](DETAILS.md#the-responses-api-and-codex-cli)).
- **Claude Code:** `ANTHROPIC_BASE_URL=http://127.0.0.1:8080`, `ANTHROPIC_MODEL` set to a Claude model name it knows
(Strata ignores the name), and any `ANTHROPIC_AUTH_TOKEN` (or the configured key).
+- **The easy way for a coding app:** `strata connect` lists them (OpenCode, Codex CLI, Claude Code, oh-my-pi, pi,
+ Hermes Agent, OpenClaw). `strata connect <app>` prints the file and environment for the running server,
+ `strata connect <app> --write` writes it after asking (and keeps a backup), `strata connect <app> --check` sends
+ one prompt and one tool call the way that app does and says whether each worked.
- **Thinking level:** `"reasoning_effort": "none" | "low" | "medium" | "high"` (default high).
-- Strata answers one request at a time; `"parallel": N` in the model's config (or setup `--parallel N`) decodes up to
- N together ([BATCHING.md](BATCHING.md)). API details: [DETAILS.md](DETAILS.md#using-it).
+- Strata answers one request at a time; `--parallel N` in `strata launch` decodes up to N together
+ ([BATCHING.md](BATCHING.md)). API details: [DETAILS.md](DETAILS.md#using-it).
+
+## 9. If the Terminal cannot run: install from source
+
+Use this only when step 2 does not work on the PC (for example a CPU without AVX2, or no internet access to GitHub
+Releases). It needs git or the zip, and on Linux `sudo` for Python and build tools.
+
+```
+git clone https://github.com/Niko1221/Strata.git
+```
+
+Without git: download https://github.com/Niko1221/Strata/archive/refs/heads/main.zip and unzip it. Pick a drive with
+enough free space: the model files go to a `Strata-data` folder **next to** the Strata folder (or `--data-dir <path>`
+on another drive). Setup's own check prints the GPU(s), driver, RAM, CPU, free disk and whether this PC can run
+Strata, and installs nothing except Python and the `.venv` it needs to run:
+
+```
+Windows: START-HERE.bat --check
+Linux: ./setup.sh --check
+```
+
+Run setup without questions, from the Strata folder (Windows: in `cmd`, or `cmd /c START-HERE.bat ...` from PowerShell).
+The model is picked as in step 3, with setup's flag names (`--family qwen|swift|coder|unsloth --model Q2_0|IQ2_XS|...`):
+
+```
+Windows: START-HERE.bat --yes --family qwen --model IQ2_XS --vision yes --no-start
+Linux: ./setup.sh --yes --family qwen --model IQ2_XS --vision yes --no-start
+```
+
+The flags (all of them: `START-HERE.bat --help`):
+
+| Flag | Meaning |
+| --- | --- |
+| `--yes` | take the recommended answer to every question (no prompts) |
+| `--family qwen\|swift\|coder\|unsloth` | the model version |
+| `--model Q2_0\|IQ2_XS\|IQ3_XXS\|IQ3_S\|IQ1_M\|UD-IQ4_XS\|UD-Q4_K_XL` | the size (the Coder is IQ1_M, Unsloth UD-IQ4_XS or UD-Q4_K_XL) |
+| `--context N` | context in tokens; default by VRAM: 32768 under 14 GB, 65536 under 20 GB, else 131072 |
+| `--vision yes\|no\|gpu\|cpu` | read pictures. Pass `yes` to get the best reader for the card (the CPU under 24 GB, the GPU from 24 GB, any make); `no` turns them off |
+| `--gpu N` / `--gpus 0,1` / `--gpus all` | one card, or several sharing the model (default: the card with the most VRAM) |
+| `--backend cuda\|hip` | NVIDIA or AMD engine; chosen by itself on a PC with only one kind of card |
+| `--data-dir PATH` | where the 70-120 GB of model files go |
+| `--port N` | the server port (default 8080) |
+| `--host H --api-key K` | listen beyond this PC; **only together with a key** |
+| `--no-start` | install only, do not start the server |
+| `--setup` | install another model or change settings of an installed one |
+| `--check` | only check the PC |
+
+`--no-start` is recommended for agents: the server runs in the foreground until its window is closed, which would
+block your shell. Start it separately:
+
+```
+Windows (PowerShell): Start-Process -FilePath ".\run-iq2_xs.bat" (opens its own window)
+Linux: nohup ./run-iq2_xs.sh > strata-server.out 2>&1 &
+```
+
+Setup prints the start script it wrote (`start script: run-<model>.bat`). The name is the family tag plus the size,
+lower case: `run-iq2_xs`, `run-swift-iq2_xs`, `run-coder-iq1_m`, `run-unsloth-ud-iq4_xs`. The engine log is
+`strata-<model>.log` in the Strata folder. Then continue with step 7.
+
+Notes:
+- **Linux:** setup uses `sudo apt` (or dnf/pacman) to install Python with venv if it is missing, and on AMD may need
+ `build-essential` and `git`. You cannot type the user's password: if a `sudo` step is needed, ask the user to run it
+ (e.g. `sudo apt install python3-venv build-essential git`) and then rerun setup.
+- **AMD on Linux:** setup installs ROCm into `.venv` (~10 GB, no sudo) unless a system ROCm 7 exists, and compiles the
+ engine for the card (10-20 minutes, once).
+- **AMD on Windows:** setup downloads the ready-made AMD engine (~550 MB, ROCm included) - nothing is compiled and only
+ the AMD driver is needed. Before the model download it runs `engine\strata-device.exe --list-devices`; if that does
+ not list the card, the driver is the problem (tell the user to update AMD Software). One card per model. Pictures use
+ the 24 GB rule like everywhere else.
+- **NVIDIA with no ready-made engine for the card:** setup offers to install build tools and compile (20-40 minutes);
+ `--yes` accepts.
+- **Mac:** `./setup.sh` builds the Metal engine on the Mac (needs Apple's Command Line Tools, `xcode-select --install`).
-## 9. When something fails
+## 10. When something fails
| Symptom | What to do |
| --- | --- |
-| `the NVIDIA driver is too old` | The user updates the driver (NVIDIA App or nvidia.com/drivers) and restarts; rerun setup. |
-| Download or install stopped | Rerun the same setup command; it continues. |
-| `port 8080 is already in use` | Strata is already running (check `/health`), or use `--port 8081`. |
-| Very slow, disk busy, or "the engine stopped unexpectedly" | Not enough free RAM: close programs, or set up a smaller size (`--setup --model Q2_0` or IQ2_XS). |
+| `the NVIDIA driver is too old` | The user updates the driver (NVIDIA App or nvidia.com/drivers) and restarts; run the command again. |
+| Download or install stopped | Run the same command again; it continues. |
+| `Strata couldn't find an engine for this computer` | The card is not one the engine packages cover, or the update feed was not reachable. Check the card against the list in step 1, check the internet, run `strata doctor`, then `strata update`. |
+| `port 8080 is already in use` | Strata is already running (check `/health` and `strata launch --status`), or use `--port 8081`. |
+| Very slow, disk busy, or "the engine stopped unexpectedly" | Not enough free RAM: close programs, or choose a smaller size (`q2_0` or `iq2_xs`). |
| `ExpertCache: cudaMalloc(...) failed: out of memory` with free VRAM (Windows) | The page file is off or tiny: set a fixed size (Custom size, the same initial and maximum size, e.g. 65536 MB) and restart; a page file Windows grows on demand ("System managed") may not grow in time (#60). |
-| `prompt ... exceeds the context` | Rerun setup with `--setup --context <bigger>`. |
+| `prompt ... exceeds the context` | Start again with a bigger `--context`. |
| No AMD GPU found (Linux) | The amdgpu driver lists no GPU; check the card and driver. Integrated GPUs are not supported. |
-| Python or build tools could not be installed | Install what the message names (links are printed), then rerun. |
+| A Mac says the app is "damaged" (the app, not the Terminal) | The file is fine: [MAC.md](MAC.md) has the one command. |
More: [TROUBLESHOOTING.md](TROUBLESHOOTING.md) and the [full table](DETAILS.md#troubleshooting). If it still fails,
-collect `strata-<model>.log` and the setup output, and suggest an issue at https://github.com/Niko1221/Strata/issues.
+run `strata doctor`, collect its output and the engine log named in the error message, and suggest an issue at
+https://github.com/Niko1221/Strata/issues.
## Alternative: the MCP server
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
index c9c6c888a..58cfe8ecc 100644
--- a/docs/INSTALL.md
+++ b/docs/INSTALL.md
@@ -1,10 +1,18 @@
# Installing Strata
-Everything about installing, starting, updating and configuring Strata, on Windows and Linux, with an NVIDIA or an
-AMD graphics card. The short version is in the [README](../README.md#install); an AI coding assistant can do all of
-this for you with [AI_SETUP.md](AI_SETUP.md).
+Everything about installing, starting, updating and configuring Strata, on Windows, Linux and Mac, with an NVIDIA
+or an AMD graphics card or an Apple Silicon Mac. The short version is in the [README](../README.md#install). An AI
+coding assistant can do all of this for you with [AI_SETUP.md](AI_SETUP.md).
-> **On this page:** [What you need](#what-you-need) · [Windows](#windows) · [Linux](#linux) ·
+There are three ways to install it. Take the first one unless you have a reason not to:
+
+1. **[The app](#the-app):** a desktop program with the engine inside. Install it, pick a model, chat.
+2. **[The Terminal](#the-terminal):** one line in a terminal, then `strata launch`. For scripts, servers and AI agents.
+3. **[From source](#from-source-windows):** `START-HERE.bat` on Windows, `./setup.sh` on Linux and Mac. It builds or
+ fetches the engine itself.
+
+> **On this page:** [What you need](#what-you-need) · [The app](#the-app) · [The Terminal](#the-terminal) ·
+> [From source: Windows](#from-source-windows) · [Linux](#from-source-linux) · [Mac](#from-source-mac) ·
> [AMD cards](#amd-cards) · [Several cards](#two-or-three-cards) · [Docker](#docker-linux) ·
> [Older CPUs](#older-cpus-experimental) · [Updating](#updating) · [Where things are stored](#where-things-are-stored) ·
> [Setup's questions](#setups-questions) · [Tuning](#tuning-for-your-pc) · [All options](#options-without-questions)
@@ -16,13 +24,15 @@ this for you with [AI_SETUP.md](AI_SETUP.md).
| **GPU** | **NVIDIA** RTX 20, 30, 40 or 50 series, **12 GB VRAM or more** (8 GB runs, slowly). Measured on an RTX 5070 and an RTX 3090; RTX 20 (Turing, since 0.1.27) was tested by a contributor on an RTX 2070. **AMD** Radeon RX 7900 XT / XTX, RX 9070 / 9070 XT and Radeon AI PRO R9700 (validated), RX 7800 XT / 7700 XT and RX 9060 XT (validated by their owners), RX 6800 / 6900 series (community-reported), with 12 GB of VRAM or more. See [AMD cards](#amd-cards). |
| **RAM** | Enough for the size you pick ([which model](MODELS.md#pick-by-ram)); **64 GB** runs every size. A big GPU makes up for less RAM - the [low-RAM mode](MODELS.md#a-big-graphics-card-and-little-ram). |
| **CPU** | x86-64 with AVX2 (any Intel/AMD desktop CPU from the last ~8 years). AVX-512 (Ryzen 7000/9000) is a bit faster. Older CPUs without AVX2 are experimental and slow: [Older CPUs](#older-cpus-experimental). |
-| **Disk** | ~70-80 GB free for the model, ~6 GB for the MTP layer (+1 GB with images). **Q2_0 on an AVX-512 CPU** also writes a one-time ~40 GB copy of its experts for the fast CPU kernel. On Linux with an AMD card, ROCm takes ~10 GB more when setup installs it. An NVMe SSD is strongly recommended: it makes the first start much faster. |
-| **OS** | Windows 10/11, or Linux (Ubuntu 22.04/24.04 and Debian 12/13 get everything installed automatically). |
+| **Disk** | The app and its engine take about 1-3 GB. ~70-80 GB free for the model, ~6 GB for the MTP layer (+1 GB with images). **Q2_0 on an AVX-512 CPU** also writes a one-time ~40 GB copy of its experts for the fast CPU kernel. On Linux with an AMD card, ROCm takes ~10 GB more when setup installs it. An NVMe SSD is strongly recommended: it makes the first start much faster. |
+| **OS** | Windows 10/11, Linux (Ubuntu 22.04/24.04 and Debian 12/13 get everything installed automatically; the app also runs on Fedora 44), or a Mac with Apple Silicon (M1 or newer) and macOS 15 or newer ([Mac](MAC.md)). |
| **Driver** | **NVIDIA:** a current driver, version 580 or newer ([nvidia.com/drivers](https://www.nvidia.com/drivers) or the NVIDIA App). **AMD:** on Linux the kernel's amdgpu driver (no ROCm install needed); on Windows a current AMD Software: Adrenalin Edition driver ([amd.com/support](https://www.amd.com/en/support)). |
-The driver is the only thing you install yourself. Everything else - Python, the engine, the model - is set up for
-you the first time: Python 3.12 if you have none (for your user account, no admin), a private Python environment in
-`.venv/`, the Strata engine, the model and the MTP draft layer. On NVIDIA it uses the ready-made engine for RTX
+The driver is the only thing you install yourself. With the app, the engine is inside the installer. With the
+Terminal, `strata launch` fetches the engine for your card from the signed update feed, and nothing is compiled.
+Installing from source sets up the rest as well, the first time: Python 3.12 if you have none (for your user
+account, no admin), a private Python environment in `.venv/`, the Strata engine, the model and the MTP draft layer.
+On NVIDIA it uses the ready-made engine for RTX
20/30/40/50 and NVIDIA's CUDA libraries from pip (~0.4 GB); if no ready-made engine fits your PC, it offers to install
the build tools (Visual Studio Build Tools + CUDA Toolkit on Windows, `build-essential` + CUDA on Ubuntu) and compiles
the engine for your GPU (asks first; 20-40 minutes once). More: [details](DETAILS.md#before-you-start).
@@ -32,7 +42,79 @@ engine or image encoder, it uses existing CMake/Ninja tools when available and i
`requirements-build.txt` into `.venv`. An older `.venv` keeps any build helpers it already has; setup does not remove
packages during an update.
-## Windows
+## The app
+
+The app is the easiest way. It has the engine inside, so there is nothing to build and no setup window.
+
+1. Download the installer for your system from the [releases page](https://github.com/Niko1221/Strata/releases/latest).
+ The page says which installer is for which graphics card (NVIDIA or AMD). Take the one that matches yours.
+ - **Windows 10 / 11:** run the `.exe`. It installs for your user and needs no administrator rights. Windows may
+ say "Windows protected your PC", because the installer is not code-signed yet. Choose **More info**, then
+ **Run anyway**.
+ - **Linux:** the `.AppImage`. Make it runnable (right-click, Properties, Permissions, "Allow executing file as
+ program", or `chmod +x`), then double-click it. It needs FUSE 2: `sudo apt install libfuse2` on Ubuntu 22.04,
+ `sudo apt install libfuse2t64` on Ubuntu 24.04 and newer, `sudo dnf install fuse fuse-libs` on Fedora. Without
+ FUSE you can still start it with `--appimage-extract-and-run` after the file name. The NVIDIA build needs
+ driver 580 or newer.
+ - **Mac:** the `.dmg`. Open it, drag Strata onto Applications, then open Strata from there. It needs Apple
+ Silicon and macOS 15 or newer. If macOS says the app is "damaged", the file is fine: see [MAC.md](MAC.md).
+2. Start Strata. The first start is a little slower on Linux and Mac, because the engine inside the app is copied
+ into your home folder.
+3. The first screen reads your PC and recommends a model that fits. Press the install button. The download is
+ big (40-70 GB). If it stops, it continues where it stopped.
+4. Chat. **Closing the window stops Strata and the engine.**
+
+**Update:** Settings, then Updates, then **Check for updates**. The buttons **Update Strata** and **Update engine**
+appear only when something is new. Your models and saved settings are kept. Unload the model first: an engine that
+is running cannot be replaced. **Roll back** in the same place brings the previous engine back.
+
+**Where things are:** the app's settings and chats are in the `.strata` folder in your user folder, the engine
+in `%LOCALAPPDATA%\Strata\components` (Linux and Mac: `~/.local/share/strata/components`), and the models in the
+`Strata` folder in your Documents (you can change the model folder in the app). Removing the app does not remove
+them. **For a bug report:** Settings, Engine, Diagnostics, View logs, **Save logs**.
+
+The app has no picture reader on the Mac yet. Everything else about pictures is in [Setup's questions](#setups-questions).
+
+## The Terminal
+
+The Terminal is the `strata` command. It uses the same engine, models and saved settings as the app, and it is the
+way for scripts, servers and AI agents. Linux (x86-64), Mac (Apple Silicon) and Windows (x64) are supported.
+
+Linux and Mac:
+
+```sh
+curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh | sh
+```
+
+Windows (PowerShell):
+
+```powershell
+irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
+```
+
+The installer finds out which computer this is, downloads the Terminal archive, checks its SHA-256, unpacks it into
+your home folder (`~/.local/share/strata/terminal`, or `%LOCALAPPDATA%\Strata\terminal` on Windows; no
+administrator rights), checks the release's signed update feed with the Terminal it just unpacked (if that check
+fails it removes what it unpacked), and puts `strata` on your PATH. On Windows, open a new terminal window
+afterwards. Then:
+
+```sh
+strata launch # guided screens: model, context, pictures, who may connect, then serve
+strata launch --model q2_0 --yes --no-tui # the same without questions (scripts, agents)
+```
+
+`strata launch` downloads the engine for your graphics card (signed, and test-started before it is used) and the
+model, starts serving on `http://127.0.0.1:8080` and shows the monitor. It never opens a browser unless you add
+`--open`. Its flags and screens: [TERMINAL_LAUNCH.md](TERMINAL_LAUNCH.md).
+
+**Update:** `strata update --check` says what would change, `strata update` installs the new engine (it is test-started
+first and switched only if that works; the previous engine is kept), `strata components rollback` goes back. It
+refuses while a model is being served. **Check your PC and the installed engine:** `strata doctor`.
+
+To read the install script before you run it, download it first (`curl -fsSL <address> -o install.sh`), read it,
+then run `sh install.sh`.
+
+## From source: Windows
1. [Download this project](https://github.com/Niko1221/Strata/archive/refs/heads/main.zip) and unzip it (or
`git clone` it).
@@ -51,7 +133,7 @@ downloaded twice. Close its window to stop the model. `SETUP.bat` (the same as `
another model or changes the settings. Starting Strata from Task Scheduler at logon needs two task settings, or the
start is 24x slower: [Running it at startup](DETAILS.md#running-it-at-startup-task-scheduler).
-## Linux
+## From source: Linux
```bash
./setup.sh
@@ -68,9 +150,16 @@ On Nix/NixOS, you can use the flake:
nix develop -c ./setup.sh
```
+## From source: Mac
+
+On an Apple Silicon Mac with macOS 15 or newer, install Apple's Command Line Tools once (`xcode-select --install`),
+then run `./setup.sh` in the Strata folder. It builds the Mac (Metal) engine on your Mac and asks the same
+questions. The app and the Terminal do not build anything. All of it, and what fits which memory size, is in
+[MAC.md](MAC.md).
+
## AMD cards
-The steps are the same as with NVIDIA: `START-HERE.bat` on Windows, `./setup.sh` on Linux. Setup finds the Radeon
+The app and the Terminal pick the AMD engine for a Radeon card by themselves. From source the steps are the same as with NVIDIA: `START-HERE.bat` on Windows, `./setup.sh` on Linux. Setup finds the Radeon
card and chooses the AMD (HIP) engine by itself on a PC with no NVIDIA card Strata can use; `--backend hip` chooses
it on a PC that has both. Supported cards: RX 7900 XT / XTX (gfx1100), RX 7800 XT / 7700 XT (gfx1101), RX 9060 XT
(gfx1200), RX 9070 / 9070 XT and Radeon AI PRO R9700 (gfx1201), and the RX 6800 / 6900 series (gfx1030). Integrated
@@ -134,7 +223,13 @@ any PC (to try it). Problems and results on real hardware are welcome as GitHub
## Updating
-**`UPDATE.bat`** (Linux: `./update.sh`) updates Strata without starting the model - for when the GPU is busy with
+**The app:** Settings, Updates, **Check for updates**, then **Update Strata** or **Update engine** (see [the app](#the-app)).
+
+**The Terminal:** `strata update --check`, then `strata update` (see [the Terminal](#the-terminal)). Add `--app` to
+download the app's installer too. Models and saved tuning presets are never downloaded, replaced or deleted by an
+update. How the signed update feed works: [packaging/UPDATES.md](../packaging/UPDATES.md).
+
+**From source: `UPDATE.bat`** (Linux: `./update.sh`) updates Strata without starting the model - for when the GPU is busy with
something else, or you just want the new version ready. In a `git clone` it runs `git pull`, then does what
`START-HERE.bat` does before a start: the engine (a new ready-made one when the new version needs it; on Linux a
compiled engine is compiled again when its source changed), the Python packages, and each installed model's settings
@@ -172,15 +267,21 @@ says so and starts the engine you had.
## Setup's questions
+These are the questions of the installer from source (`START-HERE.bat`, `./setup.sh`). The app's first screen and
+`strata launch` ask the same things in their own look.
+
- **Which model and size?** The original, Swift 1.5, the Coder or Unsloth's 4-bit, and Q2_0, IQ2_XS, IQ3_XXS or
IQ3_S - see [which model](MODELS.md).
- **How much context?** How much text it can keep in mind at once (it suggests one for your card). 384K and
512K (experimental) extend the model past its trained 262K by rope scaling - the setup turns it on itself (yarn and a
covering factor; `--rope-scaling`/`--rope-scale` override) ([details](DETAILS.md): "Context extension past
262K").
-- **Images?** Whether it should also read pictures. The image encoder (0.9 GB) runs on the GPU (0.1-0.5 s per
- picture, ~1.4 GB of VRAM kept free for it) or on the CPU (about 3-13 s per picture, nothing on the GPU); with AMD cards
- on the CPU for now. [Details](DETAILS.md#images-vision).
+- **Images?** Whether it should also read pictures. Pictures are on by default for models that can read them. The
+ image encoder (0.9 GB) runs on the CPU when your card has less than 24 GB (about 3-13 s per picture, nothing taken
+ from the GPU) and on the GPU from 24 GB (0.1-0.5 s per picture, ~1.4 GB of VRAM kept free for it). The rule is the
+ same for NVIDIA and AMD cards; `--vision gpu` and `--vision cpu` choose the reader yourself, `--vision no` turns
+ pictures off. On a Radeon AI PRO R9700 the GPU reader took 57 ms for a 260-token picture and the CPU reader 3.1 s.
+ The Mac has no picture reader yet. [Details](DETAILS.md#images-vision).
- **Experimental speed projection?** Off unless you say yes - [read what it does](DETAILS.md#experimental-speed-projection-experimental-off-by-default)
first. It changes how the model answers, and only the original model offers it.
@@ -198,6 +299,8 @@ NVIDIA cards for now. [What it measures](DETAILS.md#double-click-start-herebat).
## Options without questions
+For the Terminal, `strata launch --help` lists every flag; the most used are in [TERMINAL_LAUNCH.md](TERMINAL_LAUNCH.md). These are the options of the installer from source:
+
```
START-HERE.bat --setup install another model, or change context / images
SETUP.bat the same (double-click it)