Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docs: which cards a model uses; Cards step column widths and previews
Niko1221f533c7faa62e
diff --git a/docs/MULTI_GPU.md b/docs/MULTI_GPU.mdindex 30e316d52..e93bba2ed 100644--- a/docs/MULTI_GPU.md+++ b/docs/MULTI_GPU.md@@ -22,15 +22,21 @@ Strata can use it:GPU 1: NVIDIA GeForce GTX 1080 Ti, 11 GB VRAM - not supported - older than the RTX 20 series (compute capability 6.1; Strata needs 7.5 or newer)GPU 2: NVIDIA GeForce RTX 3090, 24 GB VRAM - can be used...- 1) GPU 0 (NVIDIA GeForce RTX 5080, 16 GB) + GPU 2 (NVIDIA GeForce RTX 3090, 24 GB) together (recommended)- 2) GPU 2 (NVIDIA GeForce RTX 3090, 24 GB) only- 3) GPU 0 (NVIDIA GeForce RTX 5080, 16 GB) only+ 1) RTX 3090 + RAM: fits: needs 44 GB of the 70 GB available (recommended)+ 2) RTX 3090 + RTX 5080 + RAM (layer split): fits: needs 44 GB of the 84 GB available+ prompts are read about 20% faster; the cards share the layers (docs/MULTI_GPU.md)+ 3) RTX 5080 + RAM: fits: needs 44 GB of the 66 GB availableWhich GPUs? [1]:```-When two or more cards can share the model, the two best together are recommended (the newest generation first:-it becomes the main card). A model installed on one card asks once, at its next start, whether to use both from-now on; the answer is kept.+**Which cards a model uses (one rule, in setup, `strata launch` and the app).** Strata first tries the model on your+**largest** card plus the computer's memory, the normal hybrid mode, and recommends that when it fits (with some RAM to+spare). Only when it does not fit one card, or would be too tight, the layer split over the two largest cards is+recommended instead. Every option is listed with plain numbers (what the model needs and what the cards and RAM give;+how much would be read from the SSD when it does not fit) and the choice is yours. The cards that count: not an+integrated chip (it shares the system memory and adds nothing beside a real card), at least 4 GB of graphics memory, and+8 GB or more to share a model. The arithmetic is the app's `fitOn` (strata_control/gpu_fit.py keeps its constants equal);+each extra card of a split costs an estimated 2 GB of its own buffers.**Choosing yourself** (at setup or at any start):diff --git a/docs/terminal-previews/launch-monitor-overview.txt b/docs/terminal-previews/launch-monitor-overview.txtindex ecf18c946..bf53d31db 100644--- a/docs/terminal-previews/launch-monitor-overview.txt+++ b/docs/terminal-previews/launch-monitor-overview.txt@@ -19,8 +19,8 @@└────────────────────────┘ └────────────────────────┘ └────────────────────────┘ └────────────────────────┘Context fill 36K / 128K 28% Recent requests############................................ TIME STATUS PROMPT REUSED OUTPUT TOK/S DURATION- Experts in VRAM 1,644 · 3.2 GB 05:09:13 Done 17,183 0 19,511 50.5 429.8 s- ##############.............................. 05:09:09 Done 626 0 11 26.8 4.0 s+ Experts in VRAM 1,644 · 3.2 GB 20:34:50 Done 17,183 0 19,511 50.5 429.8 s+ ##############.............................. 20:34:46 Done 626 0 11 26.8 4.0 sSystem RAM 58.5 / 63 GB#########################################...diff --git a/docs/terminal-previews/launch-overview-ascii.txt b/docs/terminal-previews/launch-overview-ascii.txtindex f747b1b3b..599c7245f 100644--- a/docs/terminal-previews/launch-overview-ascii.txt+++ b/docs/terminal-previews/launch-overview-ascii.txt@@ -18,8 +18,8 @@ Model state Idle Reading [Generating] Queued Error└────────────────────────┘ └────────────────────────┘ └────────────────────────┘ └────────────────────────┘Context fill 36K / 128K 28% Recent requests############................................ TIME STATUS PROMPT REUSED OUTPUT TOK/S DURATION-Experts in VRAM 1,644 · 3.2 GB 05:09:15 Done 17,183 0 19,511 50.5 429.8 s-##############.............................. 05:09:11 Done 626 0 11 26.8 4.0 s+Experts in VRAM 1,644 · 3.2 GB 20:34:53 Done 17,183 0 19,511 50.5 429.8 s+##############.............................. 20:34:49 Done 626 0 11 26.8 4.0 sSystem RAM 58.5 / 63 GB#########################################...diff --git a/docs/terminal-previews/launch-start.txt b/docs/terminal-previews/launch-start.txtindex 8597c8893..bf7713d1b 100644--- a/docs/terminal-previews/launch-start.txt+++ b/docs/terminal-previews/launch-start.txt@@ -1,7 +1,7 @@# strata launch preview: start screen with saved presets# (text of the screen as drawn at the size named in tests/terminal/launch_previews.py)- STRATA Launch RTX 5070 12 GB · 64 GB RAM · 94 GB free+ STRATA Launch RTX 5070 12 GB · 64 GB RAM · 52 GB free────────────────────────────────────────────────────────────────────────────────────────────────────────────Start from a saved preset@@ -31,6 +31,6 @@────────────────────────────────────────────────────────────────────────────────────────────────────────────- This computer: RTX 5070 12 GB · 64 GB RAM · 94 GB free. Preset: model, context, parallel requests, tuned+ This computer: RTX 5070 12 GB · 64 GB RAM · 52 GB free. Preset: model, context, parallel requests, tunedsettings, access. Shared with the Strata app.↑↓ choose Enter launch e edit preset d delete ? help q quitdiff --git a/docs/terminal-previews/launch-step1-model.txt b/docs/terminal-previews/launch-step1-model.txtindex aff17f440..8652b9c24 100644--- a/docs/terminal-previews/launch-step1-model.txt+++ b/docs/terminal-previews/launch-step1-model.txt@@ -3,7 +3,7 @@STRATA Set up Step 1 of 9: Model────────────────────────────────────────────────────────────────────────────────────────────────────────────- Which model? This computer: RTX 5070 12 GB · 64 GB RAM · 94 GB free+ Which model? This computer: RTX 5070 12 GB · 64 GB RAM · 52 GB freeModel Download Needs (RAM + VRAM) On this computerQwen3.8-Flash-Next Q2_0 66 GB 35 GB fitsQwen3.8-Flash-Next IQ2_XS 68 GB 36 GB fitsdiff --git a/docs/terminal-previews/launch-step2-folder.txt b/docs/terminal-previews/launch-step2-folder.txtindex fba6e3e04..91d366a26 100644--- a/docs/terminal-previews/launch-step2-folder.txt+++ b/docs/terminal-previews/launch-step2-folder.txt@@ -5,7 +5,8 @@────────────────────────────────────────────────────────────────────────────────────────────────────────────Where should the model files be stored?Needs about 84 GB (the download plus working room).- Free space in that folder's drive: 94 GB+ Free space in that folder's drive: 52 GB Not enough for 84 GB. You can still go on, but the download is+ likely to stop; pick a bigger drive.┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐│ C:\Users\AI-Server\Documents\Strata │└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘@@ -28,7 +29,6 @@-────────────────────────────────────────────────────────────────────────────────────────────────────────────Models are large files. Strata keeps them in one folder; the default is Strata's own data folder. Pressdiff --git a/docs/terminal-previews/launch-step4-pictures.txt b/docs/terminal-previews/launch-step4-pictures.txtindex 02dd47cbe..a6ad30175 100644--- a/docs/terminal-previews/launch-step4-pictures.txt+++ b/docs/terminal-previews/launch-step4-pictures.txt@@ -5,8 +5,7 @@────────────────────────────────────────────────────────────────────────────────────────────────────────────Should the model read pictures?› No pictures- Yes, read pictures-+ Yes, read pictures (this engine has no picture reader)@@ -33,4 +32,5 @@Pictures add a 0.9 GB download. On a card under 24 GB the picture reader runs on the CPU (a few seconds perpicture, nothing taken from the graphics card). On a bigger card it runs on the GPU and keeps about 1.4 GBfree, costing a few % of text speed. Say No if you only chat in text.+ Heads up: the engine installed here does not carry the picture reader, so the model would answer text only.↑↓ choose Enter next Esc back ? help q quitdiff --git a/docs/terminal-previews/launch-twocards-start.txt b/docs/terminal-previews/launch-twocards-start.txtindex 96b615efd..2f2f35293 100644--- a/docs/terminal-previews/launch-twocards-start.txt+++ b/docs/terminal-previews/launch-twocards-start.txt@@ -1,7 +1,7 @@# strata launch preview: start screen with saved presets# (text of the screen as drawn at the size named in tests/terminal/launch_previews.py)- STRATA Launch NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 94 GB free+ STRATA Launch NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 52 GB free────────────────────────────────────────────────────────────────────────────────────────────────────────────Start from a saved preset@@ -31,6 +31,6 @@────────────────────────────────────────────────────────────────────────────────────────────────────────────- This computer: NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 94 GB free. Preset: model, context, parallel+ This computer: NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 52 GB free. Preset: model, context, parallelrequests, tuned settings, access. Shared with the Strata app.↑↓ choose Enter launch e edit preset d delete ? help q quitdiff --git a/docs/terminal-previews/launch-twocards-step2-cards.txt b/docs/terminal-previews/launch-twocards-step2-cards.txtindex 7ac2beedd..332395e92 100644--- a/docs/terminal-previews/launch-twocards-step2-cards.txt+++ b/docs/terminal-previews/launch-twocards-step2-cards.txt@@ -4,10 +4,9 @@STRATA Set up Step 2 of 10: Cards────────────────────────────────────────────────────────────────────────────────────────────────────────────Which graphics cards should it use?- › Let Strata choose (recommended): 2 cards- Only card 0: NVIDIA RTX A4000, 16 GB- Only card 1: NVIDIA RTX A4000, 16 GB- All of them together, as one+ › RTX A4000 (card 0) + RAM (recommended) fits: needs 44 GB of the 70 GB available+ RTX A4000 (card 0) + RTX A4000 (card 1) + RAM (layer split) fits: needs 44 GB of the 84 GB available+ RTX A4000 (card 1) + RAM fits: needs 44 GB of the 70 GB available@@ -31,6 +30,7 @@────────────────────────────────────────────────────────────────────────────────────────────────────────────- This computer has several graphics cards. One card is simplest; several cards share the model so bigger- models fit. Strata's own choice is a good default.- ↑↓ choose Enter next Esc back ? help q quit+ One card plus your RAM is the normal way and keeps things simple. Two cards share the layers: more room for+ the model and prompts read about 20% faster, but a small second card can slow it down. Strata recommends one+ of them; the choice is yours.+ ↑↓ choose Enter next Esc back ? helpdiff --git a/strata_cli/launch/screens.py b/strata_cli/launch/screens.pyindex 01c4a7caa..510ff56db 100644--- a/strata_cli/launch/screens.py+++ b/strata_cli/launch/screens.py@@ -539,7 +539,8 @@ class CardsScreen(Base):items.append(Item([label, Cell(o["words"], WARN if o["verdict"] in ("spills", "tight") else "")], key=o["cards"]))if cur is None and o["recommended"] or (cur is not None and o["cards"] == list(cur)):sel = i- self.query_one(Menu).columns = [("", 58), ("", 0)]+ wide = max((len(str(i.cells[0])) for i in items), default=40)+ self.query_one(Menu).columns = [("", max(40, min(wide + 3, 70))), ("", 0)]self.query_one(Menu).set_items(items, sel)self.paint()