Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: which cards a model uses; Cards step column widths and previews

Niko1221f533c7faa62e
diff --git a/docs/MULTI_GPU.md b/docs/MULTI_GPU.md
index 30e316d52..e93bba2ed 100644
--- a/docs/MULTI_GPU.md
+++ b/docs/MULTI_GPU.md
@@ -22,15 +22,21 @@ Strata can use it:
GPU 1: NVIDIA GeForce GTX 1080 Ti, 11 GB VRAM - not supported - older than the RTX 20 series (compute capability 6.1; Strata needs 7.5 or newer)
GPU 2: NVIDIA GeForce RTX 3090, 24 GB VRAM - can be used
...
- 1) GPU 0 (NVIDIA GeForce RTX 5080, 16 GB) + GPU 2 (NVIDIA GeForce RTX 3090, 24 GB) together (recommended)
- 2) GPU 2 (NVIDIA GeForce RTX 3090, 24 GB) only
- 3) GPU 0 (NVIDIA GeForce RTX 5080, 16 GB) only
+ 1) RTX 3090 + RAM: fits: needs 44 GB of the 70 GB available (recommended)
+ 2) RTX 3090 + RTX 5080 + RAM (layer split): fits: needs 44 GB of the 84 GB available
+ prompts are read about 20% faster; the cards share the layers (docs/MULTI_GPU.md)
+ 3) RTX 5080 + RAM: fits: needs 44 GB of the 66 GB available
Which GPUs? [1]:
```
-When two or more cards can share the model, the two best together are recommended (the newest generation first:
-it becomes the main card). A model installed on one card asks once, at its next start, whether to use both from
-now on; the answer is kept.
+**Which cards a model uses (one rule, in setup, `strata launch` and the app).** Strata first tries the model on your
+**largest** card plus the computer's memory, the normal hybrid mode, and recommends that when it fits (with some RAM to
+spare). Only when it does not fit one card, or would be too tight, the layer split over the two largest cards is
+recommended instead. Every option is listed with plain numbers (what the model needs and what the cards and RAM give;
+how much would be read from the SSD when it does not fit) and the choice is yours. The cards that count: not an
+integrated chip (it shares the system memory and adds nothing beside a real card), at least 4 GB of graphics memory, and
+8 GB or more to share a model. The arithmetic is the app's `fitOn` (strata_control/gpu_fit.py keeps its constants equal);
+each extra card of a split costs an estimated 2 GB of its own buffers.
**Choosing yourself** (at setup or at any start):
diff --git a/docs/terminal-previews/launch-monitor-overview.txt b/docs/terminal-previews/launch-monitor-overview.txt
index ecf18c946..bf53d31db 100644
--- a/docs/terminal-previews/launch-monitor-overview.txt
+++ b/docs/terminal-previews/launch-monitor-overview.txt
@@ -19,8 +19,8 @@
└────────────────────────┘ └────────────────────────┘ └────────────────────────┘ └────────────────────────┘
Context fill 36K / 128K 28% Recent requests
############................................ TIME STATUS PROMPT REUSED OUTPUT TOK/S DURATION
- Experts in VRAM 1,644 · 3.2 GB 05:09:13 Done 17,183 0 19,511 50.5 429.8 s
- ##############.............................. 05:09:09 Done 626 0 11 26.8 4.0 s
+ Experts in VRAM 1,644 · 3.2 GB 20:34:50 Done 17,183 0 19,511 50.5 429.8 s
+ ##############.............................. 20:34:46 Done 626 0 11 26.8 4.0 s
System RAM 58.5 / 63 GB
#########################################...
diff --git a/docs/terminal-previews/launch-overview-ascii.txt b/docs/terminal-previews/launch-overview-ascii.txt
index f747b1b3b..599c7245f 100644
--- a/docs/terminal-previews/launch-overview-ascii.txt
+++ b/docs/terminal-previews/launch-overview-ascii.txt
@@ -18,8 +18,8 @@ Model state Idle Reading [Generating] Queued Error
└────────────────────────┘ └────────────────────────┘ └────────────────────────┘ └────────────────────────┘
Context fill 36K / 128K 28% Recent requests
############................................ TIME STATUS PROMPT REUSED OUTPUT TOK/S DURATION
-Experts in VRAM 1,644 · 3.2 GB 05:09:15 Done 17,183 0 19,511 50.5 429.8 s
-##############.............................. 05:09:11 Done 626 0 11 26.8 4.0 s
+Experts in VRAM 1,644 · 3.2 GB 20:34:53 Done 17,183 0 19,511 50.5 429.8 s
+##############.............................. 20:34:49 Done 626 0 11 26.8 4.0 s
System RAM 58.5 / 63 GB
#########################################...
diff --git a/docs/terminal-previews/launch-start.txt b/docs/terminal-previews/launch-start.txt
index 8597c8893..bf7713d1b 100644
--- a/docs/terminal-previews/launch-start.txt
+++ b/docs/terminal-previews/launch-start.txt
@@ -1,7 +1,7 @@
# strata launch preview: start screen with saved presets
# (text of the screen as drawn at the size named in tests/terminal/launch_previews.py)
- STRATA Launch RTX 5070 12 GB · 64 GB RAM · 94 GB free
+ STRATA Launch RTX 5070 12 GB · 64 GB RAM · 52 GB free
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Start from a saved preset
@@ -31,6 +31,6 @@
────────────────────────────────────────────────────────────────────────────────────────────────────────────
- This computer: RTX 5070 12 GB · 64 GB RAM · 94 GB free. Preset: model, context, parallel requests, tuned
+ This computer: RTX 5070 12 GB · 64 GB RAM · 52 GB free. Preset: model, context, parallel requests, tuned
settings, access. Shared with the Strata app.
↑↓ choose Enter launch e edit preset d delete ? help q quit
diff --git a/docs/terminal-previews/launch-step1-model.txt b/docs/terminal-previews/launch-step1-model.txt
index aff17f440..8652b9c24 100644
--- a/docs/terminal-previews/launch-step1-model.txt
+++ b/docs/terminal-previews/launch-step1-model.txt
@@ -3,7 +3,7 @@
STRATA Set up Step 1 of 9: Model
────────────────────────────────────────────────────────────────────────────────────────────────────────────
- Which model? This computer: RTX 5070 12 GB · 64 GB RAM · 94 GB free
+ Which model? This computer: RTX 5070 12 GB · 64 GB RAM · 52 GB free
Model Download Needs (RAM + VRAM) On this computer
Qwen3.8-Flash-Next Q2_0 66 GB 35 GB fits
Qwen3.8-Flash-Next IQ2_XS 68 GB 36 GB fits
diff --git a/docs/terminal-previews/launch-step2-folder.txt b/docs/terminal-previews/launch-step2-folder.txt
index fba6e3e04..91d366a26 100644
--- a/docs/terminal-previews/launch-step2-folder.txt
+++ b/docs/terminal-previews/launch-step2-folder.txt
@@ -5,7 +5,8 @@
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Where should the model files be stored?
Needs about 84 GB (the download plus working room).
- Free space in that folder's drive: 94 GB
+ Free space in that folder's drive: 52 GB Not enough for 84 GB. You can still go on, but the download is
+ likely to stop; pick a bigger drive.
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ C:\Users\AI-Server\Documents\Strata │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘
@@ -28,7 +29,6 @@
-
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Models are large files. Strata keeps them in one folder; the default is Strata's own data folder. Press
diff --git a/docs/terminal-previews/launch-step4-pictures.txt b/docs/terminal-previews/launch-step4-pictures.txt
index 02dd47cbe..a6ad30175 100644
--- a/docs/terminal-previews/launch-step4-pictures.txt
+++ b/docs/terminal-previews/launch-step4-pictures.txt
@@ -5,8 +5,7 @@
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Should the model read pictures?
› No pictures
- Yes, read pictures
-
+ Yes, read pictures (this engine has no picture reader)
@@ -33,4 +32,5 @@
Pictures add a 0.9 GB download. On a card under 24 GB the picture reader runs on the CPU (a few seconds per
picture, nothing taken from the graphics card). On a bigger card it runs on the GPU and keeps about 1.4 GB
free, costing a few % of text speed. Say No if you only chat in text.
+ Heads up: the engine installed here does not carry the picture reader, so the model would answer text only.
↑↓ choose Enter next Esc back ? help q quit
diff --git a/docs/terminal-previews/launch-twocards-start.txt b/docs/terminal-previews/launch-twocards-start.txt
index 96b615efd..2f2f35293 100644
--- a/docs/terminal-previews/launch-twocards-start.txt
+++ b/docs/terminal-previews/launch-twocards-start.txt
@@ -1,7 +1,7 @@
# strata launch preview: start screen with saved presets
# (text of the screen as drawn at the size named in tests/terminal/launch_previews.py)
- STRATA Launch NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 94 GB free
+ STRATA Launch NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 52 GB free
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Start from a saved preset
@@ -31,6 +31,6 @@
────────────────────────────────────────────────────────────────────────────────────────────────────────────
- This computer: NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 94 GB free. Preset: model, context, parallel
+ This computer: NVIDIA RTX A4000 16 GB x2 · 64 GB RAM · 52 GB free. Preset: model, context, parallel
requests, tuned settings, access. Shared with the Strata app.
↑↓ choose Enter launch e edit preset d delete ? help q quit
diff --git a/docs/terminal-previews/launch-twocards-step2-cards.txt b/docs/terminal-previews/launch-twocards-step2-cards.txt
index 7ac2beedd..332395e92 100644
--- a/docs/terminal-previews/launch-twocards-step2-cards.txt
+++ b/docs/terminal-previews/launch-twocards-step2-cards.txt
@@ -4,10 +4,9 @@
STRATA Set up Step 2 of 10: Cards
────────────────────────────────────────────────────────────────────────────────────────────────────────────
Which graphics cards should it use?
- › Let Strata choose (recommended): 2 cards
- Only card 0: NVIDIA RTX A4000, 16 GB
- Only card 1: NVIDIA RTX A4000, 16 GB
- All of them together, as one
+ › RTX A4000 (card 0) + RAM (recommended) fits: needs 44 GB of the 70 GB available
+ RTX A4000 (card 0) + RTX A4000 (card 1) + RAM (layer split) fits: needs 44 GB of the 84 GB available
+ RTX A4000 (card 1) + RAM fits: needs 44 GB of the 70 GB available
@@ -31,6 +30,7 @@
────────────────────────────────────────────────────────────────────────────────────────────────────────────
- This computer has several graphics cards. One card is simplest; several cards share the model so bigger
- models fit. Strata's own choice is a good default.
- ↑↓ choose Enter next Esc back ? help q quit
+ One card plus your RAM is the normal way and keeps things simple. Two cards share the layers: more room for
+ the model and prompts read about 20% faster, but a small second card can slow it down. Strata recommends one
+ of them; the choice is yours.
+ ↑↓ choose Enter next Esc back ? help
diff --git a/strata_cli/launch/screens.py b/strata_cli/launch/screens.py
index 01c4a7caa..510ff56db 100644
--- a/strata_cli/launch/screens.py
+++ b/strata_cli/launch/screens.py
@@ -539,7 +539,8 @@ class CardsScreen(Base):
items.append(Item([label, Cell(o["words"], WARN if o["verdict"] in ("spills", "tight") else "")], key=o["cards"]))
if cur is None and o["recommended"] or (cur is not None and o["cards"] == list(cur)):
sel = i
- self.query_one(Menu).columns = [("", 58), ("", 0)]
+ wide = max((len(str(i.cells[0])) for i in items), default=40)
+ self.query_one(Menu).columns = [("", max(40, min(wide + 3, 70))), ("", 0)]
self.query_one(Menu).set_items(items, sel)
self.paint()