Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Fix #3425: the memory tiers count RAM + VRAM less 5 GB that Windows and the system keep (setup.py and tools/strata_mcp.py, one constant each; the app and strata launch read the engine's pick; the Mac keeps its own macOS share)

Niko122176f90e58436d
diff --git a/README.md b/README.md
index eca102aa1..138657921 100644
--- a/README.md
+++ b/README.md
@@ -164,6 +164,8 @@ Gyro runs on NVIDIA and AMD cards (AMD is experimental) on Intel Arc cards on Li
### The other sizes
+The tiers count your RAM plus your VRAM, less 5 GB that Windows and the system keep for themselves. Every other size stays a choice.
+
| Your memory (RAM + VRAM) | Take | Notes |
| --- | --- | --- |
| **32 GB** | **Gyro-S**, or the **[Coder](docs/MODELS.md#coder)** for code | The Coder needs 32 GB of RAM and keeps half of the experts, so it is weaker outside code |
diff --git a/docs/MODELS.md b/docs/MODELS.md
index f3b03ba3c..f60f94c89 100644
--- a/docs/MODELS.md
+++ b/docs/MODELS.md
@@ -14,7 +14,7 @@ makers' compressions). It also runs a smaller model, [Qwen3.6-35B-A3B](#qwen36-3
The tiers count RAM and VRAM (the graphics card's memory) together, the same as the [README](../README.md#the-other-sizes).
`strata launch`, setup and the install tool of the MCP server pre-select by them; the desktop app does from 0.2.1. Every
-other size stays a choice. A PC with a card of 12 GB and 32 GB of RAM is in the 40 GB tier (32 + 12 = 44).
+other size stays a choice. Strata leaves 5 GB for Windows and the system, so the sum is RAM + VRAM minus 5 GB: a PC with a card of 12 GB and 32 GB of RAM is in the 40 GB tier (32 + 12 - 5 = 39), and a PC with a 16 GB card and 32 GB of RAM is too, not in the 48 GB tier (#3425).
| Your memory (RAM + VRAM) | Pre-selected | Notes |
| --- | --- | --- |
diff --git a/docs/release-notes-0.2.2.md b/docs/release-notes-0.2.2.md
index ebb290782..9486736b4 100644
--- a/docs/release-notes-0.2.2.md
+++ b/docs/release-notes-0.2.2.md
@@ -26,6 +26,8 @@
- **Problems with the computer's own setup are repaired or explained, not refused.** If the Windows runtime folder lists another account, Strata locks it down itself instead of stopping. A Documents folder that cannot be used (Controlled folder access, an offline OneDrive) falls back to the Strata home folder and Settings > Storage says so. A port that is taken or reserved gets one sentence with the next step. Local requests ignore a system proxy. File-in-use errors and a full disk say what happened. `strata launch` asks plain questions in a terminal that reports a tty late. `strata update` copes with a slow `--version`, a busy graphics card is not held against an engine, pre-release Control builds hand over correctly, and a wrong PC clock is named as the reason a feed or certificate looks bad. install.sh and install.ps1 give clearer messages (ARM, TLS 1.2, proxy credentials, no `tar.exe`). The Terminal parts are already in 0.2.1.3; the Documents fallback and the port handling are app parts.
- **A stale update notice goes away.** After a version change the app and the Terminal no longer keep showing an offer for a version you already have.
- **Idle GPU time was measured, with no default change.** On a two-card test PC decode leaves 47 to 49% of the card time idle (batch 51%, prompts 24 to 42%), on a one-card PC 5%. Overlap gave +0.3, +2.1 and -0.3% against 2.8% noise; pipeline windows 2 gained 5.4% on code only, cost 5 to 13% with the draft layer on the first card, and `--batch-groups 2` cost 15.3% with 4 chats, so nothing is switched on. Raw numbers are in `bench/results/2026-10-20-0.2.1-idle-gpu/`.
+- **The recommended size leaves 5 GB for Windows.** The memory tiers now count RAM + VRAM less 5 GB, so a PC with 32 GB of RAM and a 16 GB card gets the 40 GB tier instead of the 48 GB one, which had it reading part of the model from the SSD (#3425). Every other size is still a choice.
+
## Faster, with what we measured
diff --git a/setup.py b/setup.py
index 0d37d7d74..8358380b9 100644
--- a/setup.py
+++ b/setup.py
@@ -3441,6 +3441,10 @@ def mem_have(model: str, ram: float, vram: float = 0.0) -> float:
# (35 GB) as the other choice. Below 40 GB the older rules stand (Gyro-S, or the Coder for code). The Mac has its own
# rules (strata_control/mac_fit.py). tools/strata_mcp.py's recommend() holds the same floors (the app and `strata launch`).
MEMORY_TIERS = ((62.0, "qwen", "IQ3_S"), (46.0, "qwen", "IQ3_XXS"), (38.0, "swift", "IQ2_XS"))
+# #3425 (owner's rule): the tiers count RAM + VRAM less this much, which Windows and the system keep for themselves. A PC
+# with a 16 GB card and 32 GB of RAM used to land in the 48 GB tier and then read ~11 GB of it from the SSD on every answer.
+# tools/strata_mcp.py has the same constant; the Mac (strata_control/mac_fit.py) keeps its own macOS share and does not use it.
+TIER_SYSTEM_RESERVE_GB = 5.0
def tier_pick(ram: float, vram: float = 0.0):
@@ -3448,7 +3452,7 @@ def tier_pick(ram: float, vram: float = 0.0):
Gyro-S, or the Coder). With less RAM than the size's own figure the low-RAM mode starts by itself and the model is
slower. A recommendation: every size stays a choice."""
for floor, fam, m in MEMORY_TIERS:
- if ram + vram >= floor:
+ if ram + vram - TIER_SYSTEM_RESERVE_GB >= floor:
return fam, m
return None
diff --git a/tools/strata_mcp.py b/tools/strata_mcp.py
index 916bbf037..defaed0aa 100644
--- a/tools/strata_mcp.py
+++ b/tools/strata_mcp.py
@@ -118,6 +118,8 @@ FALLBACK_FAMILIES = {
}
OWN_PATH_ENGINE = (0, 1, 45) # the first engine that runs Qwen3.6-35B-A3B with pictures and parallel requests (setup.py MIN_ENGINE note)
MEMORY_TIERS = ((62.0, "qwen", "IQ3_S"), (46.0, "qwen", "IQ3_XXS"), (38.0, "swift", "IQ2_XS")) # as setup.py's: RAM + VRAM floors of the owner's 64 / 48 / 40 GB tiers
+# #3425 (owner's rule, same as setup.py's TIER_SYSTEM_RESERVE_GB): the floors count RAM + VRAM less the 5 GB Windows and the system keep
+TIER_SYSTEM_RESERVE_GB = 5.0
GYRO_ROOM_GB = 4.0 # recommend() suggests a Gyro size only when VRAM + RAM exceed its memory figure by this much
FALLBACK_CONTEXTS = [8192, 32768, 65536, 131072, 204800, 262144, 393216, 524288]
BENCH_PROMPT = ("Write a short story (about 300 words) about a lighthouse keeper who finds a message in a bottle. "
@@ -1115,10 +1117,10 @@ class Strata:
# 0.2.0, the owner's memory tiers (README "The other sizes"; setup.py MEMORY_TIERS holds the same floors): RAM + VRAM
# together. An APU's shared memory is the RAM itself, so only its BIOS carve-out counts as VRAM.
tvram = float(best.get("dedicated_gb", 0.0)) if best.get("uma") else float(vram)
- tier = next(((f, n) for lo, f, n in MEMORY_TIERS if ram + tvram >= lo), None)
+ tier = next(((f, n) for lo, f, n in MEMORY_TIERS if ram + tvram - TIER_SYSTEM_RESERVE_GB >= lo), None)
if tier and tier[1] in models and tier[0] in families:
fam, model = tier
- total = f"{ram:.0f} GB of RAM + {tvram:.0f} GB of VRAM = {ram + tvram:.0f} GB"
+ total = f"{ram:.0f} GB of RAM + {tvram:.0f} GB of VRAM = {ram + tvram:.0f} GB, {TIER_SYSTEM_RESERVE_GB:.0f} GB of it kept for Windows and the system"
if model == "IQ2_XS":
why = (f"{total}, the 40 GB tier: Swift 1.5's IQ2_XS (thinks for a shorter time)"
+ ("; Gyro-M (35 GB) is the other choice there, close to IQ3_XXS in quality but 16-21% slower at "
diff --git a/tools/test_setup_golden.json b/tools/test_setup_golden.json
index 430fc2992..e4e654c2e 100644
--- a/tools/test_setup_golden.json
+++ b/tools/test_setup_golden.json
@@ -739,11 +739,11 @@
"config": {
"args": [
"--pack",
- "<T>/data/packs/iq3_s",
+ "<T>/data/packs/iq3_xxs",
"--native",
- "<T>/models/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf",
+ "<T>/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
"--ple-gguf",
- "<T>/models/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00002-of-00002.gguf",
+ "<T>/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf",
"--expert-profile",
"<T>/data/expert-profile.bin",
"--expert-cache",
@@ -779,16 +779,16 @@
"lib_dirs": [
"<cuda>"
],
- "log": "<T>/strata-iq3_s.log",
- "model_name": "qwen3.8-flash-next-iq3_s",
+ "log": "<T>/strata-iq3_xxs.log",
+ "model_name": "qwen3.8-flash-next-iq3_xxs",
"port": 8080,
- "tokenizer": "<T>/data/packs/iq3_s/tokenizer",
+ "tokenizer": "<T>/data/packs/iq3_xxs/tokenizer",
"vision": {
"exe": "<T>/engine/strata-vision.exe",
"gpu": false,
"max_tokens": 300,
"mmproj": "<T>/models/mmproj-Qwen3.8-Flash-Next-BF16.gguf",
- "model": "<T>/models/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf",
+ "model": "<T>/models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
"threads": 6
}
}
diff --git a/tools/test_setup_gyro.py b/tools/test_setup_gyro.py
index 7b041b3f2..ae8b40cb7 100644
--- a/tools/test_setup_gyro.py
+++ b/tools/test_setup_gyro.py
@@ -62,10 +62,10 @@ class GyroCatalog(unittest.TestCase):
self.assertEqual(s.recommend(hardware(ram, vram))["family"], family, (ram, vram))
got = s.recommend(hardware(20.0, 12.0)) # 32 GB in total, the Coder's 32 GB of RAM missing
self.assertEqual((got["family"], got["model"]), ("gyro", "Gyro-S"))
- got = s.recommend(hardware(24.0, 16.0)) # 40 GB in total: the owner's 40 GB tier, Swift 1.5's IQ2_XS
+ got = s.recommend(hardware(28.0, 16.0)) # 44 GB in total, 39 after the 5 GB for the system (#3425): the owner's 40 GB tier, Swift 1.5's IQ2_XS
self.assertEqual((got["family"], got["model"]), ("swift", "IQ2_XS"))
self.assertIn("Gyro-M", got["why"]) # named as the other choice (NVIDIA, AMD and Intel Arc)
- intel = hardware(24.0, 16.0)
+ intel = hardware(28.0, 16.0)
intel["gpus"] = [{**g, "vendor": "intel"} for g in intel["gpus"]]
self.assertIn("Gyro-M", s.recommend(intel)["why"]) # the SYCL engine has the trellis kernels (0.2.2)
self.assertIsNone(s.recommend(hardware(12.0, 8.0))["family"])
diff --git a/tools/test_setup_menu.py b/tools/test_setup_menu.py
index d59e9d19c..d6b0e9398 100644
--- a/tools/test_setup_menu.py
+++ b/tools/test_setup_menu.py
@@ -103,20 +103,35 @@ class RecommendedSizes(unittest.TestCase):
for fam in ("qwen", "swift"):
n = self.names(fam)
self.assertIn("IQ3_S", n)
- self.assertEqual(setup.recommended_size(fam, n, 63.7), "IQ3_S")
+ self.assertEqual(setup.recommended_size(fam, n, 67.7), "IQ3_S")
self.assertEqual(setup.recommended_size(fam, n, 191.0), "IQ3_S") # not the largest: IQ3_S stays the pick
self.assertEqual([m for m in n if setup.size_tag(fam, m)], ["IQ3_S"]) # the others are choices, unmarked
def test_without_the_ram_the_largest_that_fits_and_iq3_s_stays_marked(self):
n = self.names("qwen")
self.assertEqual(setup.recommended_size("qwen", n, 61.0), "IQ3_XXS") # IQ3_S needs 62 GB
- self.assertEqual(setup.recommended_size("qwen", n, 50.0), "IQ3_XXS") # 48 GB tier (RAM + VRAM)
- self.assertEqual(setup.recommended_size("qwen", n, 40.0, 8.0), "IQ3_XXS") # 40 + 8 = 48 GB together
- self.assertEqual(setup.recommended_size("qwen", n, 32.0, 12.0), "IQ2_XS") # 44 GB: the 40 GB tier
+ self.assertEqual(setup.recommended_size("qwen", n, 52.0), "IQ3_XXS") # 48 GB tier (RAM + VRAM - 5 GB)
+ self.assertEqual(setup.recommended_size("qwen", n, 40.0, 11.0), "IQ3_XXS") # 40 + 11 - 5 = 46 GB together
+ self.assertEqual(setup.recommended_size("qwen", n, 32.0, 12.0), "IQ2_XS") # 44 - 5 = 39 GB: the 40 GB tier
self.assertEqual(setup.recommended_size("swift", self.names("swift"), 32.0, 12.0), "IQ2_XS")
- self.assertEqual(setup.recommended_size("qwen", n, 32.0, 32.0), "IQ3_S") # 64 GB together
+ self.assertEqual(setup.recommended_size("qwen", n, 32.0, 35.0), "IQ3_S") # 67 - 5 = 62 GB together
self.assertEqual(setup.size_tag("qwen", "IQ3_S"), " (recommended)")
+ def test_3425_the_tiers_leave_5_gb_for_windows(self):
+ # the reporter: RTX 5080 (16 GB) + 32 GB of RAM summed to 48 GB and got IQ3_XXS, which read ~11 GB from the SSD
+ n = self.names("qwen")
+ self.assertEqual(setup.TIER_SYSTEM_RESERVE_GB, 5.0)
+ self.assertEqual(setup.recommended_size("qwen", n, 31.9, 15.9), "IQ2_XS")
+ self.assertEqual(setup.tier_pick(31.9, 15.9), ("swift", "IQ2_XS"))
+ # boundaries: floors 38 / 46 / 62 plus the 5 GB
+ self.assertEqual(setup.tier_pick(43.0), ("swift", "IQ2_XS"))
+ self.assertIsNone(setup.tier_pick(42.9))
+ self.assertEqual(setup.tier_pick(51.0), ("qwen", "IQ3_XXS"))
+ self.assertEqual(setup.tier_pick(50.9), ("swift", "IQ2_XS"))
+ self.assertEqual(setup.tier_pick(67.0), ("qwen", "IQ3_S"))
+ self.assertEqual(setup.tier_pick(66.9), ("qwen", "IQ3_XXS"))
+ self.assertEqual(setup.tier_pick(40.0, 3.0), ("swift", "IQ2_XS")) # VRAM counts the same as RAM
+
def test_coder_has_nothing_marked(self):
n = self.names("coder")
self.assertEqual([setup.size_tag("coder", m) for m in n], [""] * len(n))
diff --git a/tools/test_strata_mcp.py b/tools/test_strata_mcp.py
index 62230b8ef..7716051ca 100644
--- a/tools/test_strata_mcp.py
+++ b/tools/test_strata_mcp.py
@@ -664,15 +664,34 @@ class Helpers(unittest.TestCase):
self.assertEqual(r["context"], 32768)
def test_a_pick_that_reads_from_the_ssd_says_so(self):
- # #3425: 32 GB RAM + 16 GB VRAM is the 48 GB tier; about 11 GB of IQ3_XXS would come from the SSD
+ # #3425: 36 GB RAM + 16 GB VRAM - 5 GB is the 48 GB tier; about 8 GB of IQ3_XXS would come from the SSD
s = M.Strata(HERE.parent)
gpu = [{"index": 0, "name": "x", "vram_gb": 15.9, "usable": True, "vendor": "nvidia"}]
- r = s.recommend({"ram_gb": 31.9, "gpus": gpu, "cpu": {"avx2": True}})
+ r = s.recommend({"ram_gb": 36.0, "gpus": gpu, "cpu": {"avx2": True}})
self.assertEqual((r["family"], r["model"]), ("qwen", "IQ3_XXS"))
self.assertTrue(any("read from the SSD" in n for n in r["notes"]), r["notes"])
roomy = s.recommend({"ram_gb": 64, "gpus": gpu, "cpu": {"avx2": True}})
self.assertFalse(any("read from the SSD" in n for n in roomy["notes"]), roomy["notes"])
+ def test_3425_the_tiers_leave_5_gb_for_windows(self):
+ # the reporter: RTX 5080 (16 GB) + 32 GB of RAM used to sum to 48 GB and get IQ3_XXS; now the 40 GB tier
+ s = M.Strata(HERE.parent)
+ gpu = [{"index": 0, "name": "x", "vram_gb": 15.9, "usable": True, "vendor": "nvidia"}]
+ self.assertEqual(M.TIER_SYSTEM_RESERVE_GB, 5.0)
+ r = s.recommend({"ram_gb": 31.9, "gpus": gpu, "cpu": {"avx2": True}})
+ self.assertEqual((r["family"], r["model"]), ("swift", "IQ2_XS"))
+ # boundary: 43 GB of RAM + VRAM is the lowest 40 GB tier PC, 51 GB the lowest 48 GB tier PC
+ def pick(ram, vram):
+ g = [{**gpu[0], "vram_gb": vram}]
+ r = s.recommend({"ram_gb": ram, "gpus": g, "cpu": {"avx2": True}})
+ return r["family"], r["model"]
+ self.assertEqual(pick(30.0, 13.0), ("swift", "IQ2_XS"))
+ self.assertNotEqual(pick(30.0, 12.9), ("swift", "IQ2_XS"))
+ self.assertEqual(pick(38.0, 13.0), ("qwen", "IQ3_XXS"))
+ self.assertEqual(pick(38.0, 12.9), ("swift", "IQ2_XS"))
+ self.assertEqual(pick(54.0, 13.0), ("qwen", "IQ3_S"))
+ self.assertEqual(pick(54.0, 12.9), ("qwen", "IQ3_XXS"))
+
class PhysicalCores(unittest.TestCase):
"""#3417: the hardware report's cpu.cores is the physical cores; the Work profile takes 80% of it for --pool-workers."""