Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Strix Halo: run the verify window synchronously (DS4_ASYNC_T=4 in the config env), +31% code decode
Niko122163b8a8d3cc6c
diff --git a/README.md b/README.mdindex ddc4126b5..a5baabec3 100644--- a/README.md+++ b/README.md@@ -78,7 +78,7 @@ Measured with the 2.00 bpw set on Strata 0.2.5 (Strix Halo: 0.2.7). Output is th| RTX 3060 12 GB, 46 GB RAM | 9 to 11 tokens/s | about 155 tokens/s (4K) || 1x Radeon AI PRO R9700, 251 GB RAM | 21 to 25 tokens/s | 320 to 360 tokens/s (8K) || 4x Radeon AI PRO R9700 | 74 to 77 tokens/s | 400 to 450 tokens/s (8K) |-| Ryzen AI Max+ 395 (Strix Halo), 128 GB unified | 15 to 17 tokens/s (code), 10 to 11 (chat) | 87 tokens/s (4K), 125 tokens/s (8K) |+| Ryzen AI Max+ 395 (Strix Halo), 128 GB unified | 19 to 21 tokens/s (code), 11 (chat) | 87 tokens/s (4K), 125 tokens/s (8K) |[Memory, disk and details](docs/DEEPSEEK41.md) · [Release notes](https://github.com/Niko1221/Strata/releases)diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.mdindex 1a8d1fc6a..ed49b5473 100644--- a/docs/DEEPSEEK41.md+++ b/docs/DEEPSEEK41.md@@ -42,10 +42,12 @@ perplexity +7.3% (code +3.2%, agent work +2.2%). IQ1_M: perplexity +21% (KL 0.21**On a Ryzen AI Max "Strix Halo" (Windows)** the graphics chip and the CPU share one pool of memory, so the engine counts it once: the128 GB box (a 64 GB BIOS carve-out plus the 64 GB Windows sees) keeps about 109 GB of the 2.00 set's experts in that pool (the denseweights and 12 GB for Windows and the desktop aside) and reads the other 23 GB from the SSD. Measured there (2.00 bpw, default Faster-mode, DSpark auto): answers 15-17 tokens/s on code and 10-11 on chat; a 4,000-token prompt 87 tokens/s, 8,000 tokens 125, 32,000-tokens 177 (setup writes `--prefill-chunk 8192` on this PC: 11-15% faster than the engine's 1,024-token default, decode unchanged).-The tested driver is AMD Adrenalin 26.10.1 (driver 32.0.32015.7029); the 2025-03 driver failed at 32K context, so 26.10.1 is the-minimum to use. The 32K and 64K contexts work on it; an 8-hour 32K soak passed on this box.+mode, DSpark auto): answers 19-21 tokens/s on code (about 18 on agent) and 11 on chat; a 4,000-token prompt 87 tokens/s, 8,000+tokens 125, 32,000 tokens 177 (setup writes `--prefill-chunk 8192` on this PC: 11-15% faster than the engine's 1,024-token default,+decode unchanged). The verify window runs synchronously here (setup writes `DS4_ASYNC_T=4` in the config's env; the engine's+mailbox path is sized for cards that hold most of the experts and costs this box 31% on code - `STRATA_DSV41_ASYNC_T=8` restores+the default). The tested driver is AMD Adrenalin 26.10.1 (driver 32.0.32015.7029); the 2025-03 driver failed at 32K context, so+26.10.1 is the minimum to use. The 32K and 64K contexts work on it; an 8-hour 32K soak passed on this box.121 + 111 + 8 = 239 GB for 1.75; 138 + 111 + 8 = 257 GB for 2.00; 155 + 111 + 8 = 274 GB for 2.25.diff --git a/docs/STRIX_HALO.md b/docs/STRIX_HALO.mdindex a6bdb6893..e37db19fc 100644--- a/docs/STRIX_HALO.md+++ b/docs/STRIX_HALO.md@@ -49,13 +49,21 @@ Measured on the maintainers' box (Ryzen AI Max+ 395, 128 GB unified, Windows 11,| Workload | Result || --- | --- |-| Code answer | 15-17 tokens/s |-| Chat answer | 10-11 tokens/s |+| Code answer | 19-21 tokens/s |+| Agent answer | about 18 tokens/s |+| Chat answer | 11 tokens/s || 4K prompt | about 87 tokens/s || 8K prompt | about 125 tokens/s || 32K prompt | about 177 tokens/s || Stability | 100-request, 1-hour (243 requests) and 8-hour 32K (646 requests) soaks passed on the engine line this package is built from |+The verify window runs synchronously on this PC: setup writes `DS4_ASYNC_T=4` in the config's env (`STRATA_DSV41_ASYNC_T=8`+restores the engine's default, any other value overrides). The engine's mailbox ("async") path is sized for cards that hold most+of the experts and is slower here: measured 2026-10-25 (2.00 bpw, greedy, 256 tokens): code 15.87-16.33 → 20.14-21.28 tok/s+(+31%), agent with the same accepted tokens a round 15.93 → 18.48 (+16%), chat 10.33 → 11.30 (+9%). The same bits come out; only+the scheduling changes, and the drafter's own single-token windows stay async. The 2.25 set runs without the drafter on this box+(its experts leave no room for it) and does not change.+Prompts are read in 8,192-token chunks: setup writes `--prefill-chunk 8192` on this PC (STRATA_DSV41_NO_BIG_CHUNK=1 leaves theengine's 1,024-token default). Measured 2026-10-25: 4K 78.7 → 87.2, 8K 108.9 → 125.1, 32K 155.2 → 176.9 tok/s; decode unchanged(16.53 → 16.59), because the chunk buffers take ~5 GB from the GPU expert tier and the RAM store's hit rate rises to compensate. Thediff --git a/setup.py b/setup.pyindex fe1e6204b..29cb362d9 100644--- a/setup.py+++ b/setup.py@@ -1,4 +1,4 @@-#!/usr/bin/env python3+#!/usr/bin/env python3"""Strata one-click setup and start (Windows and Linux, NVIDIA or AMD GPUs).START-HERE.bat (Windows) / ./setup.sh (Linux) - they install Python if needed and run this file@@ -8036,6 +8036,21 @@ def dsv41_prefill_chunk_args(cards, ram_gb) -> list:return ["--prefill-chunk", "8192"]+def dsv41_config_env(cards) -> dict:+ """Engine env settings for the DeepSeek engine on a unified-memory APU (Ryzen AI Max, gfx1151).++ DS4_ASYNC_T=4: the engine's windowed decode uses the mailbox ("async") path for windows up to this size, 8 by default. On the+ 128 GB Strix Halo the async path is slower than the synchronous one for the 6-row verify window: measured 2026-10-25 (2.00+ bpw, engine 0.1.51, 256-token code set, greedy): 15.87-16.33 tok/s with the default against 20.14-21.28 with 4 (+31%); the+ agent set with the same accepted tokens a round: 15.93 against 18.48 (+16%). The same bits come out - only the scheduling+ changes - and the drafter's own single-token windows (T=1) stay on the async path. A discrete card keeps the engine's+ default; STRATA_DSV41_ASYNC_T=n overrides the value, STRATA_DSV41_ASYNC_T=8 restores the default."""+ if not cards or not any(c.get("uma") for c in cards):+ return {}+ value = os.environ.get("STRATA_DSV41_ASYNC_T") or "4"+ return {} if value == "8" else {"DS4_ASYNC_T": value}++def dsv41_engine_env(lib_dirs) -> dict:env = dict(os.environ)dirs = [str(d) for d in lib_dirs or [] if d]@@ -8267,9 +8282,11 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,args = ["--pack", str(pack), "--max-context", str(ctx), "--eos", "1", "--dspark", "auto",*(["--dspark-conf", "0.65"] if md.get("recipe") == "ista" else []), *dsv41_prefill_chunk_args(cards, ram),*plan["args"]]+ env = dsv41_config_env(cards)cfg = {"exe": str(exe), "args": args, "cwd": str(ROOT), "tokenizer": str(pack / "tokenizer"),"model_name": fam["name"], "arch": fam["arch"], "log": str(home() / f"strata-{tag.lower()}.log"),- "lib_dirs": lib_dirs, "port": port, "engine_ready_s": 3600} # 300+ GB read from the files: a slow disk needs its time+ "lib_dirs": lib_dirs, "port": port, "engine_ready_s": 3600, # 300+ GB read from the files: a slow disk needs its time+ **({"env": env} if env else {})}if cuda_tk == 12:cfg["cuda"] = 12if hip:diff --git a/tests/terminal/test_dsv41_async_t.py b/tests/terminal/test_dsv41_async_t.pynew file mode 100644index 000000000..75c6cd57e--- /dev/null+++ b/tests/terminal/test_dsv41_async_t.py@@ -0,0 +1,40 @@+"""setup.dsv41_config_env: the engine env Strata writes for a unified-memory Ryzen AI Max (Strix Halo) DeepSeek install.++Measured 2026-10-25 on the 128 GB box (2.00 bpw, engine 0.1.51, greedy, 256-token sets): the engine's async window path is slower+than the synchronous one for the 6-row verify window - code 15.87-16.33 tok/s -> 20.14-21.28 (+31%), agent with the same tokens+a round 15.93 -> 18.48 (+16%). DS4_ASYNC_T=4 keeps the drafter's T=1 windows async. A discrete card keeps the engine default.+"""+from __future__ import annotations++import os+import unittest+from unittest import mock++import setup+++HALO = [{"name": "AMD Radeon 8060S", "uma": True, "vram_gb": 96.0, "dedicated_gb": 64.0, "shared_gb": 32.0}]+R9700 = [{"name": "Radeon AI PRO R9700", "vram_gb": 32.0}]+++class StrixAsyncEnv(unittest.TestCase):+ def test_a_unified_pc_gets_the_synchronous_verify_window(self):+ self.assertEqual(setup.dsv41_config_env(HALO), {"DS4_ASYNC_T": "4"})++ def test_a_discrete_card_keeps_the_engine_default(self):+ self.assertEqual(setup.dsv41_config_env(R9700), {})++ def test_no_card_is_untouched(self):+ self.assertEqual(setup.dsv41_config_env([]), {})++ def test_the_value_can_be_overridden(self):+ with mock.patch.dict(os.environ, {"STRATA_DSV41_ASYNC_T": "5"}):+ self.assertEqual(setup.dsv41_config_env(HALO), {"DS4_ASYNC_T": "5"})++ def test_eight_restores_the_engine_default(self):+ with mock.patch.dict(os.environ, {"STRATA_DSV41_ASYNC_T": "8"}):+ self.assertEqual(setup.dsv41_config_env(HALO), {})+++if __name__ == "__main__":+ unittest.main()