Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Terminal 0.2.4 / Control and engine 0.1.49: version bump for the DeepSeek V4.1-Flash release (app stays 0.2.3, Mac stays 0.1.48); release notes

Niko12218ea7fef82717
diff --git a/CMakeLists.txt b/CMakeLists.txt
index eadbffa15..c46b305c7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -8,7 +8,7 @@
# below refuses anything older: silently building for a pre-Ampere arch would produce a binary that runs
# and is wrong about its own performance, which is the worst of both outcomes.
cmake_minimum_required(VERSION 3.24)
-project(strata VERSION 0.1.48.1 LANGUAGES CXX)
+project(strata VERSION 0.1.49 LANGUAGES CXX)
# EXPERIMENTAL: the Intel Arc engine, the SYCL port in sycl/ (PR #423; icpx + oneMKL, docs/INTEL_ARC.md). It is its
# own project that takes the unported sources from src/; with this on, nothing below is configured.
option(STRATA_ENABLE_SYCL "EXPERIMENTAL: build the Intel Arc (SYCL) engine from sycl/ - docs/INTEL_ARC.md" OFF)
diff --git a/docs/release-notes-0.2.4.md b/docs/release-notes-0.2.4.md
new file mode 100644
index 000000000..15981e5f5
--- /dev/null
+++ b/docs/release-notes-0.2.4.md
@@ -0,0 +1,74 @@
+# Strata 0.2.4: DeepSeek V4.1-Flash in the Terminal
+
+Strata now runs DeepSeek V4.1-Flash on a PC with one 12 GB graphics card and 46 to 64 GB of RAM. The engine keeps the experts it uses most in RAM and streams the rest from your SSD.
+
+**DeepSeek is in the Terminal (`strata launch`, `strata setup`) in this release. The desktop app is unchanged (it stays 0.2.3) and gets DeepSeek in a later update.**
+
+## What you get
+
+- **DeepSeek V4.1-Flash** in the model list of Setup, in two sizes from IST-DAS:
+
+ | set | download | quality against the original model |
+ |---|---|---|
+ | 2.00 bits | 257 GB | perplexity +16%; code and agent text +8.5% and +5.9% |
+ | 2.25 bits | 274 GB | perplexity +7.7% to +8.3%; code and agent text +3.5% and +2.5% |
+
+ "Perplexity" is how surprised the model is by a fixed set of 4,995 test texts; lower is closer to the original. Pick 2.25 for coding and agent work when you have the disk. Setup shows both sizes and warns if your PC is short of disk or RAM. It does not stop you.
+- Setup downloads the files, builds the model folder and starts the engine. You set no flags.
+
+## Speed on real PCs
+
+Output is the speed of the answer. Prompt is how fast Strata reads what you send it.
+
+| PC | output | prompt |
+|---|---|---|
+| RTX 5070 12 GB, 64 GB RAM | about 5 tokens/s (code) | 231 tokens/s at 8K |
+| RTX 3060 12 GB, 46 GB RAM | 6 to 7.7 tokens/s | about 103 tokens/s at 8K |
+| 4x Radeon AI PRO R9700 (AMD) | about 31 tokens/s | about 360 tokens/s at 8K, 557 at 32K |
+
+These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so expect it to be a little slower. Your drive matters: the 12 GB cards spend most of each token waiting on the SSD and on the CPU, and a slow NVMe drive lowers every number above.
+
+## How it works
+
+- **A RAM expert store that sizes itself.** The model is a mixture of experts: each token uses a few of thousands of small blocks. Strata keeps the busiest ones in RAM, takes whatever RAM is free, and reads the rest from the SSD with direct reads. On the RTX 5070 PC with 64 GB this gave 4.3 to 4.9 tokens/s where memory-mapping the files gave 1.8 to 2.2.
+- **Hot-expert profiles ship with the engine,** so the first prompts do not start cold.
+- **Faster prompts.** The engine now reads long prompts layer by layer in big chunks instead of 16 tokens at a time. A 200-token prompt went from 3.6 to 6.4 tokens/s on the RTX 5070 PC, and 8K prompts reach 231 tokens/s.
+- **DSpark** (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On the RTX 3060 PC (20 threads) it raised decode by 8% on code and 26% on agent text. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There `--dspark auto` does not load the drafter.
+- **Faster CPU rows.** The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.
+- **Cards and engines:** tested on an RTX 3060 (Linux, full model), an RTX 5070 (Windows) and the Radeon AI PRO R9700 (gfx1201, Linux). The RTX 30 kernels (sm_86) were also checked on an RTX A4000. Windows on RTX 30 is built but was not run here. RX 9070 (gfx1200) is built but untested. The engine packages carry `strata-dsv41` next to `strata`.
+
+## Needs
+
+- Disk: 257 GB (2.00) or 274 GB (2.25). The download includes the Engram table and the DSpark drafter.
+- RAM: 46 GB worked on the RTX 3060; 64 GB is the tested comfortable size. More RAM means a bigger store and faster answers.
+- Graphics card: 12 GB or more.
+
+## Known limits
+
+- Prompt speed on 12 GB cards depends on the drive. A slow NVMe drive holds the prompt below the numbers above.
+- DSpark is off on PCs where the CPU is the limit.
+- Not in this release: Mac and Strix Halo. DeepSeek V4.1-Flash does not run there yet.
+- Output on 12 GB cards is 4 to 8 tokens/s. That is usable for chat and slow for long agent runs.
+- The answers can differ from run to run with the same prompt when experts move between the card and the CPU. The cause is which experts sit on the card at that moment.
+
+## Updating
+
+- **From 0.2.3 or 0.2.3.1:** run `strata update` in the Terminal. It fetches Control 0.1.49 and, on Windows NVIDIA, Linux NVIDIA and Linux AMD (Radeon AI PRO R9700 and RX 9070) PCs, the engine package that carries DeepSeek. Other engine packages stay at 0.1.48. The app is not touched.
+- **Mac:** the Mac Terminal and engine stay on 0.1.48. DeepSeek is not on the Mac yet.
+- **New installs:** the install command (`install.ps1` on Windows, `install.sh` on Linux) installs this version.
+
+## Downloads
+
+Everything from 0.2.3 is attached again, so the release page still holds every file. New in this release: `strata-control-0.1.49-windows-x86_64.zip`, `strata-control-0.1.49-linux-x86_64.tar.gz` and the three engine packages `strata-engine-0.1.49-windows-x86_64-cuda-cuda13`, `strata-engine-0.1.49-linux-x86_64-cuda-cuda13` and `strata-engine-0.1.49-linux-x86_64-hip-rx9070`. The installers, AppImages, dmg and the other engine packages are the 0.2.3 files.
+
+## Thanks
+
+- **IST-DAS** for the 2.00 and 2.25-bit sets of DeepSeek V4.1-Flash.
+- **Unsloth** for the DSpark drafter GGUF.
+- **Project Maya (mw00, MIT)**, whose DeepSeek engine this port started from.
+
+---
+
+Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going:
+
+<a href="https://buymeacoffee.com/strataengine"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="50"></a>
diff --git a/strata_cli/__init__.py b/strata_cli/__init__.py
index 4b3ec06d8..8d2b914d8 100644
--- a/strata_cli/__init__.py
+++ b/strata_cli/__init__.py
@@ -6,8 +6,8 @@ from pathlib import Path
SCHEMA_VERSION = 1
#: The version people are told: the Strata app's own (apps/desktop/src-tauri/Cargo.toml `version`; tests/terminal/test_tuiqa_version.py keeps the
-#: two equal, except for a Terminal-only hotfix such as 0.2.3.1, whose app stays 0.2.3). `version()` below is the files' build number, which the engine-compatibility checks compare, so it stays numeric.
-PRODUCT_VERSION = "0.2.3.1"
+#: two equal, except for a Terminal-only release such as 0.2.3.1 or 0.2.4, whose app stays 0.2.3). `version()` below is the files' build number, which the engine-compatibility checks compare, so it stays numeric.
+PRODUCT_VERSION = "0.2.4"
def version() -> str:
diff --git a/tests/terminal/test_tui2_023_old_launcher.py b/tests/terminal/test_tui2_023_old_launcher.py
index 30c75925f..51afb2787 100644
--- a/tests/terminal/test_tui2_023_old_launcher.py
+++ b/tests/terminal/test_tui2_023_old_launcher.py
@@ -42,7 +42,7 @@ class OldLauncherIsToldTheTruth(unittest.TestCase):
for env in ({"STRATA_NO_ACQUIRE": "1", "STRATA_HANDOVER": "1"}, {}):
done = self.run_version(tree, **env)
self.assertEqual(done.returncode, 0, done.stderr)
- self.assertTrue(done.stdout.startswith("strata 0.2.3.1 (build "), done.stdout)
+ self.assertTrue(done.stdout.startswith("strata 0.2.4 (build "), done.stdout)
elsewhere = self.staged_tree(parent="terminal") # not the updater's staging folder
self.assertEqual(self.run_version(elsewhere, STRATA_NO_ACQUIRE="1").returncode, 0)
diff --git a/tests/terminal/test_tui2_023_versions.py b/tests/terminal/test_tui2_023_versions.py
index 730f28c19..74cc2925d 100644
--- a/tests/terminal/test_tui2_023_versions.py
+++ b/tests/terminal/test_tui2_023_versions.py
@@ -20,11 +20,12 @@ def first(path, pattern):
class VersionBump(unittest.TestCase):
- def test_the_terminal_is_0_2_3_1_and_control_is_0_1_48_1_while_the_engine_stays_0_1_48(self):
- # 0.2.3.1 is a Control-only hotfix (#3810): the app stays 0.2.3 and the engine packages stay 0.1.48 (metal/CMakeLists.txt keeps 0.1.48)
- self.assertEqual(strata_cli.PRODUCT_VERSION, "0.2.3.1")
- self.assertEqual(strata_cli.version(), "0.1.48.1") # the CMake project version: Control
- self.assertEqual(first("CMakeLists.txt", r"^project\(strata VERSION ([0-9.]+) "), "0.1.48.1")
+ def test_the_terminal_is_0_2_4_and_control_is_0_1_49_while_the_app_stays_0_2_3(self):
+ # 0.2.4 is a Terminal / Control and engine release (DeepSeek V4.1-Flash): the app stays 0.2.3, the three DeepSeek engine packages are 0.1.49,
+ # the other engine packages stay 0.1.48 (metal/CMakeLists.txt keeps 0.1.48)
+ self.assertEqual(strata_cli.PRODUCT_VERSION, "0.2.4")
+ self.assertEqual(strata_cli.version(), "0.1.49") # the CMake project version: Control
+ self.assertEqual(first("CMakeLists.txt", r"^project\(strata VERSION ([0-9.]+) "), "0.1.49")
self.assertIn('STRATA_VERSION="0.1.48-metal"', (ROOT / "metal" / "CMakeLists.txt").read_text(encoding="utf-8"))
def test_every_place_the_app_version_lives_says_0_2_3(self):
@@ -48,7 +49,7 @@ class VersionBump(unittest.TestCase):
def test_version_prints_the_product_and_the_build(self):
done = subprocess.run([sys.executable, str(ROOT / "strata.py"), "--version"], capture_output=True, text=True, cwd=str(ROOT),
env={**os.environ, "STRATA_NO_HANDOVER": "1"})
- self.assertTrue(done.stdout.startswith("strata 0.2.3.1 (build 0.1.48.1;"), done.stdout)
+ self.assertTrue(done.stdout.startswith("strata 0.2.4 (build 0.1.49;"), done.stdout)
class UpdateFromTheLastRelease(test_updater.UpdaterCase):
diff --git a/tests/terminal/test_tuiqa_version.py b/tests/terminal/test_tuiqa_version.py
index c6e7fb2a1..b2087ff6f 100644
--- a/tests/terminal/test_tuiqa_version.py
+++ b/tests/terminal/test_tuiqa_version.py
@@ -16,13 +16,16 @@ import package_terminal # noqa: E402
class ProductVersion(unittest.TestCase):
- def test_it_is_the_apps_own_version_or_a_terminal_only_hotfix_of_it(self):
+ def test_it_is_the_apps_own_version_or_a_terminal_only_release_after_it(self):
cargo = ROOT / "apps" / "desktop" / "src-tauri" / "Cargo.toml"
if not cargo.is_file():
self.skipTest("no app source in this tree")
m = re.search(r'^version\s*=\s*"([^"]+)"', cargo.read_text(encoding="utf-8"), re.MULTILINE)
# 0.2.3.1 is a Control-only hotfix: the Terminal says 0.2.3.1 while the app (and its updater entry) stay 0.2.3
- self.assertRegex(strata_cli.PRODUCT_VERSION, "^" + re.escape(m.group(1)) + r"(\.[0-9]+)?$")
+ # 0.2.4 is a Terminal release: the app stays 0.2.3 (the Terminal is never older than the app)
+ as_tuple = lambda v: tuple(int(p) for p in v.split("."))
+ self.assertGreaterEqual(as_tuple(strata_cli.PRODUCT_VERSION), as_tuple(m.group(1)))
+ self.assertEqual(as_tuple(strata_cli.PRODUCT_VERSION)[:2], as_tuple(m.group(1))[:2])
def test_dash_dash_version_leads_with_it_and_keeps_the_build_number(self):
out = subprocess.run([sys.executable, str(ROOT / "strata.py"), "--version"], capture_output=True, text=True, check=True).stdout