Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

DeepSeek on a Mac: drafter off in the config and the plan, per-set measured lines, plan words for a Mac that holds every expert

Niko1221732e4bfb6066
diff --git a/docs/DEEPSEEK_MAC.md b/docs/DEEPSEEK_MAC.md
new file mode 100644
index 000000000..6d140ea67
--- /dev/null
+++ b/docs/DEEPSEEK_MAC.md
@@ -0,0 +1,82 @@
+# DeepSeek-V4.1-Flash on a Mac
+
+DeepSeek-V4.1-Flash runs on Apple Silicon Macs through its own Metal engine, `strata-dsv41`. It is a separate download
+from the Qwen engine: the app and `strata launch` fetch it the first time you pick DeepSeek. This page says what it
+needs, what it does on the machine it was measured on, and what it cannot do yet.
+
+> **Short version:** a Mac with 64 GB of memory or more, macOS 15 or newer, and about 260 GB of free disk space on a fast
+> drive. Text only (no pictures). The model does not fit in memory, so part of it is read from the SSD for every word.
+> On an M5 Max with 128 GB, DeepSeek 2.00 answers at 15-19 tokens a second and reads a 3,000-token prompt in under 8 seconds.
+
+## What it needs
+
+| | |
+|---|---|
+| Mac | Apple Silicon (M1 or newer), macOS 15 or newer. Measured on an M5 Max with 128 GB. |
+| Memory | 64 GB or more is a good fit. Less still starts, but most of the model is read from the SSD for every word, so it is much slower; setup says so and asks before it goes on. |
+| Disk | 2.00 set: 257 GB. 2.25 set: 274 GB (the 111 GB n-gram table is shared). Put the model on the internal SSD or an NVMe drive, not a USB or network drive. |
+| Pictures | Off. DeepSeek's picture reader has not been run on the Metal engine, so a Mac reads text only (Qwen on the Mac keeps its pictures). |
+
+## Install it
+
+In the app: Models, DeepSeek-V4.1-Flash, pick 2.00 bpw (the default) or 2.25 bpw, Download and prepare.
+
+In the Terminal:
+
+```sh
+strata launch --family deepseek41 --model ist-das-2.00bpw --plan # what it would do on this Mac
+strata launch --family deepseek41 --model ist-das-2.00bpw
+```
+
+Setup downloads the set from Hugging Face (pinned revision, size and SHA-256 checked, resumable), builds the pack (a few
+MB), downloads the DeepSeek engine package for macOS from the signed update feed, and runs a 16-word test question
+before it starts. Files you already have: `--gguf-dir DIR`.
+
+## What it does (M5 Max, 128 GB, 2026-10-25 and -26)
+
+| | 2.00 bpw | 2.25 bpw |
+|---|---|---|
+| Answer speed, code prompt | 19.2 tokens/s | 14.4-15.8 |
+| Answer speed, agent prompt | 19.7 | 12.4-12.6 |
+| 3K prompt | 362-377 tokens/s | 380-427 |
+| 32K prompt | 303 | 306-404 |
+| Read from the SSD per word | 40 MB | 93 MB |
+| Quality against the original model (KL, 4,995 positions) | 0.1709 | 0.1102 |
+
+The 2.25 set is the better of the two (about 7% above the original model's perplexity, against about 15%), the 2.00 set is
+faster and smaller. Both numbers come from the engine's own scoring run with a fixed expert placement
+(`--profile-readonly`); a run that learns the placement as it goes moves the last digits.
+
+How the memory is used: the engine keeps about 71 GB of the experts in memory on a 128 GB Mac (the rest of the memory is
+left to macOS and its file cache; a Mac with less memory keeps less) and reads the remaining experts from the SSD in place.
+A decode token is limited by those SSD reads and by a CPU lane that computes the experts the GPU does not hold, not by the
+GPU.
+
+## Speed setting
+
+DeepSeek's Speed choice (Faster = 5 experts a word, Full quality = 6) works as on a PC. On a Mac the engine's own default
+is Full quality: that is what was measured and gated.
+
+## Limits
+
+- Text only. A picture sent to a DeepSeek model on a Mac is refused.
+- The drafter (DSpark) is off: it was exact but not faster on the Mac.
+- Longer than 32K of context was not measured on a Mac.
+- Smaller Macs: nothing under 128 GB was run with DeepSeek. The plan screen shows how much of the model your Mac keeps in
+ memory and warns when it is under a quarter of it; that estimate is the engine's own rule, not a measurement.
+- The 1.75 bpw set shares the 2.00 set's kernel but was not run on a Mac, so it is not offered there.
+
+## Building and packaging the engine (developers)
+
+```sh
+bash packaging/mac/build_dsv41_engine.sh # configure, build, stage, package and verify the macOS package
+```
+
+It needs the Xcode Command Line Tools, cmake and ninja (a pip venv is enough), Apple's metal-cpp and the llama.cpp tree that
+setup pins (`METAL_CPP_DIR`, `STRATA_GGML_DIR`). The result is `pkg/engine/<version>-macos-arm64-metal/` with
+`strata-dsv41`, the Metal sources it compiles at start-up and `engine-component.json`; the package is unsigned, the feed key
+signs it (`packaging/UPDATES.md`). The Qwen Metal engine is not rebuilt by this: the DeepSeek runtime rules sit behind
+`STRATA_DS4_RUNTIME`, compiled only into `strata_ds4`.
+
+Diagnostics, all off by default: `DS4_CPU_LOG=1` (per-token phase times), `DS4_PROF=1` (per-kernel GPU time),
+`DS4_GPU_TOTAL=1`, `DS4_ENC_TIMES=1`, `DS4_TLOG=1`, `DS4_TRACE_DISP=1`.
diff --git a/setup.py b/setup.py
index e1cf7f10f..df935128d 100644
--- a/setup.py
+++ b/setup.py
@@ -8429,8 +8429,10 @@ def install_dsv41(a, family, gpu, hip, amd, cuda_tk, multi, port, ram, adopted,
# --dspark-conf 0.65: the drafts' confidence floor (H100 replay, DSPARK-DEFAULTS.md: within 0.01-0.02 of the per-class best,
# never below 0.3). The engine has no flag for the draft block (3 in the same table): it reads the block size from the pack,
# so only the floor is written.
- args = ["--pack", str(pack), "--max-context", str(ctx), "--eos", "1", "--vision", ds_vision, "--dspark", "auto",
- *(["--dspark-conf", "0.65"] if md.get("recipe") == "ista" else []), *dsv41_prefill_chunk_args(cards, ram),
+ # A Mac (0.2.7.1): the drafter is not loaded at all (--dspark none). It was exact there but not faster (15.1 and 11.0 tokens/s against 15.3
+ # without it on the M5 Max), and the engine's "auto" would load it on a 128 GB Mac: its 7 GB of experts take memory the trunk's experts need.
+ args = ["--pack", str(pack), "--max-context", str(ctx), "--eos", "1", "--vision", ds_vision, "--dspark", "none" if mac else "auto",
+ *(["--dspark-conf", "0.65"] if md.get("recipe") == "ista" and not mac else []), *dsv41_prefill_chunk_args(cards, ram),
*plan["args"]]
env = dsv41_config_env(cards)
cfg = {"exe": str(exe), "args": args, "cwd": str(ROOT), "tokenizer": str(pack / "tokenizer"),
diff --git a/strata_cli/launch/runner.py b/strata_cli/launch/runner.py
index 3e8d0cf1e..cd323caa6 100644
--- a/strata_cli/launch/runner.py
+++ b/strata_cli/launch/runner.py
@@ -2070,6 +2070,8 @@ class Runner:
"""The drafting line of --plan: what drafts, or why not what was asked for. Nothing is changed or downloaded."""
from strata_control import drafting
if self.multi_file(): # DeepSeek drafts with its own DSpark drafter (the --dspark auto the config writes), not MTP
+ if sys.platform == "darwin":
+ return "Drafting: off (DeepSeek's drafter was not faster on a Mac)"
return "Drafting: DSpark (used when it speeds up answers on this computer)"
try:
base, cfg = self.installed_state()
diff --git a/strata_control/dsv41_fit.py b/strata_control/dsv41_fit.py
index e04ce0cd9..06d6611a0 100644
--- a/strata_control/dsv41_fit.py
+++ b/strata_control/dsv41_fit.py
@@ -503,6 +503,14 @@ MAC_MEASURED = ("M5 Max, 128 GB, 2.25 bpw, 2026-10-25: 14.4-15.8 tokens/s for an
"experts resident and read 93.4 MB a token from the SSD")
+# What each pack did on that Mac (the owner's measurements, 2026-10-25/26; the release build's numbers replace them in MAC_MEASURED_PACK).
+MAC_MEASURED_PACK = {
+ "IQ2_XXS": ("M5 Max, 128 GB, 2.00 bpw: 19-20 tokens/s for a short answer (14-17 after a long prompt), a 3,000-token prompt at 362-377 "
+ "tokens/s, 32,000 at 303; the engine kept 71.4 GB of the experts resident and read 40 MB a token from the SSD"),
+ "IQ2_XS": MAC_MEASURED,
+}
+
+
def mac_expert_cap_gb(ram_gb: float) -> float:
"""The expert budget the engine will pick on a Mac of `ram_gb` (GiB, as the rest of the app reports memory), in
decimal GB: the RAM less what macOS keeps, less the process' own footprint. Mirrors metal_expert_cap() with its
@@ -543,7 +551,9 @@ def mac_plan(ram_gb: float, quant: str = "IQ2_XS", disk_free_gb: float | None =
lines, warns = out["lines"], out["warnings"]
lines.append(f"DeepSeek-V4.1-Flash {pack['name']}: {routed:.0f} GB of routed experts and {dense:.1f} GB of dense "
"weights, plus a 111 GB Engram table that stays on the SSD and is read a few rows for each word.")
- if ram_gb and ram_gb >= 96:
+ if ram_gb and ram_gb >= 96 and streaming <= 0.5:
+ lines.append(f"This Mac ({ram_gb:.0f} GB): the engine will keep all {routed:.0f} GB of the experts in memory.")
+ elif ram_gb and ram_gb >= 96:
lines.append(f"This Mac ({ram_gb:.0f} GB): the engine will keep about {resident:.0f} GB of the experts in memory "
f"and read the other {streaming:.0f} GB from the SSD while it answers.")
elif ram_gb:
@@ -617,7 +627,8 @@ def _plan_mac(key: str, model: str, card: dict, ram_gb: float | None, disk_free_
lines.append(f"DeepSeek-V4.1-Flash, {v['title']}: a {download:.0f} GB download ({v['set_bytes'] / GB:.0f} GB of model, "
f"{ISTA_ENGRAM_BYTES / GB:.0f} GB n-gram table, {ISTA_DSPARK_BYTES / GB:.0f} GB drafter). Quality: {v['quality']}.")
lines.extend(l for l in mp["lines"][1:] if not l.startswith("Measured on "))
- lines.append("Measured on " + MAC_MEASURED + ".")
+ lines.append("Measured on " + MAC_MEASURED_PACK[MAC_SET_PACK[key]] + ".")
+ out["measured"] = MAC_MEASURED_PACK[MAC_SET_PACK[key]]
warns.extend(mp["warnings"])
if ram and ram < MAC_SMALL_RAM_GB and mp["verdict"] != "small":
warns.append(f"This Mac has {ram:.0f} GB of memory and the set was measured on one with 128 GB: with less, more experts come from "