Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: DEEPSEEK41.md (where the model goes, install, the API)

Niko12219fcc101844ac
diff --git a/docs/DEEPSEEK41.md b/docs/DEEPSEEK41.md
new file mode 100644
index 000000000..d2d3c1631
--- /dev/null
+++ b/docs/DEEPSEEK41.md
@@ -0,0 +1,55 @@
+# DeepSeek-V4.1-Flash
+
+DeepSeek-V4.1-Flash is a large mixture-of-experts model: 40 layers, 384 experts in each (6 are used for every word), and a 203 GB
+n-gram table next to them. Strata runs it with its own engine, `strata-dsv41` (`src/dsv41`, ported from Project Maya's
+`maya-ds4`, MIT; the notice is in `third_party/project-maya`). The file is Unsloth's UD-Q8_K_XL, pinned in `setup.py`: nine
+shards, 508.4 GB, the experts as MXFP4 and the rest in BF16.
+
+This is for a computer with several graphics cards and a lot of RAM. It was run on four Radeon AI PRO R9700 (32 GB each) with
+251 GB of RAM. Strata does not refuse a smaller computer; it tells you what to expect first.
+
+## Where it goes
+
+| | size | where |
+|---|---|---|
+| routed experts (15,360 of 18.8 MB) | 288.8 GB | as many as fit in graphics memory; the rest are computed on the CPU, read in place from the model files |
+| attention and other dense weights | 16.8 GB | graphics memory, one block of layers on each card (about 3.9 GB a card on four) |
+| n-gram table (Engram) | 202.8 GB | the SSD. A few rows are read for each word it writes. It is not counted in RAM, but a slow disk slows every word, so keep the model files on an NVMe drive |
+
+Measured on the four R9700 cards: 6,197 experts (116.5 GB) in graphics memory, 9,163 (172.3 GB) on the CPU, which needs about that
+much free RAM to stay cached. `strata launch --plan` and `setup.py --check` work this out for your cards and RAM
+(`strata_control/dsv41_fit.py`; it predicted 6,240 experts on the R9700 box).
+
+With less memory than that, part of the CPU's experts are read from the SSD while the model answers. It runs, slowly. A card
+that cannot hold its share of the dense weights (about 4 GB each on four cards, 16 GB on one) can make the engine stop at the start
+with an out-of-memory message. Both are said before anything is downloaded.
+
+## Install
+
+```
+./setup.sh --family deepseek41 # downloads 508 GB (resumable, checked against pinned sizes and SHA-256)
+./setup.sh --family deepseek41 --gguf-dir DIR # files you already have: the nine shards, or the repository folder with UD-Q8_K_XL inside
+strata launch --model ud-q8_k_xl --plan # where it would go on this computer, nothing changed
+strata launch --model-path DIR --model ud-q8_k_xl # start from files you already have
+```
+
+Setup needs `strata-dsv41` next to the engine (or `STRATA_DSV41_EXE`) and says so before it downloads anything. Build it with
+`cmake -DSTRATA_BUILD_DSV41=ON`. The default context is 8192 tokens, the size it was checked at; `--context` asks for more.
+Loading takes a few minutes (the config sets `"engine_ready_s": 3600`).
+
+## Through the API
+
+The server speaks to the engine as it does for the other models, so `/v1/chat/completions`, `/v1/messages`, streaming, stop and
+`/metrics` work as usual (the answer speed is the tokens the model produced over the engine's decode time).
+
+- Thinking is on by default, as the model's template has it. `reasoning_effort: "none"` or `chat_template_kwargs.enable_thinking: false` turns it off.
+ Effort words: `low` (50), `medium` (62), `high` (75, the default), `max` (100), or a whole number 1 to 100.
+- Prompts are the GGUF's own chat template, the same bytes as DeepSeek's `encoding.py` on the cases checked. Project Maya's own template lost the
+ tool associations and later system instructions (their issue #299); Strata does not use it.
+- Tool calls: the model writes DSML (`<|DSML| calls>` ... `<|DSML| invoke name="...">` ... `<|DSML| parameter name="..." string="true">`). The
+ server turns them into OpenAI `tool_calls` and Anthropic `tool_use`, including `tool_choice`, streamed arguments and calls written inside the thinking.
+ A value marked `string="true"` stays a string.
+- Defaults when a request names none: temperature 1.0, top_p 1.0.
+
+Not in this version: pictures, the DSpark drafter, several requests at once (`--parallel` is ignored with a note), tuning
+(`--tune` is skipped with a note).