Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Release notes 0.2.2 draft: NUMA pool
Niko1221149abc896b65
diff --git a/docs/release-notes-0.2.2.md b/docs/release-notes-0.2.2.mdnew file mode 100644index 000000000..9cad27fed--- /dev/null+++ b/docs/release-notes-0.2.2.md@@ -0,0 +1,5 @@+**0.2.2 (draft).** Only the items of the `f-022-rx6000` and `f-022-numa` branches are written here; the coordinator adds the rest.++## What is new++- **The CPU expert pool splits per NUMA node on a machine with two or more NUMA nodes (built from the report in #3201, not tested on a two-socket machine).** Until now the pool kept its workers on the host thread's socket, because workers on the other socket read the experts across the link and were slower. The split gives each socket its own slice of the pool with its own task counter, and places each socket's half of every CPU-side expert in that socket's memory, so a slice reads local memory only. The reporter measured +10 to +14 % decode on a dual Xeon Gold 6226 (59.3 against 54.05 tok/s with the IOMMU in passthrough); a second reporter's dual Xeon E5-2670 v2 preferred more workers across both sockets, which the split also uses. It starts by itself when the machine has two or more NUMA nodes with CPUs and runs only when it can (Linux, two nodes, pinned threads, enough free memory per node); otherwise the pool is exactly what it was and the log says why. **A machine with one NUMA node runs the same code path as 0.2.1:** the identity gate (10 prompts, exact_pp 4K and 20K) is byte-identical and the single-request speed A/B is not slower on the one-node test boxes (numbers in the notes of the branch). `STRATA_NUMA_POOL=0` turns the split off; `STRATA_NUMA_SPLIT=1` still asks for it on any machine. The start-up log says what it did: `NUMA pool on: 2 NUMA nodes found ...` or `NUMA pool off: <reason>`. On Windows the node count is read but the split itself is Linux-only for now, so a two-node Windows PC logs the reason and runs as before. The logic runs in unit tests on made-up two-, three- and four-node machines; we have no two-socket machine, so the placement and the speed on real ones are the reporters' numbers, not ours. Do not wrap the server in `numactl --cpunodebind` or `--membind`: it pulls both halves onto one node. Thanks to baptisterajaut for the branch this is built on, and to xrip and Tworks25 for their numbers. Please send the start-up lines and a decode run with and without `STRATA_NUMA_POOL=0` from a two-socket machine.