Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
CPU pool plan: cap 32 (measured on the 32-core EPYC: 24 lost 2-5% decode, 32 and 48 equal, 63 oversubscribed lost 35%)
Niko1221036c8f6ec3b5
diff --git a/include/strata/kernels/cpu/pool_plan.hpp b/include/strata/kernels/cpu/pool_plan.hppindex df91d6ea4..0303d52e9 100644--- a/include/strata/kernels/cpu/pool_plan.hpp+++ b/include/strata/kernels/cpu/pool_plan.hpp@@ -25,7 +25,7 @@ namespace strata::kernels::cpu {/// The most default workers: past this the layer barrier costs more than the extra cores read (see the header comment)./// Measured on 6c/12t (z440), 20c (P2003), 32c/64t (r9700) and 64c (l620): notes-152-users.md.-inline constexpr int kPoolWorkersCap = 24;+inline constexpr int kPoolWorkersCap = 32;/// Cores kept free for the other threads of the engine (copy / IO threads, the GPU driver's, the OS) on a CPU with at/// least `kPoolReserveFrom` cores that the pool may use (besides the host's). Small CPUs keep every core.inline constexpr int kPoolReserveFrom = 16;diff --git a/setup.py b/setup.pyindex 5b454f325..203e05b89 100644--- a/setup.py+++ b/setup.py@@ -998,7 +998,7 @@ def two_socket_note(sockets) -> list[str]:# keeps in step with these numbers). Setup writes no worker count on an ordinary CPU: the engine picks it, from the# physical cores (not the SMT siblings), minus the host thread's core and a reserve on a big CPU, on the host thread's# socket, under a cap. What setup does write (a hybrid CPU's count, the vision encoder's threads) is capped the same way.-POOL_WORKERS_CAP = 24+POOL_WORKERS_CAP = 32POOL_RESERVE_FROM = 16POOL_RESERVE = 1VISION_THREADS_CAP = 16