Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

CPU pool plan: cap 32 (measured on the 32-core EPYC: 24 lost 2-5% decode, 32 and 48 equal, 63 oversubscribed lost 35%)

Niko1221036c8f6ec3b5
diff --git a/include/strata/kernels/cpu/pool_plan.hpp b/include/strata/kernels/cpu/pool_plan.hpp
index df91d6ea4..0303d52e9 100644
--- a/include/strata/kernels/cpu/pool_plan.hpp
+++ b/include/strata/kernels/cpu/pool_plan.hpp
@@ -25,7 +25,7 @@ namespace strata::kernels::cpu {
/// The most default workers: past this the layer barrier costs more than the extra cores read (see the header comment).
/// Measured on 6c/12t (z440), 20c (P2003), 32c/64t (r9700) and 64c (l620): notes-152-users.md.
-inline constexpr int kPoolWorkersCap = 24;
+inline constexpr int kPoolWorkersCap = 32;
/// Cores kept free for the other threads of the engine (copy / IO threads, the GPU driver's, the OS) on a CPU with at
/// least `kPoolReserveFrom` cores that the pool may use (besides the host's). Small CPUs keep every core.
inline constexpr int kPoolReserveFrom = 16;
diff --git a/setup.py b/setup.py
index 5b454f325..203e05b89 100644
--- a/setup.py
+++ b/setup.py
@@ -998,7 +998,7 @@ def two_socket_note(sockets) -> list[str]:
# keeps in step with these numbers). Setup writes no worker count on an ordinary CPU: the engine picks it, from the
# physical cores (not the SMT siblings), minus the host thread's core and a reserve on a big CPU, on the host thread's
# socket, under a cap. What setup does write (a hybrid CPU's count, the vision encoder's threads) is capped the same way.
-POOL_WORKERS_CAP = 24
+POOL_WORKERS_CAP = 32
POOL_RESERVE_FROM = 16
POOL_RESERVE = 1
VISION_THREADS_CAP = 16