Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
dsv41 CPU lane: IQ1_M uses Strata's exact rows from one token (ggml's IQ1_M dot runs at 1.8 GB/s on AMD Zen 2: _pdep_u64 is microcoded there); portable ggml-cpu build has BMI2 off (IQ1_S 2.3 -> 37 GB/s, IQ1_M 1.8 -> 32 GB/s on an EPYC 7452)
Niko12216953d896d06e
diff --git a/CMakeLists.txt b/CMakeLists.txtindex acc764463..3efdcc464 100644--- a/CMakeLists.txt+++ b/CMakeLists.txt@@ -1664,9 +1664,12 @@ if(STRATA_NATIVE_EXPERTS)set(GGML_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)if(STRATA_PORTABLE)set(GGML_NATIVE OFF CACHE BOOL "" FORCE)- foreach(_f GGML_AVX GGML_AVX2 GGML_FMA GGML_F16C GGML_BMI2)+ foreach(_f GGML_AVX GGML_AVX2 GGML_FMA GGML_F16C)set(${_f} ON CACHE BOOL "" FORCE)endforeach()+ # BMI2 off: ggml's IQ1_S / IQ1_M dots use _pdep_u64 with it, and AMD Zen 1/2 run PDEP in microcode (EPYC 7452, IQ1_M one-token+ # dot 1.8 GB/s against 32 GB/s without it; IQ1_S 2.3 against 37). Intel loses nothing measurable: no other ggml-cpu quant uses it.+ set(GGML_BMI2 OFF CACHE BOOL "" FORCE)elseif(STRATA_ISA_FLOOR)# the experimental older-CPU build (STRATA_ISA_FLOOR above): one ggml-cpu for the floor, ggml's "sandybridge"# feature set (SSE4.2 + AVX) or SSE4.2 alone - not the build host's, which may be newer than the PC it runs on.diff --git a/src/dsv41/ds4_gg_cpu.cpp b/src/dsv41/ds4_gg_cpu.cppindex e303694b3..cede307a9 100644--- a/src/dsv41/ds4_gg_cpu.cpp+++ b/src/dsv41/ds4_gg_cpu.cpp@@ -92,7 +92,10 @@ void gg_cpu_quant(int act, const float* x, int k, void* dst) {// min_tokens(type) and these rows from there. DS4_GG_IQ256=0 never takes them, =1 always (the old opt-in).static int g_iq256 = -1; // -1: from the environment and the CPU, 0 off, 1 always, 2 from min_tokens(type)void gg_cpu_set_iq256(int mode) { g_iq256 = mode; }-static int min_tokens(int type) { return type == 21 ? 1 : type == 29 ? 3 : 2; }+// IQ1_M from one token too: ggml's own IQ1_M dot (and IQ1_S's) uses _pdep_u64 where the build has BMI2, which AMD Zen 1/2 run in+// microcode - 1.8 GB/s against these rows' 33 GB/s on an EPYC 7452 (a 2.00 set decodes 3x slower on one card), 19-31 GB/s on Intel.+// The rows are bit-identical, so taking them at every token count only changes the speed.+static int min_tokens(int type) { return type == 21 ? 1 : type == 29 ? 1 : 2; }static bool own_ok(int type, int impl) {static std::once_flag once[2][32];static bool ok[2][32];