Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
dsv41 cpu lane: AVX-512 rows by default for IQ2_XXS prompt chunks (7+ tokens), measured on Zen 4
Niko1221995ff61f3c69
diff --git a/src/dsv41/ds4_gg_cpu.cpp b/src/dsv41/ds4_gg_cpu.cppindex dfab6765a..cbfcba2d0 100644--- a/src/dsv41/ds4_gg_cpu.cpp+++ b/src/dsv41/ds4_gg_cpu.cpp@@ -149,8 +149,9 @@ static bool want_512(int type, int nt) {}if (g_iq512 == 0 || !strata::kernels::cpu::iq512x_supported(type)) return false;if (g_iq512 == 1) return true;- (void) nt;- return false; // (the default rule is filled in from the Zen 4 numbers)+ // Zen 4 (Ryzen 5 7600, 6 threads): IQ2_XXS at 16 tokens 3.2 -> 4.0 GB/s over the AVX2 rows (x4.2 against ggml's dot), no gain at 4+ // tokens, IQ3_XXS the same as AVX2 at every count and slower at one - so the AVX-512 rows only for a prompt chunk's tokens on IQ2_XXS+ return nt >= 7 && type == 16;}// 0: ggml-cpu's dot, 1: the AVX2 rows, 2: the AVX-512 rowsstatic int use_iq256(int type, int nt) {