Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

dsv41 cpu lane: AVX-512 rows by default for IQ2_XXS prompt chunks (7+ tokens), measured on Zen 4

Niko1221995ff61f3c69
diff --git a/src/dsv41/ds4_gg_cpu.cpp b/src/dsv41/ds4_gg_cpu.cpp
index dfab6765a..cbfcba2d0 100644
--- a/src/dsv41/ds4_gg_cpu.cpp
+++ b/src/dsv41/ds4_gg_cpu.cpp
@@ -149,8 +149,9 @@ static bool want_512(int type, int nt) {
}
if (g_iq512 == 0 || !strata::kernels::cpu::iq512x_supported(type)) return false;
if (g_iq512 == 1) return true;
- (void) nt;
- return false; // (the default rule is filled in from the Zen 4 numbers)
+ // Zen 4 (Ryzen 5 7600, 6 threads): IQ2_XXS at 16 tokens 3.2 -> 4.0 GB/s over the AVX2 rows (x4.2 against ggml's dot), no gain at 4
+ // tokens, IQ3_XXS the same as AVX2 at every count and slower at one - so the AVX-512 rows only for a prompt chunk's tokens on IQ2_XXS
+ return nt >= 7 && type == 16;
}
// 0: ggml-cpu's dot, 1: the AVX2 rows, 2: the AVX-512 rows
static int use_iq256(int type, int nt) {