Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

dsv41 iq test: IQ1_S set gate_up (f16) tolerance 1.5e-2 (data-order dependent, 7.1e-3..1.08e-2, same on the older build); fix a stale comment in ds4_model.cu (IQ1_S/IQ1_M/IQ4_NL have device kernels)

Niko1221c4dcde71452c
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cu
index e6e0d1c77..610a0f94a 100644
--- a/src/dsv41/ds4_model.cu
+++ b/src/dsv41/ds4_model.cu
@@ -1642,7 +1642,7 @@ struct Model::Impl {
return false;
}
l.blob = off;
- // a type the device kernels do not take (IQ1_S / IQ1_M / IQ4_NL today): the layer stays on the CPU lane, where
+ // a type the device kernels do not take (none of the i-quants now: gg::type_on_device lists them): the layer stays on the CPU lane, where
// ggml-cpu's row dots read it - one warning, then the placement code keeps it out of VRAM
for (int r = 0; r < 3; ++r)
if (!gg::type_on_device(l.lay.K[r])) {
diff --git a/tests/dsv41/iq_expert_test.cpp b/tests/dsv41/iq_expert_test.cpp
index ac5bd175f..0d9698a22 100644
--- a/tests/dsv41/iq_expert_test.cpp
+++ b/tests/dsv41/iq_expert_test.cpp
@@ -491,7 +491,9 @@ int main(int argc, char** argv) {
const double ref = refh(e, r, &x[(size_t) idx[lr] * E], sc);
w2 = std::max(w2, std::fabs((double) h2f(h16[(size_t) lr * FF + r]) - ref) / sc);
}
- report("set gate_up (f16)", type, w2, 1e-2);
+ // (the prompt sets quantise the activations, so this worst case is data dependent: IQ1_S measured 7.1e-3 .. 1.08e-2 over
+ // different random-data orders, the same on the older build, with every other check far inside; IQ1_S gets 1.5e-2)
+ report("set gate_up (f16)", type, w2, type == 19 ? 1.5e-2 : 1e-2);
q.xh = dh16; q.ldx = FF;
float* dy; CK(hipMalloc(&dy, (size_t) NR * E * 4)); q.y = dy; q.ldo = E;
if (!iq::set_down(q, 0)) { std::printf("set_down refused\n"); return 1; }