Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
prompt: the RAM store's post-prompt preload is off by default - it costs a 4K prompt 4.7 s and the decode that follows gains nothing
Niko1221259532bf2e74
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cuindex 6d151af60..48d6d54f9 100644--- a/src/dsv41/ds4_model.cu+++ b/src/dsv41/ds4_model.cu@@ -6282,7 +6282,10 @@ struct Model::Impl {if (sw_inflight == 0) break; // (nothing queued: every layer's best CPU-lane expert is not hotter than its coldest VRAM one)std::this_thread::sleep_for(std::chrono::milliseconds(1));}- if (store && env_f("DS4_SETTLE_STORE", 1.0f) != 0.0f) { // the RAM store too: a window prompt fills it through the CPU+ // The RAM store's preload after a chunked prompt is off by default: measured on Strix Halo (code fixture, prompt ++ // 256 tokens, 3 runs a side) it costs the prompt 4.7 s and the decode that follows gains nothing (13.9/14.4 s with it+ // against 14.2 s without). The swaps above are what the decode actually uses; DS4_SETTLE_STORE=1 restores it.+ if (store && env_f("DS4_SETTLE_STORE", 0.0f) != 0.0f) { // the RAM store too: a window prompt fills it through the CPU// lane's picks, a chunked one streams past it - the hottest CPU-lane experts of the prompt into its free slotsstd::vector<std::pair<float, std::pair<int, int>>> hot;for (const Layer& l : L) {