Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
dsv41: the layer-major run's RAM is given back before the cache catch-up and the store fill
Niko1221dbf3bb2e643c
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cuindex f4ccb36ad..1b5215556 100644--- a/src/dsv41/ds4_model.cu+++ b/src/dsv41/ds4_model.cu@@ -7340,6 +7340,10 @@ struct Model::Impl {pos += n;st.tokens += n;cache_tick(n);+ if (lm_end_.on) { // (the layer cache's RAM back before the store and the swaps want it)+ lm_end();+ lm_end_.on = false;+ }cache_settle();const int nr = std::min(T_last, tap_cap);if (!seed_drafter(pos - nr, nr, 0, err)) return false;