Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

dsv41: the layer-major run's RAM is given back before the cache catch-up and the store fill

Niko1221dbf3bb2e643c
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cu
index f4ccb36ad..1b5215556 100644
--- a/src/dsv41/ds4_model.cu
+++ b/src/dsv41/ds4_model.cu
@@ -7340,6 +7340,10 @@ struct Model::Impl {
pos += n;
st.tokens += n;
cache_tick(n);
+ if (lm_end_.on) { // (the layer cache's RAM back before the store and the swaps want it)
+ lm_end();
+ lm_end_.on = false;
+ }
cache_settle();
const int nr = std::min(T_last, tap_cap);
if (!seed_drafter(pos - nr, nr, 0, err)) return false;