Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
metal: no command-buffer split after the fp8-rounding finishes (decode +22%: code 11.9 -> 14.6, agent 9.7 -> 12.0 tok/s); DS4_FINISH_SPLIT=1 restores it
Niko12211ece7efd5c24
diff --git a/metal/src/engine/ds4_finish.cpp b/metal/src/engine/ds4_finish.cppindex 7dda49484..23499135c 100644--- a/metal/src/engine/ds4_finish.cpp+++ b/metal/src/engine/ds4_finish.cpp@@ -165,8 +165,10 @@ bool finish(float* x, int n_vec, int len, long long stride, const FinishIn& f) {g_cx->run({d});// The fp8-rounding finishes (the KV rows, the attention output) are read stale by the next dispatch when it shares their// command buffer: end the buffer after them (committed, not waited for: the queue orders the buffers).- static const bool no_split = std::getenv("DS4_NO_FINISH_SPLIT") != nullptr;- if (f.quant && !no_split) g_cx->split_runs();+ // (it once ended the command buffer after every fp8-rounding finish, 2 a layer, against a stale read; each extra buffer costs the GPU ~0.3 ms idle, a third of a decode token.+ // The decode-path score is bit-identical without it, the dispatch barrier after every dispatch orders the readers. DS4_FINISH_SPLIT=1 brings it back.)+ static const bool split_after = std::getenv("DS4_FINISH_SPLIT") != nullptr;+ if (f.quant && split_after) g_cx->split_runs();if (const char* fm2 = std::getenv("DS4_FLUSH_FINISH_CTX")) { if (f.quant) { if (std::atoi(fm2) == 2) g_cx->split_runs(); else g_cx->flush_runs(); } }if (const char* fm = std::getenv("DS4_FLUSH_FINISH")) { // debugging: which finish calls need a drain after themconst int m = std::atoi(fm);