Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

metal: no expert-cache loan in the CLI prompt path either (the 4K+ prompt of a plain strata run aborted in ExpertCache::fill_slot_queued)

Niko1221b440640929a9
diff --git a/src/program/generate.cpp b/src/program/generate.cpp
index 3b7ad4111..da4e798e5 100644
--- a/src/program/generate.cpp
+++ b/src/program/generate.cpp
@@ -15745,6 +15745,9 @@ int main(int argc, char** argv) {
k = 0; // a fixed chunk that does not fit: its own buffers, as before
}
const int64_t blob = (int64_t) strata::kernels::cpu::expert_layout().max_blob;
+#if defined(STRATA_GPU_METAL)
+ k = 0; // Metal: no loan of expert-cache slots (the prompt buffers are the Metal window's own); the refill after it copied into given-back allocations
+#endif
if (k > 0) { // the lent slots are refilled after the prompt
const int32_t first = (int32_t) (xcache.slots() - k);
// the elastic K/V first: its hot experts move into slots of the loan, which are refilled after it