Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
metal: no expert-cache loan in the CLI prompt path either (the 4K+ prompt of a plain strata run aborted in ExpertCache::fill_slot_queued)
Niko1221b440640929a9
diff --git a/src/program/generate.cpp b/src/program/generate.cppindex 3b7ad4111..da4e798e5 100644--- a/src/program/generate.cpp+++ b/src/program/generate.cpp@@ -15745,6 +15745,9 @@ int main(int argc, char** argv) {k = 0; // a fixed chunk that does not fit: its own buffers, as before}const int64_t blob = (int64_t) strata::kernels::cpu::expert_layout().max_blob;+#if defined(STRATA_GPU_METAL)+ k = 0; // Metal: no loan of expert-cache slots (the prompt buffers are the Metal window's own); the refill after it copied into given-back allocations+#endifif (k > 0) { // the lent slots are refilled after the promptconst int32_t first = (int32_t) (xcache.slots() - k);// the elastic K/V first: its hot experts move into slots of the loan, which are refilled after it