Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

dsv41 Windows: 64 Engram reader threads (each ReadFile is synchronous, so the threads are the queue depth); DS4_ENGRAM_THREADS overrides

Niko1221aee5db5eb1a9
diff --git a/src/dsv41/ds4_model.cu b/src/dsv41/ds4_model.cu
index 1eaeccebc..4b307a026 100644
--- a/src/dsv41/ds4_model.cu
+++ b/src/dsv41/ds4_model.cu
@@ -2388,8 +2388,14 @@ struct Model::Impl {
// the Engram rows' reader: threads enough to keep an SSD's queue busy (reads from the page cache need few);
// the row cache a small share of the RAM free (DS4_ENGRAM_CACHE_MB: its size, 0 off; DS4_ENGRAM_THREADS)
const size_t row = (size_t) pk.eg_hd + pk.eg_hd / 32;
+#if defined(_WIN32)
+ // (Windows: each thread's ReadFile is synchronous, so the threads are the queue depth - a 28K-token prompt
+ // in one chunk, 1.36M rows before layer 1: 10 threads 17.1 s, 32 14.1 s, 64 13.4 s on a 20-core PC)
+ int nth = 64;
+#else
const unsigned hw = std::max(1u, std::thread::hardware_concurrency());
int nth = (int) std::min(16u, std::max(4u, hw / 2));
+#endif
if (const char* e = std::getenv("DS4_ENGRAM_THREADS")) nth = std::max(1, std::atoi(e));
const uint64_t avail = ram_avail();
uint64_t mb = avail == ~uint64_t{0} ? 64 : std::min<uint64_t>(256, std::max<uint64_t>(16, (avail >> 20) / 64));