Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
149: the Linux file-tier defaults only for the GGUF read in place
Niko1221c31367979d7c
diff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex e0861c7cf..2f90c88e8 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -262,7 +262,8 @@ engine fetches a layer's missing experts on 8 threads (`STRATA_FETCH_THREADS`) w(Windows `PrefetchVirtualMemory`). With an `experts.bin` in the pack, nothing changes. Setup does not use this yet.**Linux file tier, 0.2.0 defaults (zero-copy page-cached experts, io prefetch, the decode gate):** when experts are read-from the files (a GGUF read in place, `--mmap-experts`, or an `experts.bin` file tier), Linux now runs three things by default.+from a GGUF read in place (the native packs, also with `--mmap-experts`), Linux now runs three things by default (an+`experts.bin` pack, such as the NVFP4 ones, keeps them off unless you set them: not measured there yet).(1) **Zero-copy** (`STRATA_COLD_OVERLAP=4`): an expert of a GGUF read in place whose three tensors' pages are all in the pagecache (mincore) is computed straight from the mapping; before, every such expert was first assembled into a staging buffer(a copy of ~1.5 MB per Q2_0 expert, on the decode's critical path). Same bytes, same rows, same output (identity gatesdiff --git a/include/strata/core/expert_source.hpp b/include/strata/core/expert_source.hppindex ab7ae4a4a..e14d3f4e2 100644--- a/include/strata/core/expert_source.hpp+++ b/include/strata/core/expert_source.hpp@@ -259,6 +259,8 @@ public:virtual void prefetch_end() {}/// 149 STRATA_IO_PF_GATE: whether the decode's read-ahead runs now (the look-ahead then predicts one layer, not two)virtual bool read_ahead_open() const { return true; }+ /// 0.2.0: the source the Linux file-tier defaults apply to (the GGUF read in place); STRATA_COLD_OVERLAP's default+ virtual bool linux_file_defaults() const { return false; }/// 149-cold STRATA_COLD_OVERLAP=3: the last prefetch_begin left this expert unstaged because the page cache holds/// all of it (the GGUF read in place): its gate, up and down rows in the mapping. False: ask `blob`.virtual bool zc_roles(int64_t layer, int64_t expert, const uint8_t** p) { (void) layer; (void) expert; (void) p; return false; }@@ -992,6 +994,9 @@ public:static bool in_decode();bool gate_open() const { return !gate_on_ || gate_open_.load(std::memory_order_relaxed); }bool read_ahead_open() const override { return gate_open(); }+#if defined(__linux__)+ bool linux_file_defaults() const override { return !role_ptr_.empty(); } // the GGUF in place only (measured)+#endifbool gate_on() const { return gate_on_; }double gate_held() const { return gate_ewma_; }int64_t gate_closed_windows() const { return gate_closed_w_; }diff --git a/src/core/expert_source.cpp b/src/core/expert_source.cppindex 38b8be0be..5cd037da2 100644--- a/src/core/expert_source.cpp+++ b/src/core/expert_source.cpp@@ -1822,14 +1822,11 @@ int64_t FileExpertSource::prefetch_begin(int64_t layer, const int64_t* experts,prefetch_end();async_todo_.clear();async_fill_us_ = 0;- static const int mode = [] { // as the dispatch reads it (0.2.0: 4 by default on Linux)+ static const int mode_env = [] { // as the dispatch reads it (0.2.0: unset = 4 for the GGUF read in place on Linux)const char* v = std::getenv("STRATA_COLD_OVERLAP");-#if defined(__linux__)- return v == nullptr || v[0] == '\0' ? 4 : std::atoi(v);-#else- return v == nullptr || v[0] == '\0' ? 0 : std::atoi(v);-#endif+ return v == nullptr || v[0] == '\0' ? -1 : std::atoi(v);}();+ const int mode = mode_env >= 0 ? mode_env : (linux_file_defaults() ? 4 : 0);claim_todo(layer, experts, n, async_todo_, pending, mode >= 3);if (mode == 4) { // zero-copy only: the blobs that must be read are read here, as `prefetch` reads themconst auto t0 = std::chrono::steady_clock::now();@@ -4383,17 +4380,13 @@ void expert_pool_dispatch_multi(ExpertDispatch& d, const float* x_f, const int32};const auto c0 = std::chrono::steady_clock::now();const FileExpertSource::DecodeScope decode_scope; // 149 STRATA_IO_PF_GATE: this thread reads for the decode- // 149-cold STRATA_COLD_OVERLAP: 0 off (default), 1 overlap, 2 measure only, 3 overlap + zero-copy, 4 zero-copy only- // 0.2.0: 4 by default on Linux (zero-copy needs the page cache's residency query); elsewhere 0+ // 149-cold STRATA_COLD_OVERLAP: 0 off, 1 overlap, 2 measure only, 3 overlap + zero-copy, 4 zero-copy only.+ // 0.2.0: unset, 4 for the GGUF read in place on Linux (zero-copy needs the page cache's residency query), else 0static const int cold_env = [] {const char* v = std::getenv("STRATA_COLD_OVERLAP");-#if defined(__linux__)- return v == nullptr || v[0] == '\0' ? 4 : std::clamp(std::atoi(v), 0, 4);-#else- return v == nullptr || v[0] == '\0' ? 0 : std::clamp(std::atoi(v), 0, 4);-#endif+ return v == nullptr || v[0] == '\0' ? -1 : std::clamp(std::atoi(v), 0, 4);}();- d.cold_mode = cold_env;+ d.cold_mode = cold_env >= 0 ? cold_env : (d.src != nullptr && d.src->linux_file_defaults() ? 4 : 0);pt("begin");d.src->begin_layer(d.layers, ids, n_tok * k);pt("begun");diff --git a/src/program/generate.cpp b/src/program/generate.cppindex 95975f1ad..f5a5407c9 100644--- a/src/program/generate.cpp+++ b/src/program/generate.cpp@@ -5449,19 +5449,16 @@ int main(int argc, char** argv) {// ahead on I/O threads while the current layer computes (see FileExpertSource::set_io_prefetch). 0.2.0: on by// default on Linux, with the decode gate (STRATA_IO_PF_GATE) and zero-copy page-cached experts// (STRATA_COLD_OVERLAP=4); STRATA_IO_PREFETCH=0 / STRATA_IO_PF_GATE=0 / STRATA_COLD_OVERLAP=0 turn them off.+ // the defaults are for the GGUF read in place (what was measured); an experts.bin pack keeps them off unless setconst char* io_pf_env = std::getenv("STRATA_IO_PREFETCH");-#if defined(__linux__)- const bool io_pf_default = io_pf_env == nullptr || io_pf_env[0] == '\0';-#else- const bool io_pf_default = false;-#endif+ const bool io_pf_default = (io_pf_env == nullptr || io_pf_env[0] == '\0') && src.linux_file_defaults();if (io_pf_default || (io_pf_env != nullptr && std::atoi(io_pf_env) != 0)) {const char* th = std::getenv("STRATA_IO_PF_THREADS");src.set_io_prefetch(true, th != nullptr ? std::atoi(th) : 0);if (src.io_prefetch() && io_pf_default) {const char* cv = std::getenv("STRATA_COLD_OVERLAP");- std::fprintf(stderr, "strata generate: file tier (Linux defaults): page-cached experts computed from the "- "mapping (STRATA_COLD_OVERLAP=%s), io prefetch with whole-blob preads on %s I/O threads, "+ std::fprintf(stderr, "strata generate: file tier (Linux defaults, the GGUF read in place): page-cached experts "+ "computed from the mapping (STRATA_COLD_OVERLAP=%s), io prefetch with whole-blob preads on %s I/O threads, ""the decode's read-ahead %s; off: STRATA_COLD_OVERLAP=0, STRATA_IO_PREFETCH=0, ""STRATA_IO_PF_GATE=0\n", cv != nullptr && cv[0] != '\0' ? cv : "4", th != nullptr ? th : "8",src.gate_on() ? "only while the page cache has room for it and it pays (STRATA_IO_PF_GATE=1)"