Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
pin budget: CUDA-on-Windows default only (the HIP build keeps its pinning), docs/DETAILS.md
Niko1221805a308d2ea7
diff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex 045be3021..52bb179be 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -192,6 +192,14 @@ other ~18 GB), a 32 GB PC with a 12-16 GB GPU the Coder; IQ3_XXS on a 32 GB PC slayer has two (the share otherwise rounds to 0 at 1-2 misses). RTX 3090 + Ryzen 5 5600X: prompt read +5-7 % at 56K-117Ktokens; decode level alone, +2-3 % with `--adapt-async 1` and `--pcie-frac 0.6`([report](../bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/README.md)).+- **Pinned-memory headroom** (Windows CUDA, on by default; `STRATA_PIN_RESERVE_MIB=N`, `0` = off, sets it on any OS): the arena registers+ slice by slice until the driver refuses one, and on a 64 GB PC that left no page-locked memory at all for the few small buffers the+ engine allocates after it (the verify window's staging: `verify: mapped staging allocation failed`, exit 1, on Swift 1.5 IQ2_XS). The+ engine now holds 640 MiB (given back as soon as the arena is placed) and 160 MiB (kept until a required buffer is refused, or READY) while+ the arena pins, so the arena stops about one layer earlier; a refused required buffer gives both back and tries once more. If that still+ fails the error ends with `[pinned-memory-exhausted]` and names the `STRATA_ARENA_PIN_GIB` to start with. Linux has no such ceiling+ (the NVIDIA driver pins past `ulimit -l`), so the reserve is off there. Test knobs: `STRATA_TEST_PIN_BUDGET_MIB=N` makes the engine's own+ pinned allocations fail beyond N MiB, `STRATA_TEST_PIN_LEFT_KIB=N` lowers that ceiling after the arena to what is charged plus N KiB.- `STRATA_EXCHANGE_ROTATE=1` (opt-in): an adaptive swap hands buffer ownership over instead of copying the evictedblob into the RAM copy (equal-size blobs, fully page-locked copy). Same tokens, fewer host copies; it works with`--adapt-async 1` too. Details and the measurement: [EXCHANGE_ROTATION.md](EXCHANGE_ROTATION.md).diff --git a/include/strata/core/pin_budget.hpp b/include/strata/core/pin_budget.hppindex 1a31a8311..718299b41 100644--- a/include/strata/core/pin_budget.hpp+++ b/include/strata/core/pin_budget.hpp@@ -13,7 +13,7 @@// the engine is up; the allocation then releases it and retries once (host_alloc_guarded).// A reserve that cannot be had is shrunk, then skipped: this never fails a start.//-// Defaults: Windows 640 + 160 MiB; Linux off (the NVIDIA driver there pins past RLIMIT_MEMLOCK and has no such+// Defaults: Windows CUDA 640 + 160 MiB (not the HIP build); Linux off (the NVIDIA driver there pins past RLIMIT_MEMLOCK and has no such// ceiling; checked with a 4 GiB cudaHostRegister under `ulimit -l 1 GiB`). STRATA_PIN_RESERVE_MIB=N sets the soft// reserve (0 = off) on any OS; the hard one is a quarter of it, at least 64 MiB.//@@ -89,7 +89,7 @@ inline void free_locked(State& s, void* p) {inline uint64_t soft_reserve_default_mib() {if (const char* e = std::getenv("STRATA_PIN_RESERVE_MIB"); e != nullptr && *e != '\0')return (uint64_t) std::strtoull(e, nullptr, 10);-#if defined(_WIN32)+#if defined(_WIN32) && !defined(STRATA_USE_HIP) // CUDA on Windows (WDDM); the AMD build keeps its pinning as it wasreturn 640;#elsereturn st().sim_budget != 0 ? 640 : 0; // Linux: only when the test ceiling is on