Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

pin budget: CUDA-on-Windows default only (the HIP build keeps its pinning), docs/DETAILS.md

Niko1221805a308d2ea7
diff --git a/docs/DETAILS.md b/docs/DETAILS.md
index 045be3021..52bb179be 100644
--- a/docs/DETAILS.md
+++ b/docs/DETAILS.md
@@ -192,6 +192,14 @@ other ~18 GB), a 32 GB PC with a 12-16 GB GPU the Coder; IQ3_XXS on a 32 GB PC s
layer has two (the share otherwise rounds to 0 at 1-2 misses). RTX 3090 + Ryzen 5 5600X: prompt read +5-7 % at 56K-117K
tokens; decode level alone, +2-3 % with `--adapt-async 1` and `--pcie-frac 0.6`
([report](../bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/README.md)).
+- **Pinned-memory headroom** (Windows CUDA, on by default; `STRATA_PIN_RESERVE_MIB=N`, `0` = off, sets it on any OS): the arena registers
+ slice by slice until the driver refuses one, and on a 64 GB PC that left no page-locked memory at all for the few small buffers the
+ engine allocates after it (the verify window's staging: `verify: mapped staging allocation failed`, exit 1, on Swift 1.5 IQ2_XS). The
+ engine now holds 640 MiB (given back as soon as the arena is placed) and 160 MiB (kept until a required buffer is refused, or READY) while
+ the arena pins, so the arena stops about one layer earlier; a refused required buffer gives both back and tries once more. If that still
+ fails the error ends with `[pinned-memory-exhausted]` and names the `STRATA_ARENA_PIN_GIB` to start with. Linux has no such ceiling
+ (the NVIDIA driver pins past `ulimit -l`), so the reserve is off there. Test knobs: `STRATA_TEST_PIN_BUDGET_MIB=N` makes the engine's own
+ pinned allocations fail beyond N MiB, `STRATA_TEST_PIN_LEFT_KIB=N` lowers that ceiling after the arena to what is charged plus N KiB.
- `STRATA_EXCHANGE_ROTATE=1` (opt-in): an adaptive swap hands buffer ownership over instead of copying the evicted
blob into the RAM copy (equal-size blobs, fully page-locked copy). Same tokens, fewer host copies; it works with
`--adapt-async 1` too. Details and the measurement: [EXCHANGE_ROTATION.md](EXCHANGE_ROTATION.md).
diff --git a/include/strata/core/pin_budget.hpp b/include/strata/core/pin_budget.hpp
index 1a31a8311..718299b41 100644
--- a/include/strata/core/pin_budget.hpp
+++ b/include/strata/core/pin_budget.hpp
@@ -13,7 +13,7 @@
// the engine is up; the allocation then releases it and retries once (host_alloc_guarded).
// A reserve that cannot be had is shrunk, then skipped: this never fails a start.
//
-// Defaults: Windows 640 + 160 MiB; Linux off (the NVIDIA driver there pins past RLIMIT_MEMLOCK and has no such
+// Defaults: Windows CUDA 640 + 160 MiB (not the HIP build); Linux off (the NVIDIA driver there pins past RLIMIT_MEMLOCK and has no such
// ceiling; checked with a 4 GiB cudaHostRegister under `ulimit -l 1 GiB`). STRATA_PIN_RESERVE_MIB=N sets the soft
// reserve (0 = off) on any OS; the hard one is a quarter of it, at least 64 MiB.
//
@@ -89,7 +89,7 @@ inline void free_locked(State& s, void* p) {
inline uint64_t soft_reserve_default_mib() {
if (const char* e = std::getenv("STRATA_PIN_RESERVE_MIB"); e != nullptr && *e != '\0')
return (uint64_t) std::strtoull(e, nullptr, 10);
-#if defined(_WIN32)
+#if defined(_WIN32) && !defined(STRATA_USE_HIP) // CUDA on Windows (WDDM); the AMD build keeps its pinning as it was
return 640;
#else
return st().sim_budget != 0 ? 640 : 0; // Linux: only when the test ceiling is on