Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Route prior: the 0.2.7 default on a discrete NVIDIA card of 13 GiB or less drops from 2.0 to 1.5 (owner decision 2026-10-26). On the RTX 3060 12 GB (IQ3_XXS) the copy and code check found real defects at 2.0 and none in 19 tasks at 1.5 or 1.0; 1.5 keeps story +15% and code +37% against off. The value lives in strata::kRoutePriorSmallCard (sm120_series.hpp); scope (CUDA discrete <= 13 GiB), STRATA_ROUTE_PRIOR=0, an explicit value and STRATA_DEFAULTS=026 are unchanged. arch_defaults_test, verify.cpp comments, DETAILS.md and the 0.2.7 release notes follow.

Niko12210afaaa399edd
diff --git a/docs/DETAILS.md b/docs/DETAILS.md
index e0b9a7d12..eb9451325 100644
--- a/docs/DETAILS.md
+++ b/docs/DETAILS.md
@@ -957,7 +957,7 @@ them on your own card.
| `STRATA_PREFILL_EQUAL=1` | CUDA | A prompt segment is read in chunks of equal size, not full chunks and a short last one. Changes rounding. | |
| `STRATA_OWNED_PRICE=exact` | CUDA | The cache sizing prices the prompt path's own buffers by their real allocation. | |
| `STRATA_IQ_MT_MIN=1`, `STRATA_IQ3S_MT1=1` | CPU | See [Draft layer](#draft-layer) (reproducible output; the second gives IQ3_S the multi-token kernel for one token on Alder Lake and later without AVX-512). | |
-| `STRATA_ROUTE_PRIOR`, `STRATA_ROUTE_RESIDENT`, `STRATA_ROUTE_RESIDENT_MTP` | CUDA | Routing changes that add speed and also add KL. | Not recommended, except that 0.2.7 turns `STRATA_ROUTE_PRIOR=2.0` on by itself on a discrete NVIDIA card of 13 GiB or less (the owner approved the quality trade for 12 GB cards); `STRATA_ROUTE_PRIOR=0` turns it off. It is never a default on a Radeon, an integrated GPU or a bigger card. |
+| `STRATA_ROUTE_PRIOR`, `STRATA_ROUTE_RESIDENT`, `STRATA_ROUTE_RESIDENT_MTP` | CUDA | Routing changes that add speed and also add KL. | Not recommended, except that 0.2.7 turns `STRATA_ROUTE_PRIOR=1.5` on by itself on a discrete NVIDIA card of 13 GiB or less (the owner approved the quality trade for 12 GB cards; the first 0.2.7 candidates used 2.0, see the 12 GB default in the release notes for why it is 1.5); `STRATA_ROUTE_PRIOR=0` turns it off and an explicit value wins. It is never a default on a Radeon, an integrated GPU or a bigger card. |
| `STRATA_HOST_CORE=last\|sibling` (or `--host-core`) | Windows | `last` puts the host thread on the last physical core, away from the core that gets the GPU's interrupts. `sibling` keeps it on the first core's other hardware thread, so the workers keep every core. The default `first` leaves a hybrid CPU as it is. | One layer-split box went from a ±6% swing to steady and 8% faster. |
| `--pool-affinity measure` | Hybrid CPUs, native packs | Times the `all` and `p-cores` layouts at start (about a second) and takes the faster. The measurement is taken once; where the E-cores' speed changes while the engine runs, `--pool-affinity p-cores` is the steady choice. | |
| `STRATA_KEEP_EMPTY_TURNS=1`, `STRATA_TOPK_STREAM=0` | | See [TROUBLESHOOTING.md](TROUBLESHOOTING.md). | |
@@ -1137,7 +1137,7 @@ On 12.0 cards (sm_120) these also apply:
prompt time and start-up time, which is why smaller PCs keep the SSD reads.
A prompt chunk above 8192 (`--prefill auto:16384` or `auto:32768`) is not bit-exact on CUDA, so the default stays 8192 on
-NVIDIA. It is exact on gfx12, where 16384 is the default. `STRATA_ROUTE_PRIOR` raises KL, so below 0.2.7 it was opt-in; 0.2.7 sets 2.0 by itself only on a discrete NVIDIA card of 13 GiB or less.
+NVIDIA. It is exact on gfx12, where 16384 is the default. `STRATA_ROUTE_PRIOR` raises KL, so below 0.2.7 it was opt-in; 0.2.7 sets 1.5 by itself only on a discrete NVIDIA card of 13 GiB or less.
---
diff --git a/docs/release-notes-0.2.7.md b/docs/release-notes-0.2.7.md
index a37622782..8c0081cf4 100644
--- a/docs/release-notes-0.2.7.md
+++ b/docs/release-notes-0.2.7.md
@@ -74,10 +74,14 @@ These need no engine change; they are settings. Each was measured on the same bo
## The 12 GB default: residency-biased routing
-**On by default at 2.0 on a discrete NVIDIA card with 13 GiB of memory or less** (`STRATA_ROUTE_PRIOR=0` turns it off;
+**On by default at 1.5 on a discrete NVIDIA card with 13 GiB of memory or less** (`STRATA_ROUTE_PRIOR=0` turns it off, an explicit value wins;
not a default on a Radeon, a Strix Halo or any integrated GPU, where it was not measured). The router prefers
-experts that are already resident, so a window stops waiting on a page-in that the next window would need again. Measured
-on the RTX 3060 12 GB (IQ3_XXS), five interleaved pairs of fresh engines:
+experts that are already resident, so a window stops waiting on a page-in that the next window would need again. The first
+0.2.7 candidates used a bonus of 2.0, and the table below was measured at 2.0 on the RTX 3060 12 GB (IQ3_XXS), five interleaved
+pairs of fresh engines. The default was lowered to 1.5 by the owner on 26 October after the copy and code check on that card
+(IQ3_XXS) found real defects in the answers at 2.0 and **no defect in 19 tasks at 1.5** (or at 1.0). 1.5 keeps most of the speed:
+story **+15%** and code **+37%** against the routing off. That is 19 tasks, not a proof; a longer 61-task check of 0, 1.5 and 2.0
+is still running, and if it shows defects at 1.5 the default falls back to 1.0. The 2.0 measurements:
| axis | before | after |
|---|---|---|
@@ -94,8 +98,8 @@ document prompt), and the whole recipe - routing, a workload-fitted expert profi
896` and the English draft vocabulary - gives **+26.5%** there and **+80%** on code in a sustained 80-request session.
It works by choosing a slightly different set of experts, so replies change: the owner approved that trade for 12 GB cards.
-**Do not raise the bonus beyond 2.0.** 3.0 measures +17.7% / +24.0% on the test prompts but **-4.5% on held-out content**,
-and 4.0 breaks outright - 2.0 is the setting that generalises.
+**Do not raise the bonus to 2.0 or more.** 2.0 showed defects in the copy and code check; 3.0 measures +17.7% / +24.0% on the test
+prompts but **-4.5% on held-out content**, and 4.0 breaks outright. `STRATA_ROUTE_PRIOR=2` still works if you want the old value.
## What was tried and did not make it
diff --git a/include/strata/core/sm120_series.hpp b/include/strata/core/sm120_series.hpp
index 8a6932a65..d21d5b051 100644
--- a/include/strata/core/sm120_series.hpp
+++ b/include/strata/core/sm120_series.hpp
@@ -49,10 +49,16 @@ inline bool sm120_default(int major, int minor) { return is_sm120(major, minor)
// `total_bytes`: the card's memory (cudaDeviceProp::totalGlobalMem); `integrated`: a GPU that shares the PC's memory.
constexpr double kSmallCardGiB = 13.0; // "a 12 GB card": 12 GB cards report 11.6 - 12.0 GiB, 16 GB ones 15.6 - 16.0
-/// STRATA_ROUTE_PRIOR when the user set nothing: 2.0 (residency-biased routing, changes which experts run; owner-approved for 12 GB cards) on a
-/// discrete CUDA card of 13 GiB or less, 0 everywhere else. Measured on an RTX 3060 12 GB only.
+/// The residency-bias bonus a discrete CUDA card of 13 GiB or less gets when the user set nothing: THE one place the value lives. 1.5 since the
+/// owner decision of 2026-10-26 (it was 2.0 in the first 0.2.7 candidates): on the RTX 3060 12 GB with IQ3_XXS the copy and code check found real
+/// defects at 2.0 and none in 19 tasks at 1.5 or 1.0, and 1.5 keeps the speed (story +15%, code +37% against off). A longer 61-task check of
+/// 0 / 1.5 / 2.0 is still running; if it shows defects at 1.5 this falls back to 1.0.
+constexpr float kRoutePriorSmallCard = 1.5f;
+
+/// STRATA_ROUTE_PRIOR when the user set nothing: kRoutePriorSmallCard (residency-biased routing, changes which experts run; owner-approved for 12 GB
+/// cards) on a discrete CUDA card of 13 GiB or less, 0 everywhere else. Measured on an RTX 3060 12 GB only.
inline float route_prior_default(bool hip, bool integrated, double total_bytes, bool v026 = false) {
- return !v026 && !hip && !integrated && total_bytes > 0.0 && total_bytes / 1073741824.0 <= kSmallCardGiB ? 2.0f : 0.0f;
+ return !v026 && !hip && !integrated && total_bytes > 0.0 && total_bytes / 1073741824.0 <= kSmallCardGiB ? kRoutePriorSmallCard : 0.0f;
}
/// STRATA_GR_V3 (the stream-split hyper-connection read, another summation order) when the user set nothing: ON for a discrete compute capability
diff --git a/src/core/arch_defaults_test.cpp b/src/core/arch_defaults_test.cpp
index 51f70493b..ac80d4dd3 100644
--- a/src/core/arch_defaults_test.cpp
+++ b/src/core/arch_defaults_test.cpp
@@ -293,8 +293,8 @@ int main() {
using namespace strata;
CHECK(is_026_token("026") && is_026_token("0.2.6") && is_026_token("v0.2.6"));
CHECK(!is_026_token(nullptr) && !is_026_token("") && !is_026_token("0") && !is_026_token("027"));
- // residency-biased routing: 0 under 026 on the card that gets 2.0 otherwise
- CHECK(route_prior_default(false, false, 12.0 * GiB, false) == 2.0f);
+ // residency-biased routing: 0 under 026 on the card that gets 1.5 otherwise
+ CHECK(route_prior_default(false, false, 12.0 * GiB, false) == 1.5f);
CHECK(route_prior_default(false, false, 12.0 * GiB, true) == 0.0f);
CHECK(route_prior_default(false, false, 8.0 * GiB, true) == 0.0f);
// the wide HC up read: on for a 12.0 card on CUDA, off under 026, off in HIP, off on sm_86 / sm_121
@@ -340,11 +340,12 @@ int main() {
{
const double GiB = 1073741824.0;
using namespace strata;
- // residency-biased routing: 2.0 on a discrete CUDA card of 13 GiB or less, nothing else
- CHECK(route_prior_default(false, false, 11.9 * GiB) == 2.0f); // RTX 3060 / 5070 12 GB
- CHECK(route_prior_default(false, false, 12.0 * GiB) == 2.0f);
- CHECK(route_prior_default(false, false, 8.0 * GiB) == 2.0f); // an 8 GB card
- CHECK(route_prior_default(false, false, 13.0 * GiB) == 2.0f); // the boundary is inclusive
+ // residency-biased routing: 1.5 on a discrete CUDA card of 13 GiB or less, nothing else
+ CHECK(route_prior_default(false, false, 11.9 * GiB) == 1.5f); // RTX 3060 / 5070 12 GB
+ CHECK(route_prior_default(false, false, 12.0 * GiB) == 1.5f);
+ CHECK(route_prior_default(false, false, 8.0 * GiB) == 1.5f); // an 8 GB card
+ CHECK(route_prior_default(false, false, 13.0 * GiB) == 1.5f); // the boundary is inclusive
+ CHECK(route_prior_default(false, false, 12.0 * GiB) == strata::kRoutePriorSmallCard); // the one place the value lives (1.5 by the owner decision of 2026-10-26; 2.0 before)
CHECK(route_prior_default(false, false, 13.5 * GiB) == 0.0f);
CHECK(route_prior_default(false, false, 15.7 * GiB) == 0.0f); // 16 GB card: z440's A4000, P100
CHECK(route_prior_default(false, false, 72.0 * GiB) == 0.0f); // RTX PRO 5000
diff --git a/src/core/verify.cpp b/src/core/verify.cpp
index cdb20bef1..4d6519470 100644
--- a/src/core/verify.cpp
+++ b/src/core/verify.cpp
@@ -206,8 +206,10 @@ inline int g_side_mixer() {
#endif
}
// STRATA_ROUTE_PRIOR=lambda: residency-biased routing (the top-10 picked again with a logit bonus for the resident
-// experts; route_prior.cu). Unset: 2.0 on a DISCRETE NVIDIA card of 13 GiB or less (CUDA builds only), 0 elsewhere.
-// Measured on an RTX 3060 12 GB only (IQ3_XXS, 5 interleaved pairs: +37.6% / +35.2% decode); the owner approved the
+// experts; route_prior.cu). Unset: 1.5 (strata::kRoutePriorSmallCard, sm120_series.hpp; it was 2.0 until the owner decision of
+// 2026-10-26: defects in the copy and code check at 2.0, none in 19 tasks at 1.5) on a DISCRETE NVIDIA card of 13 GiB or less
+// (CUDA builds only), 0 elsewhere.
+// Measured on an RTX 3060 12 GB only (IQ3_XXS, 5 interleaved pairs at 2.0: +37.6% / +35.2% decode; at 1.5 story +15%, code +37%); the owner approved the
// quality trade for 12 GB cards (it changes which experts run). Not a default in a HIP build (a 12 GB Radeon, or a
// Strix Halo whose reported VRAM is a small carve-out of unified memory: unmeasured there) and not on an integrated GPU.
// STRATA_ROUTE_PRIOR=0 turns it off, any other value wins everywhere. The default is read from the device properties
@@ -2002,8 +2004,8 @@ bool Verifier::record_window(int T, cudaStream_t cs, std::string& err) {
if (!moe_route(wt, g, l, K, mb, mixed_ + t * N, cs, err, nullptr)) return false;
}
// STRATA_ROUTE_PRIOR=lambda (changes the output): the top-10 picked again with a logit bonus for the resident
- // experts; the weights stay the router's (route_prior.cu). The default is g_route_prior() above: 2.0 on a
- // discrete CUDA card of 13 GiB or less, off elsewhere (measured on an RTX 3060 12 GB: story +37.6%, code +35.2%;
+ // experts; the weights stay the router's (route_prior.cu). The default is g_route_prior() above: 1.5 on a
+ // discrete CUDA card of 13 GiB or less, off elsewhere (measured on an RTX 3060 12 GB at 2.0: story +37.6%, code +35.2%;
// on an RTX PRO 5000 72 GB -1.9% / -0.5%, on 2x RTX A4000 -4.2% / +11.0%, hence not a default on bigger cards).
const float route_prior = g_route_prior();
if (route_prior > 0.0f && NE == 512 && K == 10 && hits_.d_res != nullptr)