Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: GR_V3 and PROMPT_ATTN_IMMA defaults in DETAILS.md and the 0.2.7 release-notes draft

Niko1221407f65976341
diff --git a/docs/DETAILS.md b/docs/DETAILS.md
index afd8290a7..e5c23659f 100644
--- a/docs/DETAILS.md
+++ b/docs/DETAILS.md
@@ -919,7 +919,8 @@ measured. "Exact" means the 10-prompt and 4K/20K identity gate with the switch o
| `STRATA_B6_MMVQ_ROWS=1` | Exact multi-column MMVQ reusing the activation fragment across rows | Exact; on by default on sm_86 and sm_60 |
| `STRATA_QSA_SPLIT_ATTN=1` | Decode attention split across blocks on the int8 mma path (sm_80+, int8 KV) | Rounding level |
| `STRATA_HC_UPMIX=1` | Hyper-connection up projection and `gr_mix_r` in one kernel (sm_80+) | Exact |
-| `STRATA_PROMPT_ATTN_IMMA=1` | Int8-KV prompt attention on INT8 tensor cores | Rounding level |
+| `STRATA_PROMPT_ATTN_IMMA` | Int8-KV prompt attention on INT8 tensor cores | Rounding level. On by default since 0.2.7 on NVIDIA Ampere and newer (+1.1% to +1.8% prompt at 8K/16K on an RTX PRO 5000 and 2x RTX A4000); `=0` brings the 0.2.6 kernel back. |
+| `STRATA_GR_V3` | The hyper-connection read split over two kernels | Rounding level. On by default since 0.2.7 on a discrete compute capability 12.0 card (RTX 50, RTX PRO Blackwell) with more than 13 GiB (RTX PRO 5000: +1.2% decode with identical tokens); slower on sm_86, so off there. `=0` / `=1` forces it off / on. |
| `STRATA_HC_REQ8=1` | Hyper-connection projections as int8 plus a scale per 32 (more expert-cache slots) | Rounding level |
| `STRATA_QUANT_GATHER=1`, `STRATA_PREFILL_GROUP_GATHER=0` | Per-row gather for the MoE input quantization; the short-prompt grouped gather off | No gain measured; keep the defaults |
diff --git a/docs/release-notes-0.2.7.md b/docs/release-notes-0.2.7.md
index 8166d70db..de0295223 100644
--- a/docs/release-notes-0.2.7.md
+++ b/docs/release-notes-0.2.7.md
@@ -16,6 +16,14 @@
- Off switch: `STRATA_GR_UP_WIDE=0`. `STRATA_SM120_PERF=0` still turns off the whole Blackwell set.
- Other cards are untouched: the old kernel runs everywhere else.
+- **Two more kernel switches are on by default where they were measured.** Both sum in another order (about 1e-7), so a
+ tie between two near-equal words can flip and a reply can differ from 0.2.6 here and there.
+ - The prompt attention on int8 tensor cores (NVIDIA Ampere and newer): **+1.6% / +1.8% prompt at 8K / 16K on the RTX PRO 5000
+ and +1.1% / +1.3% on 2x RTX A4000**, decode unchanged, five pairs each. Off switch: `STRATA_PROMPT_ATTN_IMMA=0`.
+ - The hyper-connection read split over two kernels (a discrete RTX 50 / RTX PRO Blackwell card with more than 13 GiB): **+1.2%
+ decode on the RTX PRO 5000 with identical tokens**, about +1.5% together with the switch above. On 2x RTX A4000 and the
+ RTX 3060 it was slower, so those cards keep the old read. Off switch: `STRATA_GR_V3=0`.
+
## What is faster if your PC is set up for it
These need no engine change; they are settings. Each was measured on the same boxes.