Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

docs: INTEL.md section for the fused int8 expert path (numbers, quality, crossover)

Niko12216aa8e25f4929
diff --git a/docs/INTEL.md b/docs/INTEL.md
index f987a5780..421539d98 100644
--- a/docs/INTEL.md
+++ b/docs/INTEL.md
@@ -899,3 +899,26 @@ The lent slots are refilled after the prompt, so the bigger chunk costs the deco
difference 1.0 (the chunk boundary moves where the QSA block selection is made, so a different chunk size is not bit-identical;
two runs of the same size are, KL 0). With the 32K context (own prompt buffers, no borrowing) a bigger chunk is +20% on a 16K
prompt (1,236 -> 1,478-1,482) but takes 1,397 cache slots (10,926 -> 9,529) and decode after it falls 9-16%, so 4,096 stays there.
+
+## Arc Pro B70: the fused int8 expert path for the prompt (2026-10-18, branch f-151-b70, opt-in)
+
+`STRATA_PREFILL_MMQ=1` (or `auto`) computes the routed experts of the prompt with ESIMD kernels that decode the quantized weights in
+registers straight into the s8 DPAS B operand (no FP16 copy of the expert, 2.1 MB of weights read instead of 9.8 MB written and read
+back) and multiply them by q8_1-quantized activations; `sycl/src/prefill/esimd_moe.dp.cpp`, behaviour note and switches in
+[INTEL_INT8_EXPERTS.md](INTEL_INT8_EXPERTS.md). Gate/up IQ3_S, IQ3_XXS, IQ2_S and down IQ4_NL, Q2_0 are covered (47 of the 48 layers of
+the mixed IQ3_S pack; the other layer, IQ4_XS gate/up, and any other format keep the FP16 path). Off by default: the activations are
+quantized, so the bits differ from the FP16 path. `auto` times both chains on this GPU at the first layer of each format and takes int8
+below the measured crossover (130-250 rows per expert here), per group of experts.
+
+Arc Pro B70, IQ3_S pack, 32K config, `--prefill 4096`, fixed flags (`STRATA_PREFILL_CPU_SHARE=0 STRATA_ROUTE_TAIL_SKIP=0 --pcie-frac 0
+--adapt-every 0`), 5 interleaved rounds per size, prefill tok/s (medians; the noise arm is the default run again):
+
+| prompt | default | noise arm | `=1` | `=auto` |
+|---|---|---|---|---|
+| 1,024 tokens | 645.8 | 648.3 | 775.4 (+20%) | 783.5 (+21%) |
+| 4,095 tokens | 1,257.0 | 1,256.8 | 1,413.8 (+12.5%) | 1,407.7 (+12.0%) |
+
+The output tokens are identical in all runs. Quality (teacher-forced KL against the FP16 path over 6,002 positions of 28 prompts): mean
+2e-4 nats, p99 6e-3, top-1 99.7%, perplexity ratio 1.0004; a `--prefill 8192` chunk moves the first-token distribution by 1e-2 to 3e-2
+for comparison. At more than ~130 rows per expert (a chunk of 8,192 tokens or more) the int8 chain stops winning because every row block
+decodes the weights again (1.03x at 194 rows against 1.4-1.9x at 60): use `auto`.