Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docs: INTEL.md section for the fused int8 expert path (numbers, quality, crossover)
Niko12216aa8e25f4929
diff --git a/docs/INTEL.md b/docs/INTEL.mdindex f987a5780..421539d98 100644--- a/docs/INTEL.md+++ b/docs/INTEL.md@@ -899,3 +899,26 @@ The lent slots are refilled after the prompt, so the bigger chunk costs the decodifference 1.0 (the chunk boundary moves where the QSA block selection is made, so a different chunk size is not bit-identical;two runs of the same size are, KL 0). With the 32K context (own prompt buffers, no borrowing) a bigger chunk is +20% on a 16Kprompt (1,236 -> 1,478-1,482) but takes 1,397 cache slots (10,926 -> 9,529) and decode after it falls 9-16%, so 4,096 stays there.++## Arc Pro B70: the fused int8 expert path for the prompt (2026-10-18, branch f-151-b70, opt-in)++`STRATA_PREFILL_MMQ=1` (or `auto`) computes the routed experts of the prompt with ESIMD kernels that decode the quantized weights in+registers straight into the s8 DPAS B operand (no FP16 copy of the expert, 2.1 MB of weights read instead of 9.8 MB written and read+back) and multiply them by q8_1-quantized activations; `sycl/src/prefill/esimd_moe.dp.cpp`, behaviour note and switches in+[INTEL_INT8_EXPERTS.md](INTEL_INT8_EXPERTS.md). Gate/up IQ3_S, IQ3_XXS, IQ2_S and down IQ4_NL, Q2_0 are covered (47 of the 48 layers of+the mixed IQ3_S pack; the other layer, IQ4_XS gate/up, and any other format keep the FP16 path). Off by default: the activations are+quantized, so the bits differ from the FP16 path. `auto` times both chains on this GPU at the first layer of each format and takes int8+below the measured crossover (130-250 rows per expert here), per group of experts.++Arc Pro B70, IQ3_S pack, 32K config, `--prefill 4096`, fixed flags (`STRATA_PREFILL_CPU_SHARE=0 STRATA_ROUTE_TAIL_SKIP=0 --pcie-frac 0+--adapt-every 0`), 5 interleaved rounds per size, prefill tok/s (medians; the noise arm is the default run again):++| prompt | default | noise arm | `=1` | `=auto` |+|---|---|---|---|---|+| 1,024 tokens | 645.8 | 648.3 | 775.4 (+20%) | 783.5 (+21%) |+| 4,095 tokens | 1,257.0 | 1,256.8 | 1,413.8 (+12.5%) | 1,407.7 (+12.0%) |++The output tokens are identical in all runs. Quality (teacher-forced KL against the FP16 path over 6,002 positions of 28 prompts): mean+2e-4 nats, p99 6e-3, top-1 99.7%, perplexity ratio 1.0004; a `--prefill 8192` chunk moves the first-token distribution by 1e-2 to 3e-2+for comparison. At more than ~130 rows per expert (a chunk of 8,192 tokens or more) the int8 chain stops winning because every row block+decodes the weights again (1.03x at 194 rows against 1.4-1.9x at 60): use `auto`.