Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
sycl: int8 prompt experts default to auto on Battlemage (bmg-g21 / bmg-g31 by architecture id), FP16 elsewhere; STRATA_PREFILL_MMQ=0 off, =1 forced; launch failure or missing kernels fall back to the FP16 chain with one log line; load-time log line; mode-choice unit test; docs
c765c5f Niko1221 committed 1 month ago
docs: B70 follow-up measurements (GDN variant, XMX attention, attention layout, KqSplit, group size); static_assert on the group size
983fa18 Niko1221 committed 1 month ago
sycl: QSA prompt attention: probabilities in [cell][head] order with vector reads in the value phase (default, bit-identical, +0.9% prompt on the B70); STRATA_ATTN_RS=1 reduce-scatter scores measured -1.5%
5bcc208 Niko1221 committed 1 month ago
tests: iq_multi_parity under STRATA_KQ_SPLIT=1 also without V2K, with the fused gate/up SwiGLU, and with the sub-16 gate/up rows off
0e58a65 Niko1221 committed 1 month ago
pt: s3158 without the word 'demora' (the no-demo catalog test matches 'demo' inside it)
88c1502 Niko1221 committed 1 month ago
gfx12: the WMMA prompt attention and the Q2_0 fused prompt experts are default-on on Linux only; Windows HIP builds keep them opt-in until tested there
97b680e Niko1221 committed 1 month ago
Prefill planner: request-sized chunks. A prompt that fits today's chunk keeps one chunk; a longer one reads in equal chunks of at most the plan's size (no stub tail). Buffers stay sized for today's chunk.
8253477 Niko1221 committed 1 month ago
docs: INTEL.md section for the fused int8 expert path (numbers, quality, crossover)
6aa8e25 Niko1221 committed 1 month ago
Merge f-148-amd (2053388dc, without the IQ2 fused default) into f-split-int2
af3770c Niko1221 committed 1 month ago
docs: --kv-grow on HIP with numbers, STRATA_PREFILL_HELP on two R9700
d907018 Niko1221 committed 1 month ago
NVIDIA sm_86 and sm_60: the measured bitwise decode switches on by default per architecture (nv_defaults; STRATA_SM86_PERF=0 / STRATA_SM60_PERF=0 turn them off)
2725c23 Niko1221 committed 1 month ago
HIP: virtual memory ranges (hipMemAddressReserve/Create/Map/SetAccess, 2 MiB chunks) so --kv-grow can run on ROCm 6+ (STRATA_HIP_VMM=0 turns it off)
3980a5c Niko1221 committed 1 month ago
149: STRATA_COLD_OVERLAP=4 (zero-copy without the overlap); no residency query for RAM-resident experts
2044864 Niko1221 committed 1 month ago
gfx12: the 96-slot prompt ring follows the matrix-core attention's real state (STRATA_HIP_WMMA=1/auto on a card that has it, not failed)
961d37c Niko1221 committed 1 month ago
docs: AMD_HIP.md - the gfx12 matrix-core prompt attention and the Q2_0 fused prompt experts as defaults, with the 148-amd quality and speed numbers
2053388 Niko1221 committed 1 month ago
sycl: int8 DPAS expert products for IQ3_XXS and IQ2_S gate/up and Q2_0 down too (every layer type of the IQ3_S pack but one), thread tile NT=1 with MT 2/4/8, parity test over all six type pairs; STRATA_PREFILL_MMQ=1 prompts +10% (4K), +16% (2K), +20% (1K) with identical tokens
ebdfaac Niko1221 committed 1 month ago
docs: STRATA_HIP_WMMA=auto, the first-prompt warm-up, the gfx1201 hipBLASLt version note; timing: the first chunk's PLE wait on the ple line
4908dda Niko1221 committed 1 month ago
Prefill planner: refit the miss share to two rigs (#2824 SATA 2x3060, P2003 NVMe cgroup sweep); read the drive class from the GGUF path
d847105 Niko1221 committed 1 month ago
sycl: fused int8 DPAS expert products for the prompt path (ESIMD, Battlemage): IQ3_S gate/up + IQ4_NL down, int8 q8_1 activations, parity test, opt-in STRATA_PREFILL_MMQ=1; first correct version, not yet faster than dequantize + oneMKL
2856a80 Niko1221 committed 1 month ago
149: io prefetch scheduling, RAM_ADAPT2, cold-read overlap and zero-copy GGUF experts (all opt-in)
c85cc6d Niko1221 committed 1 month ago
prefill timing: the hyper-connection write-back on its own line (it was charged to the phase before it, e.g. the wait for the first chunk's PLE rows); hip_prefill_gr_write_bench
aef32e2 Niko1221 committed 1 month ago
setup: say why the page waits while the bundled engine unpacks (8 languages)
83aa59f Niko1221 committed 1 month ago