Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
dsv41 cpu lane: AVX-512 rows chosen by the detected CPU (Zen 5 family 0x1A: IQ2_XXS and IQ3_XXS; Zen 4 0x19: IQ2_XXS; others none), prompt chunks of 7+ tokens only
02b07b6 Niko1221 committed 27 days ago
metal: the IQ2_S expert kernel, tiled - 515.7 GB/s effective, 89 % of the streaming ceiling
2cc55f6 Niko1221 committed 27 days ago
metal: the IQ2_XS expert kernel, tiled - 467.8 GB/s effective at 97 MB
446a45f Niko1221 committed 27 days ago
dsv41 iq (CUDA): persistent work-item grids only for codebooks of 8 KiB+ (IQ1_M/IQ1_S/IQ2_S), lazy codebook load, small-codebook types keep the release grids and kernels (no regression with few active experts)
a70b006 Niko1221 committed 27 days ago
metal: the tiled IQ2_XXS kernel - 437 GB/s effective in the engine's 8-token window, 5x the one-token form
1d21a0f Niko1221 committed 27 days ago
metal: the IQ2_XXS expert GEMV ported - 87.7 GB/s, and why it cannot go faster as a warp-per-row kernel
9c1e61a Niko1221 committed 27 days ago
metal: ggml's i-quant grid tables in MSL, generated from the pinned header
45faf57 Niko1221 committed 27 days ago
metal: the Q8_0 weight GEMV ported - 173 GB/s, and the reference has to use the file's half-rounded scale
680724b Niko1221 committed 27 days ago
dsv41 cpu lane: AVX-512 rows by default for IQ2_XXS prompt chunks (7+ tokens), measured on Zen 4
995ff61 Niko1221 committed 27 days ago
metal: the BF16 GEMV ported and measured - 83 GB/s warp-per-row, and why the expert path cannot stop there
a445418 Niko1221 committed 27 days ago
metal: the verified dsv41 activation quantiser, in the engine's kernel tree
baf261d Niko1221 committed 27 days ago
metal: the first dsv41 kernel ported and bit-exact - the Q8 activation quantiser
6bfcfa7 Niko1221 committed 27 days ago
metal: a launch with no Metal dispatch names its own file and line
ca903c1 Niko1221 committed 27 days ago
Merge catch-up: f-dsv41-win fd19f9d6b, f-dsv41-h100 c0f944e12, f-dsv41-cuk
428950b Niko1221 committed 27 days ago
dsv41 iq (CUDA): light sets (DSpark verify, short prompts) take the persistent grid and the IQ1_M code table for experts of <= 4 rows, 4-block occupancy kept for 8+ rows; test: --bench-light
0553ba6 Niko1221 committed 27 days ago
metal: the last twelve CUDA names (dim3, funnel shift, transcendentals, ldg/ldcv, shfl_up) - no undeclared name left
aaf3aa3 Niko1221 committed 27 days ago
metal: finish the CUDA API surface the token path names, enumerated by the compiler
dc61a06 Niko1221 committed 27 days ago
metal: fp8 conversion modes, the round-to-nearest float helpers, and an mma.h placeholder
e46b149 Niko1221 committed 27 days ago
metal: the half-precision and vector-type surface (cuda_fp16.h, vector_types.h, cuda_fp8.h)
18c6075 Niko1221 committed 27 days ago
metal: the dsv41 host side compiles - ds4_model.cu goes from 20 errors to 0
a798d03 Niko1221 committed 27 days ago
metal: ds4_os.hpp takes the macOS shim at the top of the file, not inside the Windows block
bb60de0 Niko1221 committed 27 days ago
metal: a macOS stand-in for the Linux OS surface ds4_os.hpp uses (half done - not yet effective)
910af91 Niko1221 committed 27 days ago