Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
metal: the dsv41 core's CUDA surface, part one (bf16/fp8 conversions, the cuBLAS names, the compile audit)
bcb2133 Niko1221 committed 27 days ago
Merge f-dsv41-cuk into f-dsv41-rel (per-device light-set scratch and tensor-core check, faster IQ1_M/IQ2_S/IQ3_XXS token kernels)
d689e60 Niko1221 committed 27 days ago
dsv41 cpu lane: take the AVX2 rows by ggml-cpu's own AVX2/FMA/F16C answer (MSVC builds have no __builtin_cpu_supports)
cf4f8be Niko1221 committed 27 days ago
dsv41 iq (CUDA): per-device light-set scratch and tensor-core check (stream 0 is one handle on every device: the scratch of the first GPU was reused on the second), block/sub-block loop form in the token kernels; test: --devices (in-process multi-GPU, bit-compared)
1cace5c Niko1221 committed 27 days ago
dsv41 cpu lane: exact AVX2 rows for 7..64+ tokens an expert (the prompt path): decode once, u8 activations, no per-token sign; IQ1_M too
7e9b0dc Niko1221 committed 27 days ago
dsv41 win: --dspark auto (not loaded when the experts do not fit the GPU), --ids @file, h100sim
0b48e23 Niko1221 committed 27 days ago
dsv41 mac: the CPU-only build works on arm64 (x86 kernels guarded, the AVX2 row entry points answer no)
f0f056b Niko1221 committed 27 days ago
dsv41: chunked prompt for a single card (--prefill-chunk N), the expert cache learns from chunks and catches up after a prompt, chunk buffers fall back by halving, CLI feeds a whole prompt
f6e4cd6 Niko1221 committed 27 days ago
dsv41: DS4_PRED_DUMP / DS4_XIN_DUMP (router logits + projected router input; router inputs for the expert-similarity scan)
c0f944e Niko1221 committed 27 days ago
dsv41 cpu lane: the exact AVX2 rows on by default from 2 tokens (1 on IQ3_S, 3 on IQ1_M), 5-6 tokens in one pass, IQ1_M rows rewritten
6aba354 Niko1221 committed 27 days ago
dsv41 iq (CUDA): persistent token kernels (codebook loaded once a block) and IQ1_M decode through a 4096-entry code table (delta sign folded in): bit-identical, IQ1_M layer -15% sm_120 / -30% sm_86
71abb6c Niko1221 committed 27 days ago
dsv41 cpu lane: AVX2 multi-token i-quant rows in ggml's float order (bit-identical to ggml-cpu at 1..8 tokens), IQ1_M rows
cf45533 Niko1221 committed 27 days ago
dsv41 iq test: --bench-big (>=640 MiB working set, rotated experts: DRAM rates), --dump/--cmp (bit-exact golden compare incl. real shapes and grouped path)
95994c4 Niko1221 committed 27 days ago
dsv41 rel: one IQ1_M implementation in ds4_iq.cu (the textual merge kept both; port's kernels stay), win's dequant_rows (chunked-prompt dense dequant) kept and takes IQ1_S too; strix add4 dropped with win's IQ1_M decode (port's decode has no __vadd4)
ea081e3 Niko1221 committed 27 days ago
Merge f-dsv41-strix into f-dsv41-rel (HIP build fixes: portable add4 for the IQ1_M decode, strata_core link for the DXGI budget, HIP build scripts, umaprobe)
8cfe988 Niko1221 committed 27 days ago
Merge f-dsv41-h100 into f-dsv41-rel (--profile-run, DS4_TRACE on the chunked path, --spec-eval)
e5c70bb Niko1221 committed 27 days ago
dsv41 win: store test covers cancel and shrink; shrink finalizes read slots
f232cbf Niko1221 committed 27 days ago
dsv41 win: priority SSD queue with cancellation (demand before guesses), partial GPU visibility, store shrink on RAM pressure, predsim
08224f2 Niko1221 committed 27 days ago
dsv41 win: store init last, zc off by default, flow reads the store, release notes
c8ead78 Niko1221 committed 27 days ago
dsv41: --spec-eval and DS4_ROUND_LOG (DSpark acceptance per round, every draft verified)
55e1a53 Niko1221 committed 27 days ago
dsv41: DS4_TRACE also records the chunked prompt path's picks
8da4276 Niko1221 committed 27 days ago
dsv41 win: store auto size, profile preload, decode staging zc_tab (T=1), release plumbing
2e61fef Niko1221 committed 27 days ago
dsv41: --profile-run mode (feed id segments without logits, expert profile saved as it goes)
9051f31 Niko1221 committed 27 days ago
dsv41 strix: umaprobe (UMA budget, hipMalloc reach, host registration, zero-copy read bandwidth), build script
17bc654 Niko1221 committed 27 days ago