Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
KVA ships OFF in 0.2.7: STRATA_KVA unset is the exact 0.2.6 engine, STRATA_KVA=1 turns it on with an 8192-token exact tail
e3450e4 Niko1221 committed 24 days ago
App: "Faster long prompts" (KVA) is Off by default in the model's Expert settings; On writes STRATA_KVA=1, Off removes it
9760cbc Niko1221 committed 24 days ago
Terminal and the shared speed-switch table: Faster long prompts (Qwen KVA) is OFF by default, STRATA_KVA=1 turns it on
ae03ef6 Niko1221 committed 24 days ago
KVA: STRATA_KVA_SKIP=moe (opt-in) - the late layers of an approximated chunk run their whole attention half and skip only the MoE half
3171291 Niko1221 committed 24 days ago
DeepSeek engine (AMD): the matrix-core attention on by default for the chunked prompt
fd81df6 Niko1221 committed 24 days ago
docs: setup is expected to choose --ple-io ram on a RAM-rich Linux PC - say to check setup --plan rather than asserting it
879b958 Niko1221 committed 24 days ago
prompt streaming: the staging ring sizes from the VRAM room (400 -> 1200 MB on Strix Halo) and the attention slab defaults to 32 MB - 4K prompt 27.3 -> 21.0 s
67cfaf2 Niko1221 committed 24 days ago
Merge f-027-win (ctest registers ple_residency_test with the GPU-build tests)
bffb41e Niko1221 committed 24 days ago
ple_residency_test is registered with the GPU-build tests (it sat in the conversation-tests block, which is off by default, so ctest never ran it)
3e2f45e Niko1221 committed 24 days ago
docs: the 12 GB card's prompt figures are withdrawn - a five-pair repeat read +254% and -20.5% where the first read +35% and +17%, and that box's prompt swings 218-466 tok/s between arms
1ee6060 Niko1221 committed 24 days ago
dense GEMM: the chunked prompt's gemm() prefers the engine's own RDNA3 WMMA kernel (src/prefill/wmma_gemm.cu, now linked) over hipBLAS - 4.97 vs 20-40 TFLOPS at our shapes
cca3dd2 Niko1221 committed 24 days ago
Merge f-027-ctx (DeepSeek-V4.1-Flash: 512K and 1M contexts in setup, the Terminal and the app up to the pack's own limit, with a 'not tested yet' note past 256K; the app now honours a chosen DeepSeek context; idea from Project Maya v1.1.28 #442 by @needmorevram)
4329c0d Niko1221 committed 24 days ago
Tests: nobody gets a bigger DeepSeek context than before unless they choose it (8K stays the default in setup, the Terminal and the app; an installed model shows and loads the context it was prepared with)
b8946f4 Niko1221 committed 24 days ago
App: DeepSeek's context goes up to the pack's limit (512K, 1M), with a plain "not tested yet" note past 256K, in eight languages (Project Maya #442, @needmorevram)
c4c320a Niko1221 committed 24 days ago
DeepSeek-V4.1-Flash: 512K and 1M contexts in setup and the Terminal, as Project Maya v1.1.28 did (#442 by @needmorevram, MIT)
33780a4 Niko1221 committed 24 days ago
docs: the agent scenario - two conversations nearly double throughput
11b5a77 Niko1221 committed 24 days ago
docs: the routing default is +67% with two conversations in flight
f9907b6 Niko1221 committed 24 days ago
KVA: the draft layer is untouched by default (window 32768, cells built for approximated chunks, no floor); the cell floor is STRATA_KVA_DRAFT_FLOOR=1
6c00851 Niko1221 committed 24 days ago
Merge f-027-win (DeepSeek: the chunked prompt's attention slab is automatic, up to 2048 rows only where the VRAM would not hold an expert; DS4_ATTN_SLAB=0 = the 0.2.6 rule)
2043b68 Niko1221 committed 24 days ago
docs: the release notes carry the two-conversation batching gain
e66d275 Niko1221 committed 24 days ago
dense dequant: the chunked prompt's dq_rows_kernel packs each sub-block's 32 halves into four 16-byte stores and stages the warp's input through contiguous uint4 loads decoded at a relative index - the same values (bit-identical), 4K prefill 79.5 -> 88.52 tok/s, acceptance 97.4%
0305616 Niko1221 committed 24 days ago
KVA: leave the draft window alone; a per-session cell floor keeps the draft layer off approximated cells
7a34c60 Niko1221 committed 24 days ago