Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
DeepSeek engine (AMD): the small prompt chunk whenever the cards hold most of the experts, 4 cards included
88c2931 Niko1221 committed 24 days ago
DeepSeek engine (AMD): the drafter's cumulative confidence cut on by default (0.5) on HIP
e83b82a Niko1221 committed 24 days ago
DeepSeek engine (AMD): pin the CPU lane's RAM copy when the cards hold >= half the experts (DS4_COPY_PIN auto)
2247e5a Niko1221 committed 25 days ago
DeepSeek engine (AMD, diagnostic): the io threads' busy time in the prompt stream line
5b969c1 Niko1221 committed 25 days ago
DeepSeek engine (AMD, diagnostic): where copy_ahead's time goes in the prompt stream line (DS4_PROF only)
3cc132e Niko1221 committed 25 days ago
DeepSeek engine (AMD): the staging ring from each device's room (not the summed room)
3b23994 Niko1221 committed 25 days ago
DeepSeek engine (AMD): a staging ring sized from the VRAM room, and expert-tile knobs for the MarLin work
1e72cdc Niko1221 committed 25 days ago
setup: a 16,384-token prompt chunk on a unified-memory PC whose pool holds ~128 GB (16K prompt 152.2 -> 158.6 tok/s, 32K 174.0 -> 179.7, 8K flat, decode unchanged); smaller unified PCs keep 8,192
81ac4dd Niko1221 committed 25 days ago
docs: the release notes carry the residency-biased routing default
44b88bc Niko1221 committed 25 days ago
hc: the one-kernel persistent hyper-connection read builds and runs on CUDA
2395c8a Niko1221 committed 25 days ago
route: residency-biased routing is on by default at 2.0 on a card of 13 GiB or less
c30656a Niko1221 committed 25 days ago
docs: the miss split is opt-in, not a default: the text can differ (4 of 6 fixed prompts within 64 greedy tokens) even though the first-token KL is 0.0013
2465b93 Niko1221 committed 25 days ago
Revert "gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55"
1d00c01 Niko1221 committed 25 days ago
Merge f-027-async: synchronous verify window on Strix Halo (+31% code decode)
1f00c18 Niko1221 committed 25 days ago
Strix Halo: run the verify window synchronously (DS4_ASYNC_T=4 in the config env), +31% code decode
63b8a8d Niko1221 committed 25 days ago
STRATA_TRACE: one clock for the whole process, so a layer split's stages compare
3a26753 Niko1221 committed 25 days ago
STRATA_TRACE: the prompt chunk line carries its stage range and elapsed ms
14d076e Niko1221 committed 25 days ago
docs: the miss split is the gfx12 start value now (measured A/B on the shipped path: 8K +9.0%, 32K +23.6% decode)
13c4aeb Niko1221 committed 25 days ago
gfx12: native packs start the decode miss split at pcie_frac 0.15 (R9700: +9.9% decode at 8K, +25.6% at 32K, +10.4% under serve, first-token KL 0.0013 vs 0.55 with top-1 12/12); explicit --pcie-frac wins, STRATA_PCIE_FRAC_DEFAULT=old restores 0.55
d7ad4a2 Niko1221 committed 25 days ago
docs: RDNA4 decode miss split measured on the R9700 (--pcie-frac 0.15: +9.9% at 8K, +25.6% at 32K, first-token KL 0.0013, top-1 12/12)
7901bc5 Niko1221 committed 25 days ago
STRATA_TRACE: print elapsed milliseconds on each request step
91d4d7f Niko1221 committed 25 days ago