Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
layer-major prefill: a prompt that needs a run_impl feature the walk lacks takes the chunk-major path (#3976)
a732a34 Niko1221 committed 24 days ago
Test: app-ui 'Quit works once' waits for the shutdown callback (polling, up to 5 s) instead of a fixed 20 ms sleep, so a busy PC cannot fail it (test only)
0acba45 Niko1221 committed 24 days ago
layer-major prefill on a split: a stale hand_in_ left by a chunk-major prompt no longer turns layer-major off (#3976)
e7318fa Norihiro Nishimura committed 24 days ago
layer-major prefill: even super-chunks (STRATA_PREFILL_LM_EVEN, default on with layer-major) (#3976)
18c5357 Norihiro Nishimura committed 24 days ago
layer-major prefill on a split: the PLE layer is fine in a first stage that holds layers 0 and 1 (#3976)
c6f6db8 Norihiro Nishimura committed 24 days ago
layer-major prefill on a CUDA layer split: each stage runs its layers over the super-chunk and hands its rows to the next stage (#3976)
6eff377 Norihiro Nishimura committed 24 days ago
layer-major prefill: no MMQ row table without an MMQ layer; QSA shapes from core::qsa_shapes (#3976)
cc125e0 Norihiro Nishimura committed 24 days ago
layer-major prefill: the FP16 path needs no MMQ identity table or token upload buffer (#3976)
5602253 Norihiro Nishimura committed 24 days ago
layer-major prefill: the FP16 expert path for layers MMQ does not take, decline reasons name the layers (#3976)
42094da Norihiro Nishimura committed 24 days ago
layer-major prefill: STRATA_HC_Q8 dequant, guards for APR/NVFP4/WMMA layers, QSA indexer calls of this tree (#3976)
df7068b Norihiro Nishimura committed 24 days ago
prefill: layer-major prefill (STRATA_PREFILL_LAYER_MAJOR=1) (#3976, #1670)
ffe40c1 vizdatom committed 24 days ago
prefill: kernels for a layer-major expert pass (moe_shared_init, moe_scatter_add) (#3976, #1670)
a6df002 vizdatom committed 24 days ago
prefill: the per-layer walk warms this layer's and the next layer's file-tier experts (STRATA_PREFILL_WARM=1) (#3976, #1670)
162dec3 vizdatom committed 24 days ago
prefill stager: a job waits for the previous job on its buffer; skip() (#3976, #1670)
fca1936 vizdatom committed 24 days ago
Merge f-027-gh-eng-s (#3291 #3955 #3317 #4036 #2893 #3218 #3200: serve, tools and setup ports)
8486336 Niko1221 committed 24 days ago
Web chat: preview a generated svg or html block in a sandboxed frame (#1628)
0aa64b2 Niko1221 committed 24 days ago
Merge f-027-gh-eng-k2 (#2526 #1864 #1911 #4005 #3060 #4010 #3809: PR ports for the conversation cache, prefill events, expert source and batch groups)
675f7fc Niko1221 committed 24 days ago
prefill: opt-in fused int8 prompt experts for the native Q2_0 pack on sm_80+, STRATA_PF_FUSED_Q2N=1 (#3809)
c47c699 chaog992 committed 24 days ago
Web chat: draw math with the bundled KaTeX, opt-in and offline (#2972, #2974)
a965c96 Niko1221 committed 24 days ago
Web page: a translation mechanism, and a language choice in About > Settings
d5cca31 Niko1221 committed 24 days ago
docs: --prompt-cache-every does not set the size of the pieces a prompt is read in (#3311, #2834)
fa128f9 gopinath87607 committed 24 days ago
strata launch --show-key shows the key of a model setup.py set up (#3570)
47c98bb Niko1221 committed 24 days ago
docker: model files left in the older /data/models stop the start instead of being downloaded again (#3421)
10a62b8 Niko1221 committed 24 days ago
Split serving, first version: GEN 0 prefill=1 in the engine, serve/split.py (SplitEngine, config split_serve / --split-serve), tests
690172c Niko1221 committed 24 days ago
DeepSeek engine packages: add sm_89 (RTX 40) to the CUDA builds (#3935)
4a759f8 Niko1221 committed 24 days ago
docs/MULTI_GPU.md: say when a layer split pays on a slow CPU (#3930)
ab48216 Norwin committed 24 days ago