Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default)
d8f58af Niko1221 committed 1 month ago
verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it)
dcba249 Niko1221 committed 1 month ago
tools/dflash2: q8test, the drafter with Q8_0 matrices keeps its acceptance
5f426c3 Niko1221 committed 1 month ago
tools/dflash2: the torch port of the DFlash2 drafter and the scripts of the acceptance experiment (research tooling)
1282238 Niko1221 committed 1 month ago
generate: STRATA_SPEC_T_CYCLE, a bench-only cycle of the verify window size with a per-width cost line (off by default)
17c9b92 Niko1221 committed 1 month ago
verify: STRATA_TAP_FILE, an env-gated dump of the hidden states at chosen layers per verified row (DFlash2 acceptance experiment; off by default, no new graph nodes without it)
d1dec64 Niko1221 committed 1 month ago
README: RX 6000 cards get their own engine package in 0.2.2 (no installer of their own; built, not tested on a real card)
011c9e6 Niko1221 committed 1 month ago
dsv41: pack_gguf --tq-layers (a subset of the converted layers)
2355265 Niko1221 committed 1 month ago
Fix #3425: the memory tiers count RAM + VRAM less 5 GB that Windows and the system keep (setup.py and tools/strata_mcp.py, one constant each; the app and strata launch read the engine's pick; the Mac keeps its own macOS share)
76f90e5 Niko1221 committed 1 month ago
Gyro-M text: 16-21% slower than Q2_0 everywhere (setup's model list, strata_mcp, GYRO.md intro), as README and MODELS.md say
328a3f5 Niko1221 committed 1 month ago
Merge f-022-int (68864c564) into f-022-gyrocuda: release notes keep both sides
1df3a31 Niko1221 committed 1 month ago
Gyro docs: the 0.2.2 decode numbers on the pinned files (RTX 3060, 5 pairs, merged tree against f-022-int)
daa654b Niko1221 committed 1 month ago
Merge f-022-specfix: first-window HIP verify hang (PLE flag before the first launch of each window graph) + 1-token prompt starts the speculative loop
68864c5 Niko1221 committed 1 month ago
docs/GYRO.md: Gyro-M was 24% slower than Q2_0 before 0.2.2 (the measured table says 24)
2c27afe Niko1221 committed 1 month ago
Version bump for 0.2.2 (proposed): app 0.2.2, engine and Control 0.1.47
1c9bfd2 Niko1221 committed 1 month ago
Fix the gfx906 query-swizzle host guard test: the dispatch expression now spans two lines and reads kernel
be4e24d Niko1221 committed 1 month ago
docs: release notes 0.2.2 cover every merged stream (GPU roles, Gyro on Arc/Mac/NVIDIA/P100, models folder, NUMA, RX 6000, API key, own files, speed definition, fixes), what 0.2.1.1-0.2.1.4 already shipped, known limits
3a7a695 Niko1221 committed 1 month ago
docs: translated READMEs give Gyro-M's decode gap to Q2_0 as 16-21% (RTX 3060 12 GB: 16%, R9700: 21%), like the English README
4f864bc Niko1221 committed 1 month ago
Merge f-022-gyrocuda: Gyro trellis kernels faster on NVIDIA (8-lane rows on every card above Pascal): Gyro-S +17%, Gyro-M +11% on an RTX 3060, identical answers
2c8eb88 Niko1221 committed 1 month ago
Merge f-022-engfix: #3465 NaN retry reads the prompt batched, #3441/#3075 tool-call envelopes, #3503 cuda12 package keeps Pascal/Volta, #3480 queue priming respects GPU_MAX_HW_QUEUES
e0cfb80 Niko1221 committed 1 month ago
verify (HIP): the first launch of each window graph reads the PLE rows and raises the PLE flag before cudaGraphLaunch, so the launch cannot wait for a GPU that waits for the flag (first-window hang on gfx1151; STRATA_VERIFY_EARLY_PLE_FLAG=0/1/2)
86d10ed Niko1221 committed 1 month ago
dsv41: trellis gate/up fused in one pass sharing the activation fragments (token path 454 -> ~355 us a layer, outputs unchanged: ds4_tq_test ALL OK); optional 32-bit stream loads (no gain)
1c23888 Niko1221 committed 1 month ago
Merge f-022-int into f-022-gyrocuda: Pascal path kept for cc 6.0, 8-lane trellis rows for every other CUDA card
01f8c32 Niko1221 committed 1 month ago
Fix #3465: after non-finite logits in a prompt window the retry reads the whole prompt batched
894f133 Niko1221 committed 1 month ago
suffix drafter: randomized soundness test (small vocabularies, tiny tables, propose_after), clean under ASan/UBSan
8a4106a Niko1221 committed 1 month ago
generate: spec_loop_runs() names the speculative loop's start rule (+ test); STRATA_SPEC_WATCH=<s>, a stall watchdog with the stage names for the CLI's verify loop (stall report and thread dump, off by default)
4c3edf4 Niko1221 committed 1 month ago
Fix #3480: the HIP queue priming stands aside when GPU_MAX_HW_QUEUES is 1 or 2
f05725c Niko1221 committed 1 month ago
dsv41: tq_convert resumes (skips finished layers), pack_gguf --tq-experts takes several folders
e8cb1cb Niko1221 committed 1 month ago
Test: torch has one pin per Python range since #3474; compare the package set
8de1ec4 Niko1221 committed 1 month ago
Merge f-022-issues2: #3488 #3502 #3490 #3507 #3424 #3501 #3504 #3425 #3469 #3437 #3510
3fbcea2 Niko1221 committed 1 month ago