Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Commits on main
Merge f-022-gpuroles: Main / Helper / Off per graphics card in the app, TUI, launch, setup, server and Docker
e28889d Niko1221 committed 1 month ago
GPU roles: two identical cards recommend the highest number as Main, so the engine order stays [0, 1]; a model that keeps one card keeps setup's old pick; golden back to [0, 1] with roles only
890fee8 Niko1221 committed 1 month ago
Metal: Gyro dense K-quant GEMV tiles (+5% decode, 5/5 pairs, same output), GYRO.md supports the Mac
5e633b4 Niko1221 committed 1 month ago
Server: the card roles (and a stop when every card is Off) come before anything starts
7c11a19 Niko1221 committed 1 month ago
K5/K4 version 2: computed (LUT-free) codebook for TQK5X/TQK4X, point-table versions kept as TQK5T/TQK4T; CPU decoder tables per type; tests
a7c50df Niko1221 committed 1 month ago
serve: engine_ready_s in the run config (DeepSeek-V4.1-Flash reads 300 GB at start; setup writes 3600)
a4c9336 Niko1221 committed 1 month ago
dsv41: DS4_PROF events per device (invalid resource handle on several GPUs), exl3_legacy.cu in CUDA builds (reported by the CUDA lab)
2189884 Niko1221 committed 1 month ago
SYCL Gyro: trellis packs take the link share from a 12 GB/s reference (A750: 0.29 -> 0.48, decode 13.7 -> 15.8 tok/s, 5/5 pairs); Gyro memory warnings on an Arc read the real RAM
8bf77c0 Niko1221 committed 1 month ago
Thread names: the Windows half lives in pool.cpp, not in a header every engine file includes
8c41d36 Niko1221 committed 1 month ago
serve, launch: "pool_cores" in the run config and strata launch --pool-cores
2aa1ec2 Niko1221 committed 1 month ago
CPU pool: say where the workers run, --pool-cores (experimental), names for the engine's threads
7a55995 Niko1221 committed 1 month ago
Merge main (0.2.1.3 Terminal update); docs keep the 0.2.2 text
82a9725 Niko1221 committed 1 month ago
Merge main (0.2.1.2 engine hotfix) into the 0.2.1.3 Terminal update
cbfd5b9 Niko1221 committed 1 month ago
strata launch: --gpu-roles summary line in the plan and before the start; tests for the flag, the mirror and installed roles
301ab7c Niko1221 committed 1 month ago
Docs: Main, Helper, Off in MULTI_GPU.md (the main card is the last stage), Docker GPU_ROLES, wording fixes
0fce78c Niko1221 committed 1 month ago
App: GPU roles in the generic Run screen (two or more cards), preview fixture can name its cards
095e524 Niko1221 committed 1 month ago
Gyro trellis gate/up on compute capability 6.0: the group's activations in shared memory for the block's row chunks
88278af Niko1221 committed 1 month ago
Gyro trellis experts on compute capability 6.0: row chunks per block so the staged codebook is paid once per several rows; bench: rows of different lengths
c864d80 Niko1221 committed 1 month ago
Gyro trellis experts on compute capability 6.0: read-only loads for the block words, TQK6 needs three words of the stream
51f418f Niko1221 committed 1 month ago
Gyro trellis experts on compute capability 6.0: the int8 codebook staged in shared memory, the TQ down rows as 8-lane rows (bitwise the same sums)
9f38ee3 Niko1221 committed 1 month ago
tq_pascal_bench: the Gyro trellis decode kernels on synthetic blocks (bitwise vs the per-column kernels, output checksums, --bench)
b2f2149 Niko1221 committed 1 month ago
App: GPU roles section (Main / Helper / Off per card) replaces the card option list and the expert Run on picker; host writes roles plus the legacy mirror; 46 strings in 8 locales
ed61ad2 Niko1221 committed 1 month ago
SYCL Gyro: decode-once multi-entry trellis kernels, the grouped XMX prompt path takes the trellis types (prompt 40 -> 82 tok/s on the A750), tq_sycl_parity --bench; setup keeps Gyro's n-gram table on the SSD on i915
2411e7c Niko1221 committed 1 month ago
Merge f-dsv41-cudalab (trellis GEMV lab, tqenc) into f-dsv41-port
d9220bd Niko1221 committed 1 month ago