Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
Release notes 0.2.2: Gyro on a Tesla P100
Niko1221025558167775
diff --git a/docs/release-notes-0.2.2.md b/docs/release-notes-0.2.2.mdindex 15a55b096..fb5fce8fd 100644--- a/docs/release-notes-0.2.2.md+++ b/docs/release-notes-0.2.2.md@@ -17,3 +17,4 @@- **Docker image build.** The terminal stage no longer needs git in the image (it failed with `FileNotFoundError`); the commit comes in as a build argument.- **The app and the Terminal share one models folder.** Each of the two settings files now remembers when the models folder was chosen; the one changed last wins and the other follows. Nothing is moved. A folder picked in the Terminal was not seen by the app before, so the same model could be downloaded twice.- **Docker.** An old (0.1.x) volume with `/data/models`, `/data/packs` and `/data/mtp` is moved into `/data/Strata-data` by rename instead of being downloaded and packed again (it used to start a 70 GB download). A card without `--gpus` stops at once with the `--gpus all` hint instead of after minutes of setup. A missing or empty `MODEL_PATH` gets a plain message. Settings that name another image's component folder (a CUDA 12 image on a CUDA 13 volume) are made again. Tested on one RTX 3060 PC; no AMD or Intel image and no Docker Desktop.+- **Gyro decodes about 30 % faster on a Tesla P100 (source build for compute capability 6.0 only; the ready-made CUDA 13 packages have no code for that card).** On a P100, Gyro-S ran at about 70 % of the speed of the Q2_0 file of similar size: 30 / 36 / 33 tok/s against 41 / 51 / 48 (story / code / facts, median of 5 interleaved pairs, default flags). On compute capability 6.0 the engine now keeps the 4 KB trellis codebook in shared memory, reads the down rows as 8-lane rows, lets each block work on several row chunks, and copies the activations of an expert group to shared memory once per block instead of reading them from global memory for every row. Gyro-S now decodes 38.5 / 45.6 / 42.7 tok/s (+30 %; every one of 10 reps and 5 of 5 pairs faster), 0.96 / 0.90 / 0.91 of Q2_0. Prompt speed is unchanged: 4096 tokens take 14.0 s and 20000 tokens 62 s, 3 to 5 % slower than Q2_0. Answers are the same: the identity gate (10 prompts, 4096- and 20000-token prompt dumps) is byte-identical on the P100 and on an RTX 3060, and Q2_0 on the P100 did not change (0.999 / 0.998 / 1.001). Other cards run the old kernels, because the new path is picked by compute capability 6.0 at run time; `STRATA_TQ_STAGE=0` turns it off. Two attempts to speed up the prompt (a faster dp4a in llama.cpp's MMQ kernels, which this card lacks, and a register-tiled MMQ dot) were slower and are not included.