Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
amd: open two GPU queues instead of five at start (keeps the fixed prompt speed, no decode loss on R9700)
Niko1221b35a9816c798
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.mdindex 64ffc5fa2..9aec79030 100644--- a/docs/AMD_HIP.md+++ b/docs/AMD_HIP.md@@ -525,7 +525,7 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39path has two busy queues, the compute stream and the stream that copies experts to the card (about 31,000 copies andevent hand-offs per 4K prompt). The runtime opens its queues in the order the loader threads reach them, which changesevery start; when both busy queues land on the same compute pipe of the card (the card has two for this process), every- hand-off waits for a queue switch. The engine now opens five queues (four at normal priority, one at the highest) one+ hand-off waits for a queue switch. The engine now opens two queues at normal priority (more queues cost decode: five cost 2 to 3.6%) oneafter the other on the main thread before any other thread starts (`STRATA_HIP_QUEUE_PRIME=0` turns it off; any othervalue is a list of `n` / `h` letters for tests). gfx1200 and gfx1201 on Linux only; nothing else changes (the answersare identical). Not clocks, not memory placement (read speed over the expert cache was 637-640 GB/s in both kinds ofdiff --git a/src/program/generate.cpp b/src/program/generate.cppindex 702819851..711bbc663 100644--- a/src/program/generate.cpp+++ b/src/program/generate.cpp@@ -2220,7 +2220,9 @@ static double ple_direct_rows_per_s(strata::kernels::PleTable& t) {// switch queues, and a 4K prompt reads 13% slower (p620, 1,980 vs 2,290 tok/s). Opening the queues here, one after// the other on the main thread and before any other thread exists, makes the placement the same in every start (and// it is the fast one). Streams are destroyed again; the runtime keeps the queues.-// STRATA_HIP_QUEUE_PRIME unset: gfx1200 / gfx1201 on Linux only, "nnnnh"; "0": off; any other string is a spec+// Two queues are enough (R9700, "nn": 0 slow of 32 fresh starts, unprimed 10 of 32); five (nnnnh) fixed the starts too but cost+// decode 2 to 3.6% and two expert slots, four normal ones did not even fix the starts.+// STRATA_HIP_QUEUE_PRIME unset: gfx1200 / gfx1201 on Linux only, "nn"; "0": off; any other string is a spec// ('n' = a stream at the default priority, 'h' = at the highest) for experiments on other cards.static void prime_hw_queues() {const char* env = std::getenv("STRATA_HIP_QUEUE_PRIME");@@ -2239,7 +2241,7 @@ static void prime_hw_queues() {(void) cudaGetLastError();return;}- spec = "nnnnh";+ spec = "nn";#endif}if (dev != 0 && cudaSetDevice(dev) != cudaSuccess) { (void) cudaGetLastError(); return; }