Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

amd: open two GPU queues instead of five at start (keeps the fixed prompt speed, no decode loss on R9700)

Niko1221b35a9816c798
diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md
index 64ffc5fa2..9aec79030 100644
--- a/docs/AMD_HIP.md
+++ b/docs/AMD_HIP.md
@@ -525,7 +525,7 @@ an RX 9070 XT 16 GB and a Radeon AI PRO R9700 32 GB (both gfx1201), a Ryzen 9 39
path has two busy queues, the compute stream and the stream that copies experts to the card (about 31,000 copies and
event hand-offs per 4K prompt). The runtime opens its queues in the order the loader threads reach them, which changes
every start; when both busy queues land on the same compute pipe of the card (the card has two for this process), every
- hand-off waits for a queue switch. The engine now opens five queues (four at normal priority, one at the highest) one
+ hand-off waits for a queue switch. The engine now opens two queues at normal priority (more queues cost decode: five cost 2 to 3.6%) one
after the other on the main thread before any other thread starts (`STRATA_HIP_QUEUE_PRIME=0` turns it off; any other
value is a list of `n` / `h` letters for tests). gfx1200 and gfx1201 on Linux only; nothing else changes (the answers
are identical). Not clocks, not memory placement (read speed over the expert cache was 637-640 GB/s in both kinds of
diff --git a/src/program/generate.cpp b/src/program/generate.cpp
index 702819851..711bbc663 100644
--- a/src/program/generate.cpp
+++ b/src/program/generate.cpp
@@ -2220,7 +2220,9 @@ static double ple_direct_rows_per_s(strata::kernels::PleTable& t) {
// switch queues, and a 4K prompt reads 13% slower (p620, 1,980 vs 2,290 tok/s). Opening the queues here, one after
// the other on the main thread and before any other thread exists, makes the placement the same in every start (and
// it is the fast one). Streams are destroyed again; the runtime keeps the queues.
-// STRATA_HIP_QUEUE_PRIME unset: gfx1200 / gfx1201 on Linux only, "nnnnh"; "0": off; any other string is a spec
+// Two queues are enough (R9700, "nn": 0 slow of 32 fresh starts, unprimed 10 of 32); five (nnnnh) fixed the starts too but cost
+// decode 2 to 3.6% and two expert slots, four normal ones did not even fix the starts.
+// STRATA_HIP_QUEUE_PRIME unset: gfx1200 / gfx1201 on Linux only, "nn"; "0": off; any other string is a spec
// ('n' = a stream at the default priority, 'h' = at the highest) for experiments on other cards.
static void prime_hw_queues() {
const char* env = std::getenv("STRATA_HIP_QUEUE_PRIME");
@@ -2239,7 +2241,7 @@ static void prime_hw_queues() {
(void) cudaGetLastError();
return;
}
- spec = "nnnnh";
+ spec = "nn";
#endif
}
if (dev != 0 && cudaSetDevice(dev) != cudaSuccess) { (void) cudaGetLastError(); return; }