Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

[Engine] SYCL: prompt attention takes the lane-cell score phase, +5% on a prompt read

Niko1221949f77c0a80b
diff --git a/sycl/src/kernels/cuda/qsa_decode_attn.dp.cpp b/sycl/src/kernels/cuda/qsa_decode_attn.dp.cpp
index 24ea37ad9..9caef220d 100644
--- a/sycl/src/kernels/cuda/qsa_decode_attn.dp.cpp
+++ b/sycl/src/kernels/cuda/qsa_decode_attn.dp.cpp
@@ -665,8 +665,15 @@ void qsa_decode_attn_batch(const float* q, const QsaAttnPools& pools, const int3
const dpct::dim3 grid((unsigned)n_chunks, (unsigned)s.n_head_kv,
(unsigned)n_q);
dpct::queue_ptr st = strata::q_of(stream);
- // S25: STRATA_ATTN_LANECELL=1 - the score phase one cell per thread (bit-identical scores, no shuffle reductions)
- static const bool lane_cell = [] { const char* v = std::getenv("STRATA_ATTN_LANECELL"); return v && v[0] == '1'; }();
+ // S25: the score phase one cell per thread (bit-identical scores, no shuffle reductions). Prompt batches take
+ // it by default: on 2x Arc Pro B65 it reads a 4,106-token prompt 5.2% faster, 8,191 tokens 5.2% and 16,380
+ // tokens 4.8%, with the same tokens; the decode window's few queries per call are 1.4% slower with it (56.6-57.0
+ // against 55.9-56.3 tok/s), so the choice follows the query count. STRATA_ATTN_LANECELL=1/0 forces either way.
+ static const int lane_cell_env = [] {
+ const char* v = std::getenv("STRATA_ATTN_LANECELL");
+ return v != nullptr && v[0] != '\0' ? std::atoi(v) : -1;
+ }();
+ const bool lane_cell = lane_cell_env >= 0 ? lane_cell_env != 0 : n_q >= 16;
#define STRATA_ATTN_LC(M) \
{ \
auto exp_props = sycl::ext::oneapi::experimental::properties{ \