Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

KVA: say why a configured run fell back to the full path (the observability criterion: an operator must be able to tell a configured-and-declined run from one that never asked)

Niko1221053a41570c6c
diff --git a/docs/KVA.md b/docs/KVA.md
index 3e84dca30..e6c6fbf57 100644
--- a/docs/KVA.md
+++ b/docs/KVA.md
@@ -41,6 +41,9 @@ Requirements and limits for this first version:
- **The session state after a KVA read is approximate.** It must not be reused as an exact prefix: no
checkpoint of it, no follow-up prompt that reuses it. The engine prints a warning when this happens.
- The approximation is a quality tradeoff, not a lossless transformation. Measure before using it.
+- Every run says what it did:
+ - applied: `strata prefill: KVA on: split 24, suffix 2112, approximated 5120 of 7533 positions in 5 of 8 chunks, mode identity`
+ - configured but not applied: `strata prefill: KVA configured but NOT applied: <reason> (prompt … positions, min_prompt …, suffix …, split …)`
## Measured (Tesla P100 16 GB, Qwen3.8-Flash-Next IQ3_XXS, chunk 1024, expert cache 1500, MTP window 2048)
diff --git a/src/prefill/prefill.cpp b/src/prefill/prefill.cpp
index 286b215c6..f0b09e558 100644
--- a/src/prefill/prefill.cpp
+++ b/src/prefill/prefill.cpp
@@ -5279,6 +5279,18 @@ bool Prefill::run_impl(const int64_t* tokens, int64_t n, int64_t pos0, std::stri
if (kva_positions > 0)
std::fprintf(stderr, "strata prefill: KVA: the session state holds approximated cells; it is not an exact "
"prefix and must not be reused as one (no checkpoint, no follow-up reuse)\n");
+ } else if (kva.on) {
+ // KVA-8 (observability): a run that was configured for KVA but fell back completely must say so and why,
+ // or an operator cannot tell it apart from a run where the feature was never asked for.
+ const char* why = pos0 != 0 ? "a continuation read (pos0 != 0)"
+ : (LB != 0 || (LE >= 0 && LE != g.n_layers)) ? "a layer split (not the whole trunk)"
+ : kva.capture ? "capture mode"
+ : n < kva.min_prompt ? "the prompt is shorter than STRATA_KVA_MIN_PROMPT"
+ : n <= kva.suffix + 64 ? "the suffix leaves too few positions to approximate"
+ : "not applicable";
+ std::fprintf(stderr, "strata prefill: KVA configured but NOT applied: %s (prompt %lld positions, "
+ "min_prompt %lld, suffix %lld, split %lld)\n",
+ why, (long long) n, (long long) kva.min_prompt, (long long) kva.suffix, (long long) kva_split);
}
if (std::getenv("STRATA_STATE_HASH_GDN") != nullptr) { // debug: the GDN states as the prompt path leaves them
cudaStreamSynchronize(m.cs);