Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
KVA: say why a configured run fell back to the full path (the observability criterion: an operator must be able to tell a configured-and-declined run from one that never asked)
Niko1221053a41570c6c
diff --git a/docs/KVA.md b/docs/KVA.mdindex 3e84dca30..e6c6fbf57 100644--- a/docs/KVA.md+++ b/docs/KVA.md@@ -41,6 +41,9 @@ Requirements and limits for this first version:- **The session state after a KVA read is approximate.** It must not be reused as an exact prefix: nocheckpoint of it, no follow-up prompt that reuses it. The engine prints a warning when this happens.- The approximation is a quality tradeoff, not a lossless transformation. Measure before using it.+- Every run says what it did:+ - applied: `strata prefill: KVA on: split 24, suffix 2112, approximated 5120 of 7533 positions in 5 of 8 chunks, mode identity`+ - configured but not applied: `strata prefill: KVA configured but NOT applied: <reason> (prompt … positions, min_prompt …, suffix …, split …)`## Measured (Tesla P100 16 GB, Qwen3.8-Flash-Next IQ3_XXS, chunk 1024, expert cache 1500, MTP window 2048)diff --git a/src/prefill/prefill.cpp b/src/prefill/prefill.cppindex 286b215c6..f0b09e558 100644--- a/src/prefill/prefill.cpp+++ b/src/prefill/prefill.cpp@@ -5279,6 +5279,18 @@ bool Prefill::run_impl(const int64_t* tokens, int64_t n, int64_t pos0, std::striif (kva_positions > 0)std::fprintf(stderr, "strata prefill: KVA: the session state holds approximated cells; it is not an exact ""prefix and must not be reused as one (no checkpoint, no follow-up reuse)\n");+ } else if (kva.on) {+ // KVA-8 (observability): a run that was configured for KVA but fell back completely must say so and why,+ // or an operator cannot tell it apart from a run where the feature was never asked for.+ const char* why = pos0 != 0 ? "a continuation read (pos0 != 0)"+ : (LB != 0 || (LE >= 0 && LE != g.n_layers)) ? "a layer split (not the whole trunk)"+ : kva.capture ? "capture mode"+ : n < kva.min_prompt ? "the prompt is shorter than STRATA_KVA_MIN_PROMPT"+ : n <= kva.suffix + 64 ? "the suffix leaves too few positions to approximate"+ : "not applicable";+ std::fprintf(stderr, "strata prefill: KVA configured but NOT applied: %s (prompt %lld positions, "+ "min_prompt %lld, suffix %lld, split %lld)\n",+ why, (long long) n, (long long) kva.min_prompt, (long long) kva.suffix, (long long) kva_split);}if (std::getenv("STRATA_STATE_HASH_GDN") != nullptr) { // debug: the GDN states as the prompt path leaves themcudaStreamSynchronize(m.cs);