Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
STRATA_LOGITS_DUMP: a post-window logits tap for KL quality measurement (off by default); the sampler itself runs inside a CUDA graph capture, so the copy lives in the verify window's host-side epilogue where STRATA_DBG_NAN already copies from
Niko122145c9e623e235
diff --git a/docs/KVA.md b/docs/KVA.mdindex e6c6fbf57..556f71974 100644--- a/docs/KVA.md+++ b/docs/KVA.md@@ -31,6 +31,7 @@ STRATA_KVA_MIN_PROMPT=4096 \| `STRATA_KVA_CAPTURE_FP32` | off | write fp32 instead of bf16 records || `STRATA_KVA_ORACLE` | — | directory: replay a capture instead of a projector || `STRATA_KVA_GDN_NORM` | off | keep the full GDN recurrence (the A/B control for the state-only one) |+| `STRATA_LOGITS_DUMP` | — | debug (not KVA-specific): write every verify window's sampler logits to `path`, for KL / top-1 between two runs |Requirements and limits for this first version:@@ -51,13 +52,17 @@ Requirements and limits for this first version:|---:|---:|---:|---:|---:|---:|| 5,642 | 203.3 | 265.7 | 1.31× | 54 % | 0.357 || 14,581 | 202.5 | 311.2 | 1.54× | 85 % | not measured |-| 29,997 | 198.7 | 326.5 | 1.64× | 92 % | 0.451 |+| 29,997 | 199.7 | 338.3 | 1.69× | 92 % | 0.451 |- Decode throughput was unchanged within the run-to-run spread (24–28 tok/s) in every arm: KVA changes promptprocessing only.- Quality: on a 14.6K-token prompt with a fact inserted at 10/50/90 %, the KVA arm retrieved the fact at allthree positions and produced the identical answer string. On a "summarise this documentation" prompt theanswers stayed coherent and semantically identical, with different wording after 4–12 tokens.+- **Likelihood** (`STRATA_LOGITS_DUMP`): at the first generated token the approximation moves the next-token+ distribution by KL 0.018 nats at a 5.6K prompt (two control runs differ by 0.0013) and 0.020 nats at 30K+ (control floor 0.0032). The top-1 token is the same and 9 of the top 10 are shared in both cases. The effect+ does not grow with context even though the residual error does.- Layer type matters more than the count: approximating only the 6 late QSA layers cost a 0.153 residual errorat 1.08×; approximating 18 GDN layers cost 0.339 at 1.18×. A gated recurrence integrates its input over thewhole prefix, so a wrong input keeps biasing the state, while a wrong QSA K/V only misplaces attention keys.diff --git a/src/core/verify.cpp b/src/core/verify.cppindex 3d6f48b2e..b82181c25 100644--- a/src/core/verify.cpp+++ b/src/core/verify.cpp@@ -3198,6 +3198,34 @@ bool Verifier::run(int T, const int32_t* tokens, int64_t pos0, PoolMultiFn pool,for (int t = 0; t < T; ++t) out[t] = ((volatile int32_t*) h_out_)[t];if (tap_file_ != nullptr && !batch_rec_) tap_write(T, tokens, pos0);note_nonfinite(T, nullptr);+ // STRATA_LOGITS_DUMP=<path> (debug tap, off by default): the window's logits as the sampler saw them, written+ // AFTER the window (the stream is synchronised above and the graph capture is over, so a plain copy is legal -+ // the same place STRATA_DBG_NAN copies from). One record is+ // {int32 magic, int32 T, int32 n_vocab, int32 record, int64 position}, then T * n_vocab FP32 rows; row 0 is+ // the window's first position (the last prompt token of a prefill), which is what a KL comparison wants.+ if (const char* logits_path = std::getenv("STRATA_LOGITS_DUMP")) {+ static std::FILE* lf = std::fopen(logits_path, "wb");+ static int32_t lrec = 0;+ if (lf != nullptr) {+ std::vector<float> h((size_t) T * (size_t) n_vocab_);+ const cudaError_t le = cudaMemcpy(h.data(), head_logits_, h.size() * sizeof(float), cudaMemcpyDeviceToHost);+ if (le == cudaSuccess) {+ const int32_t hdr[4] = {0x53474F4C /*"LOGS"*/, T, (int32_t) n_vocab_, lrec++};+ const int64_t hpos = pos0;+ std::fwrite(hdr, sizeof hdr, 1, lf);+ std::fwrite(&hpos, sizeof hpos, 1, lf);+ std::fwrite(h.data(), sizeof(float), h.size(), lf);+ std::fflush(lf);+ } else {+ static bool said = false;+ if (!said) {+ said = true;+ std::fprintf(stderr, "strata verify: STRATA_LOGITS_DUMP copy failed: %s\n", cudaGetErrorString(le));+ (void) cudaGetLastError();+ }+ }+ }+ }if (static const bool dbg = std::getenv("STRATA_DBG_NAN") != nullptr; dbg) { // debug: the first non-finite headstatic bool reported = false;if (!reported) {