Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
C++
docs: --prompt-cache-every does not set the size of the pieces a prompt is read in (#3311, #2834)
gopinath87607fa128f91e8ad
diff --git a/docs/DETAILS.md b/docs/DETAILS.mdindex 27d0fdf3a..0e765106c 100644--- a/docs/DETAILS.md+++ b/docs/DETAILS.md@@ -621,7 +621,7 @@ system prompt with a billing header that changes on every request; the server pi| Option | Effect || --- | --- || `--prompt-cache N` | Number of checkpoints (`0` = off, which also disables parking). |-| `--prompt-cache-every N`, `--prompt-cache-root N`, `--turn-token ID` | Checkpoint interval, the system-prompt checkpoint (`0` = none) and the turn token. |+| `--prompt-cache-every N`, `--prompt-cache-root N`, `--turn-token ID` | Checkpoint interval, the system-prompt checkpoint (`0` = none) and the turn token. The interval is not a piece-size setting: [what it does](PROMPT_CACHE_TAIL.md#prompt-cache-every-does-not-set-the-size-of-the-pieces-a-prompt-is-read-in). || `--prompt-cache-spread` (opt-in) | The checkpoint that leaves is the one whose removal leaves the smallest gap, so the kept points thin out geometrically back from the end. An agent that rewrites its history a few thousand tokens back then resumes near the edit instead of at the system prompt. Same RAM. || `"strata_checkpoint": false` (per request) | A one-shot request that no later request continues saves no checkpoints and keeps no session. || `"strata_prefix": {"messages": 1}` (per request, opt-in) | One long document, many questions. Marks where the document ends: `{"messages": 1}`, `{"message": 0, "chars": 210000}` or `{"tokens": 12345}`. The checkpoint at the boundary is pinned, so 20 questions cost one read of the document. One pinned prefix at a time per engine; needs `--prompt-cache 3` or more; a malformed field is a 400. See [RESEARCH_RUNS.md](RESEARCH_RUNS.md). |diff --git a/docs/PROMPT_CACHE_TAIL.md b/docs/PROMPT_CACHE_TAIL.mdindex 836e3b9ed..8e1d3cf04 100644--- a/docs/PROMPT_CACHE_TAIL.md+++ b/docs/PROMPT_CACHE_TAIL.md@@ -31,3 +31,14 @@ This is a separate conservative alternative to the Strata-2080Ti Daily'scheckpoint-grid patch. The grid changes chunk boundaries to reach exactcheckpoint intervals and failed its local paired numerical gate. This optiondoes not claim that the original grid patch has been fixed or validated.++## `--prompt-cache-every` does not set the size of the pieces a prompt is read in++The option can look like a piece-size dial, because a mark every N tokens sounds like a cut every N tokens. In this engine it is not one: a periodic+mark is saved at a chunk boundary of the batched prefill and the read carries on, so no piece ends at it, and changing N does not change how the prompt+is split into runs. What cuts a read into runs is the list of places where a read has to stop and start again: a re-read (`STRATA_CKPT_REREAD`),+the system-prompt root (`--prompt-cache-root`), a message boundary (`STRATA_CACHE_MESSAGE_BOUNDARY`, see+[MESSAGE_BOUNDARY_CACHE.md](MESSAGE_BOUNDARY_CACHE.md)) and the start of the last turn. So the lever for a fresh long read is those cuts, not this cadence.++What `--prompt-cache-every 0` does: it skips the periodic marks and the near-tail one (`--prompt-cache-tail` needs `--prompt-cache-every` above 0), so a client+that rewrote a turn before the last one mounts a farther mark and reads more of it again. The root mark is controlled by `--prompt-cache-root`.