Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs(consolidate): correct the small-context budget examples and floor
yubr-zzh6ac6bb9bb201
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6bf29b1a..c71fd2d7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Fixed+- Fixed consolidation sending the model a prompt with no evidence when+ `consolidation.max_input_tokens` sat at the validated floor. The floor is+ meant to guarantee room for observations after the fixed system prompt, page+ conventions and structured-output schema, but it was checked against the raw+ token count while the builders run on a shrunken character budget+ (`max_input_tokens` times a chars-per-token estimate times+ `input_token_safety_margin`, default 0.8). At the old 6,500 floor the batch+ prompt had a 15,600-character budget against a 18,997-character fixed part,+ so every observation became an omission marker and the config was accepted+ anyway. The floor is now 8,350 tokens, derived from the real builders at the+ default margin, and the startup check compares the configured budget at its+ own margin with that requirement at the default, so lowering the margin+ demands a proportionally larger `max_input_tokens` instead of silently+ starving the prompt. An install configured between 6,500 and 8,350 tokens now+ fails at startup with a message naming both numbers, where it previously+ started and produced evidence-free consolidations. (#1234)- Replaced the unmaintained YAML library (`serde_yaml` and its`unsafe-libyaml` parser, RUSTSEC-2024-0449) with `serde-saphyr`. The serverreads existing page frontmatter to exactly the same values as before. Onediff --git a/crates/ai-memory-cli/src/config.rs b/crates/ai-memory-cli/src/config.rsindex 27e3632f..ab87e3de 100644--- a/crates/ai-memory-cli/src/config.rs+++ b/crates/ai-memory-cli/src/config.rs@@ -3973,6 +3973,33 @@ max_input_tokens = {tokens});}+ /// The small-context examples in `docs/install.md` and+ /// `docs/llm-providers.md` must be configs that actually load, so a doc+ /// example cannot go stale the way `max_input_tokens = 6500` did once the+ /// floor moved (#1234). These are the literal values those docs quote.+ #[test]+ fn documented_small_context_examples_load() {+ for (doc, tokens, output) in [+ ("docs/install.md", 10_000, 1_000),+ ("docs/llm-providers.md", 10_000, 1_000),+ ] {+ let tmp = TempDir::new().unwrap();+ let config_path = tmp.path().join("config.toml");+ std::fs::write(+ &config_path,+ format!(+ "[consolidation]+max_input_tokens = {tokens}+max_output_tokens = {output}+"+ ),+ )+ .unwrap();+ Config::load(Some(&config_path), Some(tmp.path().to_path_buf()))+ .unwrap_or_else(|error| panic!("{doc}'s example no longer loads: {error:#}"));+ }+ }+/// A small-context provider needs both sides of the context allocation to/// survive the config round-trip. The budget is derived from the floor/// rather than written out, so the test keeps meaning "the smallestdiff --git a/docs/install.md b/docs/install.mdindex 16670884..dd774fa5 100644--- a/docs/install.md+++ b/docs/install.md@@ -2309,7 +2309,7 @@ the real context window, with additional headroom for tokenizer variance:```bash# e.g. a model loaded with an 8k context window--e AI_MEMORY_CONSOLIDATION__MAX_INPUT_TOKENS=6500+-e AI_MEMORY_CONSOLIDATION__MAX_INPUT_TOKENS=10000-e AI_MEMORY_CONSOLIDATION__MAX_OUTPUT_TOKENS=1000```@@ -2320,9 +2320,12 @@ schema, and provider-envelope reserve. Tokenizers differ, so this is aconservative estimate rather than an exact provider token count. An automaticcheckpoint provider failure degrades to a rule-based page rather than losingthe checkpoint, but right-sized limits are what allow LLM consolidation to-succeed. Startup rejects input targets below 6,000 and output limits below-1,000 because the batch schema and a useful response cannot fit reliably below-those floors.+succeed. Startup rejects a budget that leaves too little room for observations+after the fixed prompts and the structured-output schema -- 8,350 input tokens+at the default `input_token_safety_margin` of 0.8 -- and rejects output limits+below 1,000, because the batch schema and a useful response cannot fit reliably+below those floors. Lowering the safety margin shrinks the character budget, so+a smaller margin needs a proportionally larger `max_input_tokens`.---diff --git a/docs/llm-providers.md b/docs/llm-providers.mdindex 2db90351..9f4b83d6 100644--- a/docs/llm-providers.md+++ b/docs/llm-providers.md@@ -357,7 +357,7 @@ headroom because provider tokenizers differ:```toml[consolidation]-max_input_tokens = 6500+max_input_tokens = 10000max_output_tokens = 1000```@@ -366,7 +366,9 @@ The equivalent environment variables are`AI_MEMORY_CONSOLIDATION__MAX_OUTPUT_TOKENS`. Provider failures during anautomatic PreCompact/PostCompaction checkpoint fall back to the deterministicrule-based page; admission, storage, and scope errors still fail closed. The-validated minimums are 6,500 input and 1,000 output tokens.+validated minimums are 8,350 input and 1,000 output tokens at the default+`input_token_safety_margin` of 0.8; a lower margin shrinks the character budget+and needs a proportionally larger input target.Every chat provider bounds each completion request at 300 seconds(`AI_MEMORY_LLM_TIMEOUT_SECS` to override, or `llm_timeout_secs = 900` in