Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(consolidate): soften the link quota that pushes models to invent links (#1219)
AkitaOnRailsccf31a89122b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 84390c7e..092b52ce 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]### Fixed+- Softened the consolidation prompts' wikilink quota that pushed models to+ invent links. The single and batch prompts told the model "2-5 well-chosen+ links per page beat exhaustive linking; zero links should be rare", which+ pressured it to add links just to hit the number. They now state that zero+ links is fine when nothing in the input genuinely relates, require a genuine+ reason visible in the input for every link, and forbid padding a page with+ links to hit a number; the #1207 restriction on cross-scope links whose+ target is not visible in the input is unchanged. The validated+ `consolidation.max_input_tokens` floor moved from 6,000 to 6,500 tokens so+ the larger prompts still leave room for observation evidence at the minimum.+ (#1219)- Fixed consolidation inventing `[[_global:…]]` and `[[project:…]]` wikilinks topages that never existed, which `ai-memory lint` then reported as`broken_link`. The single, batch and bootstrap prompts told the model that adiff --git a/crates/ai-memory-cli/templates/config.default.toml b/crates/ai-memory-cli/templates/config.default.tomlindex b367110e..bb1ce097 100644--- a/crates/ai-memory-cli/templates/config.default.toml+++ b/crates/ai-memory-cli/templates/config.default.toml@@ -176,7 +176,7 @@ per_user = false# LLM consolidation prompt limits. These are approximate because tokenizers# differ by provider and model. Their sum must fit the model's context window;# leave additional headroom for tokenizer variance. Small local models must-# lower both values together. Startup minimums are 6000 input and 1000 output.+# lower both values together. Startup minimums are 6500 input and 1000 output.# Consolidation turns a finished session's raw observations into wiki# pages. Off (zero-LLM) you still get session pages + search; with an# LLM you get distilled concept/gotcha/rule pages. Reach for these knobsdiff --git a/crates/ai-memory-consolidate/prompts/batch_consolidate_system.md b/crates/ai-memory-consolidate/prompts/batch_consolidate_system.mdindex 9a0a3931..b2999fd8 100644--- a/crates/ai-memory-consolidate/prompts/batch_consolidate_system.md+++ b/crates/ai-memory-consolidate/prompts/batch_consolidate_system.md@@ -84,9 +84,13 @@ input — reference it inline with a wikilink:A missing same-project target is fine: it is recorded as a pendinglink and resolves when the page appears. A missing `project:` or-`_global:` target is reported by lint as broken. 2-5 well-chosen-links per page beat exhaustive linking; zero links should be rare,-but never pad with a cross-scope link you cannot see in the input.+`_global:` target is reported by lint as broken. A few well-chosen+links beat exhaustive linking, and zero links is fine when nothing+in the input genuinely relates. Link only where the input gives a+genuine reason — a page named in the input, one you are emitting+in this reply, or a same-project page the observations clearly+relate to. Never pad a page with links to hit a number, and+never pad with a cross-scope link you cannot see in the input.## OUTPUT LANGUAGEdiff --git a/crates/ai-memory-consolidate/prompts/single_consolidate_system.md b/crates/ai-memory-consolidate/prompts/single_consolidate_system.mdindex fb9d902d..075e005d 100644--- a/crates/ai-memory-consolidate/prompts/single_consolidate_system.md+++ b/crates/ai-memory-consolidate/prompts/single_consolidate_system.md@@ -80,9 +80,13 @@ input — reference it inline with a wikilink:A missing same-project target is fine: it is recorded as a pendinglink and resolves when the page appears. A missing `project:` or-`_global:` target is reported by lint as broken. 2-5 well-chosen-links per page beat exhaustive linking; zero links should be rare,-but never pad with a cross-scope link you cannot see in the input.+`_global:` target is reported by lint as broken. A few well-chosen+links beat exhaustive linking, and zero links is fine when nothing+in the input genuinely relates. Link only where the input gives a+genuine reason — a page named in the input, one you are emitting+in this reply, or a same-project page the observations clearly+relate to. Never pad a page with links to hit a number, and+never pad with a cross-scope link you cannot see in the input.## OUTPUT LANGUAGEdiff --git a/crates/ai-memory-consolidate/src/consolidator.rs b/crates/ai-memory-consolidate/src/consolidator.rsindex c76089ea..503f57d6 100644--- a/crates/ai-memory-consolidate/src/consolidator.rs+++ b/crates/ai-memory-consolidate/src/consolidator.rs@@ -1376,7 +1376,7 @@ const MIN_OBSERVATION_RESERVE_CHARS: usize = 1_024;/// Smallest input budget that still leaves room for observations after/// the fixed prompts and structured-output schema. Below this the batch prompt/// can leave too little room for observations.-pub const MIN_CONSOLIDATION_MAX_INPUT_TOKENS: usize = 6_000;+pub const MIN_CONSOLIDATION_MAX_INPUT_TOKENS: usize = 6_500;/// Smallest useful structured-output allowance. Lower values are unlikely to/// fit even one concise batch update and its JSON framing.@@ -2974,6 +2974,42 @@ mod tests {}}+ /// #1219: a numeric quota ("2-5 links per page, zero links should be+ /// rare") pressures the model to invent links to hit the number. The+ /// prompts must state that zero links is a fine outcome, that links+ /// need a genuine reason visible in the input, and that padding to+ /// hit a number is never acceptable — while keeping the #1217+ /// cross-scope restriction intact.+ #[test]+ fn consolidation_system_prompts_do_not_pressure_invented_links() {+ for (name, prompt) in [("single", SYSTEM_PROMPT), ("batch", BATCH_SYSTEM_PROMPT)] {+ assert!(+ prompt.contains("zero links is fine"),+ "{name} prompt must permit a linkless page"+ );+ assert!(+ prompt.contains("genuine reason"),+ "{name} prompt must require a genuine reason to link"+ );+ assert!(+ prompt.contains("Never pad a page with links to hit a number"),+ "{name} prompt must forbid quota padding"+ );+ assert!(+ prompt.contains("never pad with a cross-scope link"),+ "{name} prompt must keep the #1217 cross-scope restriction"+ );+ assert!(+ !prompt.contains("zero links should be rare"),+ "{name} prompt must not stigmatise a linkless page"+ );+ assert!(+ !prompt.contains("2-5"),+ "{name} prompt must not carry a numeric link quota"+ );+ }+ }+#[test]fn build_request_elides_raw_observations_from_current_body() {let raw_dump = (0..2_000)diff --git a/docs/llm-providers.md b/docs/llm-providers.mdindex 9ce1964f..2db90351 100644--- a/docs/llm-providers.md+++ b/docs/llm-providers.md@@ -366,7 +366,7 @@ The equivalent environment variables are`AI_MEMORY_CONSOLIDATION__MAX_OUTPUT_TOKENS`. Provider failures during anautomatic PreCompact/PostCompaction checkpoint fall back to the deterministicrule-based page; admission, storage, and scope errors still fail closed. The-validated minimums are 6,000 input and 1,000 output tokens.+validated minimums are 6,500 input and 1,000 output tokens.Every chat provider bounds each completion request at 300 seconds(`AI_MEMORY_LLM_TIMEOUT_SECS` to override, or `llm_timeout_secs = 900` in