Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(bootstrap): leave headroom for the token estimate (#937)
Felipe Neves Ricardo0d8dc7cc5af2
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex adae0176..e86743a6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Fixed+- `ai-memory bootstrap` now leaves headroom for its own token estimate. It+ counts bytes ÷ 4, which undercounts non-English text and source code (about+ 40% on Portuguese mixed with code, as measured for consolidation), and it+ filled `--max-input-tokens` and `--chunk-input-tokens` to the last estimated+ token, so a chunk sized to fit a model's window could overflow it on input+ alone. Prunes and chunks now fill 80% of each budget by the estimate, the+ same default consolidation uses; a run may plan more chunks than before.+ (#937)+## [2.4.1] - 2026-09-25### Changeddiff --git a/crates/ai-memory-consolidate/src/bootstrap.rs b/crates/ai-memory-consolidate/src/bootstrap.rsindex 4ea11c0d..0936180d 100644--- a/crates/ai-memory-consolidate/src/bootstrap.rs+++ b/crates/ai-memory-consolidate/src/bootstrap.rs@@ -52,6 +52,20 @@ use crate::path_sanitize::slugify_page_path;/// only used to decide which sources to drop.const CHARS_PER_TOKEN: usize = 4;+/// Share of a token budget that bootstrap fills, by its own estimate.+///+/// [`CHARS_PER_TOKEN`] undercounts denser text: non-English prose and source+/// code measured about 40% above a bytes ÷ 4 estimate on real tokenizers+/// (#884). Filling only 80% of `--max-input-tokens` and `--chunk-input-tokens`+/// keeps a bundle the estimate calls a fit inside the provider's window,+/// the same default consolidation uses for this undercount.+const ESTIMATE_BUDGET_PERCENT: usize = 80;++/// `budget` shrunk by [`ESTIMATE_BUDGET_PERCENT`].+const fn with_estimate_margin(budget: usize) -> usize {+ budget.saturating_mul(ESTIMATE_BUDGET_PERCENT) / 100+}+/// Errors returned from [`Bootstrap::run`].#[derive(Debug, Error)]pub enum BootstrapError {@@ -1198,8 +1212,9 @@ pub fn prune_sources_to_budget(mut sources: Vec<BootstrapSource>,budget: usize,) -> (Vec<BootstrapSource>, usize, usize) {- // Reserve ~1k tokens for the prompt scaffolding itself.- let usable = budget.saturating_sub(1_000);+ // Leave headroom for the estimate's undercount, then reserve ~1k tokens+ // for the prompt scaffolding itself.+ let usable = with_estimate_margin(budget).saturating_sub(1_000);// Order: highest drop_priority FIRST → drop those when over budget.sources.sort_by_key(|s| std::cmp::Reverse(s.kind.drop_priority()));let total_count = sources.len();@@ -1298,10 +1313,11 @@ fn chunk_sources_greedy(sources: Vec<BootstrapSource>, usable: usize) -> Vec<Vecfn usable_chunk_tokens(chunk_budget: usize) -> usize {const PROMPT_RESERVE: usize = 1_000;- if chunk_budget > PROMPT_RESERVE {- chunk_budget - PROMPT_RESERVE+ let budget = with_estimate_margin(chunk_budget);+ if budget > PROMPT_RESERVE {+ budget - PROMPT_RESERVE} else {- chunk_budget+ budget}}@@ -1768,6 +1784,37 @@ mod tests {assert_eq!(kept.len(), 2);}+ /// A bundle the bytes ÷ 4 estimate puts just under the budget can exceed+ /// it on a real tokenizer (#884 measured ~1.4x on pt-BR text with code),+ /// so bootstrap fills only 80% of each budget by its own estimate.+ /// Control: a bundle well inside the margin keeps one chunk and every+ /// source.+ #[test]+ fn budgets_leave_headroom_for_the_estimate_undercount() {+ let near_limit = |tokens: usize| BootstrapSource {+ kind: SourceKind::DocFile,+ label: "docs/guia.md".into(),+ text: "x".repeat(tokens * CHARS_PER_TOKEN - 32),+ };+ // 22K estimated tokens fit the old 23K usable chunk; with the margin+ // (24K × 0.8 − 1K = 18.2K) they must be split.+ let chunks = plan_bootstrap_chunks(vec![near_limit(22_000)], DEFAULT_CHUNK_INPUT_TOKENS);+ assert!(chunks.len() > 1, "a near-limit chunk must be split");+ for chunk in &chunks {+ let t: usize = chunk.iter().map(BootstrapSource::estimated_tokens).sum();+ assert!(t <= 18_200, "each chunk stays inside the margin; got {t}");+ }+ // The prune budget gets the same margin: 145K estimated tokens fit+ // the old 149K usable prune but not 150K × 0.8 − 1K = 119K.+ let (kept, dropped, _) = prune_sources_to_budget(vec![near_limit(145_000)], 150_000);+ assert_eq!((kept.len(), dropped), (0, 1));++ let small = plan_bootstrap_chunks(vec![near_limit(10_000)], DEFAULT_CHUNK_INPUT_TOKENS);+ assert_eq!(small.len(), 1);+ let (kept, dropped, _) = prune_sources_to_budget(vec![near_limit(100_000)], 150_000);+ assert_eq!((kept.len(), dropped), (1, 0));+ }+#[test]fn plan_chunks_single_when_under_budget() {let s = BootstrapSource {diff --git a/docs/install.md b/docs/install.mdindex ff774284..9ea04206 100644--- a/docs/install.md+++ b/docs/install.md@@ -2177,6 +2177,12 @@ remote or uses a custom host/port.--force (re-bootstrap, overwrites the prior manifest)```+**Budgets and the estimate.** Bootstrap estimates tokens as bytes ÷ 4, which+undercounts non-English text and code, so it fills only 80% of+`--max-input-tokens` and of each `--chunk-input-tokens` by that estimate. The+dry run still reports tokens by the same estimate; the sources it keeps and the+chunks it plans reflect the margin.+**Cost.** With Kimi 2.6 via OpenRouter ($0.73/$3.49 per M):- 50k input tokens cap → ~$0.04 worst case input- 1-2k generated tokens → ~$0.007 output