Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: refresh current benchmark comparison
AkitaOnRailsda7e6552e060
diff --git a/docs/comparison.md b/docs/comparison.mdindex 696ded37..8f37c43e 100644--- a/docs/comparison.md+++ b/docs/comparison.md@@ -55,8 +55,10 @@ LongMemEval-S hit@5 **0.666** zero-LLM and **0.815** with the default localembeddings in the published 2.6 line. The observation stream in 3.0(on by default) raises zero-LLM to **0.951** and local embeddingsto **0.945** on all 470 questions, measured on one build with the stream off-and on. The latest release run (3.2.1) measures **0.951** zero-LLM and-**0.949** local ([report](benchmarks/longmemeval-s-3.2.1-release.md)). With capture skipped and each raw session indexed whole (component+and on. The 3.3.0 release run measures **0.951** zero-LLM and **0.945** local+at hit@5 ([report](benchmarks/longmemeval-s-3.3.0-release.md)); its local+top-ten movement is published and tracked in #1854. With capture skipped and+each raw session indexed whole (componentmode, ranking only), zero-LLM FTS scores 0.972; the[decomposition report](benchmarks/longmemeval-s-decomposition-2026-10-12.md)explains the difference.@@ -443,7 +445,7 @@ ships something ai-memory does not, it is named.- **Retrieval through capture, and no local reranker.** Through the shippedpipeline, LongMemEval-S hit@5 is 0.815 with local embeddings and 0.666zero-LLM (2.6); the 3.0 observation stream raises them to 0.945 and 0.951- (3.2.1: 0.949 and 0.951). Ranking+ (3.3.0: 0.945 and 0.951). Rankingthe same raw sessions whole scores 0.972 zero-LLM. The[decomposition](benchmarks/longmemeval-s-decomposition-2026-10-12.md) tracedthe gap to capture, and within capture not to the 2 KB privacy cap (pages holding exactly the captured text@@ -464,7 +466,7 @@ ships something ai-memory does not, it is named.| Tool | Number | Kind | Setup ||---|---|---|---|- | ai-memory 3.2 | LongMemEval-S hit@5 0.951 FTS / 0.949 local (3.0: 0.951 / 0.945; 2.6: 0.666 / 0.815) | Retrieval through capture | In-repo harness, all 470 questions; component mode FTS 0.972 |+ | ai-memory 3.3 | LongMemEval-S hit@5 0.951 FTS / 0.945 local (3.0: 0.951 / 0.945; 2.6: 0.666 / 0.815) | Retrieval through capture | In-repo harness, all 470 questions; component mode FTS 0.972; published local top-ten movement tracked in #1854 || MemPalace | 96.6% R@5 raw, 98.4% held-out (450 q) | Retrieval only | Different harness || agentmemory | 95.2% `recall_any@5`; 2026-10-09 baseline 96.2% any-hit, 84.68% strict recall-all@5 | Retrieval, component | Whole raw sessions, reranker disabled; the authors say the baseline supports no competitive claim || Mem0 | LongMemEval 94.4, LoCoMo 92.5, BEAM 64.1 (1M) / 48.6 (10M) | End-to-end QA, LLM-judged | Managed platform, top-200 budget; harness defaults to gpt-4o reader and judge; headline models not stated |diff --git a/docs/competitive-parity.md b/docs/competitive-parity.mdindex 630dfd2c..72237a0c 100644--- a/docs/competitive-parity.md+++ b/docs/competitive-parity.md@@ -47,7 +47,8 @@ None of the competitors below has more than one or two of these; none has all.#16), self-hosted, with no paid tier.- **Reproducible retrieval benchmarks**: LongMemEval-S hit@5 0.815 local and0.666 zero-LLM through the shipped 2.6 pipeline, 0.951 zero-LLM and 0.945 local with the 3.0- observation stream (3.2.1: 0.951 and 0.949, the 3.2.0 local drop recovered — it was boundary-question noise, see the 3.2.1 report), and a published pipeline-vs-component decomposition+ observation stream (3.3.0: 0.951 and 0.945; the published local top-ten+ movement is tracked in #1854), and a published pipeline-vs-component decomposition([`benchmarks/`](benchmarks/README.md)). Others publish numbers too (Mem0,Zep, Hindsight, EverOS, MemPalace, agentmemory), mostly end-to-end QA withtheir own reader and judge models. The edge is reproducibility; we claim no