Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: drop plan-specific wording and remap rewritten commit ids
AkitaOnRailsb341b04da36c
diff --git a/.gitleaksignore b/.gitleaksignoreindex de8ce7c3..94003686 100644--- a/.gitleaksignore+++ b/.gitleaksignore@@ -15,8 +15,8 @@ d7ba81976c35b0462f35a4efc53b44347c42295a:companions/ai-memory-relay/src/relay.rs# Synthetic idempotency key in a PowerShell outage test; contains no credential material.adc65423be5e8379b0f40de1d7fb38b094e0940d:tests/hooks/test_spool_ps.ps1:generic-api-key:21# Planted canary in the /admin/metrics leak test (split with concat! since); contains no credential material.-43b00a0e797c59252c1d6ac1f8b2e37c985534f2:crates/ai-memory-mcp/tests/suite/admin_metrics.rs:github-pat:31+0832025e2aaba073f07da516633bf7b70d6dce86:crates/ai-memory-mcp/tests/suite/admin_metrics.rs:github-pat:31# Synthetic credential-shaped id in an importer refusal test (split since); contains no credential material.-b19ee0c040fb766a5c7405c9e22159fdbc1e2ef4:companions/ai-memory-importer/src/agentmemory/vault.rs:github-pat:498+47523180f01f1361e030cd16d45712f7889c99a9:companions/ai-memory-importer/src/agentmemory/vault.rs:github-pat:498# Synthetic key in an importer redaction test (split since); contains no credential material.-b19ee0c040fb766a5c7405c9e22159fdbc1e2ef4:companions/ai-memory-importer/src/agentmemory/export/tests.rs:generic-api-key:272+47523180f01f1361e030cd16d45712f7889c99a9:companions/ai-memory-importer/src/agentmemory/export/tests.rs:generic-api-key:272diff --git a/docs/benchmarks/README.md b/docs/benchmarks/README.mdindex 4f6981df..eaa9989a 100644--- a/docs/benchmarks/README.md+++ b/docs/benchmarks/README.md@@ -88,8 +88,7 @@ baseline-vs-baseline determinism check is clean (all accuracy/token deltas `0.00a component number, and even our component numbers use adifferent index and document construction. Our grep engine reproducesagentmemory's grep adapter on all 470 questions, so the question set and- metric match; a joint evaluation of the two systems is pending, and no- ranking between them is claimed. The decomposition report has the+ metric match, but no ranking between the two systems is claimed. The decomposition report has thedetails.- **Regression gate.** Roadmap items 2-6 re-run this benchmark; a changethat lowers a slice materially is a regression to fix, not a note todiff --git a/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md b/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.mdindex 1f25b20a..743d84d9 100644--- a/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md+++ b/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md@@ -20,13 +20,13 @@ Every number below comes from the in-repo harness (`evals/`, see## Provenance-Decomposition runs (M1, 2026-10-10 to 2026-10-11):+Decomposition runs (2026-10-10 to 2026-10-11):- server: `release/2.7` at `7dd6b2c2` (the binary reports 2.6.3), before theobservation stream existed- harness: `420a8556` (the instruments) and `31f515dc` (adds`--http-timeout-secs`); `release/2.7` carries the same `evals/` tree as- `014e123b` and `b7ac5a70`+ `7a2c603d` and `8b797e08`- dataset: `longmemeval_s` (sha256 `08d8dad4be43ee2049a22ff5674eb86725d0ce5ff434cde2627e5e8e7e117894`);where noted, `longmemeval_s_cleaned` (Hugging Face`xiaowu0162/longmemeval-cleaned` at `98d7416c24c778c2fee6e6f3006e7a073259d48f`,@@ -40,18 +40,18 @@ Decomposition runs (M1, 2026-10-10 to 2026-10-11):Pipeline re-runs (2026-10-13, corrected harness): the FTS full-set pipelinewith reachability, both date-prefix ablations and the strat-230 local-pipeline-vs-whole-session A/B, on `release/2.7` at `85eb1723` with+pipeline-vs-whole-session A/B, on `release/2.7` at `7e08baa7` with`AI_MEMORY_RETRIEVAL__OBSERVATION_STREAM=false` on every arm (see "Harnesscorrection").Observation-stream runs (2026-10-13, corrected harness):-- build: the shipped ranking, `228d72f0` on top of `e124b123`, for the- full-set FTS and local A/Bs; `release/2.7` at `85eb1723` for the first+- build: the shipped ranking, `41793120` on top of `25594224`, for the+ full-set FTS and local A/Bs; `release/2.7` at `7e08baa7` for the firstversion. The stream is on by default in both, so the baseline side sets`AI_MEMORY_RETRIEVAL__OBSERVATION_STREAM=false` and the candidate side`=true`-- harness: `85eb1723`, which replays each question's sessions under their own+- harness: `7e08baa7`, which replays each question's sessions under their ownsession ids (see "Harness correction")- dataset, hardware and model as above (original `longmemeval_s` file)@@ -198,8 +198,8 @@ answer. Hits are still pages, and nothing new is written to the wiki. It is onby default in 2.7; setting it to `false` restores the previous rankingexactly.-Stream off → on, same build (`release/2.7` at `228d72f0` on top of-`e124b123`, the shipped ranking), same questions, all 470, corrected harness:+Stream off → on, same build (`release/2.7` at `41793120` on top of+`25594224`, the shipped ranking), same questions, all 470, corrected harness:| run | hit@5 | hit@1 | hit@10 | recall@5 | p95 ms | flips ||---|---|---|---|---|---|---|@@ -236,7 +236,7 @@ vector, like its FTS row, sees only the title-only body.In its first version the stream's vote was added on top of page FTS, so asession page whose title list shared topic words with the question stillcollected both. That version, stream off → on, corrected harness, on-`release/2.7` at `85eb1723`:+`release/2.7` at `7e08baa7`:| run | questions | hit@5 | hit@1 | hit@10 | recall@5 | p95 ms | flips ||---|---|---|---|---|---|---|---|@@ -269,7 +269,7 @@ got a session page in the later project. This was the "8 of 845 evidencesessions had no session page" the first version of this report could notexplain. A real harness mints a fresh id per session, so the replay nownamespaces each id by question (`stored_session_uuid` in-`evals/src/retrieval/ingest.rs`, harness `85eb1723`), and all 845 evidence+`evals/src/retrieval/ingest.rs`, harness `7e08baa7`), and all 845 evidencepages are present.The pipeline numbers above, and the observation-stream runs, are from the@@ -307,7 +307,7 @@ Reachability is unchanged too (0.056 / 0.102 on the session page).component measurement. ai-memory's pipeline numbers include capture, andeven its component numbers use a different index, document construction anddataset file. **The two are not like-for-like, and no ranking between the- tools is claimed here.** A joint evaluation on one harness is pending.+ tools is claimed here.**- The cleaned dataset file changes the haystacks of 454 questions but keepsevery question and evidence label. Grep and whole-session FTS score the sameon both files.@@ -366,7 +366,7 @@ $E retrieval --stratify 20 --embeddings local --candidate-server-env AI_MEMORY_RObservation stream, shipped ranking (corrected harness, stream on bydefault): the full-set FTS and local A/Bs. The first version's runs used the-same commands on `85eb1723`, plus `--stratify 30` (FTS) and `--embeddings+same commands on `7e08baa7`, plus `--stratify 30` (FTS) and `--embeddingslocal --stratify 15`:```bashdiff --git a/docs/comparison.md b/docs/comparison.mdindex f971db1a..23d0c14f 100644--- a/docs/comparison.md+++ b/docs/comparison.md@@ -52,8 +52,8 @@ ranking only), zero-LLM FTS scores 0.972; theexplains the difference. For context on the same dataset, mcp-memory-servicereports 0.804 R@5, and agentmemory publishes 95.2% `recall_any@5` from acomponent harness (whole raw sessions, BM25 plus vectors, reranker disabled).-Pipeline and component numbers are not like-for-like, and a joint evaluation-with agentmemory is pending, so this page claims no ranking between them. See+Pipeline and component numbers are not like-for-like, so this page claims no+ranking between them. See[where we're behind](#where-were-behind-or-different-by-choice).## By camp@@ -303,8 +303,7 @@ operator turns it on.the gap to capture, and within capture not to the 2 KB privacy cap (pages holding exactly the captured textscore 0.968) but the zero-LLM session page, which indexed only short turntitles. agentmemory's published 95.2% is a component number with its- reranker disabled, so it is not like-for-like with ours; a joint evaluation- is pending. The only+ reranker disabled, so it is not like-for-like with ours. The onlyreranker is LLM-as-judge, so the zero-LLM default path has none; the opt-in`answer=true` dialectic synthesis (2.4) is likewise LLM-required, off bydefault, and its end-to-end QA-accuracy result is early and small-sample (seediff --git a/docs/competitive-parity.md b/docs/competitive-parity.mdindex d38b7770..868aa5c6 100644--- a/docs/competitive-parity.md+++ b/docs/competitive-parity.md@@ -53,7 +53,7 @@ None of the competitors below has more than one or two of these; none has all.|---|---|---|---|| **mcp-memory-service** (fact-row twin) | **Yes** if you liked hook capture + typed edges + honest numbers | Files-as-truth (not opaque fact rows), handoffs, cross-project messaging, `as_of` temporal, a marginally better + reproducible number, and (new on 2.4) its aging set at parity done zero-LLM/reversible/off-by-default (per-tier decay curves, extractive compression, DBSCAN cold-cluster dedup, access boosts, 0.4–0.75-band contradiction flagging) | Cloud/multi-backend replication (Cloudflare/Milvus), graph viz. (Belief-strength + clustering consolidation are no longer a give-up: ai-memory now ships both, clustering as a zero-LLM default-off pass, belief-strength off/R2-gated) || **basic-memory** (file-first sibling) | **Yes** for multi-harness coding continuity; **no** for a personal Obsidian KB with live team co-edit | Ambient capture (no `write_note` ceremony), full lifecycle, handoffs/messaging, git-versioned truth, published numbers | Local zero-cost reranking, real-time collaborative editing, Postgres backend, `build_context` graph-walk, hosted mobile/web UX |-| **agentmemory** (ideological sibling) | **Yes** for operability + data ownership | Self-contained binary (no `iii-engine` sidecar, no ELv2 runtime), files-as-truth, SQL indexes committed in-txn, zero-LLM summaries and handoffs, typed claim-once handoffs, a multi-user auth ladder (DB tokens, OIDC), native Windows without a manual sidecar download (ours is still "Experimental"), and a 2.7 importer for its vault and `ExportData`. Both now default to auth on for new installs (theirs since 0.9.30, ours in 2.7, unreleased) | Browser edit/forget (the 2.7 web UI adds page history/diff, a session timeline, a recall trace and a health panel, but stays read-only by design), the iii console's OTEL traces (we ship explain timings + Prometheus `/admin/metrics` in 2.7), a P2P mesh-sync primitive (`ai-memory-wikisync` is our git-based answer), the opt-in local cross-encoder reranker, an npm/`npx` install (ours is a two-command native install on Linux and macOS in 2.7, no npm package), ~31 more tools (54, all visible by default; mostly speculative). Benchmarks are not like-for-like: their 95.2% LongMemEval-S is a component measurement with the reranker disabled, ours is end-to-end; a joint evaluation is pending |+| **agentmemory** (ideological sibling) | **Yes** for operability + data ownership | Self-contained binary (no `iii-engine` sidecar, no ELv2 runtime), files-as-truth, SQL indexes committed in-txn, zero-LLM summaries and handoffs, typed claim-once handoffs, a multi-user auth ladder (DB tokens, OIDC), native Windows without a manual sidecar download (ours is still "Experimental"), and a 2.7 importer for its vault and `ExportData`. Both now default to auth on for new installs (theirs since 0.9.30, ours in 2.7, unreleased) | Browser edit/forget (the 2.7 web UI adds page history/diff, a session timeline, a recall trace and a health panel, but stays read-only by design), the iii console's OTEL traces (we ship explain timings + Prometheus `/admin/metrics` in 2.7), a P2P mesh-sync primitive (`ai-memory-wikisync` is our git-based answer), the opt-in local cross-encoder reranker, an npm/`npx` install (ours is a two-command native install on Linux and macOS in 2.7, no npm package), ~31 more tools (54, all visible by default; mostly speculative). Benchmarks are not like-for-like: their 95.2% LongMemEval-S is a component measurement with the reranker disabled, ours is end-to-end || **Claude Code built-in memory** | **Yes** for teams / multi-machine / multi-harness; **not obviously** for a solo dev on one machine in Claude only | Cross-machine, cross-harness, team sharing, tool-lifecycle capture, real search | Zero setup: it's already on with no server to run (our one honest structural disadvantage for the solo case; 2.7's `install.sh` + `ai-memory setup` cut ours to two commands on Linux and macOS, but there is still a server) || **Hindsight** (pages-over-facts) | **Yes** for self-hosted/offline/team; **no** if you need their published accuracy + are fine with an LLM-required cloud | Zero-LLM, files you own, one binary, in-project team sharing (vs strict per-bank isolation), and (new on 2.4) a belief-strength `confidence` + an opt-in "dream" rewrite that borrow their model but never delete a source and keep the zero-LLM default | Their published headline accuracy, VLM/LLM extraction depth, and belief-strength that *actually moves ranking by default* (ours ships but is off/R2-gated until an eval proves it) || **Zep/Graphiti** (temporal KG) | **Yes** for self-hosting coders avoiding Neo4j+LLM (Community Edition is discontinued); **no** for enterprise graph-query/world-time | One binary on SQLite, zero-LLM, files, coding-harness native, a supported self-host | True bi-temporal (world vs observed time), Cypher/BFS graph queries, custom entity/edge types, `minRating` |@@ -214,7 +214,7 @@ implemented without a separate decision.** Ordered by leverage.`issues-agentmemory.md` now carry a dated-snapshot header (54 tools, allvisible by default; 138 REST endpoints, per their README as of 2026-10-10).Their 95.2% is a component measurement and our 0.82 a pipeline one, so- neither pairing is like-for-like; a joint evaluation is pending.+ neither pairing is like-for-like.8. **Fleet-scale trust governance: a real gap vs Caura (issue #810).** Ourmulti-user story is per-project (`pages shared, batons owned`, invariant #16)with a root→DB-user→OIDC auth ladder and an audit log, and it is deliberately@@ -246,7 +246,7 @@ implemented without a separate decision.** Ordered by leverage.or calls an LLM. The R2 harness that gates the quality work now exists andmeasures the accuracy+latency+context-tokens triple plus end-to-endQA-accuracy (the QA column is still illustrative sample-20 only).-- **2.7 (unreleased) closes several agentmemory-specific gaps** the same way:+- **2.7 (unreleased) closes several viewer, operability and migration gaps** the same way:read-only page history/diff, a session timeline, a recall trace and aroot-only health panel in the web UI; explain timings and a content-free`/admin/metrics`; auth on by default for new installs with `auth enable`diff --git a/evals/README.md b/evals/README.mdindex 1e2970ac..792bf331 100644--- a/evals/README.md+++ b/evals/README.md@@ -372,7 +372,7 @@ cargo run -p ai-memory-eval -- retrieval --sample 3 \--qa --qa-provider gemini --qa-model gemini-2.5-flash```-### `retrieval` decomposition instruments (M1)+### `retrieval` decomposition instrumentsThese flags split a hit@k number into evidence the index never saw andranking on evidence it did. All are opt-in; the defaults reproduce thediff --git a/evals/src/retrieval/ingest.rs b/evals/src/retrieval/ingest.rsindex 8faa1c29..adebd481 100644--- a/evals/src/retrieval/ingest.rs+++ b/evals/src/retrieval/ingest.rs@@ -13,7 +13,7 @@//! own), mirroring what the official LongMemEval harness exposes to//! retrievers as session timestamps.//!-//! Two ablation knobs change that shape for the M1 decomposition (defaults+//! Two ablation knobs change that shape for the decomposition (defaults//! keep every published baseline reproducible)://!//! - [`DatePrefix`] moves or drops the per-turn date prefix, which takesdiff --git a/evals/src/retrieval/mod.rs b/evals/src/retrieval/mod.rsindex 8863eeba..aaba597a 100644--- a/evals/src/retrieval/mod.rs+++ b/evals/src/retrieval/mod.rs@@ -16,7 +16,7 @@//! tokens a result would cost the agent (chars/4 over returned snippets).//! Without a candidate config the single-config path is unchanged.//!-//! ## M1: decomposing the retrieval gap+//! ## Decomposing the retrieval gap//!//! Four opt-in instruments split a hit@k number into capture loss and//! ranking loss (defaults reproduce the published baselines):