Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs: drop plan-specific wording and remap rewritten commit ids

AkitaOnRailsb341b04da36c
diff --git a/.gitleaksignore b/.gitleaksignore
index de8ce7c3..94003686 100644
--- a/.gitleaksignore
+++ b/.gitleaksignore
@@ -15,8 +15,8 @@ d7ba81976c35b0462f35a4efc53b44347c42295a:companions/ai-memory-relay/src/relay.rs
# Synthetic idempotency key in a PowerShell outage test; contains no credential material.
adc65423be5e8379b0f40de1d7fb38b094e0940d:tests/hooks/test_spool_ps.ps1:generic-api-key:21
# Planted canary in the /admin/metrics leak test (split with concat! since); contains no credential material.
-43b00a0e797c59252c1d6ac1f8b2e37c985534f2:crates/ai-memory-mcp/tests/suite/admin_metrics.rs:github-pat:31
+0832025e2aaba073f07da516633bf7b70d6dce86:crates/ai-memory-mcp/tests/suite/admin_metrics.rs:github-pat:31
# Synthetic credential-shaped id in an importer refusal test (split since); contains no credential material.
-b19ee0c040fb766a5c7405c9e22159fdbc1e2ef4:companions/ai-memory-importer/src/agentmemory/vault.rs:github-pat:498
+47523180f01f1361e030cd16d45712f7889c99a9:companions/ai-memory-importer/src/agentmemory/vault.rs:github-pat:498
# Synthetic key in an importer redaction test (split since); contains no credential material.
-b19ee0c040fb766a5c7405c9e22159fdbc1e2ef4:companions/ai-memory-importer/src/agentmemory/export/tests.rs:generic-api-key:272
+47523180f01f1361e030cd16d45712f7889c99a9:companions/ai-memory-importer/src/agentmemory/export/tests.rs:generic-api-key:272
diff --git a/docs/benchmarks/README.md b/docs/benchmarks/README.md
index 4f6981df..eaa9989a 100644
--- a/docs/benchmarks/README.md
+++ b/docs/benchmarks/README.md
@@ -88,8 +88,7 @@ baseline-vs-baseline determinism check is clean (all accuracy/token deltas `0.00
a component number, and even our component numbers use a
different index and document construction. Our grep engine reproduces
agentmemory's grep adapter on all 470 questions, so the question set and
- metric match; a joint evaluation of the two systems is pending, and no
- ranking between them is claimed. The decomposition report has the
+ metric match, but no ranking between the two systems is claimed. The decomposition report has the
details.
- **Regression gate.** Roadmap items 2-6 re-run this benchmark; a change
that lowers a slice materially is a regression to fix, not a note to
diff --git a/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md b/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md
index 1f25b20a..743d84d9 100644
--- a/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md
+++ b/docs/benchmarks/longmemeval-s-decomposition-2026-10-12.md
@@ -20,13 +20,13 @@ Every number below comes from the in-repo harness (`evals/`, see
## Provenance
-Decomposition runs (M1, 2026-10-10 to 2026-10-11):
+Decomposition runs (2026-10-10 to 2026-10-11):
- server: `release/2.7` at `7dd6b2c2` (the binary reports 2.6.3), before the
observation stream existed
- harness: `420a8556` (the instruments) and `31f515dc` (adds
`--http-timeout-secs`); `release/2.7` carries the same `evals/` tree as
- `014e123b` and `b7ac5a70`
+ `7a2c603d` and `8b797e08`
- dataset: `longmemeval_s` (sha256 `08d8dad4be43ee2049a22ff5674eb86725d0ce5ff434cde2627e5e8e7e117894`);
where noted, `longmemeval_s_cleaned` (Hugging Face
`xiaowu0162/longmemeval-cleaned` at `98d7416c24c778c2fee6e6f3006e7a073259d48f`,
@@ -40,18 +40,18 @@ Decomposition runs (M1, 2026-10-10 to 2026-10-11):
Pipeline re-runs (2026-10-13, corrected harness): the FTS full-set pipeline
with reachability, both date-prefix ablations and the strat-230 local
-pipeline-vs-whole-session A/B, on `release/2.7` at `85eb1723` with
+pipeline-vs-whole-session A/B, on `release/2.7` at `7e08baa7` with
`AI_MEMORY_RETRIEVAL__OBSERVATION_STREAM=false` on every arm (see "Harness
correction").
Observation-stream runs (2026-10-13, corrected harness):
-- build: the shipped ranking, `228d72f0` on top of `e124b123`, for the
- full-set FTS and local A/Bs; `release/2.7` at `85eb1723` for the first
+- build: the shipped ranking, `41793120` on top of `25594224`, for the
+ full-set FTS and local A/Bs; `release/2.7` at `7e08baa7` for the first
version. The stream is on by default in both, so the baseline side sets
`AI_MEMORY_RETRIEVAL__OBSERVATION_STREAM=false` and the candidate side
`=true`
-- harness: `85eb1723`, which replays each question's sessions under their own
+- harness: `7e08baa7`, which replays each question's sessions under their own
session ids (see "Harness correction")
- dataset, hardware and model as above (original `longmemeval_s` file)
@@ -198,8 +198,8 @@ answer. Hits are still pages, and nothing new is written to the wiki. It is on
by default in 2.7; setting it to `false` restores the previous ranking
exactly.
-Stream off → on, same build (`release/2.7` at `228d72f0` on top of
-`e124b123`, the shipped ranking), same questions, all 470, corrected harness:
+Stream off → on, same build (`release/2.7` at `41793120` on top of
+`25594224`, the shipped ranking), same questions, all 470, corrected harness:
| run | hit@5 | hit@1 | hit@10 | recall@5 | p95 ms | flips |
|---|---|---|---|---|---|---|
@@ -236,7 +236,7 @@ vector, like its FTS row, sees only the title-only body.
In its first version the stream's vote was added on top of page FTS, so a
session page whose title list shared topic words with the question still
collected both. That version, stream off → on, corrected harness, on
-`release/2.7` at `85eb1723`:
+`release/2.7` at `7e08baa7`:
| run | questions | hit@5 | hit@1 | hit@10 | recall@5 | p95 ms | flips |
|---|---|---|---|---|---|---|---|
@@ -269,7 +269,7 @@ got a session page in the later project. This was the "8 of 845 evidence
sessions had no session page" the first version of this report could not
explain. A real harness mints a fresh id per session, so the replay now
namespaces each id by question (`stored_session_uuid` in
-`evals/src/retrieval/ingest.rs`, harness `85eb1723`), and all 845 evidence
+`evals/src/retrieval/ingest.rs`, harness `7e08baa7`), and all 845 evidence
pages are present.
The pipeline numbers above, and the observation-stream runs, are from the
@@ -307,7 +307,7 @@ Reachability is unchanged too (0.056 / 0.102 on the session page).
component measurement. ai-memory's pipeline numbers include capture, and
even its component numbers use a different index, document construction and
dataset file. **The two are not like-for-like, and no ranking between the
- tools is claimed here.** A joint evaluation on one harness is pending.
+ tools is claimed here.**
- The cleaned dataset file changes the haystacks of 454 questions but keeps
every question and evidence label. Grep and whole-session FTS score the same
on both files.
@@ -366,7 +366,7 @@ $E retrieval --stratify 20 --embeddings local --candidate-server-env AI_MEMORY_R
Observation stream, shipped ranking (corrected harness, stream on by
default): the full-set FTS and local A/Bs. The first version's runs used the
-same commands on `85eb1723`, plus `--stratify 30` (FTS) and `--embeddings
+same commands on `7e08baa7`, plus `--stratify 30` (FTS) and `--embeddings
local --stratify 15`:
```bash
diff --git a/docs/comparison.md b/docs/comparison.md
index f971db1a..23d0c14f 100644
--- a/docs/comparison.md
+++ b/docs/comparison.md
@@ -52,8 +52,8 @@ ranking only), zero-LLM FTS scores 0.972; the
explains the difference. For context on the same dataset, mcp-memory-service
reports 0.804 R@5, and agentmemory publishes 95.2% `recall_any@5` from a
component harness (whole raw sessions, BM25 plus vectors, reranker disabled).
-Pipeline and component numbers are not like-for-like, and a joint evaluation
-with agentmemory is pending, so this page claims no ranking between them. See
+Pipeline and component numbers are not like-for-like, so this page claims no
+ranking between them. See
[where we're behind](#where-were-behind-or-different-by-choice).
## By camp
@@ -303,8 +303,7 @@ operator turns it on.
the gap to capture, and within capture not to the 2 KB privacy cap (pages holding exactly the captured text
score 0.968) but the zero-LLM session page, which indexed only short turn
titles. agentmemory's published 95.2% is a component number with its
- reranker disabled, so it is not like-for-like with ours; a joint evaluation
- is pending. The only
+ reranker disabled, so it is not like-for-like with ours. The only
reranker is LLM-as-judge, so the zero-LLM default path has none; the opt-in
`answer=true` dialectic synthesis (2.4) is likewise LLM-required, off by
default, and its end-to-end QA-accuracy result is early and small-sample (see
diff --git a/docs/competitive-parity.md b/docs/competitive-parity.md
index d38b7770..868aa5c6 100644
--- a/docs/competitive-parity.md
+++ b/docs/competitive-parity.md
@@ -53,7 +53,7 @@ None of the competitors below has more than one or two of these; none has all.
|---|---|---|---|
| **mcp-memory-service** (fact-row twin) | **Yes** if you liked hook capture + typed edges + honest numbers | Files-as-truth (not opaque fact rows), handoffs, cross-project messaging, `as_of` temporal, a marginally better + reproducible number, and (new on 2.4) its aging set at parity done zero-LLM/reversible/off-by-default (per-tier decay curves, extractive compression, DBSCAN cold-cluster dedup, access boosts, 0.4–0.75-band contradiction flagging) | Cloud/multi-backend replication (Cloudflare/Milvus), graph viz. (Belief-strength + clustering consolidation are no longer a give-up: ai-memory now ships both, clustering as a zero-LLM default-off pass, belief-strength off/R2-gated) |
| **basic-memory** (file-first sibling) | **Yes** for multi-harness coding continuity; **no** for a personal Obsidian KB with live team co-edit | Ambient capture (no `write_note` ceremony), full lifecycle, handoffs/messaging, git-versioned truth, published numbers | Local zero-cost reranking, real-time collaborative editing, Postgres backend, `build_context` graph-walk, hosted mobile/web UX |
-| **agentmemory** (ideological sibling) | **Yes** for operability + data ownership | Self-contained binary (no `iii-engine` sidecar, no ELv2 runtime), files-as-truth, SQL indexes committed in-txn, zero-LLM summaries and handoffs, typed claim-once handoffs, a multi-user auth ladder (DB tokens, OIDC), native Windows without a manual sidecar download (ours is still "Experimental"), and a 2.7 importer for its vault and `ExportData`. Both now default to auth on for new installs (theirs since 0.9.30, ours in 2.7, unreleased) | Browser edit/forget (the 2.7 web UI adds page history/diff, a session timeline, a recall trace and a health panel, but stays read-only by design), the iii console's OTEL traces (we ship explain timings + Prometheus `/admin/metrics` in 2.7), a P2P mesh-sync primitive (`ai-memory-wikisync` is our git-based answer), the opt-in local cross-encoder reranker, an npm/`npx` install (ours is a two-command native install on Linux and macOS in 2.7, no npm package), ~31 more tools (54, all visible by default; mostly speculative). Benchmarks are not like-for-like: their 95.2% LongMemEval-S is a component measurement with the reranker disabled, ours is end-to-end; a joint evaluation is pending |
+| **agentmemory** (ideological sibling) | **Yes** for operability + data ownership | Self-contained binary (no `iii-engine` sidecar, no ELv2 runtime), files-as-truth, SQL indexes committed in-txn, zero-LLM summaries and handoffs, typed claim-once handoffs, a multi-user auth ladder (DB tokens, OIDC), native Windows without a manual sidecar download (ours is still "Experimental"), and a 2.7 importer for its vault and `ExportData`. Both now default to auth on for new installs (theirs since 0.9.30, ours in 2.7, unreleased) | Browser edit/forget (the 2.7 web UI adds page history/diff, a session timeline, a recall trace and a health panel, but stays read-only by design), the iii console's OTEL traces (we ship explain timings + Prometheus `/admin/metrics` in 2.7), a P2P mesh-sync primitive (`ai-memory-wikisync` is our git-based answer), the opt-in local cross-encoder reranker, an npm/`npx` install (ours is a two-command native install on Linux and macOS in 2.7, no npm package), ~31 more tools (54, all visible by default; mostly speculative). Benchmarks are not like-for-like: their 95.2% LongMemEval-S is a component measurement with the reranker disabled, ours is end-to-end |
| **Claude Code built-in memory** | **Yes** for teams / multi-machine / multi-harness; **not obviously** for a solo dev on one machine in Claude only | Cross-machine, cross-harness, team sharing, tool-lifecycle capture, real search | Zero setup: it's already on with no server to run (our one honest structural disadvantage for the solo case; 2.7's `install.sh` + `ai-memory setup` cut ours to two commands on Linux and macOS, but there is still a server) |
| **Hindsight** (pages-over-facts) | **Yes** for self-hosted/offline/team; **no** if you need their published accuracy + are fine with an LLM-required cloud | Zero-LLM, files you own, one binary, in-project team sharing (vs strict per-bank isolation), and (new on 2.4) a belief-strength `confidence` + an opt-in "dream" rewrite that borrow their model but never delete a source and keep the zero-LLM default | Their published headline accuracy, VLM/LLM extraction depth, and belief-strength that *actually moves ranking by default* (ours ships but is off/R2-gated until an eval proves it) |
| **Zep/Graphiti** (temporal KG) | **Yes** for self-hosting coders avoiding Neo4j+LLM (Community Edition is discontinued); **no** for enterprise graph-query/world-time | One binary on SQLite, zero-LLM, files, coding-harness native, a supported self-host | True bi-temporal (world vs observed time), Cypher/BFS graph queries, custom entity/edge types, `minRating` |
@@ -214,7 +214,7 @@ implemented without a separate decision.** Ordered by leverage.
`issues-agentmemory.md` now carry a dated-snapshot header (54 tools, all
visible by default; 138 REST endpoints, per their README as of 2026-10-10).
Their 95.2% is a component measurement and our 0.82 a pipeline one, so
- neither pairing is like-for-like; a joint evaluation is pending.
+ neither pairing is like-for-like.
8. **Fleet-scale trust governance: a real gap vs Caura (issue #810).** Our
multi-user story is per-project (`pages shared, batons owned`, invariant #16)
with a root→DB-user→OIDC auth ladder and an audit log, and it is deliberately
@@ -246,7 +246,7 @@ implemented without a separate decision.** Ordered by leverage.
or calls an LLM. The R2 harness that gates the quality work now exists and
measures the accuracy+latency+context-tokens triple plus end-to-end
QA-accuracy (the QA column is still illustrative sample-20 only).
-- **2.7 (unreleased) closes several agentmemory-specific gaps** the same way:
+- **2.7 (unreleased) closes several viewer, operability and migration gaps** the same way:
read-only page history/diff, a session timeline, a recall trace and a
root-only health panel in the web UI; explain timings and a content-free
`/admin/metrics`; auth on by default for new installs with `auth enable`
diff --git a/evals/README.md b/evals/README.md
index 1e2970ac..792bf331 100644
--- a/evals/README.md
+++ b/evals/README.md
@@ -372,7 +372,7 @@ cargo run -p ai-memory-eval -- retrieval --sample 3 \
--qa --qa-provider gemini --qa-model gemini-2.5-flash
```
-### `retrieval` decomposition instruments (M1)
+### `retrieval` decomposition instruments
These flags split a hit@k number into evidence the index never saw and
ranking on evidence it did. All are opt-in; the defaults reproduce the
diff --git a/evals/src/retrieval/ingest.rs b/evals/src/retrieval/ingest.rs
index 8faa1c29..adebd481 100644
--- a/evals/src/retrieval/ingest.rs
+++ b/evals/src/retrieval/ingest.rs
@@ -13,7 +13,7 @@
//! own), mirroring what the official LongMemEval harness exposes to
//! retrievers as session timestamps.
//!
-//! Two ablation knobs change that shape for the M1 decomposition (defaults
+//! Two ablation knobs change that shape for the decomposition (defaults
//! keep every published baseline reproducible):
//!
//! - [`DatePrefix`] moves or drops the per-turn date prefix, which takes
diff --git a/evals/src/retrieval/mod.rs b/evals/src/retrieval/mod.rs
index 8863eeba..aaba597a 100644
--- a/evals/src/retrieval/mod.rs
+++ b/evals/src/retrieval/mod.rs
@@ -16,7 +16,7 @@
//! tokens a result would cost the agent (chars/4 over returned snippets).
//! Without a candidate config the single-config path is unchanged.
//!
-//! ## M1: decomposing the retrieval gap
+//! ## Decomposing the retrieval gap
//!
//! Four opt-in instruments split a hit@k number into capture loss and
//! ranking loss (defaults reproduce the published baselines):