Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

docs: add competitive parity & migration-worthiness audit

AkitaOnRails3387263419aa
diff --git a/AGENTS.md b/AGENTS.md
index bd22afa5..57363f62 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -543,6 +543,10 @@ Additional boundary rules:
against other memory tools (camps, migration notes, how the field validates
the file-first/pages-over-facts approach). Analysis behind it:
`research-2026-landscape.md`.
+- [`docs/competitive-parity.md`](docs/competitive-parity.md) — self-critical
+ internal audit: per-competitor migration-worthiness (do we do the basics +
+ add enough to justify switching?), the "did we copy without improving?"
+ borrowed-ideas verdicts, and documented gap-fill recommendations.
- [`docs/lifecycle-ops.md`](docs/lifecycle-ops.md) — read before touching
purge/rename/backup/restore/reset/reindex/restore-page.
- [`docs/auto-improvement-loop.md`](docs/auto-improvement-loop.md) —
diff --git a/docs/competitive-parity.md b/docs/competitive-parity.md
new file mode 100644
index 00000000..c00b6a85
--- /dev/null
+++ b/docs/competitive-parity.md
@@ -0,0 +1,150 @@
+# Competitive parity & migration-worthiness
+
+A self-critical internal audit, September 2026. Sibling to the public
+[`comparison.md`](comparison.md) (the fair pitch) and
+[`research-2026-landscape.md`](research-2026-landscape.md) (what each competitor
+does + what to borrow). This doc asks the harder, honest question the pitch
+cannot:
+
+> **In the goals where we overlap a competitor, are we actually *better* — or did
+> we copy their ideas without improving on them? If I already use competitor X,
+> is ai-memory worth migrating *to*? What do I gain that X doesn't have, and what
+> do I give up?**
+
+The premise being tested: *we do the same basic stuff and have extra to justify
+the switch.* Where that premise holds, migration is real; where we only matched
+(or fell behind), it isn't. Every ai-memory claim below was verified against the
+code, not the marketing; competitor claims against their primary sources. Goals
+that simply differ from ours are called out as "different buyer — not a migration
+target," not forced into a comparison.
+
+## The migration bar
+
+A competitor's user should switch only if ai-memory (a) does the basic thing they
+rely on **at parity or better**, and (b) adds something they can't get where they
+are. So each verdict rests on two honest ledgers: **our verified moat** (the
+"extra") and **our real gaps** (where "better" is not yet true).
+
+## ai-memory's verified moat (the "extra") — real in code
+
+None of the competitors below has more than one or two of these; none has all.
+
+- **Zero-LLM default path.** Capture, FTS5 + entity + link RRF retrieval, local
+ embeddings, and rule-based summaries all work with no API key (invariant #13).
+- **Files are the source of truth.** A git-backed markdown wiki, committed every
+ consolidation pass — `grep`/diff/Obsidian/rsync it. The DB is a rebuildable
+ index. (Letta's own "Is a Filesystem All You Need?" concession validates this.)
+- **One self-contained binary** — bundled SQLite, vendored libgit2, no sidecar,
+ no external DB, no graph server.
+- **Cross-harness lifecycle capture** across 20+ harnesses through a typed
+ sanitizer boundary (invariants #5/#6).
+- **Typed claim-once handoffs + cross-project agent messaging** — protocols, not
+ "both agents read the same files."
+- **Multi-user shared-within-project** ("pages shared, batons owned", invariant
+ #16) — self-hosted, no paid tier.
+- **A single honest, reproducible benchmark** (0.823 hit@5 local; 0.668 zero-LLM).
+
+## Migration verdicts, by competitor
+
+| From | Migrate to ai-memory? | You gain | You give up |
+|---|---|---|---|
+| **mcp-memory-service** (fact-row twin) | **Yes** if you liked hook capture + typed edges + honest numbers | Files-as-truth (not opaque fact rows), handoffs, cross-project messaging, `as_of` temporal, a marginally better + reproducible number | Belief-strength + clustering consolidation, cloud/multi-backend replication (Cloudflare/Milvus), graph viz |
+| **basic-memory** (file-first sibling) | **Yes** for multi-harness coding continuity; **no** for a personal Obsidian KB with live team co-edit | Ambient capture (no `write_note` ceremony), full lifecycle, handoffs/messaging, git-versioned truth, published numbers | Local zero-cost reranking, real-time collaborative editing, Postgres backend, `build_context` graph-walk, hosted mobile/web UX |
+| **agentmemory** (ideological sibling) | **Yes** for operability + data ownership | Self-contained binary (no `iii-engine` sidecar), files-as-truth, SQL indexes committed in-txn, typed handoffs, Windows parity, fuller auth | ~13 pts raw R@5 (they rerank), a P2P mesh-sync primitive, ~31 tools of (mostly speculative) surface |
+| **Claude Code built-in memory** | **Yes** for teams / multi-machine / multi-harness; **not obviously** for a solo dev on one machine in Claude only | Cross-machine, cross-harness, team sharing, tool-lifecycle capture, real search | Zero setup — it's already on with no server to run (our one honest structural disadvantage for the solo case) |
+| **Hindsight** (pages-over-facts) | **Yes** for self-hosted/offline/team; **no** if you need their published accuracy + are fine with an LLM-required cloud | Zero-LLM, files you own, one binary, in-project team sharing (vs strict per-bank isolation) | Their published headline accuracy, VLM/LLM extraction depth, belief-strength that actually moves ranking |
+| **Zep/Graphiti** (temporal KG) | **Yes** for self-hosting coders avoiding Neo4j+LLM (Community Edition is discontinued); **no** for enterprise graph-query/world-time | One binary on SQLite, zero-LLM, files, coding-harness native, a supported self-host | True bi-temporal (world vs observed time), Cypher/BFS graph queries, custom entity/edge types, `minRating` |
+| **Mem0** | **Mostly no — different buyer** (app end-user personalization). Yes only if you (mis)used it as coding-session memory | Zero-LLM, per-repo, cross-harness, editable pages, no API spend | Mem0's SDK/integration ecosystem + managed cloud personalization |
+| **Letta / MemOS** | **No** if you're building your agent *on* Letta; **yes** if you only wanted your existing coding agent to remember | Additive memory under your existing harness, zero-LLM, no runtime adoption | The ADE + agent-framework machinery (which you don't need unless building on Letta) |
+| **Mem0/Zep for apps; OpenViking, Supermemory** | different buyer (app/user personalization, hosted context DB) — **not migration targets** | — | — |
+
+Net: the premise holds where it should. For a **self-hosted, multi-harness,
+team, zero-LLM, data-ownership** coding-memory use case, ai-memory does the basics
+and adds a moat none of them fully have — migration is justified. It does *not*
+hold for app/user personalization (Mem0/Supermemory), enterprise graph queries
+(Zep Cloud), agent-building runtimes (Letta), or a solo dev happy with Claude's
+built-in — those are different buyers, and we should say so rather than overclaim.
+
+## Did we copy without improving? — the borrowed-ideas audit
+
+The sharp finding: ai-memory did **not** cargo-cult this field. Of the ideas it
+took, most are at parity-or-better; the exceptions are **honestly-scoped-but-
+unfinished substrate**, each deliberately gated behind "prove it with an eval
+first." The risk is not shallow copying — it is that borrowed ideas sit **dormant**
+until the eval harness (R2) that would justify activating them gets built.
+
+| Borrowed idea | From | Verdict | Detail |
+|---|---|---|---|
+| Local embeddings (all-MiniLM-L6-v2) | mcp-memory-service | **BETTER** | Pure-Rust candle (no ONNX/C++ per target), checksum-pinned, default-on with background backfill |
+| Hook capture | mcp-memory-service | **BETTER** | 20+ harnesses + typed sanitizer boundary + backpressure vs Claude-Code-centric |
+| Versioned supersession | agentmemory | **BETTER** | A concurrent divergent write supersedes but the loser stays reachable (invariant #16); theirs flips `isLatest` |
+| Retention-as-formula (decay) | agentmemory | **BETTER** | Real SQL decay math vs in-memory O(N²) Jaccard capped at 1000 rows |
+| Git-as-snapshot | agentmemory | **BETTER** | The wiki files *are* the truth (diffable), not a `state.json` dump |
+| Pages-over-facts / living pages | Hindsight | **PARITY** (native) | Same architecture, file-first vs their Postgres-first; `settled_first` briefing shipped |
+| Cross-session experience/abstraction pass | Letta/Hindsight/surveys | **REAL, ~parity** | ≥2-session evidence guard is the genuine article — but LLM-required and **off by default** |
+| Typed relation edges | graph camp | **SHALLOW on retrieval** | Data model + `contradicts`→lint is real, but retrieval **does not weight edge type** (explain-only) |
+| Belief-strength consolidation | Hindsight / mcp-memory-service | **BEHIND** | `page_evidence` substrate shipped but **ranking-inert**; we still correct via binary supersession, not evidence-weighted belief |
+| bi-temporal-lite | Zep/Graphiti | **BEHIND by design** | Ingestion-time only; world-time validity deferred (honestly, because dating facts needs an LLM and breaks the zero-LLM path) |
+
+## Honest gaps (consolidated) and recommended adjustments
+
+Documented recommendations only — **nothing here is scheduled or to be
+implemented without a separate decision.** Ordered by leverage.
+
+1. **Build the R2 reproducible eval harness (highest leverage).** It is the
+ meta-blocker: belief-strength ranking, typed-edge rank weighting, and a
+ published accuracy/latency/context-tokens *triple* are all gated on "prove it
+ with a number." Every quality gap below is dammed behind this one missing
+ piece. Nothing else in this list should ship before it exists.
+2. **A zero-LLM / local reranker (candle cross-encoder).** The single most-cited
+ weakness: our only reranker is LLM-as-judge, so the zero-LLM default path —
+ our headline — has *no* reranking, while basic-memory ships a local one and
+ agentmemory/Hindsight's score lead is largely reranking (0.823 vs 0.952 same
+ dataset). Closes a gap on our own keyless turf. Gate on R2.
+3. **Activate the dormant substrate (once R2 exists):** wire `page_evidence`
+ confidence into `PageAuthority` (clamped) — belief-strength; and add typed-edge
+ rank weighting + a `contradicts` authority cap — turning typed edges from
+ explain-only into a retriever. Both are designed in
+ [`design-hindsight-borrowings.md`](design-hindsight-borrowings.md); both are
+ inert today.
+4. **Sharpen the cross-machine story.** `comparison.md`'s "cross-machine *by
+ construction*" is overstated — it's one shared server + manual git/rsync, with
+ **no automatic replication** (agentmemory ships a P2P primitive; mcp-memory-
+ service has Cloudflare/Milvus). Either reframe as "one server, many machines"
+ + an explicit git/rsync note, or design a bounded `ai-memory sync` over the git
+ wiki (push/pull the source-of-truth dir, rebuild the index) — a cleaner,
+ file-first P2P story than an opaque KV LWW.
+5. **Close the setup-friction gap vs the built-in.** Our one honest structural
+ weakness for the solo user. Much is already done (`run --autowire`, `doctor`,
+ `backfill`, `install-self-routing`); surface it as *the* onboarding path in the
+ README hero, add a one-line `ai-memory run claude` quickstart, and ship a
+ "coming from Claude built-in?" importer (the `~/.claude/.../memory/` format is
+ nearly identical markdown + `type` frontmatter — a cheap, strong on-ramp).
+6. **Smaller, competitor-specific closes:** a graph-walk retrieval tool
+ (basic-memory's `build_context`/`memory://`, borrowed-but-unshipped); MCP tool
+ behavior hints (readOnly/destructive/idempotent) on the 23-tool surface; a
+ clustering/dedup consolidation pass (mcp-memory-service's DBSCAN); a read-only
+ graph visualization in `/web`; a first-class TS SDK story for ecosystem parity.
+7. **Fix doc staleness the audit surfaced** (accuracy, not features):
+ `research-basic-memory.md` understates basic-memory's shipped cross-encoder
+ reranking + Teams tier; `research-agentmemory.md` says 53 tools / 124 endpoints
+ (now 54 / 130); `comparison.md` attributes agentmemory's `0.967` to
+ LongMemEval-S when it's their in-house `coding-agent-life-v1` (the like-for-like
+ LongMemEval-S is 0.952 vs our 0.823).
+
+## Bottom line
+
+- **The moat is real and verified:** zero-LLM default, files-as-truth, one binary,
+ cross-harness capture, claim-once handoffs + messaging, and multi-user
+ shared-within-project. On operability, data ownership, and cross-agent/team
+ continuity, ai-memory is genuinely ahead of every overlapping competitor.
+- **The honest weakness is retrieval quality**, and it is *self-inflicted by
+ discipline, not by copying*: belief-strength and typed-edge ranking are built
+ but switched off, no local reranker exists, and there is no published
+ comparable number — all because the R2 eval harness that would justify turning
+ them on has not been built. **Build R2, then activate what's already there.**
+- **Migration answer:** for the self-hosted, multi-harness, team, zero-LLM,
+ own-your-data coding-memory user, yes — we do the basics and add what they
+ can't get elsewhere. For app personalization, enterprise graph queries, agent
+ runtimes, or a solo dev content with the built-in, no — and we should keep
+ saying so.
diff --git a/docs/research-2026-landscape.md b/docs/research-2026-landscape.md
index 736711b1..fae94107 100644
--- a/docs/research-2026-landscape.md
+++ b/docs/research-2026-landscape.md
@@ -248,6 +248,12 @@ standalone deep-dive; this section is its record.
## 5. What this means for ai-memory - analysis and recommendations
+> Companion audit: [`competitive-parity.md`](competitive-parity.md) turns this
+> "what to borrow" lens around and asks the harder one - in the goals where we
+> *overlap* a competitor, are we actually better or did we copy without
+> improving, and is ai-memory worth migrating *to*? It is the self-critical
+> record behind this section's recommendations.
+
The May research led us to build: versioned supersession, retention
formulas, hybrid RRF retrieval, opt-in LLM consolidation, handoffs as a
protocol, single self-contained binary, typed scope isolation. Every one