Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

feat/personal memory vault (#1093)

Ruben Lütkeeccd6fee4f5e
diff --git a/docs/adr/0044-personal-memory-vault.md b/docs/adr/0044-personal-memory-vault.md
new file mode 100644
index 0000000000..60c8d7a5f0
--- /dev/null
+++ b/docs/adr/0044-personal-memory-vault.md
@@ -0,0 +1,75 @@
+# ADR-0044 — A personal memory vault replaces the wiki vault
+
+**Status:** Accepted · **Date:** 2026-11-02
+
+## Context
+
+The main assistant's long-term memory is an Obsidian-style Markdown vault
+(`wiki/obsidian-vault`, ADR-0013) written by a two-stage LLM curator that
+rewrites whole pages, plus five side stores that grew next to it: the
+learned notebooks (`society/jarvis/MEMORY.md`, `USER.md`), `core_memory.json`,
+`data/workspace/USER.md`, the identity card and the contacts mirror. Six
+stores, ~30k lines, and still no answer to "what do you know about me?", no
+history of how a fact changed, and no export.
+
+What the assistants that do this well converged on in 2025–2026
+(ChatGPT saved memories and "dreaming", Claude memory and its memory tool,
+Meta Muse's `MEMORY.md`, Gemini memory, Mem0, Zep/Graphiti, Letta,
+Supermemory):
+
+1. **Atomic memories, not pages.** One sentence per memory, typed, with its
+ source. A page is a view, never the unit of storage.
+2. **A small profile always in the prompt, the rest searched per turn.**
+3. **Supersede, never overwrite.** A changed fact closes the old one with a
+ date (bi-temporal validity) instead of deleting it, so the assistant knows
+ both "now" and "before".
+4. **Write in the background, reconcile against what exists.** A candidate
+ fact is compared with its nearest memories and becomes ADD, UPDATE,
+ INVALIDATE or NOOP.
+5. **The user sees and controls everything:** view, edit, pin, forget, a
+ per-chat off switch.
+6. **Portable.** No provider format has won; the most complete open proposal
+ is MacPaw's Portable Memory (`.mem`): a folder of JSONL per kind, a
+ manifest, an audit log, tombstones applied first, checksums,
+ content-derived ids so a re-import merges.
+
+## Decision
+
+1. **One store.** `dodaios/memory/vault/` owns the main assistant's
+ long-term memory in one SQLite file, `data/memory/vault.db`, that belongs
+ to the user. Tables: `memory`, `entity`, `memory_entity`, `audit`,
+ `tombstone`, and an FTS5 index. Embeddings, when present, are an optional
+ extra table; search works without them.
+2. **The memory row.** `id`, `content_hash`, `kind`, `text` (one sentence),
+ `entities`, `scope`, `valid_from`/`valid_to`, `recorded_at`/`expired_at`,
+ `supersedes_id`, `origin` (user / inferred / imported), `source`,
+ `confidence`, `sensitivity`, `pinned`, `use_count`, `last_used_at`.
+ Kinds: `profile`, `preference`, `relationship`, `project`, `procedural`,
+ `episode`, `reference`, `ephemeral`.
+3. **Writes.** An explicit "remember / forget" is applied at once without a
+ model call. Everything else goes through one background extractor that
+ reconciles each candidate against the nearest current memories. Nothing
+ is deleted unless the user asks; a forget leaves a tombstone and purges
+ every derived copy.
+4. **Reads.** A budgeted profile block (pinned, profile, preference,
+ procedural, active projects) sits in the cached system prompt; a hybrid
+ search adds the few memories relevant to the turn. Voice engines get the
+ same profile block and the same search tool.
+5. **Export and import.** A `.mem`-compatible bundle (lossless) and a
+ Markdown rendering (readable) from the Memory section; import is
+ idempotent. The old vault, notebooks, core memory and USER.md are imported
+ once, then the wiki subsystem is removed.
+6. **Scope.** This ADR covers the main assistant. Agent notebooks keep their
+ format and move out of the vault folder when the vault is removed.
+
+Supersedes ADR-0013 (knowledge wiki) and ADR-0015 (Obsidian setup wizard)
+once the removal chunk lands. The plan lives in
+`docs/memory-vault-plan.md`.
+
+## Consequences
+
+- One place to look, one file to back up, one button to export.
+- The Obsidian integration goes away. A Markdown export still opens in
+ Obsidian, but it is a copy, not the store.
+- Removing ~30k lines of wiki code and ~21k lines of its tests is done in a
+ separate, last chunk, after the import has run on real data.
diff --git a/docs/memory-vault-plan.md b/docs/memory-vault-plan.md
new file mode 100644
index 0000000000..fd28f7797a
--- /dev/null
+++ b/docs/memory-vault-plan.md
@@ -0,0 +1,37 @@
+# Personal memory vault — build plan
+
+Decision: ADR-0044. Each chunk ships as its own pull request and leaves the
+app working.
+
+| # | Chunk | Done when |
+|---|-------|-----------|
+| 1 | **Core store** — `dodaios/memory/vault/`: schema, add/supersede/forget/pin, search (FTS5 BM25 + entity + recency), history, profile block, audit, tombstones, `.mem` bundle + Markdown export, idempotent import | unit tests cover every write path and an export → import round trip |
+| 2 | **Writes** — explicit remember/forget tools on the new store; one background extractor (candidate facts → nearest memories → ADD / UPDATE / INVALIDATE / NOOP) on the existing background provider chain and call cap; sensitivity rules; off switch per chat | a conversation that states, changes and revokes a fact leaves exactly the right current and closed rows |
+| 3 | **Reads** — profile block in the cached system prompt, per-turn relevant memories in the turn context, `memory-search` / `memory-history` tools, the same block for the realtime and GPT-Live engines; replaces the wiki injector, identity card and core-memory block | the main bot answers "what do you know about me?" and "where did I live before?" from the store, in text and voice |
+| 4 | **Import of today's memory** — one-time importer for the wiki vault pages, the learned notebooks, `core_memory.json` and `USER.md`; keeps a tar.gz of the old vault | a copy of the real vault imports with no secrets and no duplicates |
+| 5 | **Memory section** — new UI: profile, all memories by kind, timeline of changes, edit / pin / forget, export and import buttons | visible in the running desktop app |
+| 6 | **Removal** — delete the wiki subsystem, its routes, tools, Obsidian setup, Command Deck wiki graph, config sections and tests; move agent notebooks to `data/agents/` | suite green, no import of `dodaios.memory.wiki` left |
+
+## Store layout (chunk 1)
+
+- `data/memory/vault.db` — SQLite, WAL.
+- `memory` rows are never updated in place except for `pinned`, use
+ counters and closing (`valid_to`, `expired_at`). An edit is a new row that
+ supersedes the old one.
+- `current` = `expired_at IS NULL AND (valid_to IS NULL OR valid_to > now)`.
+- `tombstone(id, content_hash, forgotten_at, reason)`: a forgotten memory's
+ text is purged from `memory`, the FTS index and the audit log.
+
+## Bundle format (`.mem`-compatible)
+
+```
+manifest.json format, version, created_at, counts, app
+items/<kind>.jsonl one memory per line, all fields
+entities.jsonl
+audit/log.jsonl
+audit/tombstones.jsonl applied first on import
+CHECKSUMS sha256 per file
+```
+
+Import matches on `id`, then on `content_hash`, so the same bundle imported
+twice changes nothing.
diff --git a/dodaios/memory/vault/__init__.py b/dodaios/memory/vault/__init__.py
new file mode 100644
index 0000000000..7cc8b59e9a
--- /dev/null
+++ b/dodaios/memory/vault/__init__.py
@@ -0,0 +1,6 @@
+"""Personal memory vault: the main assistant's long-term memory (ADR-0044)."""
+
+from dodaios.memory.vault.models import KINDS, Memory, SearchHit, WriteResult
+from dodaios.memory.vault.store import MemoryVault
+
+__all__ = ["KINDS", "Memory", "MemoryVault", "SearchHit", "WriteResult"]
diff --git a/dodaios/memory/vault/models.py b/dodaios/memory/vault/models.py
new file mode 100644
index 0000000000..53c0e6c220
--- /dev/null
+++ b/dodaios/memory/vault/models.py
@@ -0,0 +1,110 @@
+"""Data types of the personal memory vault (ADR-0044)."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+import unicodedata
+from dataclasses import asdict, dataclass, field
+from typing import Any
+
+#: What a memory is about. The profile block and the UI group by it.
+KINDS: tuple[str, ...] = (
+ "profile", # who the user is: name, role, home, languages
+ "preference", # likes, dislikes, how they want things done
+ "relationship", # people and how they relate to the user
+ "project", # ongoing goals, plans, deadlines, decisions
+ "procedural", # standing rules and corrections for the assistant
+ "episode", # a notable event or conversation
+ "reference", # where things are: accounts, places, files
+ "ephemeral", # true for a short while; carries an end date
+)
+
+#: Who put the memory there.
+ORIGINS: tuple[str, ...] = ("user", "inferred", "imported")
+
+#: ``sensitive`` memories never enter the always-on profile block.
+SENSITIVITIES: tuple[str, ...] = ("normal", "sensitive")
+
+_SPACE = re.compile(r"\s+")
+
+
+def normalize_text(text: str) -> str:
+ """One canonical spelling of a memory sentence, for duplicate detection."""
+ folded = unicodedata.normalize("NFKC", text).casefold()
+ folded = _SPACE.sub(" ", folded).strip()
+ return folded.rstrip(" .!")
+
+
+def content_hash(kind: str, text: str) -> str:
+ """Stable id of what a memory says, independent of when it was said."""
+ digest = hashlib.sha256(f"{kind}\n{normalize_text(text)}".encode()).hexdigest()
+ return f"sha256:{digest}"
+
+
+@dataclass(frozen=True)
+class Memory:
+ """One atomic memory: a single sentence about the user or their world."""
+
+ id: str
+ content_hash: str
+ kind: str
+ text: str
+ recorded_at: str
+ origin: str
+ entities: tuple[str, ...] = ()
+ scope: str = "global"
+ valid_from: str | None = None
+ valid_to: str | None = None
+ expired_at: str | None = None
+ supersedes_id: str | None = None
+ source: dict[str, Any] = field(default_factory=dict)
+ confidence: float = 1.0
+ sensitivity: str = "normal"
+ pinned: bool = False
+ use_count: int = 0
+ last_used_at: str | None = None
+
+ def is_current(self, now: str) -> bool:
+ """True while the memory is neither replaced nor past its end date."""
+ if self.expired_at is not None:
+ return False
+ return self.valid_to is None or self.valid_to > now
+
+ def to_dict(self) -> dict[str, Any]:
+ data = asdict(self)
+ data["entities"] = list(self.entities)
+ return data
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> Memory:
+ known = {f for f in cls.__dataclass_fields__}
+ values = {k: v for k, v in data.items() if k in known}
+ values["entities"] = tuple(values.get("entities") or ())
+ values["pinned"] = bool(values.get("pinned", False))
+ values["source"] = dict(values.get("source") or {})
+ return cls(**values)
+
+ def source_json(self) -> str:
+ return json.dumps(self.source, ensure_ascii=False, sort_keys=True)
+
+
+@dataclass(frozen=True)
+class WriteResult:
+ """What a write did.
+
+ ``action`` is one of ``added``, ``existing`` (the same memory was already
+ current), ``superseded``, ``refused`` (it looked like a secret) or
+ ``blocked`` (the user had told the assistant to forget it).
+ """
+
+ action: str
+ memory: Memory | None = None
+ reason: str = ""
+
+
+@dataclass(frozen=True)
+class SearchHit:
+ memory: Memory
+ score: float
diff --git a/dodaios/memory/vault/portable.py b/dodaios/memory/vault/portable.py
new file mode 100644
index 0000000000..84755bac86
--- /dev/null
+++ b/dodaios/memory/vault/portable.py
@@ -0,0 +1,299 @@
+"""Export and import of the personal memory vault (ADR-0044).
+
+Two outputs:
+
+- **Bundle** (lossless, re-importable), shaped after the open Portable Memory
+ proposal (``.mem``): ``manifest.json``, ``items/<kind>.jsonl``,
+ ``entities.jsonl``, ``audit/log.jsonl``, ``audit/tombstones.jsonl`` and
+ ``CHECKSUMS``. As a folder or a zip file.
+- **Markdown** (readable): ``MEMORY.md`` plus one file per kind, which opens in
+ any Markdown editor, Obsidian included.
+
+Import applies tombstones first, then inserts every memory whose id and
+content are unknown, so importing the same bundle twice changes nothing.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import io
+import json
+import zipfile
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+from dodaios.memory.vault.models import KINDS, Memory
+from dodaios.memory.vault.store import MemoryVault
+
+FORMAT = "portable-memory"
+FORMAT_VERSION = "1.0"
+APP = "dodaios"
+
+_KIND_TITLES = {
+ "profile": "About me",
+ "preference": "Preferences",
+ "relationship": "People",
+ "project": "Projects",
+ "procedural": "Standing instructions",
+ "episode": "Moments",
+ "reference": "Where things are",
+ "ephemeral": "For now",
+}
+
+
+class BundleError(ValueError):
+ """The bundle is not one this app can read, or it was altered."""
+
+
+@dataclass
+class ImportReport:
+ added: int = 0
+ skipped: int = 0
+ forgotten: int = 0
+ errors: list[str] = field(default_factory=list)
+
+ def to_dict(self) -> dict[str, Any]:
+ return {
+ "added": self.added,
+ "skipped": self.skipped,
+ "forgotten": self.forgotten,
+ "errors": list(self.errors),
+ }
+
+
+# -- bundle --------------------------------------------------------------
+
+
+def bundle_files(vault: MemoryVault) -> dict[str, bytes]:
+ """Every file of a bundle, keyed by its path inside the bundle."""
+ memories = vault.list(include_closed=True)
+ files: dict[str, bytes] = {}
+ for kind in KINDS:
+ rows = [m.to_dict() for m in memories if m.kind == kind]
+ if rows:
+ files[f"items/{kind}.jsonl"] = _jsonl(rows)
+ entities = [
+ {"id": e["id"], "name": e["name"], "created_at": e["created_at"]} for e in vault.entities()
+ ]
+ files["entities.jsonl"] = _jsonl(entities)
+ files["audit/log.jsonl"] = _jsonl(vault.audit_log())
+ files["audit/tombstones.jsonl"] = _jsonl(vault.tombstones())
+ stats = vault.stats()
+ manifest = {
+ "format": FORMAT,
+ "version": FORMAT_VERSION,
+ "app": APP,
+ "vault_id": vault.vault_id,
+ "created_at": vault.now(),
+ "counts": {
+ "memories": len(memories),
+ "current": stats["current"],
+ "entities": stats["entities"],
+ "tombstones": stats["forgotten"],
+ },
+ "kinds": list(KINDS),
+ }
+ files["manifest.json"] = json.dumps(manifest, ensure_ascii=False, indent=2).encode()
+ files["CHECKSUMS"] = "".join(
+ f"{hashlib.sha256(data).hexdigest()} {name}\n" for name, data in sorted(files.items())
+ ).encode()
+ return files
+
+
+def export_bundle(vault: MemoryVault, dest: str | Path) -> Path:
+ """Write a bundle. A path ending in ``.zip`` gets a zip file, else a folder."""
+ dest = Path(dest)
+ files = bundle_files(vault)
+ if dest.suffix.lower() == ".zip":
+ dest.parent.mkdir(parents=True, exist_ok=True)
+ dest.write_bytes(_zip(files))
+ return dest
+ for name, data in files.items():
+ target = dest / name
+ target.parent.mkdir(parents=True, exist_ok=True)
+ target.write_bytes(data)
+ return dest
+
+
+def export_zip_bytes(vault: MemoryVault) -> bytes:
+ return _zip(bundle_files(vault))
+
+
+def read_bundle(src: str | Path | bytes) -> dict[str, bytes]:
+ """Load a bundle from a folder, a zip path or zip bytes and verify it."""
+ if isinstance(src, bytes):
+ files = _unzip(src)
+ else:
+ path = Path(src)
+ if path.is_dir():
+ files = {
+ p.relative_to(path).as_posix(): p.read_bytes()
+ for p in path.rglob("*")
+ if p.is_file()
+ }
+ else:
+ files = _unzip(path.read_bytes())
+ _verify(files)
+ return files
+
+
+def import_bundle(vault: MemoryVault, src: str | Path | bytes) -> ImportReport:
+ files = read_bundle(src)
+ report = ImportReport()
+ for row in _rows(files.get("audit/tombstones.jsonl", b"")):
+ try:
+ vault.apply_tombstone(
+ str(row["id"]),
+ str(row["content_hash"]),
+ str(row.get("forgotten_at") or vault.now()),
+ str(row.get("reason") or "imported"),
+ )
+ report.forgotten += 1
+ except (KeyError, TypeError) as exc:
+ report.errors.append(f"tombstone: {exc}")
+ memories: list[Memory] = []
+ for name in sorted(files):
+ if name.startswith("items/") and name.endswith(".jsonl"):
+ for row in _rows(files[name]):
+ try:
+ memories.append(Memory.from_dict(row))
+ except (TypeError, ValueError) as exc:
+ report.errors.append(f"{name}: {exc}")
+ # Older versions first, so a row is inserted after the one it replaces.
+ memories.sort(key=lambda m: (m.recorded_at, m.id))
+ for memory in memories:
+ if vault.has(memory.id) or vault.has_hash_version(memory):
+ report.skipped += 1
+ continue
+ try:
+ if vault.insert_imported(memory):
+ report.added += 1
+ else:
+ report.skipped += 1
+ except ValueError as exc:
+ report.errors.append(f"{memory.id}: {exc}")
+ return report
+
+
+# -- markdown ------------------------------------------------------------
+
+
+def markdown_files(vault: MemoryVault) -> dict[str, str]:
+ """A readable copy: ``MEMORY.md`` and one page per kind."""
+ current = vault.list()
+ stats = vault.stats()
+ files: dict[str, str] = {}
+ index = [
+ "# My memory",
+ "",
+ f"Exported {vault.now()} · {stats['current']} current memories.",
+ "",
+ ]
+ for kind in KINDS:
+ items = [m for m in current if m.kind == kind]
+ if not items:
+ continue
+ title = _KIND_TITLES[kind]
+ index.append(f"- [{title}]({kind}.md) — {len(items)}")
+ page = [
+ "---",
+ f"kind: {kind}",
+ f"count: {len(items)}",
+ "---",
+ "",
+ f"# {title}",
+ "",
+ ]
+ for m in sorted(items, key=lambda m: m.recorded_at):
+ line = f"- {m.text}"
+ details = []
+ if m.entities:
+ details.append(", ".join(m.entities))
+ if m.valid_from:
+ details.append(f"since {m.valid_from[:10]}")
+ if m.valid_to:
+ details.append(f"until {m.valid_to[:10]}")
+ if m.pinned:
+ details.append("pinned")
+ if details:
+ line += f" *({'; '.join(details)})*"
+ page.append(f"{line} <!-- {m.id} -->")
+ files[f"{kind}.md"] = "\n".join(page) + "\n"
+ files["MEMORY.md"] = "\n".join(index) + "\n"
+ return files
+
+
+def export_markdown(vault: MemoryVault, dest: str | Path) -> Path:
+ dest = Path(dest)
+ dest.mkdir(parents=True, exist_ok=True)
+ for name, text in markdown_files(vault).items():
+ (dest / name).write_text(text, encoding="utf-8")
+ return dest
+
+
+# -- helpers -------------------------------------------------------------
+
+
+def _jsonl(rows: list[dict[str, Any]]) -> bytes:
+ return "".join(json.dumps(r, ensure_ascii=False, sort_keys=True) + "\n" for r in rows).encode()
+
+
+def _rows(data: bytes) -> list[dict[str, Any]]:
+ return [json.loads(line) for line in data.decode("utf-8").splitlines() if line.strip()]
+
+
+def _zip(files: dict[str, bytes]) -> bytes:
+ buffer = io.BytesIO()
+ with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as zf:
+ for name in sorted(files):
+ zf.writestr(name, files[name])
+ return buffer.getvalue()
+
+
+def _unzip(data: bytes) -> dict[str, bytes]:
+ try:
+ with zipfile.ZipFile(io.BytesIO(data)) as zf:
+ files: dict[str, bytes] = {}
+ for info in zf.infolist():
+ if info.is_dir():
+ continue
+ name = info.filename.replace("\\", "/")
+ if name.startswith("/") or ".." in name.split("/"):
+ raise BundleError(f"unsafe path in bundle: {name}")
+ if info.file_size > 200 * 1024 * 1024:
+ raise BundleError(f"file too large in bundle: {name}")
+ files[name] = zf.read(info)
+ # A zip of the folder may wrap everything in one top folder.
+ if "manifest.json" not in files:
+ roots = {n.split("/", 1)[0] for n in files if "/" in n}
+ if len(roots) == 1:
+ root = roots.pop() + "/"
+ files = {n[len(root) :]: d for n, d in files.items() if n.startswith(root)}
+ return files
+ except zipfile.BadZipFile as exc:
+ raise BundleError("not a zip file") from exc
+
+
+def _verify(files: dict[str, bytes]) -> None:
+ if "manifest.json" not in files:
+ raise BundleError("no manifest.json; not a memory bundle")
+ try:
+ manifest = json.loads(files["manifest.json"])
+ except json.JSONDecodeError as exc:
+ raise BundleError("manifest.json is not JSON") from exc
+ if manifest.get("format") != FORMAT:
+ raise BundleError(f"unknown bundle format {manifest.get('format')!r}")
+ if str(manifest.get("version", "")).split(".")[0] != FORMAT_VERSION.split(".")[0]:
+ raise BundleError(f"unsupported bundle version {manifest.get('version')!r}")
+ checksums = files.get("CHECKSUMS")
+ if checksums is None:
+ return
+ for line in checksums.decode("utf-8").splitlines():
+ if not line.strip():
+ continue
+ digest, _, name = line.partition(" ")
+ if name not in files:
+ raise BundleError(f"missing file listed in CHECKSUMS: {name}")
+ if hashlib.sha256(files[name]).hexdigest() != digest:
+ raise BundleError(f"checksum mismatch: {name}")
diff --git a/dodaios/memory/vault/store.py b/dodaios/memory/vault/store.py
new file mode 100644
index 0000000000..761f290a08
--- /dev/null
+++ b/dodaios/memory/vault/store.py
@@ -0,0 +1,786 @@
+"""The personal memory vault: one SQLite file that holds what the assistant
+knows about the user (ADR-0044).
+
+Rules the store enforces:
+
+- A memory row is never rewritten. An edit or a changed fact is a new row
+ that supersedes the old one; the old one is closed with a date.
+- Forgetting purges the text everywhere (row, search index, audit log) and
+ leaves a tombstone, so an inferred write can never bring it back.
+- Secrets are refused at the door.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import math
+import re
+import secrets
+import sqlite3
+import threading
+from collections.abc import Callable, Iterable, Sequence
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any
+
+from dodaios.memory.vault.models import (
+ KINDS,
+ ORIGINS,
+ SENSITIVITIES,
+ Memory,
+ SearchHit,
+ WriteResult,
+ content_hash,
+)
+
+log = logging.getLogger(__name__)
+
+SCHEMA_VERSION = 1
+
+_SCHEMA = """
+CREATE TABLE IF NOT EXISTS meta (
+ key TEXT PRIMARY KEY,
+ value TEXT NOT NULL
+);
+CREATE TABLE IF NOT EXISTS memory (
+ id TEXT PRIMARY KEY,
+ content_hash TEXT NOT NULL,
+ kind TEXT NOT NULL,
+ text TEXT NOT NULL,
+ scope TEXT NOT NULL DEFAULT 'global',
+ valid_from TEXT,
+ valid_to TEXT,
+ recorded_at TEXT NOT NULL,
+ expired_at TEXT,
+ supersedes_id TEXT,
+ origin TEXT NOT NULL,
+ source TEXT NOT NULL DEFAULT '{}',
+ confidence REAL NOT NULL DEFAULT 1.0,
+ sensitivity TEXT NOT NULL DEFAULT 'normal',
+ pinned INTEGER NOT NULL DEFAULT 0,
+ use_count INTEGER NOT NULL DEFAULT 0,
+ last_used_at TEXT
+);
+CREATE INDEX IF NOT EXISTS memory_hash ON memory(content_hash);
+CREATE INDEX IF NOT EXISTS memory_kind ON memory(kind);
+CREATE INDEX IF NOT EXISTS memory_supersedes ON memory(supersedes_id);
+CREATE TABLE IF NOT EXISTS entity (
+ id TEXT PRIMARY KEY,
+ name TEXT NOT NULL,
+ name_key TEXT NOT NULL UNIQUE,
+ created_at TEXT NOT NULL
+);
+CREATE TABLE IF NOT EXISTS memory_entity (
+ memory_id TEXT NOT NULL,
+ entity_id TEXT NOT NULL,
+ PRIMARY KEY (memory_id, entity_id)
+);
+CREATE INDEX IF NOT EXISTS memory_entity_by_entity ON memory_entity(entity_id);
+CREATE TABLE IF NOT EXISTS audit (
+ seq INTEGER PRIMARY KEY AUTOINCREMENT,
+ at TEXT NOT NULL,
+ action TEXT NOT NULL,
+ memory_id TEXT,
+ detail TEXT NOT NULL DEFAULT '{}'
+);
+CREATE TABLE IF NOT EXISTS tombstone (
+ id TEXT PRIMARY KEY,
+ content_hash TEXT NOT NULL,
+ forgotten_at TEXT NOT NULL,
+ reason TEXT NOT NULL DEFAULT ''
+);
+CREATE INDEX IF NOT EXISTS tombstone_hash ON tombstone(content_hash);
+CREATE VIRTUAL TABLE IF NOT EXISTS memory_fts USING fts5(
+ id UNINDEXED,
+ text,
+ entities,
+ tokenize = 'unicode61 remove_diacritics 2'
+);
+"""
+
+_COLUMNS = (
+ "id, content_hash, kind, text, scope, valid_from, valid_to, recorded_at, "
+ "expired_at, supersedes_id, origin, source, confidence, sensitivity, pinned, "
+ "use_count, last_used_at"
+)
+
+_CURRENT = "m.expired_at IS NULL AND (m.valid_to IS NULL OR m.valid_to > ?)"
+
+_TOKEN = re.compile(r"\w+", re.UNICODE)
+
+#: Order of kinds in the always-on profile block.
+_PROFILE_ORDER = ("profile", "preference", "procedural", "relationship", "project", "reference")
+
+_PROFILE_HEADINGS = {
+ "profile": "About the user",
+ "preference": "Preferences",
+ "procedural": "Standing instructions",
+ "relationship": "People",
+ "project": "Current projects",
+ "reference": "Where things are",
+ "pinned": "Pinned",
+}
+
+
+def utc_now() -> str:
+ return datetime.now(UTC).isoformat(timespec="seconds")
+
+
+def _new_id() -> str:
+ stamp = datetime.now(UTC).strftime("%Y%m%d%H%M%S")
+ return f"mem_{stamp}_{secrets.token_hex(6)}"
+
+
+def _secret_check(text: str) -> bool:
+ # The secret detector still lives with the wiki; ADR-0044 chunk 6 moves it.
+ from dodaios.memory.wiki.secret_guard import contains_secret
+
+ return contains_secret(text)
+
+
+def _clean_entities(names: Iterable[str]) -> tuple[str, ...]:
+ seen: dict[str, str] = {}
+ for name in names:
+ label = " ".join(str(name).split())
+ if label and label.casefold() not in seen:
+ seen[label.casefold()] = label
+ return tuple(seen.values())
+
+
+class MemoryVault:
+ """Read/write handle for ``vault.db``. Safe to share between threads."""
+
+ def __init__(self, path: str | Path, *, clock: Callable[[], str] = utc_now) -> None:
+ self.path = Path(path)
+ self._clock = clock
+ self._lock = threading.RLock()
+ if str(path) != ":memory:":
+ self.path.parent.mkdir(parents=True, exist_ok=True)
+ self._db = sqlite3.connect(str(path), check_same_thread=False)
+ self._db.row_factory = sqlite3.Row
+ with self._lock:
+ if str(path) != ":memory:":
+ self._db.execute("PRAGMA journal_mode=WAL")
+ self._db.execute("PRAGMA foreign_keys=ON")
+ self._db.executescript(_SCHEMA)
+ self._db.execute(
+ "INSERT OR IGNORE INTO meta(key, value) VALUES ('schema_version', ?)",
+ (str(SCHEMA_VERSION),),
+ )
+ self._db.execute(
+ "INSERT OR IGNORE INTO meta(key, value) VALUES ('vault_id', ?)",
+ (f"vault_{secrets.token_hex(8)}",),
+ )
+ self._db.commit()
+
+ # -- lifecycle ---------------------------------------------------------
+
+ def close(self) -> None:
+ with self._lock:
+ self._db.close()
+
+ def __enter__(self) -> MemoryVault:
+ return self
+
+ def __exit__(self, *_exc: object) -> None:
+ self.close()
+
+ @property
+ def vault_id(self) -> str:
+ row = self._db.execute("SELECT value FROM meta WHERE key='vault_id'").fetchone()
+ return str(row["value"])
+
+ def now(self) -> str:
+ return self._clock()
+
+ # -- writes ------------------------------------------------------------
+
+ def remember(
+ self,
+ text: str,
+ kind: str = "profile",
+ *,
+ entities: Sequence[str] = (),
+ origin: str = "user",
+ source: dict[str, Any] | None = None,
+ scope: str = "global",
+ confidence: float = 1.0,
+ sensitivity: str = "normal",
+ pinned: bool = False,
+ valid_from: str | None = None,
+ valid_to: str | None = None,
+ ) -> WriteResult:
+ """Store one memory, or return the identical one that is already current."""
+ text = " ".join(text.split())
+ self._validate(text, kind, origin, sensitivity)
+ if _secret_check(text):
+ return WriteResult("refused", reason="looks like a secret")
+ digest = content_hash(kind, text)
+ with self._lock:
+ now = self.now()
+ existing = self._current_by_hash(digest, scope, now)
+ if existing is not None:
+ if pinned and not existing.pinned:
+ self.pin(existing.id, True)
+ existing = self.get(existing.id)
+ return WriteResult("existing", existing)
+ if self._is_tombstoned(digest):
+ if origin != "user":
+ return WriteResult("blocked", reason="the user asked to forget this")
+ self._db.execute("DELETE FROM tombstone WHERE content_hash=?", (digest,))
+ if kind == "ephemeral" and valid_to is None:
+ raise ValueError("an ephemeral memory needs valid_to")
+ memory = Memory(
+ id=_new_id(),
+ content_hash=digest,
+ kind=kind,
+ text=text,
+ recorded_at=now,
+ origin=origin,
+ entities=_clean_entities(entities),
+ scope=scope,
+ valid_from=valid_from or now,
+ valid_to=valid_to,
+ source=dict(source or {}),
+ confidence=max(0.0, min(1.0, float(confidence))),
+ sensitivity=sensitivity,
+ pinned=pinned,
+ )
+ self._insert(memory)
+ self._audit("add", memory.id, {"kind": kind, "origin": origin})
+ self._db.commit()
+ return WriteResult("added", memory)
+
+ def supersede(
+ self,
+ old_id: str,
+ text: str,
+ *,
+ kind: str | None = None,
+ entities: Sequence[str] | None = None,
+ origin: str = "user",
+ source: dict[str, Any] | None = None,
+ confidence: float | None = None,
+ valid_from: str | None = None,
+ valid_to: str | None = None,
+ ) -> WriteResult:
+ """Replace a memory with a newer version and close the old one."""
+ text = " ".join(text.split())
+ with self._lock:
+ old = self.get(old_id)
+ if old is None:
+ raise KeyError(old_id)
+ now = self.now()
+ if not old.is_current(now):
+ raise ValueError(f"{old_id} is no longer current")
+ new_kind = kind or old.kind
+ self._validate(text, new_kind, origin, old.sensitivity)
+ if _secret_check(text):
+ return WriteResult("refused", reason="looks like a secret")
+ digest = content_hash(new_kind, text)
+ if digest == old.content_hash:
+ return WriteResult("existing", old)
+ if self._is_tombstoned(digest) and origin != "user":
+ return WriteResult("blocked", reason="the user asked to forget this")
+ memory = Memory(
+ id=_new_id(),
+ content_hash=digest,
+ kind=new_kind,
+ text=text,
+ recorded_at=now,
+ origin=origin,
+ entities=_clean_entities(old.entities if entities is None else entities),
+ scope=old.scope,
+ valid_from=valid_from or now,
+ valid_to=valid_to,
+ supersedes_id=old.id,
+ source=dict(source or {}),
+ confidence=old.confidence if confidence is None else confidence,
+ sensitivity=old.sensitivity,
+ pinned=old.pinned,
+ )
+ self._close(old.id, valid_to=memory.valid_from, expired_at=now)
+ self._insert(memory)
+ self._audit("supersede", memory.id, {"replaces": old.id, "origin": origin})
+ self._db.commit()
+ return WriteResult("superseded", memory)
+
+ def invalidate(self, memory_id: str, *, reason: str = "", when: str | None = None) -> bool:
+ """Mark a fact as no longer true without a replacement ("I quit that job")."""
+ with self._lock:
+ memory = self.get(memory_id)
+ now = self.now()
+ if memory is None or not memory.is_current(now):
+ return False
+ self._close(memory_id, valid_to=when or now, expired_at=None)
+ self._audit("invalidate", memory_id, {"reason": reason})
+ self._db.commit()
+ return True
+
+ def forget(self, memory_id: str, *, reason: str = "user request") -> int:
+ """Erase a memory and every earlier and later version of it.
+
+ Returns how many rows were erased. The text is gone from the row, the
+ search index and the audit log; a tombstone per version remains.
+ """
+ with self._lock:
+ chain = self._chain_ids(memory_id)
+ if not chain:
+ return 0
+ now = self.now()
+ for mid in chain:
+ row = self._db.execute(
+ "SELECT content_hash FROM memory WHERE id=?", (mid,)
+ ).fetchone()
+ self._db.execute(
+ "INSERT OR REPLACE INTO tombstone(id, content_hash, forgotten_at, reason) "
+ "VALUES (?, ?, ?, ?)",
+ (mid, row["content_hash"], now, reason),
+ )
+ self._purge(mid)
+ self._audit("forget", None, {"count": len(chain), "reason": reason})
+ self._db.commit()
+ return len(chain)
+
+ def pin(self, memory_id: str, pinned: bool = True) -> bool:
+ with self._lock:
+ cur = self._db.execute(
+ "UPDATE memory SET pinned=? WHERE id=?", (1 if pinned else 0, memory_id)
+ )
+ if cur.rowcount:
+ self._audit("pin" if pinned else "unpin", memory_id, {})
+ self._db.commit()
+ return bool(cur.rowcount)
+
+ def mark_used(self, memory_ids: Iterable[str]) -> None:
+ ids = list(memory_ids)
+ if not ids:
+ return
+ with self._lock:
+ now = self.now()
+ self._db.executemany(
+ "UPDATE memory SET use_count = use_count + 1, last_used_at=? WHERE id=?",
+ [(now, mid) for mid in ids],
+ )
+ self._db.commit()
+
+ # -- reads -------------------------------------------------------------
+
+ def get(self, memory_id: str) -> Memory | None:
+ row = self._db.execute(
+ f"SELECT {_COLUMNS} FROM memory WHERE id=?", # noqa: S608
+ (memory_id,),
+ ).fetchone()
+ return self._row_to_memory(row) if row else None
+
+ def list(
+ self,
+ *,
+ kind: str | None = None,
+ scope: str | None = None,
+ include_closed: bool = False,
+ limit: int | None = None,
+ ) -> list[Memory]:
+ """Memories, newest first. Current ones only unless asked otherwise."""
+ where: list[str] = []
+ args: list[Any] = []
+ if not include_closed:
+ where.append(_CURRENT)
+ args.append(self.now())
+ if kind:
+ where.append("m.kind = ?")
+ args.append(kind)
+ if scope:
+ where.append("m.scope IN ('global', ?)")
+ args.append(scope)
+ sql = f"SELECT {_prefixed()} FROM memory m" # noqa: S608
+ if where:
+ sql += " WHERE " + " AND ".join(where)
+ sql += " ORDER BY m.recorded_at DESC, m.id DESC"
+ if limit:
+ sql += f" LIMIT {int(limit)}"
+ return [self._row_to_memory(r) for r in self._db.execute(sql, args)]
+
+ def history(self, memory_id: str) -> list[Memory]:
+ """Every version of a memory, oldest first."""
+ ids = self._chain_ids(memory_id)
+ found = {m.id: m for m in (self.get(i) for i in ids) if m is not None}
+ # Follow the supersede links; time alone cannot order two versions
+ # recorded in the same second.
+ depth: dict[str, int] = {}
+
+ def _depth(mid: str) -> int:
+ if mid not in depth:
+ prev = found[mid].supersedes_id
+ depth[mid] = 0 if prev not in found else _depth(prev) + 1
+ return depth[mid]
+
+ return sorted(found.values(), key=lambda m: (_depth(m.id), m.recorded_at, m.id))
+
+ def about(self, entity: str, *, include_closed: bool = True) -> list[Memory]:
+ """Every memory that names an entity, oldest first ("where did I live before?")."""
+ rows = self._db.execute(
+ f"SELECT {_prefixed()} FROM memory m " # noqa: S608
+ "JOIN memory_entity me ON me.memory_id = m.id "
+ "JOIN entity e ON e.id = me.entity_id WHERE e.name_key = ? "
+ "ORDER BY m.recorded_at, m.id",
+ (entity.casefold().strip(),),
+ )
+ memories = [self._row_to_memory(r) for r in rows]
+ if include_closed:
+ return memories
+ now = self.now()
+ return [m for m in memories if m.is_current(now)]
+
+ def entities(self) -> list[dict[str, Any]]:
+ rows = self._db.execute(
+ "SELECT e.id, e.name, e.created_at, COUNT(me.memory_id) AS uses FROM entity e "
+ "LEFT JOIN memory_entity me ON me.entity_id = e.id GROUP BY e.id ORDER BY e.name"
+ )
+ return [dict(r) for r in rows]
+
+ def search(
+ self,
+ query: str,
+ *,
+ limit: int = 8,
+ kinds: Sequence[str] | None = None,
+ scope: str | None = None,
+ include_closed: bool = False,
+ ) -> list[SearchHit]:
+ """Keyword search with entity and recency boosts. Best hit first.
+
+ Each word matches as a prefix, any word may match (OR), and BM25
+ ranks the rows; a hit that names an entity from the query, or that
+ is pinned or recent, moves up.
+ """
+ tokens = [t for t in _TOKEN.findall(query.casefold()) if len(t) > 1]
+ if not tokens:
+ return []
+ match = " OR ".join(f'"{t}"*' for t in dict.fromkeys(tokens))
+ where = ["memory_fts MATCH ?"]
+ args: list[Any] = [match]
+ if not include_closed:
+ where.append(_CURRENT)
+ args.append(self.now())
+ if kinds:
+ where.append(f"m.kind IN ({','.join('?' * len(kinds))})")
+ args.extend(kinds)
+ if scope:
+ where.append("m.scope IN ('global', ?)")
+ args.append(scope)
+ sql = (
+ f"SELECT {_prefixed()}, bm25(memory_fts) AS rank FROM memory_fts " # noqa: S608
+ "JOIN memory m ON m.id = memory_fts.id WHERE "
+ + " AND ".join(where)
+ + " ORDER BY rank LIMIT ?"
+ )
+ args.append(max(limit * 4, 20))
+ try:
+ rows = self._db.execute(sql, args).fetchall()
+ except sqlite3.OperationalError:
+ log.warning("memory vault: search failed for %r", query, exc_info=True)
+ return []
+ query_words = set(tokens)
+ hits: list[SearchHit] = []
+ for row in rows:
+ memory = self._row_to_memory(row)
+ score = -float(row["rank"]) # bm25 is lower-is-better
+ entity_words = {w for e in memory.entities for w in _TOKEN.findall(e.casefold())}
+ if entity_words & query_words:
+ score *= 1.5
+ if memory.pinned:
+ score *= 1.2
+ score *= _recency(memory.recorded_at, self.now())
+ hits.append(SearchHit(memory, score))
+ hits.sort(key=lambda h: h.score, reverse=True)
+ return hits[:limit]
+
+ def profile_block(self, *, max_chars: int = 2400, scope: str | None = None) -> str:
+ """The always-on summary for the system prompt.
+
+ Pinned memories first, then the stable kinds in a fixed order. Episodes
+ and sensitive memories stay out; they reach the model through search.
+ A budget cut drops whole lines, oldest last-used first.
+ """
+ current = [
+ m
+ for m in self.list(scope=scope)
+ if m.sensitivity == "normal" and (m.pinned or m.kind in _PROFILE_ORDER)
+ ]
+ if not current:
+ return ""
+ groups: dict[str, list[Memory]] = {}
+ for m in current:
+ groups.setdefault("pinned" if m.pinned else m.kind, []).append(m)
+ lines: list[str] = []
+ budget = max_chars
+ for key in ("pinned", *_PROFILE_ORDER):
+ items = sorted(
+ groups.get(key, []),
+ key=lambda m: (m.use_count, m.recorded_at),
+ reverse=True,
+ )
+ if not items:
+ continue
+ heading = f"{_PROFILE_HEADINGS[key]}:"
+ if len(heading) + 1 > budget:
+ break
+ section = [heading]
+ used = len(heading) + 1
+ for m in items:
+ line = f"- {m.text}"
+ if used + len(line) + 1 > budget:
+ break
+ section.append(line)
+ used += len(line) + 1
+ if len(section) > 1:
+ lines.extend(section)
+ budget -= used
+ return "\n".join(lines)
+
+ def stats(self) -> dict[str, Any]:
+ now = self.now()
+ by_kind = {
+ r["kind"]: r["n"]
+ for r in self._db.execute(
+ f"SELECT kind, COUNT(*) AS n FROM memory m WHERE {_CURRENT} GROUP BY kind", # noqa: S608
+ (now,),
+ )
+ }
+ total = self._db.execute("SELECT COUNT(*) FROM memory").fetchone()[0]
+ forgotten = self._db.execute("SELECT COUNT(*) FROM tombstone").fetchone()[0]
+ return {
+ "current": sum(by_kind.values()),
+ "by_kind": by_kind,
+ "all_versions": total,
+ "forgotten": forgotten,
+ "entities": self._db.execute("SELECT COUNT(*) FROM entity").fetchone()[0],
+ }
+
+ def audit_log(self, *, limit: int | None = None) -> list[dict[str, Any]]:
+ sql = "SELECT seq, at, action, memory_id, detail FROM audit ORDER BY seq"
+ if limit:
+ sql += f" DESC LIMIT {int(limit)}"
+ rows = [dict(r) for r in self._db.execute(sql)]
+ for r in rows:
+ r["detail"] = json.loads(r["detail"] or "{}")
+ return rows
+
+ def tombstones(self) -> list[dict[str, Any]]:
+ return [
+ dict(r)
+ for r in self._db.execute(
+ "SELECT id, content_hash, forgotten_at, reason FROM tombstone "
+ "ORDER BY forgotten_at, id"
+ )
+ ]
+
+ # -- import support (used by vault.portable) ----------------------------
+
+ def has(self, memory_id: str) -> bool:
+ return (
+ self._db.execute("SELECT 1 FROM memory WHERE id=?", (memory_id,)).fetchone() is not None
+ )
+
+ def has_hash_version(self, memory: Memory) -> bool:
+ """True if the same statement is already here, current or as the same version."""
+ row = self._db.execute(
+ f"SELECT 1 FROM memory m WHERE m.content_hash=? AND m.scope=? " # noqa: S608
+ f"AND (m.recorded_at=? OR {_CURRENT})",
+ (memory.content_hash, memory.scope, memory.recorded_at, self.now()),
+ ).fetchone()
+ return row is not None
+
+ def insert_imported(self, memory: Memory) -> bool:
+ """Insert a memory exactly as exported. False if it is known or forgotten."""
+ with self._lock:
+ if self.has(memory.id) or self._is_tombstoned_id(memory.id):
+ return False
+ self._validate(memory.text, memory.kind, memory.origin, memory.sensitivity)
+ if _secret_check(memory.text):
+ return False
+ self._insert(memory)
+ self._audit("import", memory.id, {"kind": memory.kind})
+ self._db.commit()
+ return True
+
+ def apply_tombstone(self, memory_id: str, digest: str, forgotten_at: str, reason: str) -> None:
+ with self._lock:
+ if self.has(memory_id):
+ self._purge(memory_id)
+ for row in self._db.execute(
+ "SELECT id FROM memory WHERE content_hash=?", (digest,)
+ ).fetchall():
+ self._purge(row["id"])
+ self._db.execute(
+ "INSERT OR IGNORE INTO tombstone(id, content_hash, forgotten_at, reason) "
+ "VALUES (?, ?, ?, ?)",
+ (memory_id, digest, forgotten_at, reason),
+ )
+ self._db.commit()
+
+ # -- internals ---------------------------------------------------------
+
+ def _validate(self, text: str, kind: str, origin: str, sensitivity: str) -> None:
+ if not text:
+ raise ValueError("a memory needs text")
+ if len(text) > 2000:
+ raise ValueError("a memory is one statement; split long text")
+ if kind not in KINDS:
+ raise ValueError(f"unknown kind {kind!r}")
+ if origin not in ORIGINS:
+ raise ValueError(f"unknown origin {origin!r}")
+ if sensitivity not in SENSITIVITIES:
+ raise ValueError(f"unknown sensitivity {sensitivity!r}")
+
+ def _insert(self, m: Memory) -> None:
+ self._db.execute(
+ f"INSERT INTO memory({_COLUMNS}) VALUES ({','.join('?' * 17)})", # noqa: S608
+ (
+ m.id,
+ m.content_hash,
+ m.kind,
+ m.text,
+ m.scope,
+ m.valid_from,
+ m.valid_to,
+ m.recorded_at,
+ m.expired_at,
+ m.supersedes_id,
+ m.origin,
+ m.source_json(),
+ m.confidence,
+ m.sensitivity,
+ 1 if m.pinned else 0,
+ m.use_count,
+ m.last_used_at,
+ ),
+ )
+ self._db.execute(
+ "INSERT INTO memory_fts(id, text, entities) VALUES (?, ?, ?)",
+ (m.id, m.text, " ".join(m.entities)),
+ )
+ for name in m.entities:
+ key = name.casefold()
+ row = self._db.execute("SELECT id FROM entity WHERE name_key=?", (key,)).fetchone()
+ if row is None:
+ entity_id = f"ent_{secrets.token_hex(6)}"
+ self._db.execute(
+ "INSERT INTO entity(id, name, name_key, created_at) VALUES (?, ?, ?, ?)",
+ (entity_id, name, key, m.recorded_at),
+ )
+ else:
+ entity_id = row["id"]
+ self._db.execute(
+ "INSERT OR IGNORE INTO memory_entity(memory_id, entity_id) VALUES (?, ?)",
+ (m.id, entity_id),
+ )
+
+ def _close(self, memory_id: str, *, valid_to: str | None, expired_at: str | None) -> None:
+ self._db.execute(
+ "UPDATE memory SET valid_to = COALESCE(?, valid_to), expired_at = ? WHERE id=?",
+ (valid_to, expired_at, memory_id),
+ )
+
+ def _purge(self, memory_id: str) -> None:
+ self._db.execute("DELETE FROM memory WHERE id=?", (memory_id,))
+ self._db.execute("DELETE FROM memory_fts WHERE id=?", (memory_id,))
+ self._db.execute("DELETE FROM memory_entity WHERE memory_id=?", (memory_id,))
+ self._db.execute(
+ "UPDATE audit SET detail='{}', memory_id=NULL WHERE memory_id=?", (memory_id,)
+ )
+ self._db.execute("DELETE FROM entity WHERE id NOT IN (SELECT entity_id FROM memory_entity)")
+ # Rows that pointed at a forgotten version keep pointing at nothing.
+ self._db.execute("UPDATE memory SET supersedes_id=NULL WHERE supersedes_id=?", (memory_id,))
+
+ def _chain_ids(self, memory_id: str) -> list[str]:
+ if not self.has(memory_id):
+ return []
+ seen = {memory_id}
+ frontier = [memory_id]
+ while frontier:
+ mid = frontier.pop()
+ row = self._db.execute("SELECT supersedes_id FROM memory WHERE id=?", (mid,)).fetchone()
+ linked = [row["supersedes_id"]] if row and row["supersedes_id"] else []
+ linked += [
+ r["id"]
+ for r in self._db.execute("SELECT id FROM memory WHERE supersedes_id=?", (mid,))
+ ]
+ for other in linked:
+ if other not in seen:
+ seen.add(other)
+ frontier.append(other)
+ return sorted(seen)
+
+ def _current_by_hash(self, digest: str, scope: str, now: str) -> Memory | None:
+ row = self._db.execute(
+ f"SELECT {_prefixed()} FROM memory m " # noqa: S608 - constant columns
+ f"WHERE m.content_hash=? AND m.scope=? AND {_CURRENT}",
+ (digest, scope, now),
+ ).fetchone()
+ return self._row_to_memory(row) if row else None
+
+ def _is_tombstoned(self, digest: str) -> bool:
+ return (
+ self._db.execute("SELECT 1 FROM tombstone WHERE content_hash=?", (digest,)).fetchone()
+ is not None
+ )
+
+ def _is_tombstoned_id(self, memory_id: str) -> bool:
+ return (
+ self._db.execute("SELECT 1 FROM tombstone WHERE id=?", (memory_id,)).fetchone()
+ is not None
+ )
+
+ def _audit(self, action: str, memory_id: str | None, detail: dict[str, Any]) -> None:
+ self._db.execute(
+ "INSERT INTO audit(at, action, memory_id, detail) VALUES (?, ?, ?, ?)",
+ (self.now(), action, memory_id, json.dumps(detail, ensure_ascii=False)),
+ )
+
+ def _row_to_memory(self, row: sqlite3.Row) -> Memory:
+ names = [
+ r["name"]
+ for r in self._db.execute(
+ "SELECT e.name FROM memory_entity me JOIN entity e ON e.id = me.entity_id "
+ "WHERE me.memory_id=? ORDER BY e.name",
+ (row["id"],),
+ )
+ ]
+ return Memory(
+ id=row["id"],
+ content_hash=row["content_hash"],
+ kind=row["kind"],
+ text=row["text"],
+ recorded_at=row["recorded_at"],
+ origin=row["origin"],
+ entities=tuple(names),
+ scope=row["scope"],
+ valid_from=row["valid_from"],
+ valid_to=row["valid_to"],
+ expired_at=row["expired_at"],
+ supersedes_id=row["supersedes_id"],
+ source=json.loads(row["source"] or "{}"),
+ confidence=float(row["confidence"]),
+ sensitivity=row["sensitivity"],
+ pinned=bool(row["pinned"]),
+ use_count=int(row["use_count"]),
+ last_used_at=row["last_used_at"],
+ )
+
+
+def _prefixed() -> str:
+ return ", ".join(f"m.{c.strip()}" for c in _COLUMNS.split(","))
+
+
+def _recency(recorded_at: str, now: str) -> float:
+ """1.0 for today, sliding to 0.7 for memories a few years old."""
+ try:
+ age_days = (
+ datetime.fromisoformat(now) - datetime.fromisoformat(recorded_at)
+ ).total_seconds() / 86400
+ except ValueError: # an unparsable stamp gets no recency penalty, by design
+ return 1.0
+ return 0.7 + 0.3 * math.exp(-max(age_days, 0.0) / 365.0)
diff --git a/tests/unit/memory/vault/__init__.py b/tests/unit/memory/vault/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/unit/memory/vault/test_portable.py b/tests/unit/memory/vault/test_portable.py
new file mode 100644
index 0000000000..90d5c57b1a
--- /dev/null
+++ b/tests/unit/memory/vault/test_portable.py
@@ -0,0 +1,111 @@
+"""Export and import of the personal memory vault (ADR-0044)."""
+
+from __future__ import annotations
+
+import io
+import json
+import zipfile
+
+import pytest
+
+from dodaios.memory.vault import MemoryVault
+from dodaios.memory.vault.portable import (
+ BundleError,
+ export_bundle,
+ export_markdown,
+ export_zip_bytes,
+ import_bundle,
+ read_bundle,
+)
+
+
+def _clock() -> str:
+ return "2026-03-01T10:00:00+00:00"
+
+
+def source(tmp_path) -> MemoryVault:
+ v = MemoryVault(tmp_path / "a.db", clock=_clock)
+ old = v.remember("I live in Hamburg.", "profile", entities=["Hamburg"]).memory
+ v.supersede(old.id, "I live in Berlin.", entities=["Berlin"])
+ v.remember("Prefers German replies.", "preference", pinned=True)
+ gone = v.remember("Secret crush on Alex.", "relationship", entities=["Alex"]).memory
+ v.forget(gone.id)
+ yield v
+ v.close()
+
+
+def test_folder_round_trip_is_lossless_and_idempotent(source, tmp_path):
+ bundle = export_bundle(source, tmp_path / "bundle")
+ assert (bundle / "manifest.json").exists()
+ assert (bundle / "items" / "profile.jsonl").exists()
+
+ with MemoryVault(tmp_path / "b.db", clock=_clock) as target:
+ first = import_bundle(target, bundle)
+ assert first.added == 3 and not first.errors
+ assert {m.text for m in target.list()} == {"I live in Berlin.", "Prefers German replies."}
+ current = next(m for m in target.list() if m.kind == "profile")
+ assert [m.text for m in target.history(current.id)] == [
+ "I live in Hamburg.",
+ "I live in Berlin.",
+ ]
+ assert any(m.pinned for m in target.list())
+
+ second = import_bundle(target, bundle)
+ assert second.added == 0 and second.skipped == 3
+ assert target.stats()["all_versions"] == 3
+
+
+def test_tombstones_travel_and_erase_on_the_other_side(source, tmp_path):
+ with MemoryVault(tmp_path / "b.db", clock=_clock) as target:
+ target.remember("Secret crush on Alex.", "relationship")
+ import_bundle(target, export_zip_bytes(source))
+ assert target.search("Alex") == []
+ assert target.remember(
+ "Secret crush on Alex.", "relationship", origin="inferred"
+ ).action == ("blocked")
+
+
+def test_zip_file_export_and_wrapped_folder_zip(source, tmp_path):
+ path = export_bundle(source, tmp_path / "memory.zip")
+ assert read_bundle(path)["manifest.json"]
+
+ buffer = io.BytesIO()
+ with zipfile.ZipFile(buffer, "w") as zf:
+ for name, data in read_bundle(path).items():
+ zf.writestr(f"my-memory/{name}", data)
+ assert "manifest.json" in read_bundle(buffer.getvalue())
+
+
+def test_tampered_bundle_is_rejected(source, tmp_path):
+ bundle = export_bundle(source, tmp_path / "bundle")
+ items = bundle / "items" / "profile.jsonl"
+ items.write_text(
+ items.read_text(encoding="utf-8").replace("Berlin", "Munich"), encoding="utf-8"
+ )
+ with pytest.raises(BundleError, match="checksum"):
+ read_bundle(bundle)
+
+
+def test_foreign_or_unsafe_bundles_are_rejected(tmp_path):
+ with pytest.raises(BundleError):
+ read_bundle(b"not a zip")
+ buffer = io.BytesIO()
+ with zipfile.ZipFile(buffer, "w") as zf:
+ zf.writestr("manifest.json", json.dumps({"format": "other", "version": "1"}))
+ with pytest.raises(BundleError, match="format"):
+ read_bundle(buffer.getvalue())
+ buffer = io.BytesIO()
+ with zipfile.ZipFile(buffer, "w") as zf:
+ zf.writestr("../evil.txt", "x")
+ with pytest.raises(BundleError, match="unsafe"):
+ read_bundle(buffer.getvalue())
+
+
+def test_markdown_export_lists_current_memories(source, tmp_path):
+ out = export_markdown(source, tmp_path / "md")
+ index = (out / "MEMORY.md").read_text(encoding="utf-8")
+ assert "[About me](profile.md)" in index
+ profile = (out / "profile.md").read_text(encoding="utf-8")
+ assert "I live in Berlin." in profile and "Hamburg" not in profile
+ assert "Alex" not in "".join(p.read_text(encoding="utf-8") for p in out.iterdir())
diff --git a/tests/unit/memory/vault/test_store.py b/tests/unit/memory/vault/test_store.py
new file mode 100644
index 0000000000..47bccc83b1
--- /dev/null
+++ b/tests/unit/memory/vault/test_store.py
@@ -0,0 +1,173 @@
+"""Personal memory vault store (ADR-0044)."""
+
+from __future__ import annotations
+
+import pytest
+
+from dodaios.memory.vault import MemoryVault
+
+
+class Clock:
+ def __init__(self) -> None:
+ self.value = "2026-01-01T09:00:00+00:00"
+
+ def __call__(self) -> str:
+ return self.value
+
+
+def clock() -> Clock:
+ return Clock()
+
+
+def vault(tmp_path, clock) -> MemoryVault:
+ v = MemoryVault(tmp_path / "vault.db", clock=clock)
+ yield v
+ v.close()
+
+
+def test_remember_then_same_statement_is_not_duplicated(vault):
+ first = vault.remember("I live in Hamburg.", "profile", entities=["Hamburg"])
+ again = vault.remember("i live in hamburg", "profile")
+ assert first.action == "added"
+ assert again.action == "existing"
+ assert again.memory.id == first.memory.id
+ assert vault.stats()["current"] == 1
+
+
+def test_supersede_keeps_history_and_answers_both_now_and_before(vault, clock):
+ old = vault.remember("I live in Hamburg.", "profile", entities=["Hamburg"]).memory
+ clock.value = "2026-06-01T09:00:00+00:00"
+ new = vault.supersede(old.id, "I live in Berlin.", entities=["Berlin"]).memory
+
+ current = vault.list()
+ assert [m.text for m in current] == ["I live in Berlin."]
+ closed = vault.get(old.id)
+ assert closed.valid_to == "2026-06-01T09:00:00+00:00"
+ assert closed.expired_at is not None
+ assert new.supersedes_id == old.id
+ assert [m.text for m in vault.history(new.id)] == ["I live in Hamburg.", "I live in Berlin."]
+ assert [m.text for m in vault.about("hamburg")] == ["I live in Hamburg."]
+
+
+def test_invalidate_closes_without_replacement(vault):
+ m = vault.remember("I work at Acme.", "profile", entities=["Acme"]).memory
+ assert vault.invalidate(m.id, reason="quit")
+ assert vault.list() == []
+ assert vault.list(include_closed=True)[0].id == m.id
+ assert not vault.invalidate(m.id)
+
+
+def test_forget_purges_every_version_and_blocks_inferred_rewrite(vault):
+ old = vault.remember("My ex is called Sam.", "relationship", entities=["Sam"]).memory
+ new = vault.supersede(old.id, "My ex Sam moved to Paris.").memory
+
+ assert vault.forget(new.id) == 2
+ assert vault.get(old.id) is None and vault.get(new.id) is None
+ assert vault.search("Sam") == []
+ assert vault.entities() == []
+ assert all("Sam" not in str(row["detail"]) for row in vault.audit_log())
+ assert len(vault.tombstones()) == 2
+
+ blocked = vault.remember("My ex Sam moved to Paris.", "relationship", origin="inferred")
+ assert blocked.action == "blocked"
+ # The user saying it again is their choice.
+ assert vault.remember("My ex Sam moved to Paris.", "relationship").action == "added"
+
+
+def test_secrets_are_refused(vault):
+ fake_key = "sk-" + "proj-" + "abcdefghijklmnopqrstuvwxyz0123456789ABCD"
+ result = vault.remember(f"My OpenAI key is {fake_key}")
+ assert result.action == "refused"
+ assert vault.stats()["current"] == 0
+
+
+def test_ephemeral_memory_expires(vault, clock):
+ vault.remember(
+ "I have a dentist appointment tomorrow.",
+ "ephemeral",
+ valid_to="2026-01-02T23:59:59+00:00",
+ )
+ assert len(vault.list()) == 1
+ clock.value = "2026-01-03T00:00:00+00:00"
+ assert vault.list() == []
+ with pytest.raises(ValueError):
+ vault.remember("Something short-lived.", "ephemeral")
+
+
+def test_search_ranks_entity_matches_and_hides_closed_rows(vault):
+ vault.remember("Viktoria is my sister.", "relationship", entities=["Viktoria"])
+ vault.remember("I had dinner with friends in San Francisco.", "episode")
+ old = vault.remember("My favourite food is sushi.", "preference").memory
+ vault.supersede(old.id, "My favourite food is ramen.")
+
+ hits = vault.search("who is viktoria")
+ assert hits[0].memory.text == "Viktoria is my sister."
+ assert [h.memory.text for h in vault.search("favourite food")] == [
+ "My favourite food is ramen."
+ ]
+ assert len(vault.search("favourite food", include_closed=True)) == 2
+ assert vault.search(" ") == []
+ assert vault.search('"unbalanced') == []
+
+
+def test_search_ignores_accents(vault):
+ vault.remember("My favourite place is the Café de Flore.", "preference")
+ assert vault.search("cafe")[0].memory.text == "My favourite place is the Café de Flore."
+
+
+def test_profile_block_order_budget_and_exclusions(vault):
+ vault.remember("Name: Ruben.", "profile")
+ vault.remember("Prefers short answers.", "preference")
+ vault.remember("Never send emails without asking.", "procedural")
+ vault.remember("Went hiking last Sunday.", "episode")
+ vault.remember("Has a chronic illness.", "profile", sensitivity="sensitive")
+ vault.remember("Wife is called Lea.", "relationship", pinned=True)
+
+ block = vault.profile_block()
+ assert block.index("Pinned:") < block.index("About the user:") < block.index("Preferences:")
+ assert "Went hiking" not in block
+ assert "chronic" not in block
+ assert block.count("Wife is called Lea.") == 1
+
+ small = vault.profile_block(max_chars=40)
+ assert len(small) <= 40
+ assert small.startswith("Pinned:")
+
+
+def test_scope_keeps_projects_apart(vault):
+ vault.remember("Uses tabs.", "preference", scope="project:a")
+ vault.remember("Uses spaces.", "preference", scope="project:b")
+ vault.remember("Likes dark mode.", "preference")
+ texts = {m.text for m in vault.list(scope="project:a")}
+ assert texts == {"Uses tabs.", "Likes dark mode."}
+
+
+def test_validation(vault):
+ with pytest.raises(ValueError):
+ vault.remember("", "profile")
+ with pytest.raises(ValueError):
+ vault.remember("x", "diary")
+ with pytest.raises(ValueError):
+ vault.remember("x", "profile", origin="robot")
+ with pytest.raises(KeyError):
+ vault.supersede("mem_missing", "y")
+
+
+def test_pin_and_usage_counters(vault):
+ m = vault.remember("Drinks coffee black.", "preference").memory
+ assert vault.pin(m.id)
+ vault.mark_used([m.id, m.id])
+ got = vault.get(m.id)
+ assert got.pinned and got.use_count == 2 and got.last_used_at
+
+
+def test_reopen_keeps_data(tmp_path, clock):
+ path = tmp_path / "vault.db"
+ with MemoryVault(path, clock=clock) as v:
+ v.remember("Born in 1990.", "profile")
+ vault_id = v.vault_id
+ with MemoryVault(path, clock=clock) as v:
+ assert v.vault_id == vault_id
+ assert v.list()[0].text == "Born in 1990."