Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
feat/personal memory vault (#1093)
Ruben Lütkeeccd6fee4f5e
diff --git a/docs/adr/0044-personal-memory-vault.md b/docs/adr/0044-personal-memory-vault.mdnew file mode 100644index 0000000000..60c8d7a5f0--- /dev/null+++ b/docs/adr/0044-personal-memory-vault.md@@ -0,0 +1,75 @@+# ADR-0044 — A personal memory vault replaces the wiki vault++**Status:** Accepted · **Date:** 2026-11-02++## Context++The main assistant's long-term memory is an Obsidian-style Markdown vault+(`wiki/obsidian-vault`, ADR-0013) written by a two-stage LLM curator that+rewrites whole pages, plus five side stores that grew next to it: the+learned notebooks (`society/jarvis/MEMORY.md`, `USER.md`), `core_memory.json`,+`data/workspace/USER.md`, the identity card and the contacts mirror. Six+stores, ~30k lines, and still no answer to "what do you know about me?", no+history of how a fact changed, and no export.++What the assistants that do this well converged on in 2025–2026+(ChatGPT saved memories and "dreaming", Claude memory and its memory tool,+Meta Muse's `MEMORY.md`, Gemini memory, Mem0, Zep/Graphiti, Letta,+Supermemory):++1. **Atomic memories, not pages.** One sentence per memory, typed, with its+ source. A page is a view, never the unit of storage.+2. **A small profile always in the prompt, the rest searched per turn.**+3. **Supersede, never overwrite.** A changed fact closes the old one with a+ date (bi-temporal validity) instead of deleting it, so the assistant knows+ both "now" and "before".+4. **Write in the background, reconcile against what exists.** A candidate+ fact is compared with its nearest memories and becomes ADD, UPDATE,+ INVALIDATE or NOOP.+5. **The user sees and controls everything:** view, edit, pin, forget, a+ per-chat off switch.+6. **Portable.** No provider format has won; the most complete open proposal+ is MacPaw's Portable Memory (`.mem`): a folder of JSONL per kind, a+ manifest, an audit log, tombstones applied first, checksums,+ content-derived ids so a re-import merges.++## Decision++1. **One store.** `dodaios/memory/vault/` owns the main assistant's+ long-term memory in one SQLite file, `data/memory/vault.db`, that belongs+ to the user. Tables: `memory`, `entity`, `memory_entity`, `audit`,+ `tombstone`, and an FTS5 index. Embeddings, when present, are an optional+ extra table; search works without them.+2. **The memory row.** `id`, `content_hash`, `kind`, `text` (one sentence),+ `entities`, `scope`, `valid_from`/`valid_to`, `recorded_at`/`expired_at`,+ `supersedes_id`, `origin` (user / inferred / imported), `source`,+ `confidence`, `sensitivity`, `pinned`, `use_count`, `last_used_at`.+ Kinds: `profile`, `preference`, `relationship`, `project`, `procedural`,+ `episode`, `reference`, `ephemeral`.+3. **Writes.** An explicit "remember / forget" is applied at once without a+ model call. Everything else goes through one background extractor that+ reconciles each candidate against the nearest current memories. Nothing+ is deleted unless the user asks; a forget leaves a tombstone and purges+ every derived copy.+4. **Reads.** A budgeted profile block (pinned, profile, preference,+ procedural, active projects) sits in the cached system prompt; a hybrid+ search adds the few memories relevant to the turn. Voice engines get the+ same profile block and the same search tool.+5. **Export and import.** A `.mem`-compatible bundle (lossless) and a+ Markdown rendering (readable) from the Memory section; import is+ idempotent. The old vault, notebooks, core memory and USER.md are imported+ once, then the wiki subsystem is removed.+6. **Scope.** This ADR covers the main assistant. Agent notebooks keep their+ format and move out of the vault folder when the vault is removed.++Supersedes ADR-0013 (knowledge wiki) and ADR-0015 (Obsidian setup wizard)+once the removal chunk lands. The plan lives in+`docs/memory-vault-plan.md`.++## Consequences++- One place to look, one file to back up, one button to export.+- The Obsidian integration goes away. A Markdown export still opens in+ Obsidian, but it is a copy, not the store.+- Removing ~30k lines of wiki code and ~21k lines of its tests is done in a+ separate, last chunk, after the import has run on real data.diff --git a/docs/memory-vault-plan.md b/docs/memory-vault-plan.mdnew file mode 100644index 0000000000..fd28f7797a--- /dev/null+++ b/docs/memory-vault-plan.md@@ -0,0 +1,37 @@+# Personal memory vault — build plan++Decision: ADR-0044. Each chunk ships as its own pull request and leaves the+app working.++| # | Chunk | Done when |+|---|-------|-----------|+| 1 | **Core store** — `dodaios/memory/vault/`: schema, add/supersede/forget/pin, search (FTS5 BM25 + entity + recency), history, profile block, audit, tombstones, `.mem` bundle + Markdown export, idempotent import | unit tests cover every write path and an export → import round trip |+| 2 | **Writes** — explicit remember/forget tools on the new store; one background extractor (candidate facts → nearest memories → ADD / UPDATE / INVALIDATE / NOOP) on the existing background provider chain and call cap; sensitivity rules; off switch per chat | a conversation that states, changes and revokes a fact leaves exactly the right current and closed rows |+| 3 | **Reads** — profile block in the cached system prompt, per-turn relevant memories in the turn context, `memory-search` / `memory-history` tools, the same block for the realtime and GPT-Live engines; replaces the wiki injector, identity card and core-memory block | the main bot answers "what do you know about me?" and "where did I live before?" from the store, in text and voice |+| 4 | **Import of today's memory** — one-time importer for the wiki vault pages, the learned notebooks, `core_memory.json` and `USER.md`; keeps a tar.gz of the old vault | a copy of the real vault imports with no secrets and no duplicates |+| 5 | **Memory section** — new UI: profile, all memories by kind, timeline of changes, edit / pin / forget, export and import buttons | visible in the running desktop app |+| 6 | **Removal** — delete the wiki subsystem, its routes, tools, Obsidian setup, Command Deck wiki graph, config sections and tests; move agent notebooks to `data/agents/` | suite green, no import of `dodaios.memory.wiki` left |++## Store layout (chunk 1)++- `data/memory/vault.db` — SQLite, WAL.+- `memory` rows are never updated in place except for `pinned`, use+ counters and closing (`valid_to`, `expired_at`). An edit is a new row that+ supersedes the old one.+- `current` = `expired_at IS NULL AND (valid_to IS NULL OR valid_to > now)`.+- `tombstone(id, content_hash, forgotten_at, reason)`: a forgotten memory's+ text is purged from `memory`, the FTS index and the audit log.++## Bundle format (`.mem`-compatible)++```+manifest.json format, version, created_at, counts, app+items/<kind>.jsonl one memory per line, all fields+entities.jsonl+audit/log.jsonl+audit/tombstones.jsonl applied first on import+CHECKSUMS sha256 per file+```++Import matches on `id`, then on `content_hash`, so the same bundle imported+twice changes nothing.diff --git a/dodaios/memory/vault/__init__.py b/dodaios/memory/vault/__init__.pynew file mode 100644index 0000000000..7cc8b59e9a--- /dev/null+++ b/dodaios/memory/vault/__init__.py@@ -0,0 +1,6 @@+"""Personal memory vault: the main assistant's long-term memory (ADR-0044)."""++from dodaios.memory.vault.models import KINDS, Memory, SearchHit, WriteResult+from dodaios.memory.vault.store import MemoryVault++__all__ = ["KINDS", "Memory", "MemoryVault", "SearchHit", "WriteResult"]diff --git a/dodaios/memory/vault/models.py b/dodaios/memory/vault/models.pynew file mode 100644index 0000000000..53c0e6c220--- /dev/null+++ b/dodaios/memory/vault/models.py@@ -0,0 +1,110 @@+"""Data types of the personal memory vault (ADR-0044)."""++from __future__ import annotations++import hashlib+import json+import re+import unicodedata+from dataclasses import asdict, dataclass, field+from typing import Any++#: What a memory is about. The profile block and the UI group by it.+KINDS: tuple[str, ...] = (+ "profile", # who the user is: name, role, home, languages+ "preference", # likes, dislikes, how they want things done+ "relationship", # people and how they relate to the user+ "project", # ongoing goals, plans, deadlines, decisions+ "procedural", # standing rules and corrections for the assistant+ "episode", # a notable event or conversation+ "reference", # where things are: accounts, places, files+ "ephemeral", # true for a short while; carries an end date+)++#: Who put the memory there.+ORIGINS: tuple[str, ...] = ("user", "inferred", "imported")++#: ``sensitive`` memories never enter the always-on profile block.+SENSITIVITIES: tuple[str, ...] = ("normal", "sensitive")++_SPACE = re.compile(r"\s+")+++def normalize_text(text: str) -> str:+ """One canonical spelling of a memory sentence, for duplicate detection."""+ folded = unicodedata.normalize("NFKC", text).casefold()+ folded = _SPACE.sub(" ", folded).strip()+ return folded.rstrip(" .!")+++def content_hash(kind: str, text: str) -> str:+ """Stable id of what a memory says, independent of when it was said."""+ digest = hashlib.sha256(f"{kind}\n{normalize_text(text)}".encode()).hexdigest()+ return f"sha256:{digest}"+++@dataclass(frozen=True)+class Memory:+ """One atomic memory: a single sentence about the user or their world."""++ id: str+ content_hash: str+ kind: str+ text: str+ recorded_at: str+ origin: str+ entities: tuple[str, ...] = ()+ scope: str = "global"+ valid_from: str | None = None+ valid_to: str | None = None+ expired_at: str | None = None+ supersedes_id: str | None = None+ source: dict[str, Any] = field(default_factory=dict)+ confidence: float = 1.0+ sensitivity: str = "normal"+ pinned: bool = False+ use_count: int = 0+ last_used_at: str | None = None++ def is_current(self, now: str) -> bool:+ """True while the memory is neither replaced nor past its end date."""+ if self.expired_at is not None:+ return False+ return self.valid_to is None or self.valid_to > now++ def to_dict(self) -> dict[str, Any]:+ data = asdict(self)+ data["entities"] = list(self.entities)+ return data++ @classmethod+ def from_dict(cls, data: dict[str, Any]) -> Memory:+ known = {f for f in cls.__dataclass_fields__}+ values = {k: v for k, v in data.items() if k in known}+ values["entities"] = tuple(values.get("entities") or ())+ values["pinned"] = bool(values.get("pinned", False))+ values["source"] = dict(values.get("source") or {})+ return cls(**values)++ def source_json(self) -> str:+ return json.dumps(self.source, ensure_ascii=False, sort_keys=True)+++@dataclass(frozen=True)+class WriteResult:+ """What a write did.++ ``action`` is one of ``added``, ``existing`` (the same memory was already+ current), ``superseded``, ``refused`` (it looked like a secret) or+ ``blocked`` (the user had told the assistant to forget it).+ """++ action: str+ memory: Memory | None = None+ reason: str = ""+++@dataclass(frozen=True)+class SearchHit:+ memory: Memory+ score: floatdiff --git a/dodaios/memory/vault/portable.py b/dodaios/memory/vault/portable.pynew file mode 100644index 0000000000..84755bac86--- /dev/null+++ b/dodaios/memory/vault/portable.py@@ -0,0 +1,299 @@+"""Export and import of the personal memory vault (ADR-0044).++Two outputs:++- **Bundle** (lossless, re-importable), shaped after the open Portable Memory+ proposal (``.mem``): ``manifest.json``, ``items/<kind>.jsonl``,+ ``entities.jsonl``, ``audit/log.jsonl``, ``audit/tombstones.jsonl`` and+ ``CHECKSUMS``. As a folder or a zip file.+- **Markdown** (readable): ``MEMORY.md`` plus one file per kind, which opens in+ any Markdown editor, Obsidian included.++Import applies tombstones first, then inserts every memory whose id and+content are unknown, so importing the same bundle twice changes nothing.+"""++from __future__ import annotations++import hashlib+import io+import json+import zipfile+from dataclasses import dataclass, field+from pathlib import Path+from typing import Any++from dodaios.memory.vault.models import KINDS, Memory+from dodaios.memory.vault.store import MemoryVault++FORMAT = "portable-memory"+FORMAT_VERSION = "1.0"+APP = "dodaios"++_KIND_TITLES = {+ "profile": "About me",+ "preference": "Preferences",+ "relationship": "People",+ "project": "Projects",+ "procedural": "Standing instructions",+ "episode": "Moments",+ "reference": "Where things are",+ "ephemeral": "For now",+}+++class BundleError(ValueError):+ """The bundle is not one this app can read, or it was altered."""+++@dataclass+class ImportReport:+ added: int = 0+ skipped: int = 0+ forgotten: int = 0+ errors: list[str] = field(default_factory=list)++ def to_dict(self) -> dict[str, Any]:+ return {+ "added": self.added,+ "skipped": self.skipped,+ "forgotten": self.forgotten,+ "errors": list(self.errors),+ }+++# -- bundle --------------------------------------------------------------+++def bundle_files(vault: MemoryVault) -> dict[str, bytes]:+ """Every file of a bundle, keyed by its path inside the bundle."""+ memories = vault.list(include_closed=True)+ files: dict[str, bytes] = {}+ for kind in KINDS:+ rows = [m.to_dict() for m in memories if m.kind == kind]+ if rows:+ files[f"items/{kind}.jsonl"] = _jsonl(rows)+ entities = [+ {"id": e["id"], "name": e["name"], "created_at": e["created_at"]} for e in vault.entities()+ ]+ files["entities.jsonl"] = _jsonl(entities)+ files["audit/log.jsonl"] = _jsonl(vault.audit_log())+ files["audit/tombstones.jsonl"] = _jsonl(vault.tombstones())+ stats = vault.stats()+ manifest = {+ "format": FORMAT,+ "version": FORMAT_VERSION,+ "app": APP,+ "vault_id": vault.vault_id,+ "created_at": vault.now(),+ "counts": {+ "memories": len(memories),+ "current": stats["current"],+ "entities": stats["entities"],+ "tombstones": stats["forgotten"],+ },+ "kinds": list(KINDS),+ }+ files["manifest.json"] = json.dumps(manifest, ensure_ascii=False, indent=2).encode()+ files["CHECKSUMS"] = "".join(+ f"{hashlib.sha256(data).hexdigest()} {name}\n" for name, data in sorted(files.items())+ ).encode()+ return files+++def export_bundle(vault: MemoryVault, dest: str | Path) -> Path:+ """Write a bundle. A path ending in ``.zip`` gets a zip file, else a folder."""+ dest = Path(dest)+ files = bundle_files(vault)+ if dest.suffix.lower() == ".zip":+ dest.parent.mkdir(parents=True, exist_ok=True)+ dest.write_bytes(_zip(files))+ return dest+ for name, data in files.items():+ target = dest / name+ target.parent.mkdir(parents=True, exist_ok=True)+ target.write_bytes(data)+ return dest+++def export_zip_bytes(vault: MemoryVault) -> bytes:+ return _zip(bundle_files(vault))+++def read_bundle(src: str | Path | bytes) -> dict[str, bytes]:+ """Load a bundle from a folder, a zip path or zip bytes and verify it."""+ if isinstance(src, bytes):+ files = _unzip(src)+ else:+ path = Path(src)+ if path.is_dir():+ files = {+ p.relative_to(path).as_posix(): p.read_bytes()+ for p in path.rglob("*")+ if p.is_file()+ }+ else:+ files = _unzip(path.read_bytes())+ _verify(files)+ return files+++def import_bundle(vault: MemoryVault, src: str | Path | bytes) -> ImportReport:+ files = read_bundle(src)+ report = ImportReport()+ for row in _rows(files.get("audit/tombstones.jsonl", b"")):+ try:+ vault.apply_tombstone(+ str(row["id"]),+ str(row["content_hash"]),+ str(row.get("forgotten_at") or vault.now()),+ str(row.get("reason") or "imported"),+ )+ report.forgotten += 1+ except (KeyError, TypeError) as exc:+ report.errors.append(f"tombstone: {exc}")+ memories: list[Memory] = []+ for name in sorted(files):+ if name.startswith("items/") and name.endswith(".jsonl"):+ for row in _rows(files[name]):+ try:+ memories.append(Memory.from_dict(row))+ except (TypeError, ValueError) as exc:+ report.errors.append(f"{name}: {exc}")+ # Older versions first, so a row is inserted after the one it replaces.+ memories.sort(key=lambda m: (m.recorded_at, m.id))+ for memory in memories:+ if vault.has(memory.id) or vault.has_hash_version(memory):+ report.skipped += 1+ continue+ try:+ if vault.insert_imported(memory):+ report.added += 1+ else:+ report.skipped += 1+ except ValueError as exc:+ report.errors.append(f"{memory.id}: {exc}")+ return report+++# -- markdown ------------------------------------------------------------+++def markdown_files(vault: MemoryVault) -> dict[str, str]:+ """A readable copy: ``MEMORY.md`` and one page per kind."""+ current = vault.list()+ stats = vault.stats()+ files: dict[str, str] = {}+ index = [+ "# My memory",+ "",+ f"Exported {vault.now()} · {stats['current']} current memories.",+ "",+ ]+ for kind in KINDS:+ items = [m for m in current if m.kind == kind]+ if not items:+ continue+ title = _KIND_TITLES[kind]+ index.append(f"- [{title}]({kind}.md) — {len(items)}")+ page = [+ "---",+ f"kind: {kind}",+ f"count: {len(items)}",+ "---",+ "",+ f"# {title}",+ "",+ ]+ for m in sorted(items, key=lambda m: m.recorded_at):+ line = f"- {m.text}"+ details = []+ if m.entities:+ details.append(", ".join(m.entities))+ if m.valid_from:+ details.append(f"since {m.valid_from[:10]}")+ if m.valid_to:+ details.append(f"until {m.valid_to[:10]}")+ if m.pinned:+ details.append("pinned")+ if details:+ line += f" *({'; '.join(details)})*"+ page.append(f"{line} <!-- {m.id} -->")+ files[f"{kind}.md"] = "\n".join(page) + "\n"+ files["MEMORY.md"] = "\n".join(index) + "\n"+ return files+++def export_markdown(vault: MemoryVault, dest: str | Path) -> Path:+ dest = Path(dest)+ dest.mkdir(parents=True, exist_ok=True)+ for name, text in markdown_files(vault).items():+ (dest / name).write_text(text, encoding="utf-8")+ return dest+++# -- helpers -------------------------------------------------------------+++def _jsonl(rows: list[dict[str, Any]]) -> bytes:+ return "".join(json.dumps(r, ensure_ascii=False, sort_keys=True) + "\n" for r in rows).encode()+++def _rows(data: bytes) -> list[dict[str, Any]]:+ return [json.loads(line) for line in data.decode("utf-8").splitlines() if line.strip()]+++def _zip(files: dict[str, bytes]) -> bytes:+ buffer = io.BytesIO()+ with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as zf:+ for name in sorted(files):+ zf.writestr(name, files[name])+ return buffer.getvalue()+++def _unzip(data: bytes) -> dict[str, bytes]:+ try:+ with zipfile.ZipFile(io.BytesIO(data)) as zf:+ files: dict[str, bytes] = {}+ for info in zf.infolist():+ if info.is_dir():+ continue+ name = info.filename.replace("\\", "/")+ if name.startswith("/") or ".." in name.split("/"):+ raise BundleError(f"unsafe path in bundle: {name}")+ if info.file_size > 200 * 1024 * 1024:+ raise BundleError(f"file too large in bundle: {name}")+ files[name] = zf.read(info)+ # A zip of the folder may wrap everything in one top folder.+ if "manifest.json" not in files:+ roots = {n.split("/", 1)[0] for n in files if "/" in n}+ if len(roots) == 1:+ root = roots.pop() + "/"+ files = {n[len(root) :]: d for n, d in files.items() if n.startswith(root)}+ return files+ except zipfile.BadZipFile as exc:+ raise BundleError("not a zip file") from exc+++def _verify(files: dict[str, bytes]) -> None:+ if "manifest.json" not in files:+ raise BundleError("no manifest.json; not a memory bundle")+ try:+ manifest = json.loads(files["manifest.json"])+ except json.JSONDecodeError as exc:+ raise BundleError("manifest.json is not JSON") from exc+ if manifest.get("format") != FORMAT:+ raise BundleError(f"unknown bundle format {manifest.get('format')!r}")+ if str(manifest.get("version", "")).split(".")[0] != FORMAT_VERSION.split(".")[0]:+ raise BundleError(f"unsupported bundle version {manifest.get('version')!r}")+ checksums = files.get("CHECKSUMS")+ if checksums is None:+ return+ for line in checksums.decode("utf-8").splitlines():+ if not line.strip():+ continue+ digest, _, name = line.partition(" ")+ if name not in files:+ raise BundleError(f"missing file listed in CHECKSUMS: {name}")+ if hashlib.sha256(files[name]).hexdigest() != digest:+ raise BundleError(f"checksum mismatch: {name}")diff --git a/dodaios/memory/vault/store.py b/dodaios/memory/vault/store.pynew file mode 100644index 0000000000..761f290a08--- /dev/null+++ b/dodaios/memory/vault/store.py@@ -0,0 +1,786 @@+"""The personal memory vault: one SQLite file that holds what the assistant+knows about the user (ADR-0044).++Rules the store enforces:++- A memory row is never rewritten. An edit or a changed fact is a new row+ that supersedes the old one; the old one is closed with a date.+- Forgetting purges the text everywhere (row, search index, audit log) and+ leaves a tombstone, so an inferred write can never bring it back.+- Secrets are refused at the door.+"""++from __future__ import annotations++import json+import logging+import math+import re+import secrets+import sqlite3+import threading+from collections.abc import Callable, Iterable, Sequence+from datetime import UTC, datetime+from pathlib import Path+from typing import Any++from dodaios.memory.vault.models import (+ KINDS,+ ORIGINS,+ SENSITIVITIES,+ Memory,+ SearchHit,+ WriteResult,+ content_hash,+)++log = logging.getLogger(__name__)++SCHEMA_VERSION = 1++_SCHEMA = """+CREATE TABLE IF NOT EXISTS meta (+ key TEXT PRIMARY KEY,+ value TEXT NOT NULL+);+CREATE TABLE IF NOT EXISTS memory (+ id TEXT PRIMARY KEY,+ content_hash TEXT NOT NULL,+ kind TEXT NOT NULL,+ text TEXT NOT NULL,+ scope TEXT NOT NULL DEFAULT 'global',+ valid_from TEXT,+ valid_to TEXT,+ recorded_at TEXT NOT NULL,+ expired_at TEXT,+ supersedes_id TEXT,+ origin TEXT NOT NULL,+ source TEXT NOT NULL DEFAULT '{}',+ confidence REAL NOT NULL DEFAULT 1.0,+ sensitivity TEXT NOT NULL DEFAULT 'normal',+ pinned INTEGER NOT NULL DEFAULT 0,+ use_count INTEGER NOT NULL DEFAULT 0,+ last_used_at TEXT+);+CREATE INDEX IF NOT EXISTS memory_hash ON memory(content_hash);+CREATE INDEX IF NOT EXISTS memory_kind ON memory(kind);+CREATE INDEX IF NOT EXISTS memory_supersedes ON memory(supersedes_id);+CREATE TABLE IF NOT EXISTS entity (+ id TEXT PRIMARY KEY,+ name TEXT NOT NULL,+ name_key TEXT NOT NULL UNIQUE,+ created_at TEXT NOT NULL+);+CREATE TABLE IF NOT EXISTS memory_entity (+ memory_id TEXT NOT NULL,+ entity_id TEXT NOT NULL,+ PRIMARY KEY (memory_id, entity_id)+);+CREATE INDEX IF NOT EXISTS memory_entity_by_entity ON memory_entity(entity_id);+CREATE TABLE IF NOT EXISTS audit (+ seq INTEGER PRIMARY KEY AUTOINCREMENT,+ at TEXT NOT NULL,+ action TEXT NOT NULL,+ memory_id TEXT,+ detail TEXT NOT NULL DEFAULT '{}'+);+CREATE TABLE IF NOT EXISTS tombstone (+ id TEXT PRIMARY KEY,+ content_hash TEXT NOT NULL,+ forgotten_at TEXT NOT NULL,+ reason TEXT NOT NULL DEFAULT ''+);+CREATE INDEX IF NOT EXISTS tombstone_hash ON tombstone(content_hash);+CREATE VIRTUAL TABLE IF NOT EXISTS memory_fts USING fts5(+ id UNINDEXED,+ text,+ entities,+ tokenize = 'unicode61 remove_diacritics 2'+);+"""++_COLUMNS = (+ "id, content_hash, kind, text, scope, valid_from, valid_to, recorded_at, "+ "expired_at, supersedes_id, origin, source, confidence, sensitivity, pinned, "+ "use_count, last_used_at"+)++_CURRENT = "m.expired_at IS NULL AND (m.valid_to IS NULL OR m.valid_to > ?)"++_TOKEN = re.compile(r"\w+", re.UNICODE)++#: Order of kinds in the always-on profile block.+_PROFILE_ORDER = ("profile", "preference", "procedural", "relationship", "project", "reference")++_PROFILE_HEADINGS = {+ "profile": "About the user",+ "preference": "Preferences",+ "procedural": "Standing instructions",+ "relationship": "People",+ "project": "Current projects",+ "reference": "Where things are",+ "pinned": "Pinned",+}+++def utc_now() -> str:+ return datetime.now(UTC).isoformat(timespec="seconds")+++def _new_id() -> str:+ stamp = datetime.now(UTC).strftime("%Y%m%d%H%M%S")+ return f"mem_{stamp}_{secrets.token_hex(6)}"+++def _secret_check(text: str) -> bool:+ # The secret detector still lives with the wiki; ADR-0044 chunk 6 moves it.+ from dodaios.memory.wiki.secret_guard import contains_secret++ return contains_secret(text)+++def _clean_entities(names: Iterable[str]) -> tuple[str, ...]:+ seen: dict[str, str] = {}+ for name in names:+ label = " ".join(str(name).split())+ if label and label.casefold() not in seen:+ seen[label.casefold()] = label+ return tuple(seen.values())+++class MemoryVault:+ """Read/write handle for ``vault.db``. Safe to share between threads."""++ def __init__(self, path: str | Path, *, clock: Callable[[], str] = utc_now) -> None:+ self.path = Path(path)+ self._clock = clock+ self._lock = threading.RLock()+ if str(path) != ":memory:":+ self.path.parent.mkdir(parents=True, exist_ok=True)+ self._db = sqlite3.connect(str(path), check_same_thread=False)+ self._db.row_factory = sqlite3.Row+ with self._lock:+ if str(path) != ":memory:":+ self._db.execute("PRAGMA journal_mode=WAL")+ self._db.execute("PRAGMA foreign_keys=ON")+ self._db.executescript(_SCHEMA)+ self._db.execute(+ "INSERT OR IGNORE INTO meta(key, value) VALUES ('schema_version', ?)",+ (str(SCHEMA_VERSION),),+ )+ self._db.execute(+ "INSERT OR IGNORE INTO meta(key, value) VALUES ('vault_id', ?)",+ (f"vault_{secrets.token_hex(8)}",),+ )+ self._db.commit()++ # -- lifecycle ---------------------------------------------------------++ def close(self) -> None:+ with self._lock:+ self._db.close()++ def __enter__(self) -> MemoryVault:+ return self++ def __exit__(self, *_exc: object) -> None:+ self.close()++ @property+ def vault_id(self) -> str:+ row = self._db.execute("SELECT value FROM meta WHERE key='vault_id'").fetchone()+ return str(row["value"])++ def now(self) -> str:+ return self._clock()++ # -- writes ------------------------------------------------------------++ def remember(+ self,+ text: str,+ kind: str = "profile",+ *,+ entities: Sequence[str] = (),+ origin: str = "user",+ source: dict[str, Any] | None = None,+ scope: str = "global",+ confidence: float = 1.0,+ sensitivity: str = "normal",+ pinned: bool = False,+ valid_from: str | None = None,+ valid_to: str | None = None,+ ) -> WriteResult:+ """Store one memory, or return the identical one that is already current."""+ text = " ".join(text.split())+ self._validate(text, kind, origin, sensitivity)+ if _secret_check(text):+ return WriteResult("refused", reason="looks like a secret")+ digest = content_hash(kind, text)+ with self._lock:+ now = self.now()+ existing = self._current_by_hash(digest, scope, now)+ if existing is not None:+ if pinned and not existing.pinned:+ self.pin(existing.id, True)+ existing = self.get(existing.id)+ return WriteResult("existing", existing)+ if self._is_tombstoned(digest):+ if origin != "user":+ return WriteResult("blocked", reason="the user asked to forget this")+ self._db.execute("DELETE FROM tombstone WHERE content_hash=?", (digest,))+ if kind == "ephemeral" and valid_to is None:+ raise ValueError("an ephemeral memory needs valid_to")+ memory = Memory(+ id=_new_id(),+ content_hash=digest,+ kind=kind,+ text=text,+ recorded_at=now,+ origin=origin,+ entities=_clean_entities(entities),+ scope=scope,+ valid_from=valid_from or now,+ valid_to=valid_to,+ source=dict(source or {}),+ confidence=max(0.0, min(1.0, float(confidence))),+ sensitivity=sensitivity,+ pinned=pinned,+ )+ self._insert(memory)+ self._audit("add", memory.id, {"kind": kind, "origin": origin})+ self._db.commit()+ return WriteResult("added", memory)++ def supersede(+ self,+ old_id: str,+ text: str,+ *,+ kind: str | None = None,+ entities: Sequence[str] | None = None,+ origin: str = "user",+ source: dict[str, Any] | None = None,+ confidence: float | None = None,+ valid_from: str | None = None,+ valid_to: str | None = None,+ ) -> WriteResult:+ """Replace a memory with a newer version and close the old one."""+ text = " ".join(text.split())+ with self._lock:+ old = self.get(old_id)+ if old is None:+ raise KeyError(old_id)+ now = self.now()+ if not old.is_current(now):+ raise ValueError(f"{old_id} is no longer current")+ new_kind = kind or old.kind+ self._validate(text, new_kind, origin, old.sensitivity)+ if _secret_check(text):+ return WriteResult("refused", reason="looks like a secret")+ digest = content_hash(new_kind, text)+ if digest == old.content_hash:+ return WriteResult("existing", old)+ if self._is_tombstoned(digest) and origin != "user":+ return WriteResult("blocked", reason="the user asked to forget this")+ memory = Memory(+ id=_new_id(),+ content_hash=digest,+ kind=new_kind,+ text=text,+ recorded_at=now,+ origin=origin,+ entities=_clean_entities(old.entities if entities is None else entities),+ scope=old.scope,+ valid_from=valid_from or now,+ valid_to=valid_to,+ supersedes_id=old.id,+ source=dict(source or {}),+ confidence=old.confidence if confidence is None else confidence,+ sensitivity=old.sensitivity,+ pinned=old.pinned,+ )+ self._close(old.id, valid_to=memory.valid_from, expired_at=now)+ self._insert(memory)+ self._audit("supersede", memory.id, {"replaces": old.id, "origin": origin})+ self._db.commit()+ return WriteResult("superseded", memory)++ def invalidate(self, memory_id: str, *, reason: str = "", when: str | None = None) -> bool:+ """Mark a fact as no longer true without a replacement ("I quit that job")."""+ with self._lock:+ memory = self.get(memory_id)+ now = self.now()+ if memory is None or not memory.is_current(now):+ return False+ self._close(memory_id, valid_to=when or now, expired_at=None)+ self._audit("invalidate", memory_id, {"reason": reason})+ self._db.commit()+ return True++ def forget(self, memory_id: str, *, reason: str = "user request") -> int:+ """Erase a memory and every earlier and later version of it.++ Returns how many rows were erased. The text is gone from the row, the+ search index and the audit log; a tombstone per version remains.+ """+ with self._lock:+ chain = self._chain_ids(memory_id)+ if not chain:+ return 0+ now = self.now()+ for mid in chain:+ row = self._db.execute(+ "SELECT content_hash FROM memory WHERE id=?", (mid,)+ ).fetchone()+ self._db.execute(+ "INSERT OR REPLACE INTO tombstone(id, content_hash, forgotten_at, reason) "+ "VALUES (?, ?, ?, ?)",+ (mid, row["content_hash"], now, reason),+ )+ self._purge(mid)+ self._audit("forget", None, {"count": len(chain), "reason": reason})+ self._db.commit()+ return len(chain)++ def pin(self, memory_id: str, pinned: bool = True) -> bool:+ with self._lock:+ cur = self._db.execute(+ "UPDATE memory SET pinned=? WHERE id=?", (1 if pinned else 0, memory_id)+ )+ if cur.rowcount:+ self._audit("pin" if pinned else "unpin", memory_id, {})+ self._db.commit()+ return bool(cur.rowcount)++ def mark_used(self, memory_ids: Iterable[str]) -> None:+ ids = list(memory_ids)+ if not ids:+ return+ with self._lock:+ now = self.now()+ self._db.executemany(+ "UPDATE memory SET use_count = use_count + 1, last_used_at=? WHERE id=?",+ [(now, mid) for mid in ids],+ )+ self._db.commit()++ # -- reads -------------------------------------------------------------++ def get(self, memory_id: str) -> Memory | None:+ row = self._db.execute(+ f"SELECT {_COLUMNS} FROM memory WHERE id=?", # noqa: S608+ (memory_id,),+ ).fetchone()+ return self._row_to_memory(row) if row else None++ def list(+ self,+ *,+ kind: str | None = None,+ scope: str | None = None,+ include_closed: bool = False,+ limit: int | None = None,+ ) -> list[Memory]:+ """Memories, newest first. Current ones only unless asked otherwise."""+ where: list[str] = []+ args: list[Any] = []+ if not include_closed:+ where.append(_CURRENT)+ args.append(self.now())+ if kind:+ where.append("m.kind = ?")+ args.append(kind)+ if scope:+ where.append("m.scope IN ('global', ?)")+ args.append(scope)+ sql = f"SELECT {_prefixed()} FROM memory m" # noqa: S608+ if where:+ sql += " WHERE " + " AND ".join(where)+ sql += " ORDER BY m.recorded_at DESC, m.id DESC"+ if limit:+ sql += f" LIMIT {int(limit)}"+ return [self._row_to_memory(r) for r in self._db.execute(sql, args)]++ def history(self, memory_id: str) -> list[Memory]:+ """Every version of a memory, oldest first."""+ ids = self._chain_ids(memory_id)+ found = {m.id: m for m in (self.get(i) for i in ids) if m is not None}+ # Follow the supersede links; time alone cannot order two versions+ # recorded in the same second.+ depth: dict[str, int] = {}++ def _depth(mid: str) -> int:+ if mid not in depth:+ prev = found[mid].supersedes_id+ depth[mid] = 0 if prev not in found else _depth(prev) + 1+ return depth[mid]++ return sorted(found.values(), key=lambda m: (_depth(m.id), m.recorded_at, m.id))++ def about(self, entity: str, *, include_closed: bool = True) -> list[Memory]:+ """Every memory that names an entity, oldest first ("where did I live before?")."""+ rows = self._db.execute(+ f"SELECT {_prefixed()} FROM memory m " # noqa: S608+ "JOIN memory_entity me ON me.memory_id = m.id "+ "JOIN entity e ON e.id = me.entity_id WHERE e.name_key = ? "+ "ORDER BY m.recorded_at, m.id",+ (entity.casefold().strip(),),+ )+ memories = [self._row_to_memory(r) for r in rows]+ if include_closed:+ return memories+ now = self.now()+ return [m for m in memories if m.is_current(now)]++ def entities(self) -> list[dict[str, Any]]:+ rows = self._db.execute(+ "SELECT e.id, e.name, e.created_at, COUNT(me.memory_id) AS uses FROM entity e "+ "LEFT JOIN memory_entity me ON me.entity_id = e.id GROUP BY e.id ORDER BY e.name"+ )+ return [dict(r) for r in rows]++ def search(+ self,+ query: str,+ *,+ limit: int = 8,+ kinds: Sequence[str] | None = None,+ scope: str | None = None,+ include_closed: bool = False,+ ) -> list[SearchHit]:+ """Keyword search with entity and recency boosts. Best hit first.++ Each word matches as a prefix, any word may match (OR), and BM25+ ranks the rows; a hit that names an entity from the query, or that+ is pinned or recent, moves up.+ """+ tokens = [t for t in _TOKEN.findall(query.casefold()) if len(t) > 1]+ if not tokens:+ return []+ match = " OR ".join(f'"{t}"*' for t in dict.fromkeys(tokens))+ where = ["memory_fts MATCH ?"]+ args: list[Any] = [match]+ if not include_closed:+ where.append(_CURRENT)+ args.append(self.now())+ if kinds:+ where.append(f"m.kind IN ({','.join('?' * len(kinds))})")+ args.extend(kinds)+ if scope:+ where.append("m.scope IN ('global', ?)")+ args.append(scope)+ sql = (+ f"SELECT {_prefixed()}, bm25(memory_fts) AS rank FROM memory_fts " # noqa: S608+ "JOIN memory m ON m.id = memory_fts.id WHERE "+ + " AND ".join(where)+ + " ORDER BY rank LIMIT ?"+ )+ args.append(max(limit * 4, 20))+ try:+ rows = self._db.execute(sql, args).fetchall()+ except sqlite3.OperationalError:+ log.warning("memory vault: search failed for %r", query, exc_info=True)+ return []+ query_words = set(tokens)+ hits: list[SearchHit] = []+ for row in rows:+ memory = self._row_to_memory(row)+ score = -float(row["rank"]) # bm25 is lower-is-better+ entity_words = {w for e in memory.entities for w in _TOKEN.findall(e.casefold())}+ if entity_words & query_words:+ score *= 1.5+ if memory.pinned:+ score *= 1.2+ score *= _recency(memory.recorded_at, self.now())+ hits.append(SearchHit(memory, score))+ hits.sort(key=lambda h: h.score, reverse=True)+ return hits[:limit]++ def profile_block(self, *, max_chars: int = 2400, scope: str | None = None) -> str:+ """The always-on summary for the system prompt.++ Pinned memories first, then the stable kinds in a fixed order. Episodes+ and sensitive memories stay out; they reach the model through search.+ A budget cut drops whole lines, oldest last-used first.+ """+ current = [+ m+ for m in self.list(scope=scope)+ if m.sensitivity == "normal" and (m.pinned or m.kind in _PROFILE_ORDER)+ ]+ if not current:+ return ""+ groups: dict[str, list[Memory]] = {}+ for m in current:+ groups.setdefault("pinned" if m.pinned else m.kind, []).append(m)+ lines: list[str] = []+ budget = max_chars+ for key in ("pinned", *_PROFILE_ORDER):+ items = sorted(+ groups.get(key, []),+ key=lambda m: (m.use_count, m.recorded_at),+ reverse=True,+ )+ if not items:+ continue+ heading = f"{_PROFILE_HEADINGS[key]}:"+ if len(heading) + 1 > budget:+ break+ section = [heading]+ used = len(heading) + 1+ for m in items:+ line = f"- {m.text}"+ if used + len(line) + 1 > budget:+ break+ section.append(line)+ used += len(line) + 1+ if len(section) > 1:+ lines.extend(section)+ budget -= used+ return "\n".join(lines)++ def stats(self) -> dict[str, Any]:+ now = self.now()+ by_kind = {+ r["kind"]: r["n"]+ for r in self._db.execute(+ f"SELECT kind, COUNT(*) AS n FROM memory m WHERE {_CURRENT} GROUP BY kind", # noqa: S608+ (now,),+ )+ }+ total = self._db.execute("SELECT COUNT(*) FROM memory").fetchone()[0]+ forgotten = self._db.execute("SELECT COUNT(*) FROM tombstone").fetchone()[0]+ return {+ "current": sum(by_kind.values()),+ "by_kind": by_kind,+ "all_versions": total,+ "forgotten": forgotten,+ "entities": self._db.execute("SELECT COUNT(*) FROM entity").fetchone()[0],+ }++ def audit_log(self, *, limit: int | None = None) -> list[dict[str, Any]]:+ sql = "SELECT seq, at, action, memory_id, detail FROM audit ORDER BY seq"+ if limit:+ sql += f" DESC LIMIT {int(limit)}"+ rows = [dict(r) for r in self._db.execute(sql)]+ for r in rows:+ r["detail"] = json.loads(r["detail"] or "{}")+ return rows++ def tombstones(self) -> list[dict[str, Any]]:+ return [+ dict(r)+ for r in self._db.execute(+ "SELECT id, content_hash, forgotten_at, reason FROM tombstone "+ "ORDER BY forgotten_at, id"+ )+ ]++ # -- import support (used by vault.portable) ----------------------------++ def has(self, memory_id: str) -> bool:+ return (+ self._db.execute("SELECT 1 FROM memory WHERE id=?", (memory_id,)).fetchone() is not None+ )++ def has_hash_version(self, memory: Memory) -> bool:+ """True if the same statement is already here, current or as the same version."""+ row = self._db.execute(+ f"SELECT 1 FROM memory m WHERE m.content_hash=? AND m.scope=? " # noqa: S608+ f"AND (m.recorded_at=? OR {_CURRENT})",+ (memory.content_hash, memory.scope, memory.recorded_at, self.now()),+ ).fetchone()+ return row is not None++ def insert_imported(self, memory: Memory) -> bool:+ """Insert a memory exactly as exported. False if it is known or forgotten."""+ with self._lock:+ if self.has(memory.id) or self._is_tombstoned_id(memory.id):+ return False+ self._validate(memory.text, memory.kind, memory.origin, memory.sensitivity)+ if _secret_check(memory.text):+ return False+ self._insert(memory)+ self._audit("import", memory.id, {"kind": memory.kind})+ self._db.commit()+ return True++ def apply_tombstone(self, memory_id: str, digest: str, forgotten_at: str, reason: str) -> None:+ with self._lock:+ if self.has(memory_id):+ self._purge(memory_id)+ for row in self._db.execute(+ "SELECT id FROM memory WHERE content_hash=?", (digest,)+ ).fetchall():+ self._purge(row["id"])+ self._db.execute(+ "INSERT OR IGNORE INTO tombstone(id, content_hash, forgotten_at, reason) "+ "VALUES (?, ?, ?, ?)",+ (memory_id, digest, forgotten_at, reason),+ )+ self._db.commit()++ # -- internals ---------------------------------------------------------++ def _validate(self, text: str, kind: str, origin: str, sensitivity: str) -> None:+ if not text:+ raise ValueError("a memory needs text")+ if len(text) > 2000:+ raise ValueError("a memory is one statement; split long text")+ if kind not in KINDS:+ raise ValueError(f"unknown kind {kind!r}")+ if origin not in ORIGINS:+ raise ValueError(f"unknown origin {origin!r}")+ if sensitivity not in SENSITIVITIES:+ raise ValueError(f"unknown sensitivity {sensitivity!r}")++ def _insert(self, m: Memory) -> None:+ self._db.execute(+ f"INSERT INTO memory({_COLUMNS}) VALUES ({','.join('?' * 17)})", # noqa: S608+ (+ m.id,+ m.content_hash,+ m.kind,+ m.text,+ m.scope,+ m.valid_from,+ m.valid_to,+ m.recorded_at,+ m.expired_at,+ m.supersedes_id,+ m.origin,+ m.source_json(),+ m.confidence,+ m.sensitivity,+ 1 if m.pinned else 0,+ m.use_count,+ m.last_used_at,+ ),+ )+ self._db.execute(+ "INSERT INTO memory_fts(id, text, entities) VALUES (?, ?, ?)",+ (m.id, m.text, " ".join(m.entities)),+ )+ for name in m.entities:+ key = name.casefold()+ row = self._db.execute("SELECT id FROM entity WHERE name_key=?", (key,)).fetchone()+ if row is None:+ entity_id = f"ent_{secrets.token_hex(6)}"+ self._db.execute(+ "INSERT INTO entity(id, name, name_key, created_at) VALUES (?, ?, ?, ?)",+ (entity_id, name, key, m.recorded_at),+ )+ else:+ entity_id = row["id"]+ self._db.execute(+ "INSERT OR IGNORE INTO memory_entity(memory_id, entity_id) VALUES (?, ?)",+ (m.id, entity_id),+ )++ def _close(self, memory_id: str, *, valid_to: str | None, expired_at: str | None) -> None:+ self._db.execute(+ "UPDATE memory SET valid_to = COALESCE(?, valid_to), expired_at = ? WHERE id=?",+ (valid_to, expired_at, memory_id),+ )++ def _purge(self, memory_id: str) -> None:+ self._db.execute("DELETE FROM memory WHERE id=?", (memory_id,))+ self._db.execute("DELETE FROM memory_fts WHERE id=?", (memory_id,))+ self._db.execute("DELETE FROM memory_entity WHERE memory_id=?", (memory_id,))+ self._db.execute(+ "UPDATE audit SET detail='{}', memory_id=NULL WHERE memory_id=?", (memory_id,)+ )+ self._db.execute("DELETE FROM entity WHERE id NOT IN (SELECT entity_id FROM memory_entity)")+ # Rows that pointed at a forgotten version keep pointing at nothing.+ self._db.execute("UPDATE memory SET supersedes_id=NULL WHERE supersedes_id=?", (memory_id,))++ def _chain_ids(self, memory_id: str) -> list[str]:+ if not self.has(memory_id):+ return []+ seen = {memory_id}+ frontier = [memory_id]+ while frontier:+ mid = frontier.pop()+ row = self._db.execute("SELECT supersedes_id FROM memory WHERE id=?", (mid,)).fetchone()+ linked = [row["supersedes_id"]] if row and row["supersedes_id"] else []+ linked += [+ r["id"]+ for r in self._db.execute("SELECT id FROM memory WHERE supersedes_id=?", (mid,))+ ]+ for other in linked:+ if other not in seen:+ seen.add(other)+ frontier.append(other)+ return sorted(seen)++ def _current_by_hash(self, digest: str, scope: str, now: str) -> Memory | None:+ row = self._db.execute(+ f"SELECT {_prefixed()} FROM memory m " # noqa: S608 - constant columns+ f"WHERE m.content_hash=? AND m.scope=? AND {_CURRENT}",+ (digest, scope, now),+ ).fetchone()+ return self._row_to_memory(row) if row else None++ def _is_tombstoned(self, digest: str) -> bool:+ return (+ self._db.execute("SELECT 1 FROM tombstone WHERE content_hash=?", (digest,)).fetchone()+ is not None+ )++ def _is_tombstoned_id(self, memory_id: str) -> bool:+ return (+ self._db.execute("SELECT 1 FROM tombstone WHERE id=?", (memory_id,)).fetchone()+ is not None+ )++ def _audit(self, action: str, memory_id: str | None, detail: dict[str, Any]) -> None:+ self._db.execute(+ "INSERT INTO audit(at, action, memory_id, detail) VALUES (?, ?, ?, ?)",+ (self.now(), action, memory_id, json.dumps(detail, ensure_ascii=False)),+ )++ def _row_to_memory(self, row: sqlite3.Row) -> Memory:+ names = [+ r["name"]+ for r in self._db.execute(+ "SELECT e.name FROM memory_entity me JOIN entity e ON e.id = me.entity_id "+ "WHERE me.memory_id=? ORDER BY e.name",+ (row["id"],),+ )+ ]+ return Memory(+ id=row["id"],+ content_hash=row["content_hash"],+ kind=row["kind"],+ text=row["text"],+ recorded_at=row["recorded_at"],+ origin=row["origin"],+ entities=tuple(names),+ scope=row["scope"],+ valid_from=row["valid_from"],+ valid_to=row["valid_to"],+ expired_at=row["expired_at"],+ supersedes_id=row["supersedes_id"],+ source=json.loads(row["source"] or "{}"),+ confidence=float(row["confidence"]),+ sensitivity=row["sensitivity"],+ pinned=bool(row["pinned"]),+ use_count=int(row["use_count"]),+ last_used_at=row["last_used_at"],+ )+++def _prefixed() -> str:+ return ", ".join(f"m.{c.strip()}" for c in _COLUMNS.split(","))+++def _recency(recorded_at: str, now: str) -> float:+ """1.0 for today, sliding to 0.7 for memories a few years old."""+ try:+ age_days = (+ datetime.fromisoformat(now) - datetime.fromisoformat(recorded_at)+ ).total_seconds() / 86400+ except ValueError: # an unparsable stamp gets no recency penalty, by design+ return 1.0+ return 0.7 + 0.3 * math.exp(-max(age_days, 0.0) / 365.0)diff --git a/tests/unit/memory/vault/__init__.py b/tests/unit/memory/vault/__init__.pynew file mode 100644index 0000000000..e69de29bb2diff --git a/tests/unit/memory/vault/test_portable.py b/tests/unit/memory/vault/test_portable.pynew file mode 100644index 0000000000..90d5c57b1a--- /dev/null+++ b/tests/unit/memory/vault/test_portable.py@@ -0,0 +1,111 @@+"""Export and import of the personal memory vault (ADR-0044)."""++from __future__ import annotations++import io+import json+import zipfile++import pytest++from dodaios.memory.vault import MemoryVault+from dodaios.memory.vault.portable import (+ BundleError,+ export_bundle,+ export_markdown,+ export_zip_bytes,+ import_bundle,+ read_bundle,+)+++def _clock() -> str:+ return "2026-03-01T10:00:00+00:00"+++def source(tmp_path) -> MemoryVault:+ v = MemoryVault(tmp_path / "a.db", clock=_clock)+ old = v.remember("I live in Hamburg.", "profile", entities=["Hamburg"]).memory+ v.supersede(old.id, "I live in Berlin.", entities=["Berlin"])+ v.remember("Prefers German replies.", "preference", pinned=True)+ gone = v.remember("Secret crush on Alex.", "relationship", entities=["Alex"]).memory+ v.forget(gone.id)+ yield v+ v.close()+++def test_folder_round_trip_is_lossless_and_idempotent(source, tmp_path):+ bundle = export_bundle(source, tmp_path / "bundle")+ assert (bundle / "manifest.json").exists()+ assert (bundle / "items" / "profile.jsonl").exists()++ with MemoryVault(tmp_path / "b.db", clock=_clock) as target:+ first = import_bundle(target, bundle)+ assert first.added == 3 and not first.errors+ assert {m.text for m in target.list()} == {"I live in Berlin.", "Prefers German replies."}+ current = next(m for m in target.list() if m.kind == "profile")+ assert [m.text for m in target.history(current.id)] == [+ "I live in Hamburg.",+ "I live in Berlin.",+ ]+ assert any(m.pinned for m in target.list())++ second = import_bundle(target, bundle)+ assert second.added == 0 and second.skipped == 3+ assert target.stats()["all_versions"] == 3+++def test_tombstones_travel_and_erase_on_the_other_side(source, tmp_path):+ with MemoryVault(tmp_path / "b.db", clock=_clock) as target:+ target.remember("Secret crush on Alex.", "relationship")+ import_bundle(target, export_zip_bytes(source))+ assert target.search("Alex") == []+ assert target.remember(+ "Secret crush on Alex.", "relationship", origin="inferred"+ ).action == ("blocked")+++def test_zip_file_export_and_wrapped_folder_zip(source, tmp_path):+ path = export_bundle(source, tmp_path / "memory.zip")+ assert read_bundle(path)["manifest.json"]++ buffer = io.BytesIO()+ with zipfile.ZipFile(buffer, "w") as zf:+ for name, data in read_bundle(path).items():+ zf.writestr(f"my-memory/{name}", data)+ assert "manifest.json" in read_bundle(buffer.getvalue())+++def test_tampered_bundle_is_rejected(source, tmp_path):+ bundle = export_bundle(source, tmp_path / "bundle")+ items = bundle / "items" / "profile.jsonl"+ items.write_text(+ items.read_text(encoding="utf-8").replace("Berlin", "Munich"), encoding="utf-8"+ )+ with pytest.raises(BundleError, match="checksum"):+ read_bundle(bundle)+++def test_foreign_or_unsafe_bundles_are_rejected(tmp_path):+ with pytest.raises(BundleError):+ read_bundle(b"not a zip")+ buffer = io.BytesIO()+ with zipfile.ZipFile(buffer, "w") as zf:+ zf.writestr("manifest.json", json.dumps({"format": "other", "version": "1"}))+ with pytest.raises(BundleError, match="format"):+ read_bundle(buffer.getvalue())+ buffer = io.BytesIO()+ with zipfile.ZipFile(buffer, "w") as zf:+ zf.writestr("../evil.txt", "x")+ with pytest.raises(BundleError, match="unsafe"):+ read_bundle(buffer.getvalue())+++def test_markdown_export_lists_current_memories(source, tmp_path):+ out = export_markdown(source, tmp_path / "md")+ index = (out / "MEMORY.md").read_text(encoding="utf-8")+ assert "[About me](profile.md)" in index+ profile = (out / "profile.md").read_text(encoding="utf-8")+ assert "I live in Berlin." in profile and "Hamburg" not in profile+ assert "Alex" not in "".join(p.read_text(encoding="utf-8") for p in out.iterdir())diff --git a/tests/unit/memory/vault/test_store.py b/tests/unit/memory/vault/test_store.pynew file mode 100644index 0000000000..47bccc83b1--- /dev/null+++ b/tests/unit/memory/vault/test_store.py@@ -0,0 +1,173 @@+"""Personal memory vault store (ADR-0044)."""++from __future__ import annotations++import pytest++from dodaios.memory.vault import MemoryVault+++class Clock:+ def __init__(self) -> None:+ self.value = "2026-01-01T09:00:00+00:00"++ def __call__(self) -> str:+ return self.value+++def clock() -> Clock:+ return Clock()+++def vault(tmp_path, clock) -> MemoryVault:+ v = MemoryVault(tmp_path / "vault.db", clock=clock)+ yield v+ v.close()+++def test_remember_then_same_statement_is_not_duplicated(vault):+ first = vault.remember("I live in Hamburg.", "profile", entities=["Hamburg"])+ again = vault.remember("i live in hamburg", "profile")+ assert first.action == "added"+ assert again.action == "existing"+ assert again.memory.id == first.memory.id+ assert vault.stats()["current"] == 1+++def test_supersede_keeps_history_and_answers_both_now_and_before(vault, clock):+ old = vault.remember("I live in Hamburg.", "profile", entities=["Hamburg"]).memory+ clock.value = "2026-06-01T09:00:00+00:00"+ new = vault.supersede(old.id, "I live in Berlin.", entities=["Berlin"]).memory++ current = vault.list()+ assert [m.text for m in current] == ["I live in Berlin."]+ closed = vault.get(old.id)+ assert closed.valid_to == "2026-06-01T09:00:00+00:00"+ assert closed.expired_at is not None+ assert new.supersedes_id == old.id+ assert [m.text for m in vault.history(new.id)] == ["I live in Hamburg.", "I live in Berlin."]+ assert [m.text for m in vault.about("hamburg")] == ["I live in Hamburg."]+++def test_invalidate_closes_without_replacement(vault):+ m = vault.remember("I work at Acme.", "profile", entities=["Acme"]).memory+ assert vault.invalidate(m.id, reason="quit")+ assert vault.list() == []+ assert vault.list(include_closed=True)[0].id == m.id+ assert not vault.invalidate(m.id)+++def test_forget_purges_every_version_and_blocks_inferred_rewrite(vault):+ old = vault.remember("My ex is called Sam.", "relationship", entities=["Sam"]).memory+ new = vault.supersede(old.id, "My ex Sam moved to Paris.").memory++ assert vault.forget(new.id) == 2+ assert vault.get(old.id) is None and vault.get(new.id) is None+ assert vault.search("Sam") == []+ assert vault.entities() == []+ assert all("Sam" not in str(row["detail"]) for row in vault.audit_log())+ assert len(vault.tombstones()) == 2++ blocked = vault.remember("My ex Sam moved to Paris.", "relationship", origin="inferred")+ assert blocked.action == "blocked"+ # The user saying it again is their choice.+ assert vault.remember("My ex Sam moved to Paris.", "relationship").action == "added"+++def test_secrets_are_refused(vault):+ fake_key = "sk-" + "proj-" + "abcdefghijklmnopqrstuvwxyz0123456789ABCD"+ result = vault.remember(f"My OpenAI key is {fake_key}")+ assert result.action == "refused"+ assert vault.stats()["current"] == 0+++def test_ephemeral_memory_expires(vault, clock):+ vault.remember(+ "I have a dentist appointment tomorrow.",+ "ephemeral",+ valid_to="2026-01-02T23:59:59+00:00",+ )+ assert len(vault.list()) == 1+ clock.value = "2026-01-03T00:00:00+00:00"+ assert vault.list() == []+ with pytest.raises(ValueError):+ vault.remember("Something short-lived.", "ephemeral")+++def test_search_ranks_entity_matches_and_hides_closed_rows(vault):+ vault.remember("Viktoria is my sister.", "relationship", entities=["Viktoria"])+ vault.remember("I had dinner with friends in San Francisco.", "episode")+ old = vault.remember("My favourite food is sushi.", "preference").memory+ vault.supersede(old.id, "My favourite food is ramen.")++ hits = vault.search("who is viktoria")+ assert hits[0].memory.text == "Viktoria is my sister."+ assert [h.memory.text for h in vault.search("favourite food")] == [+ "My favourite food is ramen."+ ]+ assert len(vault.search("favourite food", include_closed=True)) == 2+ assert vault.search(" ") == []+ assert vault.search('"unbalanced') == []+++def test_search_ignores_accents(vault):+ vault.remember("My favourite place is the Café de Flore.", "preference")+ assert vault.search("cafe")[0].memory.text == "My favourite place is the Café de Flore."+++def test_profile_block_order_budget_and_exclusions(vault):+ vault.remember("Name: Ruben.", "profile")+ vault.remember("Prefers short answers.", "preference")+ vault.remember("Never send emails without asking.", "procedural")+ vault.remember("Went hiking last Sunday.", "episode")+ vault.remember("Has a chronic illness.", "profile", sensitivity="sensitive")+ vault.remember("Wife is called Lea.", "relationship", pinned=True)++ block = vault.profile_block()+ assert block.index("Pinned:") < block.index("About the user:") < block.index("Preferences:")+ assert "Went hiking" not in block+ assert "chronic" not in block+ assert block.count("Wife is called Lea.") == 1++ small = vault.profile_block(max_chars=40)+ assert len(small) <= 40+ assert small.startswith("Pinned:")+++def test_scope_keeps_projects_apart(vault):+ vault.remember("Uses tabs.", "preference", scope="project:a")+ vault.remember("Uses spaces.", "preference", scope="project:b")+ vault.remember("Likes dark mode.", "preference")+ texts = {m.text for m in vault.list(scope="project:a")}+ assert texts == {"Uses tabs.", "Likes dark mode."}+++def test_validation(vault):+ with pytest.raises(ValueError):+ vault.remember("", "profile")+ with pytest.raises(ValueError):+ vault.remember("x", "diary")+ with pytest.raises(ValueError):+ vault.remember("x", "profile", origin="robot")+ with pytest.raises(KeyError):+ vault.supersede("mem_missing", "y")+++def test_pin_and_usage_counters(vault):+ m = vault.remember("Drinks coffee black.", "preference").memory+ assert vault.pin(m.id)+ vault.mark_used([m.id, m.id])+ got = vault.get(m.id)+ assert got.pinned and got.use_count == 2 and got.last_used_at+++def test_reopen_keeps_data(tmp_path, clock):+ path = tmp_path / "vault.db"+ with MemoryVault(path, clock=clock) as v:+ v.remember("Born in 1990.", "profile")+ vault_id = v.vault_id+ with MemoryVault(path, clock=clock) as v:+ assert v.vault_id == vault_id+ assert v.list()[0].text == "Born in 1990."