Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
feat(importer): import agentmemory vaults and exports; doctor flags leftovers
AkitaOnRails47523180f01f
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 4fbfdff9..eb8d575c 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -101,6 +101,13 @@ jobs:uv run --no-project python tests/e2e/external_relay_smoke.py--ai-memory-bin "${{ github.workspace }}/target/debug/ai-memory"--relay-bin "${{ github.workspace }}/target/debug/ai-memory-relay"+ # The companion job runs the importer's unit tests without a server;+ # its agentmemory end-to-end tests need the binary built above.+ - name: agentmemory import against the real server+ env:+ CARGO_TARGET_DIR: ${{ github.workspace }}/target+ AI_MEMORY_IMPORTER_E2E_BIN: ${{ github.workspace }}/target/debug/ai-memory+ run: cargo test --locked --manifest-path companions/ai-memory-importer/Cargo.toml e2e# The Jev reranker adapter under docs/examples is stdlib Python with its# own unittest suite; nothing else runs it. `unittest discover` exits 5# when it finds no tests, so the step skips cleanly until a test filediff --git a/CHANGELOG.md b/CHANGELOG.mdindex e6a56008..58f5c056 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -178,6 +178,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0a Cordis profile patch rather than a fixed config file, so the command prints aready-to-merge `- insert:` fragment for `@deepseek-ai/dsh-mcp-client` andrefuses `--apply`. (#1193)+- The `ai-memory-importer` companion imports agentmemory data. `agentmemory-vault`+ reads its Obsidian vault export; `agentmemory-export` reads its `ExportData`+ JSON file, from disk only, and needs a mapping file from agentmemory's+ project names (or exact session working directories) to ai-memory+ workspaces and projects. `--apply` refuses to run while any record is+ unmapped. Pages land under `imported/agentmemory/` (`--dest-prefix`), never+ under `sessions/` or another reserved family, and links to pages that were+ not imported become plain text. Every write goes through `memory_write_page`+ with `create_only` or `expected_page_id`, and the importer refuses servers+ older than 2.7. A memory's versions are written oldest first so ai-memory+ keeps the supersession chain, and `forgetAfter` becomes `expires_at`. Each+ page carries a stable UUIDv5 ingest key, so an interrupted import resumes+ without duplicates and a rerun writes nothing. Text is redacted before+ anything is planned or sent. Observations are dropped unless+ `--replay-observations` replays them as `external-import` sessions through+ `/hook/batch`, and records with no ai-memory counterpart are counted in the+ manifest. The cookbook has a "Coming from agentmemory" section.+- `ai-memory doctor` reports agentmemory configuration left behind after a+ migration: an MCP server entry in any harness config `install-mcp` knows (or+ in a project's `.mcp.json`, `.cursor/mcp.json`, `.gemini/settings.json` or+ `.vscode/mcp.json`), and agentmemory guideline blocks in rules files such as+ `AGENTS.md`, `GEMINI.md`, `.cursor/rules/` and `.kiro/steering/`. It names+ each file and the key or lines, prints the fix, and warns when a block names+ `memory_consolidate`, which ai-memory also has with different arguments. The+ check only reads; it never edits or deletes another tool's files. `doctor+ --json` carries the findings as `agentmemory_residue`.### Changed- Documented hand-written MCP recipes for Factory Droid, Warp, Cline, Kilodiff --git a/companions/ai-memory-importer/Cargo.lock b/companions/ai-memory-importer/Cargo.lockindex f228fa05..57aac610 100644--- a/companions/ai-memory-importer/Cargo.lock+++ b/companions/ai-memory-importer/Cargo.lock@@ -35,6 +35,7 @@ dependencies = ["tempfile","tokio","url",+ "uuid","walkdir",]@@ -1166,6 +1167,12 @@ dependencies = ["serde",]+[[package]]+name = "sha1_smol"+version = "1.0.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d"+[[package]]name = "sha2"version = "0.10.9"@@ -1472,6 +1479,17 @@ version = "0.2.2"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"+[[package]]+name = "uuid"+version = "1.28.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "7cc1186384beb7dd8eedea376413fd654937285ea6c9cfbb928dc3043ea4b606"+dependencies = [+ "js-sys",+ "sha1_smol",+ "wasm-bindgen",+]+[[package]]name = "version_check"version = "0.9.5"diff --git a/companions/ai-memory-importer/Cargo.toml b/companions/ai-memory-importer/Cargo.tomlindex ce1d161d..31fac97f 100644--- a/companions/ai-memory-importer/Cargo.toml+++ b/companions/ai-memory-importer/Cargo.toml@@ -19,6 +19,8 @@ serde-saphyr = { version = "1.3", default-features = false, features = ["deseriasha2 = "0.10"tokio = { version = "1", features = ["macros", "rt-multi-thread"] }url = "2"+# Stable ingest keys for agentmemory records (UUIDv5 of their ids).+uuid = { version = "1", features = ["v5"] }walkdir = "2"[dev-dependencies]diff --git a/companions/ai-memory-importer/README.md b/companions/ai-memory-importer/README.mdindex 78882427..c686584c 100644--- a/companions/ai-memory-importer/README.md+++ b/companions/ai-memory-importer/README.md@@ -49,6 +49,76 @@ observation names `external:<source>` and assistant/system observations retainextension provenance. A reader can therefore distinguish an explicit importand its source without adding product-specific agent kinds to core.+### agentmemory++Two routes read what agentmemory exports. Both write through the MCP+`memory_write_page` tool with explicit `workspace` and `project`, every write+conditional: `create_only` for a page's first version and `expected_page_id`+for each later one. They refuse a server older than ai-memory 2.7, which would+ignore those arguments. Pages land under `--dest-prefix` (default+`imported/agentmemory`), never under ai-memory's own `sessions/` or any other+reserved family.++`agentmemory-vault --dir <vault>` reads the Obsidian vault agentmemory's+Obsidian export writes: `memories/`, `lessons/`, `crystals/` and `sessions/`,+each page with YAML frontmatter and `[[id]]` wikilinks. The generated `MOC.md`+index is skipped. A wikilink to another imported page is rewritten to its new+path; a link to anything else becomes plain text. `--workspace` and+`--project` name the one destination.++`agentmemory-export --file <export.json> --mapping <mapping.json>` reads the+`ExportData` JSON file agentmemory's export command writes. It takes a file+only: the importer never contacts an agentmemory server and never handles its+credentials. The mapping file is required, because agentmemory names a+project by its directory basename and two repositories can share one:++```json+{+ "projects": {+ "api": {"workspace": "acme", "project": "acme-api"},+ "web": {"workspace": "acme", "project": "acme-web"}+ },+ "cwds": {+ "/home/me/other/api": {"workspace": "other", "project": "other-api"}+ },+ "unscoped": {"workspace": "acme", "project": "acme-misc"}+}+```++`cwds` matches a session's exact working directory and wins over `projects`;+records tied to that session (its summary, the memories that cite it) follow+it. `unscoped` receives records that name no project. A record the mapping+does not cover is listed in the dry run, and `--apply` refuses to run until the+mapping covers it.++| agentmemory record | ai-memory page | `kind` / `tier` |+| --- | --- | --- |+| memory (every version) | `memories/<first-version-id>.md` | `preference`, `decision` (architecture), `workflow`, `pattern`, `bug`, `fact` / `semantic`, or `procedural` for workflow and pattern |+| lesson | `lessons/<id>.md` | `lesson` / `semantic` |+| insight | `insights/<id>.md` | `insight` / `semantic` |+| semantic memory | `semantic/<id>.md` | `fact` / `semantic` |+| procedural memory | `procedural/<id>.md` | `procedure` / `procedural` |+| crystal | `crystals/<id>.md` | `crystal` / `episodic` |+| session and its summary | `sessions/<id>.md` | `session` / `episodic` |++The paths are relative to `--dest-prefix`. A memory's versions are written+oldest first to one path, each conditional on the one before, so ai-memory+keeps its own supersession chain. `forgetAfter` becomes `expires_at`, so an+expired memory is hidden from normal reads as soon as it lands. Concepts and+tags become page tags, after the `agentmemory-import` tag every page carries.+Related records become wikilinks (qualified with `workspace/project:` when the+target landed in another project), or plain ids when they were not imported.+Deleted lessons and insights are skipped.++Compressed observations are dropped by default; the session summaries carry+their value. `--replay-observations` replays each session's observations+through `/hook/batch` as one `external-import` session (`source_event`+`agentmemory-observation`, at most 254 per session), so the server sanitizer+applies a second time. Graph nodes and edges, actions, signals, routines,+sentinels, sketches, checkpoints, facets, access logs and any key a newer+export adds are not imported; the manifest counts each under `dropped`. An+export page whose `pagination.hasMore` is true needs `--allow-partial`.+Claude memory graph and Qdrant imports remain roadmap items; there are no codestubs for them.@@ -101,6 +171,31 @@ stubs for them.- There is no inbox or watch-folder mode. Import is an explicit one-shotoperation.+The agentmemory routes keep the same dry-run default, `--apply` plus+`--manifest-out`, auth and path rules, with these differences:++- Writes use MCP `memory_write_page` only, each conditional, so the server+ refuses a page that appears or changes during the import. A destination page this import did not write is never+ replaced; the run stops and names it.+- `--dest-prefix` must be relative and portable. `..`, absolute paths, a+ Windows drive, and a first segment that is reserved (`sessions`,+ `session-logs`, `procedures`, `decisions`, `gotchas`, `profile`) or starts+ with `_` or `.` are refused. Every planned path is checked to stay under the+ prefix, and two records that map to one path abort the plan.+- Vault folders and entries that are symlinks are refused; hidden entries and+ non-Markdown files are skipped and counted. Pages are capped at 1 MiB each+ and 100,000 per vault; an export file at 256 MiB.+- Every page and replayed observation is redacted client-side before the plan,+ the manifest or any request exists. Record ids that look like credentials+ are refused.+- Each record has a stable ingest key, a UUIDv5 of its agentmemory id, written+ on the page's last line. A rerun finds the key in the page the server+ already holds and skips it, so an interrupted import resumes without+ duplicates and a finished one writes nothing. The manifest records each+ version as `written`, `present` (already there) or `failed`.+- Without `--create-destination`, every destination must already exist; the+ check is a read, which never creates a scope.+## UsageDry-run with a summary:@@ -172,6 +267,30 @@ destination must already exist unless `--create-destination` is passed. Sourcefiles are read once; adapters should write their completed generic JSON exportbefore invoking the importer.+## agentmemory usage++```bash+# Obsidian vault: dry run, then apply.+cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-vault --dir ~/.agentmemory/vault --workspace default --project my-project+AI_MEMORY_AUTH_TOKEN=... \+cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-vault --dir ~/.agentmemory/vault --workspace default --project my-project \+ --apply --manifest-out /tmp/agentmemory-vault-manifest.json++# ExportData JSON: dry run (lists unmapped projects and dropped counts), then apply.+cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-export --file agentmemory-export.json --mapping mapping.json+AI_MEMORY_AUTH_TOKEN=... \+cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-export --file agentmemory-export.json --mapping mapping.json \+ --apply --manifest-out /tmp/agentmemory-export-manifest.json+```++Options shared by both: `--dest-prefix`, `--server-url`, `--create-destination`+and `--show-body`. `agentmemory-export` adds `--replay-observations` and+`--allow-partial`.+## ValidationRun these from the repository root:@@ -182,6 +301,16 @@ cargo test --manifest-path companions/ai-memory-importer/Cargo.tomlcargo clippy --manifest-path companions/ai-memory-importer/Cargo.toml --all-targets -- -D warnings```+The agentmemory end-to-end tests start a real `ai-memory serve` in a+temporary data directory when `AI_MEMORY_IMPORTER_E2E_BIN` names the binary+(CI sets it); without it they report a skip:++```bash+cargo build --bin ai-memory+AI_MEMORY_IMPORTER_E2E_BIN="$PWD/target/debug/ai-memory" \+ cargo test --manifest-path companions/ai-memory-importer/Cargo.toml e2e+```+Root hygiene checks remain separate:```bashdiff --git a/companions/ai-memory-importer/src/agentmemory/e2e.rs b/companions/ai-memory-importer/src/agentmemory/e2e.rsnew file mode 100644index 00000000..b1c5a9f2--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/e2e.rs@@ -0,0 +1,257 @@+//! End-to-end runs of both agentmemory routes against a real `ai-memory+//! serve` in a temporary data directory. Set `AI_MEMORY_IMPORTER_E2E_BIN` to+//! the `ai-memory` binary to run them (CI builds it next to the relay smoke);+//! without it they report a skip and pass, so the companion's own+//! `cargo test` stays independent of the root build.++use std::net::TcpListener;+use std::path::{Path, PathBuf};+use std::process::{Child, Command, Stdio};+use std::time::{Duration, Instant};++use serde_json::{Value, json};+use tempfile::TempDir;++use super::DEFAULT_DEST_PREFIX;+use super::export::{self, ExportArgs};+use super::vault::{self, VaultArgs};+use crate::ImportClient;+use crate::mcp::tool_json;++const BIN_ENV: &str = "AI_MEMORY_IMPORTER_E2E_BIN";++struct Server {+ child: Child,+ url: String,+ _root: TempDir,+}++impl Drop for Server {+ fn drop(&mut self) {+ let _ = self.child.kill();+ let _ = self.child.wait();+ }+}++/// Start an isolated server, or `None` when no binary is configured.+async fn start() -> Option<Server> {+ let Some(bin) = std::env::var_os(BIN_ENV).map(PathBuf::from) else {+ eprintln!("skipped: set {BIN_ENV} to an ai-memory binary to run this test");+ return None;+ };+ let root = tempfile::tempdir().unwrap();+ let home = root.path().join("home");+ let data = root.path().join("data");+ std::fs::create_dir_all(&home).unwrap();+ let gitconfig = root.path().join("gitconfig");+ std::fs::write(&gitconfig, "").unwrap();+ let port = TcpListener::bind("127.0.0.1:0")+ .unwrap()+ .local_addr()+ .unwrap()+ .port();+ let child = Command::new(&bin)+ .arg("--data-dir")+ .arg(&data)+ .args(["serve", "--transport", "http", "--bind"])+ .arg(format!("127.0.0.1:{port}"))+ .arg("--no-watcher")+ .current_dir(root.path())+ .env_clear()+ .env("PATH", std::env::var_os("PATH").unwrap_or_default())+ .env("HOME", &home)+ .env("XDG_CONFIG_HOME", home.join(".config"))+ .env("XDG_DATA_HOME", home.join(".local/share"))+ .env("GIT_CONFIG_NOSYSTEM", "1")+ .env("GIT_CONFIG_GLOBAL", &gitconfig)+ .env("AI_MEMORY_HOME", &home)+ .env("AI_MEMORY_DATA_DIR", &data)+ .env("AI_MEMORY_EMBEDDING_PROVIDER", "none")+ .env("AI_MEMORY_BACKFILL_ON_START", "false")+ .env("AI_MEMORY_CONSOLIDATE_ON_SESSION_END", "false")+ .env("AI_MEMORY_AUTO_IMPROVE__SCHEDULER__ENABLED", "false")+ .env("RUST_LOG", "off")+ .stdin(Stdio::null())+ .stdout(Stdio::null())+ .stderr(Stdio::null())+ .spawn()+ .expect("spawn ai-memory serve");+ let server = Server {+ child,+ url: format!("http://127.0.0.1:{port}"),+ _root: root,+ };+ let client = ImportClient::new(&server.url).unwrap();+ let deadline = Instant::now() + Duration::from_secs(60);+ while client.ensure_conditional_writes().await.is_err() {+ assert!(+ Instant::now() < deadline,+ "ai-memory serve never became ready"+ );+ tokio::time::sleep(Duration::from_millis(200)).await;+ }+ Some(server)+}++async fn tool(server: &Server, name: &str, arguments: Value) -> Value {+ let client = ImportClient::new(&server.url).unwrap();+ let result = client.call_tool(name, arguments).await.unwrap();+ tool_json(&result).unwrap_or(result)+}++fn manifest(path: &Path) -> Value {+ serde_json::from_slice(&std::fs::read(path).unwrap()).unwrap()+}++#[tokio::test]+async fn e2e_vault_import_is_searchable_and_idempotent() {+ let Some(server) = start().await else { return };+ let vault_dir = tempfile::tempdir().unwrap();+ vault::tests::fixture_vault(vault_dir.path());+ let manifest_path = vault_dir.path().join("manifest.json");+ let args = |project: &str, create: bool| VaultArgs {+ dir: vault_dir.path().to_path_buf(),+ workspace: Some("e2e".into()),+ project: Some(project.into()),+ dest_prefix: DEFAULT_DEST_PREFIX.into(),+ server_url: server.url.clone(),+ apply: true,+ manifest_out: Some(manifest_path.clone()),+ create_destination: create,+ show_body: false,+ };++ let error = vault::run(args("missing", false)).await.unwrap_err();+ assert!(+ error.to_string().contains("must already exist"),+ "{error:#}"+ );++ vault::run(args("vault", true)).await.unwrap();+ let first = manifest(&manifest_path);+ assert_eq!(first["written_versions"], 5);++ let hits = tool(+ &server,+ "memory_query",+ json!({"workspace": "e2e", "project": "vault", "query": "Single writer owns the store"}),+ )+ .await;+ assert!(+ hits.to_string()+ .contains("imported/agentmemory/memories/mem-alpha.md"),+ "{hits}"+ );+ let page = tool(+ &server,+ "memory_read_page",+ json!({"workspace": "e2e", "project": "vault", "path": "imported/agentmemory/memories/mem-alpha.md"}),+ )+ .await;+ let body = page["body"].as_str().unwrap();+ assert!(+ body.contains("[[imported/agentmemory/memories/mem-beta]]"),+ "{body}"+ );++ vault::run(args("vault", false)).await.unwrap();+ let second = manifest(&manifest_path);+ assert_eq!(second["written_versions"], 0, "a rerun writes 0");+ assert_eq!(second["present_versions"], 5);+}++#[tokio::test]+async fn e2e_export_import_rebuilds_chains_and_honours_ttl() {+ let Some(server) = start().await else { return };+ let mut data = export::tests::fixture_export();+ // A TTL far in the future stays visible; the fixture's past one is hidden.+ data["memories"][5]["forgetAfter"] = json!("2099-01-01T00:00:00Z");+ let fixture = export::tests::fixture_files(&data, &export::tests::fixture_mapping());+ let args = || ExportArgs {+ file: fixture.file.clone(),+ mapping: fixture.mapping.clone(),+ dest_prefix: DEFAULT_DEST_PREFIX.into(),+ server_url: server.url.clone(),+ apply: true,+ manifest_out: Some(fixture.manifest.clone()),+ create_destination: true,+ show_body: false,+ replay_observations: true,+ allow_partial: false,+ };+ export::run(args()).await.unwrap();+ let first = manifest(&fixture.manifest);+ assert_eq!(first["status"], "imported");+ assert_eq!(first["written_versions"], 14);+ assert!(+ first["hook_sessions"]+ .as_array()+ .unwrap()+ .iter()+ .all(|s| s["status"] == "imported")+ );++ // The chain: three versions written, each conditional on the one before,+ // and the server's latest version is the last one.+ let chain = first["pages"]+ .as_array()+ .unwrap()+ .iter()+ .find(|p| p["path"] == "imported/agentmemory/memories/mem-q1.md")+ .unwrap();+ let ids: Vec<&str> = chain["versions"]+ .as_array()+ .unwrap()+ .iter()+ .map(|v| v["page_id"].as_str().unwrap())+ .collect();+ assert_eq!(ids.len(), 3);+ assert_eq!(+ ids.iter().collect::<std::collections::BTreeSet<_>>().len(),+ 3+ );+ let latest = tool(+ &server,+ "memory_read_page",+ json!({"workspace": "acme", "project": "acme-api", "path": "imported/agentmemory/memories/mem-q1.md"}),+ )+ .await;+ assert_eq!(latest["page_id"], ids[2]);+ assert!(+ latest["body"]+ .as_str()+ .unwrap()+ .contains("Queue bound is 4096")+ );++ let query = |workspace: &'static str, project: &'static str, text: &'static str| {+ tool(+ &server,+ "memory_query",+ json!({"workspace": workspace, "project": project, "query": text}),+ )+ };+ let expired = query("acme", "acme-api", "Staging host is temporary").await;+ assert!(!expired.to_string().contains("mem-ttl.md"), "{expired}");+ let live = query("acme", "acme-web", "Run the web build with npm ci").await;+ assert!(live.to_string().contains("mem-web.md"), "{live}");+ let other = query("other", "other-api", "Tabs in the other api").await;+ assert!(other.to_string().contains("mem-other-api.md"), "{other}");+ // The replayed observations landed as the project's latest completed+ // session.+ let observations = tool(+ &server,+ "memory_read_session_observations",+ json!({"workspace": "acme", "project": "acme-web"}),+ )+ .await;+ assert!(+ observations.to_string().contains("Looked at CSS"),+ "{observations}"+ );++ export::run(args()).await.unwrap();+ let second = manifest(&fixture.manifest);+ assert_eq!(second["written_versions"], 0, "a rerun writes 0");+ assert_eq!(second["present_versions"], 14);+}diff --git a/companions/ai-memory-importer/src/agentmemory/export.rs b/companions/ai-memory-importer/src/agentmemory/export.rsnew file mode 100644index 00000000..5f87c70c--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/export.rs@@ -0,0 +1,1443 @@+//! `agentmemory-export <file.json>`: import the `ExportData` JSON agentmemory's+//! `export` command writes. File input only: the importer never contacts an+//! agentmemory server, so it never handles that server's credentials.+//!+//! - Every record gets a stable ingest key (a UUIDv5 of its agentmemory id).+//! - A required mapping file names the ai-memory `workspace`/`project` for+//! each agentmemory project. agentmemory keys projects by directory+//! basename, which collides across repositories, so the mapping can also+//! key by exact session `cwd`. Any record left unmapped makes `--apply`+//! refuse to run.+//! - Memory versions are written oldest first to one path, each conditional on+//! the version before it, so ai-memory builds its own supersession chain.+//! - `forgetAfter` becomes `expires_at`.+//! - Sessions and their summaries become `<prefix>/sessions/<id>.md`.+//! - Compressed observations are dropped unless `--replay-observations`+//! replays them as `external-import` sessions through `/hook/batch`, where+//! the server sanitizer applies again.+//! - Everything without an ai-memory equivalent (graph, actions, signals,+//! routines, ...) is dropped and counted in the manifest.+//! - All text is redacted client-side before the plan, the manifest or any+//! request exists.++use std::collections::{BTreeMap, BTreeSet, HashMap};+use std::fs;+use std::path::{Path, PathBuf};++use anyhow::{Context, Result, anyhow, bail};+use clap::Parser;+use serde::Deserialize;+use serde::de::DeserializeOwned;+use serde_json::{Value, json};++use super::{+ DEFAULT_DEST_PREFIX, Destination, HookSessionEntry, MAX_EXPLICIT_TITLE_CHARS, MAX_TITLE_CHARS,+ Manifest, PlannedPage, PlannedVersion, RunStatus, apply_pages, ingest_key, link_target,+ memory_kind_tier, one_line, preflight, print_done, print_plan, provenance_line, tags_from,+ validate_dest_prefix, validate_id, validate_planned_path,+};+use crate::{+ DEFAULT_SERVER_URL, ImportClient, MAX_EXTENSION_MESSAGE_BYTES, PlannedHookEvent,+ planned_hook_event, sha256_hex, slugify, stable_external_session_id, truncate_utf8,+ validate_label,+};++const IMPORT_VERSION: &str = "agentmemory-export-v1";+/// The `source` every replayed observation session names.+const SOURCE: &str = "agentmemory";+/// Bound on the export file.+const MAX_EXPORT_BYTES: u64 = 256 * 1024 * 1024;+/// Bound on the mapping file.+const MAX_MAPPING_BYTES: u64 = 1024 * 1024;+/// One hook batch holds at most 256 items: a session start, its+/// observations, and a session end.+const MAX_REPLAYED_OBSERVATIONS: usize = 254;+/// Bound on a replayed observation title, in bytes.+const MAX_OBSERVATION_TITLE_BYTES: usize = 160;++/// Top-level keys this route reads. Every other key is dropped and counted.+const READ_KEYS: &[&str] = &[+ "version",+ "exportedAt",+ "pagination",+ "sessions",+ "observations",+ "memories",+ "summaries",+ "lessons",+ "insights",+ "semanticMemories",+ "proceduralMemories",+ "crystals",+];++#[derive(Parser, Debug, Clone)]+pub(crate) struct ExportArgs {+ /// The `ExportData` JSON file agentmemory's `export` wrote.+ #[arg(long)]+ pub(crate) file: PathBuf,+ /// JSON file mapping agentmemory projects to ai-memory workspace/project.+ #[arg(long)]+ pub(crate) mapping: PathBuf,+ /// Wiki path prefix every imported page lands under.+ #[arg(long, default_value = DEFAULT_DEST_PREFIX)]+ pub(crate) dest_prefix: String,+ #[arg(long, env = "AI_MEMORY_SERVER_URL", default_value = DEFAULT_SERVER_URL)]+ pub(crate) server_url: String,+ /// Write the pages. Without this flag the command only prints a plan.+ #[arg(long)]+ pub(crate) apply: bool,+ /// Durable import manifest. Required with --apply.+ #[arg(long)]+ pub(crate) manifest_out: Option<PathBuf>,+ /// Let the first write create a missing destination workspace/project.+ #[arg(long)]+ pub(crate) create_destination: bool,+ /// Print every planned page body during the dry run.+ #[arg(long)]+ pub(crate) show_body: bool,+ /// Replay compressed observations as `external-import` sessions through+ /// `/hook/batch` instead of dropping them.+ #[arg(long)]+ pub(crate) replay_observations: bool,+ /// Import an export page whose `pagination.hasMore` says more follows.+ #[arg(long)]+ pub(crate) allow_partial: bool,+}++/// The mapping file: agentmemory project basename (and, optionally, exact+/// session cwd) to ai-memory workspace/project.+#[derive(Debug, Deserialize)]+#[serde(deny_unknown_fields)]+struct MappingFile {+ #[serde(default)]+ projects: BTreeMap<String, Target>,+ #[serde(default)]+ cwds: BTreeMap<String, Target>,+ /// Destination for records that name no project at all.+ #[serde(default)]+ unscoped: Option<Target>,+}++#[derive(Debug, Deserialize)]+#[serde(deny_unknown_fields)]+struct Target {+ workspace: String,+ project: String,+}++pub(crate) struct Mapping {+ projects: BTreeMap<String, Destination>,+ cwds: BTreeMap<String, Destination>,+ unscoped: Option<Destination>,+}++const NO_PROJECT: &str = "(no project)";++impl Mapping {+ pub(crate) fn load(path: &Path) -> Result<Self> {+ let meta =+ fs::metadata(path).with_context(|| format!("read mapping {}", path.display()))?;+ if !meta.is_file() || meta.len() > MAX_MAPPING_BYTES {+ bail!("--mapping must be a regular file of at most {MAX_MAPPING_BYTES} bytes");+ }+ let raw: MappingFile = serde_json::from_slice(&fs::read(path)?)+ .map_err(|e| anyhow!("parse mapping file: {e}"))?;+ let convert = |target: Target| -> Result<Destination> {+ Ok(Destination {+ workspace: validate_label(&target.workspace, "mapping workspace", 128)?.to_owned(),+ project: validate_label(&target.project, "mapping project", 128)?.to_owned(),+ })+ };+ Ok(Self {+ projects: raw+ .projects+ .into_iter()+ .map(|(k, v)| Ok((k, convert(v)?)))+ .collect::<Result<_>>()?,+ cwds: raw+ .cwds+ .into_iter()+ .map(|(k, v)| Ok((k, convert(v)?)))+ .collect::<Result<_>>()?,+ unscoped: raw.unscoped.map(convert).transpose()?,+ })+ }++ /// The destination for a record, or the label of the unmapped project.+ fn resolve(&self, project: Option<&str>, cwd: Option<&str>) -> Result<Destination, String> {+ if let Some(found) = cwd.filter(|c| !c.is_empty()).and_then(|c| self.cwds.get(c)) {+ return Ok(found.clone());+ }+ match project.map(str::trim).filter(|p| !p.is_empty()) {+ Some(project) => self+ .projects+ .get(project)+ .cloned()+ .ok_or_else(|| project.to_owned()),+ None => self.unscoped.clone().ok_or_else(|| NO_PROJECT.to_owned()),+ }+ }+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Session {+ id: String,+ #[serde(default)]+ project: String,+ #[serde(default)]+ cwd: String,+ #[serde(default)]+ started_at: String,+ #[serde(default)]+ ended_at: Option<String>,+ #[serde(default)]+ status: String,+ #[serde(default)]+ observation_count: u64,+ #[serde(default)]+ model: Option<String>,+ #[serde(default)]+ tags: Vec<String>,+ #[serde(default)]+ first_prompt: Option<String>,+ #[serde(default)]+ summary: Option<String>,+ #[serde(default)]+ commit_shas: Vec<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Observation {+ id: String,+ #[serde(default)]+ timestamp: String,+ #[serde(default)]+ title: String,+ #[serde(default)]+ subtitle: Option<String>,+ #[serde(default)]+ facts: Vec<String>,+ #[serde(default)]+ narrative: String,+ #[serde(default)]+ files: Vec<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Memory {+ id: String,+ #[serde(default)]+ created_at: String,+ #[serde(rename = "type", default)]+ memory_type: String,+ #[serde(default)]+ title: String,+ #[serde(default)]+ content: String,+ #[serde(default)]+ concepts: Vec<String>,+ #[serde(default)]+ files: Vec<String>,+ #[serde(default)]+ session_ids: Vec<String>,+ #[serde(default)]+ version: Option<u64>,+ #[serde(default)]+ parent_id: Option<String>,+ #[serde(default)]+ supersedes: Vec<String>,+ #[serde(default)]+ related_ids: Vec<String>,+ #[serde(default)]+ forget_after: Option<String>,+ #[serde(default)]+ project: Option<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Summary {+ session_id: String,+ #[serde(default)]+ project: Option<String>,+ #[serde(default)]+ title: String,+ #[serde(default)]+ narrative: String,+ #[serde(default)]+ key_decisions: Vec<String>,+ #[serde(default)]+ files_modified: Vec<String>,+ #[serde(default)]+ concepts: Vec<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Lesson {+ id: String,+ #[serde(default)]+ content: String,+ #[serde(default)]+ context: String,+ #[serde(default)]+ confidence: Option<f64>,+ #[serde(default)]+ reinforcements: Option<u64>,+ #[serde(default)]+ source_ids: Vec<String>,+ #[serde(default)]+ project: Option<String>,+ #[serde(default)]+ tags: Vec<String>,+ #[serde(default)]+ deleted: Option<bool>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Insight {+ id: String,+ #[serde(default)]+ title: String,+ #[serde(default)]+ content: String,+ #[serde(default)]+ confidence: Option<f64>,+ #[serde(default)]+ reinforcements: Option<u64>,+ #[serde(default)]+ source_memory_ids: Vec<String>,+ #[serde(default)]+ source_lesson_ids: Vec<String>,+ #[serde(default)]+ source_crystal_ids: Vec<String>,+ #[serde(default)]+ project: Option<String>,+ #[serde(default)]+ tags: Vec<String>,+ #[serde(default)]+ deleted: Option<bool>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct SemanticMemory {+ id: String,+ #[serde(default)]+ fact: String,+ #[serde(default)]+ confidence: Option<f64>,+ #[serde(default)]+ source_session_ids: Vec<String>,+ #[serde(default)]+ source_memory_ids: Vec<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct ProceduralMemory {+ id: String,+ #[serde(default)]+ name: String,+ #[serde(default)]+ steps: Vec<String>,+ #[serde(default)]+ trigger_condition: String,+ #[serde(default)]+ expected_outcome: Option<String>,+ #[serde(default)]+ source_session_ids: Vec<String>,+ #[serde(default)]+ tags: Vec<String>,+ #[serde(default)]+ concepts: Vec<String>,+}++#[derive(Debug, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Crystal {+ id: String,+ #[serde(default)]+ narrative: String,+ #[serde(default)]+ key_outcomes: Vec<String>,+ #[serde(default)]+ files_affected: Vec<String>,+ #[serde(default)]+ lessons: Vec<String>,+ #[serde(default)]+ source_action_ids: Vec<String>,+ #[serde(default)]+ session_id: Option<String>,+ #[serde(default)]+ project: Option<String>,+}++#[derive(Debug, Default, Deserialize)]+#[serde(rename_all = "camelCase")]+struct Pagination {+ #[serde(default)]+ has_more: bool,+}++/// One replayed observation session.+#[derive(Debug, Clone)]+pub(crate) struct HookSession {+ source_session_id: String,+ stable_session_id: String,+ destination: Destination,+ events: Vec<PlannedHookEvent>,+}++#[derive(Debug)]+pub(crate) struct ExportPlan {+ pages: Vec<PlannedPage>,+ hook_sessions: Vec<HookSession>,+ dropped: BTreeMap<String, usize>,+ unmapped: BTreeMap<String, usize>,+ partial: bool,+ source_sha256: String,+}++pub(crate) async fn run(args: ExportArgs) -> Result<()> {+ if args.apply && args.manifest_out.is_none() {+ bail!("--apply requires --manifest-out <path>");+ }+ let prefix = validate_dest_prefix(&args.dest_prefix)?;+ let mapping = Mapping::load(&args.mapping)?;+ let plan = plan(+ &args.file,+ &mapping,+ &prefix,+ args.replay_observations,+ &args.server_url,+ )?;+ let mut manifest = Manifest::new(+ IMPORT_VERSION,+ args.file.to_string_lossy().into_owned(),+ plan.source_sha256.clone(),+ &plan.pages,+ );+ manifest.dropped = plan.dropped.clone();+ manifest.unmapped_projects = plan.unmapped.clone();+ manifest.hook_sessions = plan+ .hook_sessions+ .iter()+ .map(|session| HookSessionEntry {+ source_session_id: session.source_session_id.clone(),+ stable_session_id: session.stable_session_id.clone(),+ workspace: session.destination.workspace.clone(),+ project: session.destination.project.clone(),+ planned_events: session.events.len(),+ accepted_events: 0,+ status: RunStatus::Planned,+ })+ .collect();++ if !args.apply {+ print_plan(&plan.pages, &manifest, args.show_body);+ for session in &plan.hook_sessions {+ println!(+ " replay session {} -> {}/{} ({} hook events)",+ session.source_session_id,+ session.destination.workspace,+ session.destination.project,+ session.events.len()+ );+ }+ if plan.partial {+ println!(" this export page says more data follows; --apply needs --allow-partial");+ }+ if let Some(path) = &args.manifest_out {+ manifest.write(path)?;+ }+ return Ok(());+ }++ let manifest_path = args+ .manifest_out+ .as_deref()+ .ok_or_else(|| anyhow!("--apply requires --manifest-out <path>"))?;+ // Refusals that need no server come before the client exists.+ if !plan.unmapped.is_empty() {+ let names: Vec<_> = plan.unmapped.keys().map(String::as_str).collect();+ let error = anyhow!(+ "the mapping file does not cover agentmemory project(s) {names:?}; add them (or `unscoped`) and rerun"+ );+ return Err(manifest.fail(manifest_path, error));+ }+ if plan.partial && !args.allow_partial {+ let error = anyhow!(+ "this export page says more data follows (pagination.hasMore); export everything or pass --allow-partial"+ );+ return Err(manifest.fail(manifest_path, error));+ }+ manifest.write(manifest_path)?;++ let client = ImportClient::new(&args.server_url)?;+ let destinations: BTreeSet<Destination> = plan+ .pages+ .iter()+ .map(|page| page.destination.clone())+ .chain(plan.hook_sessions.iter().map(|s| s.destination.clone()))+ .collect();+ if let Err(error) = preflight(&client, &destinations, args.create_destination).await {+ return Err(manifest.fail(manifest_path, error));+ }+ apply_pages(&client, &plan.pages, &mut manifest, manifest_path).await?;+ for (index, session) in plan.hook_sessions.iter().enumerate() {+ let delivery = match client.post_hook_batch(&session.events).await {+ Ok(delivery) => delivery,+ Err(error) => return Err(manifest.fail(manifest_path, error)),+ };+ let entry = &mut manifest.hook_sessions[index];+ entry.accepted_events = delivery.ack.accepted_count();+ if !delivery.status.is_success() || !delivery.ack.accepted_every_event(session.events.len())+ {+ entry.status = RunStatus::Failed;+ let error = anyhow!(+ "hook batch for session {} returned HTTP {} and accepted {} of {} events; rerun to resume",+ session.source_session_id,+ delivery.status,+ entry.accepted_events,+ session.events.len()+ );+ return Err(manifest.fail(manifest_path, error));+ }+ entry.status = RunStatus::Imported;+ manifest.write(manifest_path)?;+ }+ manifest.status = RunStatus::Imported;+ manifest.write(manifest_path)?;+ print_done(&manifest);+ if !plan.hook_sessions.is_empty() {+ println!(+ "replayed {} observation session(s) through /hook/batch",+ plan.hook_sessions.len()+ );+ }+ Ok(())+}++/// Parse one array the route reads; a malformed record fails the plan.+fn records<T: DeserializeOwned>(+ export: &serde_json::Map<String, Value>,+ key: &str,+) -> Result<Vec<T>> {+ match export.get(key) {+ None | Some(Value::Null) => Ok(Vec::new()),+ Some(Value::Array(items)) => items+ .iter()+ .enumerate()+ .map(|(index, item)| {+ serde_json::from_value(item.clone())+ .map_err(|e| anyhow!("{key}[{index}] is not a valid agentmemory record: {e}"))+ })+ .collect(),+ Some(_) => bail!("{key} must be an array"),+ }+}++/// How many records a dropped top-level value holds.+fn record_count(value: &Value) -> usize {+ match value {+ Value::Array(items) => items.len(),+ Value::Object(map) => map.len(),+ Value::Null => 0,+ _ => 1,+ }+}++/// What the mapping file is consulted with for one record.+struct Scope {+ project: Option<String>,+ cwd: Option<String>,+}++fn refs(ids: &[String]) -> Vec<&str> {+ ids.iter().map(String::as_str).collect()+}++/// A loose RFC 3339 / `YYYY-MM-DD` shape check; the server parses it exactly.+fn looks_like_timestamp(value: &str) -> bool {+ let bytes = value.as_bytes();+ let date = bytes.len() >= 10+ && bytes[..10].iter().enumerate().all(|(i, b)| match i {+ 4 | 7 => *b == b'-',+ _ => b.is_ascii_digit(),+ });+ date && (bytes.len() == 10 || (bytes.len() <= 40 && bytes[10] == b'T'))+}++/// Pages and replay sessions grouped per destination, plus the link table.+struct Builder<'a> {+ prefix: &'a str,+ pages: Vec<PlannedPage>,+ paths: HashMap<String, String>,+ /// agentmemory id → (destination, path) of the imported page.+ links: HashMap<String, (Destination, String)>,+ unmapped: BTreeMap<String, usize>,+}++impl Builder<'_> {+ fn path(&mut self, folder: &str, id: &str) -> Result<String> {+ let path = format!("{}/{folder}/{}.md", self.prefix, slugify(id));+ validate_planned_path(&path, self.prefix)?;+ if let Some(other) = self.paths.insert(path.clone(), id.to_owned()) {+ bail!("duplicate destination path collision: {other} and {id} both map to {path}");+ }+ Ok(path)+ }++ fn unmapped(&mut self, label: String, count: usize) {+ *self.unmapped.entry(label).or_default() += count;+ }++ /// A rendered reference to another agentmemory record: a wikilink when+ /// it was imported (scoped when it landed in another project), plain text+ /// otherwise.+ fn link(&self, from: &Destination, id: &str) -> String {+ match self.links.get(id) {+ Some((to, path)) if to == from => format!("[[{}]]", link_target(path)),+ Some((to, path)) => {+ format!("[[{}/{}:{}]]", to.workspace, to.project, link_target(path))+ }+ None => format!("`{id}`"),+ }+ }+}++fn bullet_list(out: &mut String, heading: &str, items: &[String]) {+ if items.is_empty() {+ return;+ }+ out.push_str(&format!("\n## {heading}\n"));+ for item in items {+ out.push_str(&format!("- {item}\n"));+ }+}++fn code_list(out: &mut String, heading: &str, items: &[String]) {+ let items: Vec<String> = items.iter().map(|item| format!("`{item}`")).collect();+ bullet_list(out, heading, &items);+}++/// Sanitize an assembled body and close it with its provenance line.+fn finish_body(body: &str, record: &str, id: &str, version: Option<u64>, key: &str) -> String {+ format!(+ "{}\n\n---\n{}\n",+ ai_memory_client::sanitize_external_text(body).trim_end(),+ provenance_line(record, id, version, key)+ )+}++fn single_version(+ record: &'static str,+ id: &str,+ title: String,+ body: &str,+ kind: &str,+ tier: &'static str,+ tags: &[String],+) -> PlannedVersion {+ let key = ingest_key(record, id);+ PlannedVersion {+ body: finish_body(body, record, id, None, &key),+ key,+ source_id: id.to_owned(),+ version: None,+ title: ai_memory_client::sanitize_external_text(&title),+ kind: kind.to_owned(),+ tier,+ tags: tags_from(tags.iter().map(String::as_str)),+ expires_at: None,+ }+}++/// Read, validate, map and plan a whole export. No network, no writes.+pub(crate) fn plan(+ file: &Path,+ mapping: &Mapping,+ prefix: &str,+ replay_observations: bool,+ server_url: &str,+) -> Result<ExportPlan> {+ let meta = fs::metadata(file).with_context(|| format!("read export {}", file.display()))?;+ if !meta.is_file() {+ bail!("--file must be a regular file");+ }+ if meta.len() > MAX_EXPORT_BYTES {+ bail!(+ "export file is {} bytes; maximum is {MAX_EXPORT_BYTES}",+ meta.len()+ );+ }+ let bytes = fs::read(file).with_context(|| format!("read {}", file.display()))?;+ let source_sha256 = sha256_hex(&bytes);+ let export: Value = serde_json::from_slice(&bytes)+ .map_err(|_| anyhow!("parse agentmemory export JSON: not valid JSON"))?;+ let Value::Object(export) = export else {+ bail!("an agentmemory export is a JSON object");+ };++ let mut dropped: BTreeMap<String, usize> = BTreeMap::new();+ for (key, value) in &export {+ if !READ_KEYS.contains(&key.as_str()) {+ let count = record_count(value);+ if count > 0 {+ dropped.insert(key.clone(), count);+ }+ }+ }+ let partial = match export.get("pagination") {+ Some(value) if !value.is_null() => {+ serde_json::from_value::<Pagination>(value.clone())+ .map_err(|e| anyhow!("pagination is malformed: {e}"))?+ .has_more+ }+ _ => false,+ };++ let sessions: Vec<Session> = records(&export, "sessions")?;+ let memories: Vec<Memory> = records(&export, "memories")?;+ let summaries: Vec<Summary> = records(&export, "summaries")?;+ let lessons: Vec<Lesson> = records(&export, "lessons")?;+ let insights: Vec<Insight> = records(&export, "insights")?;+ let semantic: Vec<SemanticMemory> = records(&export, "semanticMemories")?;+ let procedural: Vec<ProceduralMemory> = records(&export, "proceduralMemories")?;+ let crystals: Vec<Crystal> = records(&export, "crystals")?;+ let observations: BTreeMap<String, Vec<Observation>> = match export.get("observations") {+ None | Some(Value::Null) => BTreeMap::new(),+ Some(Value::Object(map)) => map+ .iter()+ .map(|(session, items)| {+ let parsed: Vec<Observation> = serde_json::from_value(items.clone())+ .map_err(|e| anyhow!("observations[{session:?}] is malformed: {e}"))?;+ Ok((session.clone(), parsed))+ })+ .collect::<Result<_>>()?,+ Some(_) => bail!("observations must be an object keyed by session id"),+ };++ let mut by_session: HashMap<&str, &Session> = HashMap::new();+ for session in &sessions {+ validate_id(&session.id, "session")?;+ if by_session.insert(session.id.as_str(), session).is_some() {+ bail!("duplicate session id {:?}", session.id);+ }+ }+ // A record's mapping key: its own project, else the project of the first+ // session it cites, plus that session's cwd, so a cwd mapping follows+ // every record tied to the session.+ let scope_for = |project: Option<&'_ str>, session_ids: &[&str]| -> Scope {+ let session = session_ids+ .iter()+ .find_map(|id| by_session.get(id).copied());+ Scope {+ project: project+ .filter(|p| !p.trim().is_empty())+ .or_else(|| session.map(|s| s.project.as_str()))+ .map(str::to_owned),+ cwd: session.map(|s| s.cwd.clone()),+ }+ };++ let mut builder = Builder {+ prefix,+ pages: Vec::new(),+ paths: HashMap::new(),+ links: HashMap::new(),+ unmapped: BTreeMap::new(),+ };++ // Memories: group versions into lineages through parentId/supersedes.+ let mut memory_index: HashMap<&str, usize> = HashMap::new();+ for (index, memory) in memories.iter().enumerate() {+ validate_id(&memory.id, "memory")?;+ if memory_index.insert(memory.id.as_str(), index).is_some() {+ bail!("duplicate memory id {:?}", memory.id);+ }+ if let Some(ttl) = memory.forget_after.as_deref().filter(|t| !t.is_empty())+ && !looks_like_timestamp(ttl)+ {+ bail!(+ "memory {:?} has a forgetAfter that is not a timestamp",+ memory.id+ );+ }+ }+ let mut parent: Vec<usize> = (0..memories.len()).collect();+ fn find(parent: &mut [usize], mut i: usize) -> usize {+ while parent[i] != i {+ parent[i] = parent[parent[i]];+ i = parent[i];+ }+ i+ }+ for (index, memory) in memories.iter().enumerate() {+ for other in memory.parent_id.iter().chain(&memory.supersedes) {+ if let Some(&other) = memory_index.get(other.as_str()) {+ let (a, b) = (find(&mut parent, index), find(&mut parent, other));+ parent[a] = b;+ }+ }+ }+ let mut lineages: BTreeMap<usize, Vec<usize>> = BTreeMap::new();+ for index in 0..memories.len() {+ let root = find(&mut parent, index);+ lineages.entry(root).or_default().push(index);+ }+ let mut lineages: Vec<Vec<usize>> = lineages.into_values().collect();+ for lineage in &mut lineages {+ lineage.sort_by(|&a, &b| {+ let (a, b) = (&memories[a], &memories[b]);+ (a.version.unwrap_or(1), &a.created_at, &a.id).cmp(&(+ b.version.unwrap_or(1),+ &b.created_at,+ &b.id,+ ))+ });+ }+ // Oldest lineage first keeps the plan order stable across runs.+ lineages.sort_by(|a, b| memories[a[0]].id.cmp(&memories[b[0]].id));++ let mut memory_plans = Vec::new();+ for lineage in &lineages {+ let newest = &memories[*lineage.last().unwrap_or(&lineage[0])];+ let scope = scope_for(newest.project.as_deref(), &refs(&newest.session_ids));+ match mapping.resolve(scope.project.as_deref(), scope.cwd.as_deref()) {+ Ok(destination) => {+ let path = builder.path("memories", &memories[lineage[0]].id)?;+ for &index in lineage {+ builder.links.insert(+ memories[index].id.clone(),+ (destination.clone(), path.clone()),+ );+ }+ memory_plans.push((lineage, destination, path));+ }+ Err(label) => builder.unmapped(label, lineage.len()),+ }+ }++ // Single-version records: resolve, reserve a path and a link first, so+ // bodies can link to each other.+ struct Single<'a> {+ record: &'static str,+ id: &'a str,+ destination: Destination,+ path: String,+ }+ let mut singles: Vec<Single> = Vec::new();+ let reserve = |builder: &mut Builder,+ record: &'static str,+ folder: &str,+ id: &str,+ scope: Scope|+ -> Result<Option<(Destination, String)>> {+ validate_id(id, record)?;+ match mapping.resolve(scope.project.as_deref(), scope.cwd.as_deref()) {+ Ok(destination) => {+ let path = builder.path(folder, id)?;+ builder+ .links+ .insert(id.to_owned(), (destination.clone(), path.clone()));+ Ok(Some((destination, path)))+ }+ Err(label) => {+ builder.unmapped(label, 1);+ Ok(None)+ }+ }+ };+ let mut lessons_deleted = 0usize;+ for lesson in &lessons {+ if lesson.deleted == Some(true) {+ lessons_deleted += 1;+ continue;+ }+ let scope = scope_for(lesson.project.as_deref(), &[]);+ if let Some((destination, path)) =+ reserve(&mut builder, "lesson", "lessons", &lesson.id, scope)?+ {+ singles.push(Single {+ record: "lesson",+ id: &lesson.id,+ destination,+ path,+ });+ }+ }+ let mut insights_deleted = 0usize;+ for insight in &insights {+ if insight.deleted == Some(true) {+ insights_deleted += 1;+ continue;+ }+ let scope = scope_for(insight.project.as_deref(), &[]);+ if let Some((destination, path)) =+ reserve(&mut builder, "insight", "insights", &insight.id, scope)?+ {+ singles.push(Single {+ record: "insight",+ id: &insight.id,+ destination,+ path,+ });+ }+ }+ for fact in &semantic {+ let scope = scope_for(None, &refs(&fact.source_session_ids));+ if let Some((destination, path)) =+ reserve(&mut builder, "semantic-memory", "semantic", &fact.id, scope)?+ {+ singles.push(Single {+ record: "semantic-memory",+ id: &fact.id,+ destination,+ path,+ });+ }+ }+ for procedure in &procedural {+ let scope = scope_for(None, &refs(&procedure.source_session_ids));+ if let Some((destination, path)) = reserve(+ &mut builder,+ "procedural-memory",+ "procedural",+ &procedure.id,+ scope,+ )? {+ singles.push(Single {+ record: "procedural-memory",+ id: &procedure.id,+ destination,+ path,+ });+ }+ }+ for crystal in &crystals {+ let scope = scope_for(+ crystal.project.as_deref(),+ &crystal+ .session_id+ .iter()+ .map(String::as_str)+ .collect::<Vec<_>>(),+ );+ if let Some((destination, path)) =+ reserve(&mut builder, "crystal", "crystals", &crystal.id, scope)?+ {+ singles.push(Single {+ record: "crystal",+ id: &crystal.id,+ destination,+ path,+ });+ }+ }+ // Sessions and summaries share one page per session id.+ let mut summary_by_session: HashMap<&str, &Summary> = HashMap::new();+ for summary in &summaries {+ validate_id(&summary.session_id, "summary session")?;+ summary_by_session.insert(summary.session_id.as_str(), summary);+ }+ let mut session_ids: Vec<&str> = sessions.iter().map(|s| s.id.as_str()).collect();+ for summary in &summaries {+ if !by_session.contains_key(summary.session_id.as_str()) {+ session_ids.push(summary.session_id.as_str());+ }+ }+ let mut session_destinations: HashMap<&str, Destination> = HashMap::new();+ for &id in &session_ids {+ // A session record's own project wins over its summary's.+ let project = by_session.get(id).map(|s| s.project.as_str()).or_else(|| {+ summary_by_session+ .get(id)+ .and_then(|s| s.project.as_deref())+ });+ let scope = scope_for(project, &[id]);+ if let Some((destination, path)) = reserve(&mut builder, "session", "sessions", id, scope)?+ {+ session_destinations.insert(id, destination.clone());+ singles.push(Single {+ record: "session",+ id,+ destination,+ path,+ });+ }+ }+ if lessons_deleted > 0 {+ dropped.insert("lessons (deleted)".into(), lessons_deleted);+ }+ if insights_deleted > 0 {+ dropped.insert("insights (deleted)".into(), insights_deleted);+ }++ // Now render bodies, with every link target known.+ for (lineage, destination, path) in memory_plans {+ let mut versions = Vec::with_capacity(lineage.len());+ for &index in lineage {+ let memory = &memories[index];+ let (kind, tier) = memory_kind_tier(&memory.memory_type);+ let title = if memory.title.trim().is_empty() {+ one_line(&memory.content, MAX_TITLE_CHARS)+ } else {+ one_line(&memory.title, MAX_EXPLICIT_TITLE_CHARS)+ };+ let mut body = format!("# {title}\n\n{}\n", memory.content.trim());+ code_list(&mut body, "Files", &memory.files);+ let related: Vec<String> = memory+ .related_ids+ .iter()+ .filter(|id| !lineage.iter().any(|&i| &memories[i].id == *id))+ .map(|id| builder.link(&destination, id))+ .collect();+ bullet_list(&mut body, "Related", &related);+ let key = ingest_key("memory", &memory.id);+ versions.push(PlannedVersion {+ body: finish_body(&body, "memory", &memory.id, memory.version, &key),+ key,+ source_id: memory.id.clone(),+ version: memory.version,+ title: ai_memory_client::sanitize_external_text(&title),+ kind: kind.to_owned(),+ tier,+ tags: tags_from(memory.concepts.iter().map(String::as_str)),+ expires_at: memory.forget_after.clone().filter(|t| !t.is_empty()),+ });+ }+ builder.pages.push(PlannedPage {+ record: "memory",+ destination,+ path,+ versions,+ });+ }++ let lesson_by_id: HashMap<&str, &Lesson> = lessons.iter().map(|l| (l.id.as_str(), l)).collect();+ let insight_by_id: HashMap<&str, &Insight> =+ insights.iter().map(|i| (i.id.as_str(), i)).collect();+ let semantic_by_id: HashMap<&str, &SemanticMemory> =+ semantic.iter().map(|s| (s.id.as_str(), s)).collect();+ let procedural_by_id: HashMap<&str, &ProceduralMemory> =+ procedural.iter().map(|p| (p.id.as_str(), p)).collect();+ let crystal_by_id: HashMap<&str, &Crystal> =+ crystals.iter().map(|c| (c.id.as_str(), c)).collect();+ for single in &singles {+ let to = &single.destination;+ let links =+ |ids: &[String]| -> Vec<String> { ids.iter().map(|id| builder.link(to, id)).collect() };+ let version = match single.record {+ "lesson" => {+ let lesson = lesson_by_id[single.id];+ let title = format!("Lesson: {}", one_line(&lesson.content, MAX_TITLE_CHARS));+ let mut body = format!("# {title}\n\n{}\n", lesson.content.trim());+ if !lesson.context.trim().is_empty() {+ body.push_str(&format!("\n## Context\n{}\n", lesson.context.trim()));+ }+ body.push_str(&strength_line(lesson.confidence, lesson.reinforcements));+ bullet_list(&mut body, "Sources", &links(&lesson.source_ids));+ single_version(+ "lesson",+ single.id,+ title,+ &body,+ "lesson",+ "semantic",+ &lesson.tags,+ )+ }+ "insight" => {+ let insight = insight_by_id[single.id];+ let title = if insight.title.trim().is_empty() {+ one_line(&insight.content, MAX_TITLE_CHARS)+ } else {+ one_line(&insight.title, MAX_EXPLICIT_TITLE_CHARS)+ };+ let mut body = format!("# {title}\n\n{}\n", insight.content.trim());+ body.push_str(&strength_line(insight.confidence, insight.reinforcements));+ let sources: Vec<String> = insight+ .source_memory_ids+ .iter()+ .chain(&insight.source_lesson_ids)+ .chain(&insight.source_crystal_ids)+ .cloned()+ .collect();+ bullet_list(&mut body, "Sources", &links(&sources));+ single_version(+ "insight",+ single.id,+ title,+ &body,+ "insight",+ "semantic",+ &insight.tags,+ )+ }+ "semantic-memory" => {+ let fact = semantic_by_id[single.id];+ let title = one_line(&fact.fact, MAX_TITLE_CHARS);+ let mut body = format!("# {title}\n\n{}\n", fact.fact.trim());+ body.push_str(&strength_line(fact.confidence, None));+ bullet_list(&mut body, "Sources", &links(&fact.source_memory_ids));+ single_version(+ "semantic-memory",+ single.id,+ title,+ &body,+ "fact",+ "semantic",+ &[],+ )+ }+ "procedural-memory" => {+ let procedure = procedural_by_id[single.id];+ let title = one_line(&procedure.name, MAX_EXPLICIT_TITLE_CHARS);+ let mut body = format!("# {title}\n");+ if !procedure.trigger_condition.trim().is_empty() {+ body.push_str(&format!(+ "\n**When:** {}\n",+ procedure.trigger_condition.trim()+ ));+ }+ if !procedure.steps.is_empty() {+ body.push_str("\n## Steps\n");+ for (n, step) in procedure.steps.iter().enumerate() {+ body.push_str(&format!("{}. {step}\n", n + 1));+ }+ }+ if let Some(outcome) = procedure+ .expected_outcome+ .as_deref()+ .filter(|o| !o.trim().is_empty())+ {+ body.push_str(&format!("\n**Expected outcome:** {}\n", outcome.trim()));+ }+ let tags: Vec<String> = procedure+ .tags+ .iter()+ .chain(&procedure.concepts)+ .cloned()+ .collect();+ single_version(+ "procedural-memory",+ single.id,+ title,+ &body,+ "procedure",+ "procedural",+ &tags,+ )+ }+ "crystal" => {+ let crystal = crystal_by_id[single.id];+ let title = format!("Crystal: {}", one_line(&crystal.narrative, MAX_TITLE_CHARS));+ let mut body = format!("# {title}\n\n{}\n", crystal.narrative.trim());+ bullet_list(&mut body, "Key outcomes", &crystal.key_outcomes);+ bullet_list(&mut body, "Lessons", &crystal.lessons);+ code_list(&mut body, "Files", &crystal.files_affected);+ bullet_list(+ &mut body,+ "Source actions",+ &links(&crystal.source_action_ids),+ );+ single_version(+ "crystal",+ single.id,+ title,+ &body,+ "crystal",+ "episodic",+ &[],+ )+ }+ _ => {+ let session = by_session.get(single.id).copied();+ let summary = summary_by_session.get(single.id).copied();+ let (title, body, tags) = session_page(single.id, session, summary);+ single_version(+ "session", single.id, title, &body, "session", "episodic", &tags,+ )+ }+ };+ builder.pages.push(PlannedPage {+ record: single.record,+ destination: single.destination.clone(),+ path: single.path.clone(),+ versions: vec![version],+ });+ }++ // Observations: counted as dropped, or planned as replay sessions.+ let total_observations: usize = observations.values().map(Vec::len).sum();+ let mut hook_sessions = Vec::new();+ if !replay_observations {+ if total_observations > 0 {+ dropped.insert("observations".into(), total_observations);+ }+ } else {+ let mut without_session = 0usize;+ let mut over_cap = 0usize;+ for (session_id, items) in &observations {+ if items.is_empty() {+ continue;+ }+ let Some(session) = by_session.get(session_id.as_str()) else {+ without_session += items.len();+ continue;+ };+ let Some(destination) = session_destinations.get(session_id.as_str()) else {+ // The session page is unmapped; its label is already counted.+ *builder+ .unmapped+ .entry(+ mapping+ .resolve(Some(&session.project), Some(&session.cwd))+ .err()+ .unwrap_or_default(),+ )+ .or_default() += items.len();+ continue;+ };+ let mut sorted: Vec<&Observation> = items.iter().collect();+ sorted.sort_by(|a, b| (&a.timestamp, &a.id).cmp(&(&b.timestamp, &b.id)));+ if sorted.len() > MAX_REPLAYED_OBSERVATIONS {+ over_cap += sorted.len() - MAX_REPLAYED_OBSERVATIONS;+ sorted.truncate(MAX_REPLAYED_OBSERVATIONS);+ }+ hook_sessions.push(replay_session(server_url, session, destination, &sorted)?);+ }+ if without_session > 0 {+ dropped.insert(+ "observations without a session record".into(),+ without_session,+ );+ }+ if over_cap > 0 {+ dropped.insert(+ "observations over the per-session replay cap".into(),+ over_cap,+ );+ }+ }+ if partial {+ dropped.insert("export pages after this one (pagination.hasMore)".into(), 1);+ }++ Ok(ExportPlan {+ pages: builder.pages,+ hook_sessions,+ dropped,+ unmapped: builder.unmapped,+ partial,+ source_sha256,+ })+}++fn strength_line(confidence: Option<f64>, reinforcements: Option<u64>) -> String {+ match (confidence, reinforcements) {+ (Some(c), Some(r)) => {+ format!("\nConfidence {c:.2}, reinforced {r} time(s) in agentmemory.\n")+ }+ (Some(c), None) => format!("\nConfidence {c:.2} in agentmemory.\n"),+ (None, Some(r)) => format!("\nReinforced {r} time(s) in agentmemory.\n"),+ (None, None) => String::new(),+ }+}++fn session_page(+ id: &str,+ session: Option<&Session>,+ summary: Option<&Summary>,+) -> (String, String, Vec<String>) {+ let title = summary+ .map(|s| s.title.trim())+ .filter(|t| !t.is_empty())+ .map(|t| one_line(t, MAX_EXPLICIT_TITLE_CHARS))+ .or_else(|| {+ session+ .and_then(|s| s.first_prompt.as_deref())+ .filter(|p| !p.trim().is_empty())+ .map(|p| one_line(p, MAX_TITLE_CHARS))+ })+ .unwrap_or_else(|| format!("agentmemory session {}", one_line(id, MAX_TITLE_CHARS)));+ let mut body = format!("# {title}\n\n");+ if let Some(session) = session {+ let mut meta = vec![+ format!("- Project: {}", session.project),+ format!("- Status: {}", session.status),+ format!("- Started: {}", session.started_at),+ ];+ if let Some(ended) = session.ended_at.as_deref() {+ meta.push(format!("- Ended: {ended}"));+ }+ meta.push(format!("- Observations: {}", session.observation_count));+ if !session.cwd.is_empty() {+ meta.push(format!("- Working directory: `{}`", session.cwd));+ }+ if let Some(model) = session.model.as_deref() {+ meta.push(format!("- Model: {model}"));+ }+ body.push_str(&meta.join("\n"));+ body.push('\n');+ }+ if let Some(summary) = summary {+ if !summary.narrative.trim().is_empty() {+ body.push_str(&format!("\n## Summary\n{}\n", summary.narrative.trim()));+ }+ bullet_list(&mut body, "Key decisions", &summary.key_decisions);+ code_list(&mut body, "Files modified", &summary.files_modified);+ } else if let Some(text) = session+ .and_then(|s| s.summary.as_deref())+ .filter(|s| !s.trim().is_empty())+ {+ body.push_str(&format!("\n## Summary\n{}\n", text.trim()));+ }+ if let Some(prompt) = session+ .and_then(|s| s.first_prompt.as_deref())+ .filter(|p| !p.trim().is_empty())+ {+ body.push_str(&format!("\n## First prompt\n{}\n", prompt.trim()));+ }+ if let Some(session) = session {+ code_list(&mut body, "Commits", &session.commit_shas);+ }+ let mut tags: Vec<String> = summary.map(|s| s.concepts.clone()).unwrap_or_default();+ if let Some(session) = session {+ tags.extend(session.tags.iter().cloned());+ }+ (title, body, tags)+}++/// Plan one observation session as an ordered `external-import` hook batch.+fn replay_session(+ server_url: &str,+ session: &Session,+ destination: &Destination,+ observations: &[&Observation],+) -> Result<HookSession> {+ let stable = stable_external_session_id(+ &destination.workspace,+ &destination.project,+ SOURCE,+ &session.id,+ );+ let at = |value: &str| -> Value {+ if looks_like_timestamp(value) {+ json!(value)+ } else {+ Value::Null+ }+ };+ let mut events = Vec::with_capacity(observations.len() + 2);+ let mut start = json!({+ "session_id": stable,+ "model": format!("external:{SOURCE}"),+ "external_source": SOURCE,+ "external_session_id": session.id,+ });+ if let Value::String(ts) = at(&session.started_at) {+ start["occurred_at"] = json!(ts);+ }+ events.push(planned_hook_event(+ server_url,+ &destination.workspace,+ &destination.project,+ &stable,+ 0,+ "session-start",+ None,+ None,+ start,+ )?);+ for (n, observation) in observations.iter().enumerate() {+ validate_id(&observation.id, "observation")?;+ let mut text = String::new();+ if let Some(subtitle) = observation+ .subtitle+ .as_deref()+ .filter(|s| !s.trim().is_empty())+ {+ text.push_str(subtitle.trim());+ text.push_str("\n\n");+ }+ text.push_str(observation.narrative.trim());+ for fact in &observation.facts {+ text.push_str(&format!("\n- {fact}"));+ }+ if !observation.files.is_empty() {+ text.push_str(&format!("\nFiles: {}", observation.files.join(", ")));+ }+ let message = truncate_utf8(+ &ai_memory_client::sanitize_external_text(&text),+ MAX_EXTENSION_MESSAGE_BYTES,+ );+ let title = truncate_utf8(+ &ai_memory_client::sanitize_external_text(&one_line(+ &observation.title,+ MAX_TITLE_CHARS,+ )),+ MAX_OBSERVATION_TITLE_BYTES,+ );+ let mut body = json!({+ "session_id": stable,+ "title": if title.is_empty() { "agentmemory observation".to_owned() } else { title },+ "message": if message.trim().is_empty() { "(empty observation)".to_owned() } else { message },+ "external_source": SOURCE,+ });+ if let Value::String(ts) = at(&observation.timestamp) {+ body["occurred_at"] = json!(ts);+ }+ events.push(planned_hook_event(+ server_url,+ &destination.workspace,+ &destination.project,+ &stable,+ n + 1,+ "external.observation",+ None,+ Some("agentmemory-observation"),+ body,+ )?);+ }+ let mut end = json!({+ "session_id": stable,+ "external_source": SOURCE,+ "external_session_id": session.id,+ });+ let ended = session+ .ended_at+ .clone()+ .or_else(|| observations.last().map(|o| o.timestamp.clone()))+ .unwrap_or_default();+ if let Value::String(ts) = at(&ended) {+ end["occurred_at"] = json!(ts);+ }+ events.push(planned_hook_event(+ server_url,+ &destination.workspace,+ &destination.project,+ &stable,+ observations.len() + 1,+ "session-end",+ None,+ None,+ end,+ )?);+ Ok(HookSession {+ source_session_id: session.id.clone(),+ stable_session_id: stable,+ destination: destination.clone(),+ events,+ })+}++#[cfg(test)]+pub(crate) mod tests;diff --git a/companions/ai-memory-importer/src/agentmemory/export/tests.rs b/companions/ai-memory-importer/src/agentmemory/export/tests.rsnew file mode 100644index 00000000..c3d51cb4--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/export/tests.rs@@ -0,0 +1,491 @@+use super::*;+use crate::agentmemory::mock::{Failure, MockServer};+use tempfile::tempdir;++/// A synthetic `ExportData` in agentmemory's documented shape. Nothing here is+/// real user data. Two repositories share the basename `api`; the mapping+/// tells them apart by session cwd.+pub(crate) fn fixture_export() -> Value {+ json!({+ "version": "0.9.30",+ "exportedAt": "2026-10-01T00:00:00.000Z",+ "sessions": [+ {"id": "ses_1", "project": "api", "cwd": "/work/acme/api", "startedAt": "2026-03-01T10:00:00.000Z",+ "endedAt": "2026-03-01T11:00:00.000Z", "status": "completed", "observationCount": 3,+ "firstPrompt": "Why does the queue stall?", "commitShas": ["abc1234"]},+ {"id": "ses_2", "project": "api", "cwd": "/work/other/api", "startedAt": "2026-03-02T10:00:00.000Z",+ "status": "completed", "observationCount": 0},+ {"id": "ses_3", "project": "web", "cwd": "/work/acme/web", "startedAt": "2026-03-03T10:00:00.000Z",+ "status": "abandoned", "observationCount": 1}+ ],+ "observations": {+ "ses_1": [+ {"id": "obs_2", "sessionId": "ses_1", "timestamp": "2026-03-01T10:20:00.000Z", "type": "edit",+ "title": "Bounded the queue", "facts": ["capacity 1024"], "narrative": "Set a bound.",+ "concepts": ["queue"], "files": ["src/queue.rs"], "importance": 6},+ {"id": "obs_1", "sessionId": "ses_1", "timestamp": "2026-03-01T10:10:00.000Z", "type": "read",+ "title": "Read the queue", "facts": [], "narrative": "token=github_pat_1234567890abcdefghijklmnop",+ "concepts": [], "files": [], "importance": 3},+ {"id": "obs_3", "sessionId": "ses_1", "timestamp": "2026-03-01T10:30:00.000Z", "type": "test",+ "title": "Ran the tests", "facts": [], "narrative": "All green.", "concepts": [], "files": [], "importance": 4}+ ],+ "ses_3": [+ {"id": "obs_4", "sessionId": "ses_3", "timestamp": "2026-03-03T10:05:00.000Z", "type": "read",+ "title": "Looked at CSS", "facts": [], "narrative": "Nothing.", "concepts": [], "files": [], "importance": 1}+ ]+ },+ "memories": [+ // Newest first on purpose: the importer must reorder.+ {"id": "mem_q3", "createdAt": "2026-03-05T00:00:00.000Z", "updatedAt": "2026-03-05T00:00:00.000Z",+ "type": "architecture", "title": "Queue bound is 4096", "content": "Raised again after load tests.",+ "concepts": ["queue"], "files": ["src/queue.rs"], "sessionIds": ["ses_1"], "strength": 0.9,+ "version": 3, "parentId": "mem_q2", "supersedes": ["mem_q2"], "isLatest": true,+ "relatedIds": ["mem_ttl", "mem_web", "mem_missing"]},+ {"id": "mem_q1", "createdAt": "2026-03-01T00:00:00.000Z", "updatedAt": "2026-03-01T00:00:00.000Z",+ "type": "architecture", "title": "Queue bound is 1024", "content": "First bound.",+ "concepts": ["queue"], "files": [], "sessionIds": ["ses_1"], "strength": 0.5,+ "version": 1, "isLatest": false},+ {"id": "mem_q2", "createdAt": "2026-03-03T00:00:00.000Z", "updatedAt": "2026-03-03T00:00:00.000Z",+ "type": "architecture", "title": "Queue bound is 2048", "content": "Raised once.",+ "concepts": ["queue"], "files": [], "sessionIds": ["ses_1"], "strength": 0.7,+ "version": 2, "parentId": "mem_q1", "supersedes": ["mem_q1"], "isLatest": false},+ {"id": "mem_ttl", "createdAt": "2026-03-01T00:00:00.000Z", "updatedAt": "2026-03-01T00:00:00.000Z",+ "type": "fact", "title": "Staging host is temporary", "content": "Gone after the migration.",+ "concepts": [], "files": [], "sessionIds": ["ses_1"], "strength": 0.4, "version": 1,+ "isLatest": true, "forgetAfter": "2026-04-01T00:00:00.000Z"},+ {"id": "mem_other_api", "createdAt": "2026-03-02T00:00:00.000Z", "updatedAt": "2026-03-02T00:00:00.000Z",+ "type": "preference", "title": "Tabs in the other api", "content": "This is the other repository.",+ "concepts": [], "files": [], "sessionIds": ["ses_2"], "strength": 0.4, "version": 1, "isLatest": true},+ {"id": "mem_web", "createdAt": "2026-03-03T00:00:00.000Z", "updatedAt": "2026-03-03T00:00:00.000Z",+ "type": "workflow", "title": "Run the web build with npm ci", "content": "npm ci then npm run build.",+ "concepts": ["build"], "files": [], "sessionIds": ["ses_3"], "strength": 0.6, "version": 1,+ "isLatest": true, "project": "web"}+ ],+ "summaries": [+ {"sessionId": "ses_1", "project": "api", "createdAt": "2026-03-01T11:00:00.000Z",+ "title": "Bounded the job queue", "narrative": "Found the stall and bounded the queue.",+ "keyDecisions": ["Bound the queue"], "filesModified": ["src/queue.rs"], "concepts": ["queue"],+ "observationCount": 3}+ ],+ "lessons": [+ {"id": "les_1", "content": "Always bound in-memory queues.", "context": "From the stall.",+ "confidence": 0.8, "reinforcements": 2, "source": "crystal", "sourceIds": ["crys_1"],+ "project": "api", "tags": ["queues"], "createdAt": "2026-03-04T00:00:00.000Z",+ "updatedAt": "2026-03-04T00:00:00.000Z", "decayRate": 0.01},+ {"id": "les_gone", "content": "Deleted lesson.", "context": "", "confidence": 0.1,+ "reinforcements": 0, "source": "manual", "sourceIds": [], "tags": [],+ "createdAt": "2026-03-04T00:00:00.000Z", "updatedAt": "2026-03-04T00:00:00.000Z",+ "decayRate": 0.01, "deleted": true}+ ],+ "insights": [+ {"id": "ins_1", "title": "Backpressure beats retries", "content": "Bounded queues removed the retry storm.",+ "confidence": 0.7, "reinforcements": 1, "sourceConceptCluster": [], "sourceMemoryIds": ["mem_q3"],+ "sourceLessonIds": ["les_1"], "sourceCrystalIds": [], "project": "api", "tags": [],+ "createdAt": "2026-03-06T00:00:00.000Z", "updatedAt": "2026-03-06T00:00:00.000Z", "decayRate": 0.01}+ ],+ "semanticMemories": [+ {"id": "sem_1", "fact": "The queue lives in src/queue.rs.", "confidence": 0.9,+ "sourceSessionIds": ["ses_1"], "sourceMemoryIds": [], "accessCount": 1,+ "lastAccessedAt": "2026-03-06T00:00:00.000Z", "strength": 0.8,+ "createdAt": "2026-03-06T00:00:00.000Z", "updatedAt": "2026-03-06T00:00:00.000Z"}+ ],+ "proceduralMemories": [+ {"id": "proc_1", "name": "Release the api", "steps": ["Run tests", "Tag"], "triggerCondition": "Before a release",+ "frequency": 2, "sourceSessionIds": ["ses_1"], "strength": 0.5,+ "createdAt": "2026-03-06T00:00:00.000Z", "updatedAt": "2026-03-06T00:00:00.000Z"}+ ],+ "crystals": [+ {"id": "crys_1", "narrative": "Fixed the queue stall.", "keyOutcomes": ["queue bounded"],+ "filesAffected": ["src/queue.rs"], "lessons": ["bound queues"], "sourceActionIds": ["act_1"],+ "sessionId": "ses_1", "createdAt": "2026-03-04T00:00:00.000Z"}+ ],+ "graphNodes": [{"id": "n1"}, {"id": "n2"}],+ "graphEdges": [{"id": "e1"}],+ "actions": [{"id": "act_1"}, {"id": "act_2"}],+ "signals": [{"id": "sig_1"}],+ "routines": [{"id": "rt_1"}],+ "accessLogs": [{"memoryId": "mem_q3", "count": 2, "lastAt": "2026-03-06T00:00:00.000Z", "recent": []}],+ "aFieldFromAFutureVersion": [1, 2]+ })+}++pub(crate) fn fixture_mapping() -> Value {+ json!({+ "projects": {+ "api": {"workspace": "acme", "project": "acme-api"},+ "web": {"workspace": "acme", "project": "acme-web"}+ },+ "cwds": {+ "/work/other/api": {"workspace": "other", "project": "other-api"}+ }+ })+}++pub(crate) struct Fixture {+ pub(crate) _dir: tempfile::TempDir,+ pub(crate) file: PathBuf,+ pub(crate) mapping: PathBuf,+ pub(crate) manifest: PathBuf,+}++pub(crate) fn fixture_files(export: &Value, mapping: &Value) -> Fixture {+ let dir = tempdir().unwrap();+ let file = dir.path().join("export.json");+ let mapping_path = dir.path().join("mapping.json");+ fs::write(&file, serde_json::to_vec_pretty(export).unwrap()).unwrap();+ fs::write(&mapping_path, serde_json::to_vec_pretty(mapping).unwrap()).unwrap();+ Fixture {+ manifest: dir.path().join("manifest.json"),+ file,+ mapping: mapping_path,+ _dir: dir,+ }+}++fn plan_fixture(export: &Value, mapping: &Value, replay: bool) -> ExportPlan {+ let files = fixture_files(export, mapping);+ let mapping = Mapping::load(&files.mapping).unwrap();+ plan(+ &files.file,+ &mapping,+ DEFAULT_DEST_PREFIX,+ replay,+ crate::DEFAULT_SERVER_URL,+ )+ .unwrap()+}++fn args(fixture: &Fixture, url: &str) -> ExportArgs {+ ExportArgs {+ file: fixture.file.clone(),+ mapping: fixture.mapping.clone(),+ dest_prefix: DEFAULT_DEST_PREFIX.into(),+ server_url: url.into(),+ apply: true,+ manifest_out: Some(fixture.manifest.clone()),+ create_destination: true,+ show_body: false,+ replay_observations: false,+ allow_partial: false,+ }+}++fn read_manifest(path: &Path) -> Value {+ serde_json::from_slice(&fs::read(path).unwrap()).unwrap()+}++const CHAIN: &str = "imported/agentmemory/memories/mem-q1.md";++#[test]+fn manifest_counts_equal_the_fixture() {+ let plan = plan_fixture(&fixture_export(), &fixture_mapping(), false);+ let manifest = Manifest::new(IMPORT_VERSION, String::new(), String::new(), &plan.pages);+ // Four memory lineages (q1..q3, ttl, other-api, web), six versions.+ assert_eq!(manifest.planned["memory_pages"], 4);+ assert_eq!(manifest.planned["memory_versions"], 6);+ assert_eq!(manifest.planned["lesson_pages"], 1);+ assert_eq!(manifest.planned["insight_pages"], 1);+ assert_eq!(manifest.planned["semantic-memory_pages"], 1);+ assert_eq!(manifest.planned["procedural-memory_pages"], 1);+ assert_eq!(manifest.planned["crystal_pages"], 1);+ assert_eq!(manifest.planned["session_pages"], 3);+ let expected: BTreeMap<String, usize> = [+ ("observations", 4),+ ("graphNodes", 2),+ ("graphEdges", 1),+ ("actions", 2),+ ("signals", 1),+ ("routines", 1),+ ("accessLogs", 1),+ ("aFieldFromAFutureVersion", 2),+ ("lessons (deleted)", 1),+ ]+ .into_iter()+ .map(|(k, v)| (k.to_owned(), v))+ .collect();+ assert_eq!(plan.dropped, expected);+ assert!(plan.unmapped.is_empty());+ for page in &plan.pages {+ assert!(+ page.path.starts_with("imported/agentmemory/"),+ "{}",+ page.path+ );+ }+}++#[test]+fn versions_are_ordered_oldest_first_on_one_path_and_ttl_maps_to_expires_at() {+ let plan = plan_fixture(&fixture_export(), &fixture_mapping(), false);+ let chain = plan.pages.iter().find(|p| p.path == CHAIN).unwrap();+ let ids: Vec<_> = chain+ .versions+ .iter()+ .map(|v| v.source_id.as_str())+ .collect();+ assert_eq!(ids, ["mem_q1", "mem_q2", "mem_q3"]);+ assert_eq!(chain.versions[2].kind, "decision");+ let ttl = plan+ .pages+ .iter()+ .find(|p| p.path.ends_with("mem-ttl.md"))+ .unwrap();+ assert_eq!(+ ttl.versions[0].expires_at.as_deref(),+ Some("2026-04-01T00:00:00.000Z")+ );+ // Related links: same scope as a plain link, another scope qualified,+ // a missing record as plain text.+ let body = &chain.versions[2].body;+ assert!(+ body.contains("[[imported/agentmemory/memories/mem-ttl]]"),+ "{body}"+ );+ assert!(+ body.contains("[[acme/acme-web:imported/agentmemory/memories/mem-web]]"),+ "{body}"+ );+ assert!(body.contains("`mem_missing`"), "{body}");+}++#[test]+fn colliding_basenames_split_by_cwd() {+ // Newer exports carry `project` on memories too; the cited session's cwd+ // still decides which repository it was.+ let mut export = fixture_export();+ export["memories"][4]["project"] = json!("api");+ let plan = plan_fixture(&export, &fixture_mapping(), false);+ let other = plan+ .pages+ .iter()+ .find(|p| p.path.ends_with("mem-other-api.md"))+ .unwrap();+ assert_eq!(other.destination.workspace, "other");+ assert_eq!(other.destination.project, "other-api");+ let ours = plan.pages.iter().find(|p| p.path == CHAIN).unwrap();+ assert_eq!(ours.destination.project, "acme-api");+}++#[test]+fn text_is_redacted_before_the_plan_exists() {+ let mut export = fixture_export();+ export["memories"][0]["content"] = json!("key: sk-1234567890abcdefghijklmnop end");+ let plan = plan_fixture(&export, &fixture_mapping(), true);+ let all = format!(+ "{:?}{}",+ plan.pages,+ serde_json::to_string(+ &plan+ .hook_sessions+ .iter()+ .map(|s| &s.events)+ .collect::<Vec<_>>()+ )+ .unwrap()+ );+ assert!(!all.contains("sk-1234567890abcdefghijklmnop"));+ assert!(!all.contains("github_pat_1234567890abcdefghijklmnop"));+ assert!(all.contains("REDACTED"));+}++#[test]+fn malformed_inputs_are_refused_before_planning() {+ let files = fixture_files(&fixture_export(), &json!({"projects": {}, "surprise": 1}));+ assert!(+ Mapping::load(&files.mapping).is_err(),+ "unknown mapping keys fail closed"+ );+ let files = fixture_files(+ &fixture_export(),+ &json!({"projects": {"api": {"workspace": "a/b", "project": "p"}}}),+ );+ assert!(+ Mapping::load(&files.mapping).is_err(),+ "a path separator is not a workspace"+ );++ let mapping = fixture_mapping();+ for (mutate, needle) in [+ (+ json!({"memories": [{"id": "sk-1234567890abcdefghijklmnop"}]}),+ "credential",+ ),+ (+ json!({"memories": [{"id": "m", "forgetAfter": "next tuesday"}]}),+ "forgetAfter",+ ),+ (json!({"memories": [{"id": "m"}, {"id": "m"}]}), "duplicate"),+ (json!({"memories": "nope"}), "array"),+ (json!({"sessions": [{"project": "api"}]}), "sessions[0]"),+ ] {+ let files = fixture_files(&mutate, &mapping);+ let loaded = Mapping::load(&files.mapping).unwrap();+ let error = plan(+ &files.file,+ &loaded,+ DEFAULT_DEST_PREFIX,+ false,+ crate::DEFAULT_SERVER_URL,+ )+ .unwrap_err();+ assert!(format!("{error:#}").contains(needle), "{needle}: {error:#}");+ }+ // Two ids whose slugs collide.+ let files = fixture_files(+ &json!({"lessons": [{"id": "A B"}, {"id": "a-b"}]}),+ &json!({"unscoped": {"workspace": "w", "project": "p"}}),+ );+ let loaded = Mapping::load(&files.mapping).unwrap();+ let error = plan(+ &files.file,+ &loaded,+ DEFAULT_DEST_PREFIX,+ false,+ crate::DEFAULT_SERVER_URL,+ )+ .unwrap_err();+ assert!(error.to_string().contains("collision"), "{error}");+}++#[tokio::test]+async fn apply_rebuilds_the_supersession_chain_conditionally() {+ let fixture = fixture_files(&fixture_export(), &fixture_mapping());+ let server = MockServer::start(&[]);+ run(args(&fixture, &server.url)).await.unwrap();+ let state = server.state();+ let chain = state.chain("acme", "acme-api", CHAIN);+ assert_eq!(chain.len(), 3, "chain length equals the version count");+ assert_eq!(chain[0].arguments["create_only"], true);+ assert_eq!(+ chain[1].arguments["expected_page_id"],+ chain[0].page_id.as_str()+ );+ assert_eq!(+ chain[2].arguments["expected_page_id"],+ chain[1].page_id.as_str()+ );+ assert!(chain[2].body.contains("Queue bound is 4096"));+ let ttl = state.chain(+ "acme",+ "acme-api",+ "imported/agentmemory/memories/mem-ttl.md",+ );+ assert_eq!(ttl[0].arguments["expires_at"], "2026-04-01T00:00:00.000Z");+ assert!(+ state.hook_batches.is_empty(),+ "observations are dropped by default"+ );+ assert!(+ state+ .pages+ .keys()+ .all(|(_, _, path)| !path.starts_with("sessions/"))+ );+ drop(state);+ let manifest = read_manifest(&fixture.manifest);+ assert_eq!(manifest["status"], "imported");+ assert_eq!(manifest["written_versions"], 14);+ assert_eq!(manifest["dropped"]["observations"], 4);+ assert_eq!(manifest["dropped"]["signals"], 1);+}++#[tokio::test]+async fn an_unmapped_project_aborts_live_mode_before_any_request() {+ let mut mapping = fixture_mapping();+ mapping["projects"].as_object_mut().unwrap().remove("web");+ let fixture = fixture_files(&fixture_export(), &mapping);+ let server = MockServer::start(&[]);+ let error = run(args(&fixture, &server.url)).await.unwrap_err();+ assert!(error.to_string().contains("web"), "{error:#}");+ assert!(+ server.state().requests.is_empty(),+ "no request before the refusal"+ );+ let manifest = read_manifest(&fixture.manifest);+ assert_eq!(manifest["status"], "failed");+ // mem_web and the ses_3 session page: every unmapped record is counted.+ assert_eq!(manifest["unmapped_projects"]["web"], 2);++ // The dry run reports the same gap and succeeds.+ let mut dry = args(&fixture, &server.url);+ dry.apply = false;+ run(dry).await.unwrap();+ assert!(server.state().requests.is_empty());+}++#[tokio::test]+async fn resume_after_an_injected_failure_writes_no_duplicates() {+ // Write 2 is the chain's second version: fail it before and after the+ // server stores it.+ for failure in [Failure::Before(2), Failure::After(2), Failure::After(7)] {+ let fixture = fixture_files(&fixture_export(), &fixture_mapping());+ let server = MockServer::start(&[]);+ server.state().failure = Some(failure);+ assert!(run(args(&fixture, &server.url)).await.is_err());+ assert_eq!(read_manifest(&fixture.manifest)["status"], "failed");+ server.state().failure = None;+ run(args(&fixture, &server.url)).await.unwrap();+ assert_eq!(server.state().stored_versions(), 14, "{failure:?}");+ assert_eq!(+ server.state().chain("acme", "acme-api", CHAIN).len(),+ 3,+ "{failure:?}"+ );+ // And a third run writes nothing at all.+ run(args(&fixture, &server.url)).await.unwrap();+ assert_eq!(server.state().stored_versions(), 14, "{failure:?}");+ assert_eq!(read_manifest(&fixture.manifest)["written_versions"], 0);+ }+}++#[tokio::test]+async fn observations_replay_as_ordered_external_import_sessions_when_asked() {+ let fixture = fixture_files(&fixture_export(), &fixture_mapping());+ let server = MockServer::start(&[]);+ let mut replay = args(&fixture, &server.url);+ replay.replay_observations = true;+ run(replay).await.unwrap();+ let state = server.state();+ assert_eq!(+ state.hook_batches.len(),+ 2,+ "one batch per session with observations"+ );+ let batch = state.hook_batches[0].as_array().unwrap();+ assert_eq!(batch.len(), 5, "start + 3 observations + end");+ let urls: Vec<&str> = batch+ .iter()+ .map(|item| item["url"].as_str().unwrap())+ .collect();+ assert!(urls[0].contains("event=session-start"));+ assert!(urls[4].contains("event=session-end"));+ assert!(urls.iter().all(|url| url.contains("agent=external-import")));+ assert!(urls[1].contains("source_event=agentmemory-observation"));+ // Oldest observation first, redacted.+ assert_eq!(batch[1]["body"]["title"], "Read the queue");+ assert!(+ !batch[1]["body"]["message"]+ .as_str()+ .unwrap()+ .contains("github_pat_")+ );+ assert_eq!(batch[2]["body"]["title"], "Bounded the queue");+ drop(state);+ let manifest = read_manifest(&fixture.manifest);+ assert!(manifest["dropped"].get("observations").is_none());+ assert_eq!(manifest["hook_sessions"][0]["status"], "imported");+}++#[tokio::test]+async fn a_partial_export_page_needs_allow_partial() {+ let mut export = fixture_export();+ export["pagination"] = json!({"offset": 0, "limit": 3, "total": 9, "hasMore": true});+ let fixture = fixture_files(&export, &fixture_mapping());+ let server = MockServer::start(&[]);+ let error = run(args(&fixture, &server.url)).await.unwrap_err();+ assert!(error.to_string().contains("--allow-partial"), "{error:#}");+ assert!(server.state().requests.is_empty());+ let mut partial = args(&fixture, &server.url);+ partial.allow_partial = true;+ run(partial).await.unwrap();+}diff --git a/companions/ai-memory-importer/src/agentmemory/mock.rs b/companions/ai-memory-importer/src/agentmemory/mock.rsnew file mode 100644index 00000000..31102857--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/mock.rs@@ -0,0 +1,251 @@+//! A stateful stand-in for the ai-memory MCP and hook endpoints the+//! agentmemory routes call. It enforces the server's conditional-write+//! semantics (`create_only`, `expected_page_id`), keeps each path's version+//! chain, and can fail a chosen write before or after applying it.++use std::collections::{BTreeMap, BTreeSet};+use std::net::TcpListener;+use std::sync::{Arc, Mutex};+use std::thread;++use serde_json::{Value, json};++use crate::test_http::{read_http_request, respond_json};++/// One stored version of a page.+#[derive(Debug, Clone)]+pub(crate) struct StoredVersion {+ pub(crate) page_id: String,+ pub(crate) body: String,+ pub(crate) arguments: Value,+}++/// Where and when the mock fails a `memory_write_page` call (1-based count of+/// write calls).+#[derive(Debug, Clone, Copy)]+pub(crate) enum Failure {+ /// Answer HTTP 500 without storing anything.+ Before(usize),+ /// Store the write, then answer HTTP 500, as a crash after the server+ /// committed but before the client recorded it would look.+ After(usize),+}++#[derive(Debug, Default)]+pub(crate) struct MockState {+ /// `(workspace, project, path)` → versions, oldest first.+ pub(crate) pages: BTreeMap<(String, String, String), Vec<StoredVersion>>,+ pub(crate) scopes: BTreeSet<(String, String)>,+ /// Every request path the mock saw, in order.+ pub(crate) requests: Vec<String>,+ pub(crate) write_calls: usize,+ pub(crate) failure: Option<Failure>,+ /// Advertise `create_only` in `tools/list` (a 2.7+ server).+ pub(crate) conditional: bool,+ pub(crate) hook_batches: Vec<Value>,+ next_id: usize,+}++impl MockState {+ pub(crate) fn chain(&self, workspace: &str, project: &str, path: &str) -> Vec<StoredVersion> {+ self.pages+ .get(&(workspace.into(), project.into(), path.into()))+ .cloned()+ .unwrap_or_default()+ }++ pub(crate) fn stored_versions(&self) -> usize {+ self.pages.values().map(Vec::len).sum()+ }+}++pub(crate) struct MockServer {+ pub(crate) url: String,+ pub(crate) state: Arc<Mutex<MockState>>,+}++impl MockServer {+ /// Start a mock that knows `scopes` and advertises conditional writes.+ pub(crate) fn start(scopes: &[(&str, &str)]) -> Self {+ let state = Arc::new(Mutex::new(MockState {+ conditional: true,+ scopes: scopes+ .iter()+ .map(|(w, p)| ((*w).to_owned(), (*p).to_owned()))+ .collect(),+ ..MockState::default()+ }));+ let listener = TcpListener::bind("127.0.0.1:0").unwrap();+ let url = format!("http://{}", listener.local_addr().unwrap());+ let shared = Arc::clone(&state);+ thread::spawn(move || {+ for stream in listener.incoming() {+ let Ok(mut stream) = stream else { return };+ let (headers, body) = read_http_request(&mut stream);+ let path = headers+ .split_whitespace()+ .nth(1)+ .unwrap_or_default()+ .to_owned();+ let (status, reply) = handle(&shared, &path, &body);+ respond_json(&mut stream, status, &reply.to_string());+ }+ });+ Self { url, state }+ }++ pub(crate) fn state(&self) -> std::sync::MutexGuard<'_, MockState> {+ self.state.lock().unwrap()+ }+}++fn handle(state: &Mutex<MockState>, path: &str, body: &[u8]) -> (&'static str, Value) {+ let mut state = state.lock().unwrap();+ state.requests.push(path.to_owned());+ let request: Value = serde_json::from_slice(body).unwrap_or(Value::Null);+ if path.starts_with("/hook/batch") {+ let accepted = request.as_array().map_or(0, Vec::len);+ state.hook_batches.push(request);+ return ("200 OK", json!({"accepted": accepted}));+ }+ match request["method"].as_str() {+ Some("tools/list") => {+ let mut properties = json!({"path": {}, "body": {}});+ if state.conditional {+ properties["create_only"] = json!({"type": "boolean"});+ properties["expected_page_id"] = json!({"type": "string"});+ }+ (+ "200 OK",+ rpc(+ json!({"tools": [{"name": "memory_write_page", "inputSchema": {"properties": properties}}]}),+ ),+ )+ }+ Some("tools/call") => {+ let name = request["params"]["name"].as_str().unwrap_or_default();+ let arguments = request["params"]["arguments"].clone();+ let scope = (+ arguments["workspace"]+ .as_str()+ .unwrap_or_default()+ .to_owned(),+ arguments["project"].as_str().unwrap_or_default().to_owned(),+ );+ match name {+ "memory_recent" => {+ if state.scopes.contains(&scope) {+ ("200 OK", tool_text(&json!([])))+ } else {+ (+ "200 OK",+ rpc(+ json!({"isError": true, "content": [{"type": "text", "text": "scope not found"}]}),+ ),+ )+ }+ }+ "memory_read_page" => {+ let key = (+ scope.0,+ scope.1,+ arguments["path"].as_str().unwrap_or_default().to_owned(),+ );+ match state.pages.get(&key).and_then(|chain| chain.last()) {+ Some(latest) => (+ "200 OK",+ tool_text(+ &json!({"path": key.2, "page_id": latest.page_id, "body": latest.body}),+ ),+ ),+ None => (+ "200 OK",+ rpc(+ json!({"isError": true, "content": [{"type": "text", "text": "page not found"}]}),+ ),+ ),+ }+ }+ "memory_write_page" => write(&mut state, scope, arguments),+ other => (+ "200 OK",+ rpc(+ json!({"isError": true, "content": [{"type": "text", "text": format!("unknown tool {other}")}]}),+ ),+ ),+ }+ }+ _ => ("400 Bad Request", json!({"error": "bad request"})),+ }+}++fn write(+ state: &mut MockState,+ scope: (String, String),+ arguments: Value,+) -> (&'static str, Value) {+ state.write_calls += 1;+ let call = state.write_calls;+ if matches!(state.failure, Some(Failure::Before(n)) if n == call) {+ return ("500 Internal Server Error", json!({"error": "injected"}));+ }+ let key = (+ scope.0.clone(),+ scope.1.clone(),+ arguments["path"].as_str().unwrap_or_default().to_owned(),+ );+ let current = state+ .pages+ .get(&key)+ .and_then(|chain| chain.last())+ .map(|v| v.page_id.clone());+ let create_only = arguments["create_only"].as_bool() == Some(true);+ let expected = arguments["expected_page_id"].as_str().map(str::to_owned);+ let holds = if create_only {+ current.is_none()+ } else if let Some(expected) = &expected {+ current.as_deref() == Some(expected.as_str())+ } else {+ true+ };+ if !holds {+ return (+ "200 OK",+ json!({"jsonrpc": "2.0", "id": 1, "error": {+ "code": -32600,+ "message": "precondition failed",+ "data": {"reason": "precondition_failed", "path": key.2, "current_page_id": current},+ }}),+ );+ }+ state.next_id += 1;+ let page_id = format!("page-{}", state.next_id);+ state.scopes.insert(scope);+ state+ .pages+ .entry(key.clone())+ .or_default()+ .push(StoredVersion {+ page_id: page_id.clone(),+ body: arguments["body"].as_str().unwrap_or_default().to_owned(),+ arguments,+ });+ if matches!(state.failure, Some(Failure::After(n)) if n == call) {+ return (+ "500 Internal Server Error",+ json!({"error": "injected after commit"}),+ );+ }+ (+ "200 OK",+ tool_text(&json!({"page_id": page_id, "path": key.2})),+ )+}++fn rpc(result: Value) -> Value {+ json!({"jsonrpc": "2.0", "id": 1, "result": result})+}++fn tool_text(value: &Value) -> Value {+ rpc(json!({"content": [{"type": "text", "text": value.to_string()}]}))+}diff --git a/companions/ai-memory-importer/src/agentmemory/mod.rs b/companions/ai-memory-importer/src/agentmemory/mod.rsnew file mode 100644index 00000000..54ebe9c6--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/mod.rs@@ -0,0 +1,702 @@+//! Shared pieces of the two agentmemory routes: the Obsidian vault+//! (`agentmemory-vault`) and the `ExportData` JSON file (`agentmemory-export`).+//!+//! Both plan every page before any request, write through the conditional+//! MCP tools (`create_only` for a page's first version, `expected_page_id` for+//! each later one), and never write under ai-memory's own reserved paths: the+//! destination is always below `--dest-prefix` (default+//! `imported/agentmemory`).+//!+//! Each page version ends with a provenance line naming its stable ingest key+//! (a UUIDv5 of the agentmemory record id). A rerun finds the key in the page+//! the server already holds and skips what is there, so an interrupted import+//! resumes without duplicates and a finished one writes nothing.++pub(crate) mod export;+pub(crate) mod vault;++use std::collections::{BTreeMap, BTreeSet};+use std::path::Path;++use anyhow::{Context, Result, bail};+use serde::Serialize;+use uuid::Uuid;++use crate::ImportClient;+use crate::mcp::{McpWrite, Precondition, PreconditionFailed};++/// Where imported pages land unless `--dest-prefix` says otherwise.+pub(crate) const DEFAULT_DEST_PREFIX: &str = "imported/agentmemory";+/// Every page carries this tag so an import can be found and reviewed.+pub(crate) const IMPORT_TAG: &str = "agentmemory-import";+/// Bound on the tags one page carries.+const MAX_TAGS: usize = 16;+/// Bound on one tag, in characters.+const MAX_TAG_CHARS: usize = 64;+/// Bound on a derived title, in characters.+pub(crate) const MAX_TITLE_CHARS: usize = 80;+/// Bound on an explicit page title, in characters.+pub(crate) const MAX_EXPLICIT_TITLE_CHARS: usize = 200;+/// Bound on an agentmemory id, in bytes.+const MAX_ID_BYTES: usize = 256;++/// First path segments a destination prefix may never start with: ai-memory's+/// own session pages and maintained families. Any segment starting with `_`+/// or `.` (`_rules`, `_prompts`, `_global`, `.git`, ...) is refused as well.+const RESERVED_FIRST_SEGMENTS: &[&str] = &[+ "sessions",+ "session-logs",+ "procedures",+ "decisions",+ "gotchas",+ "profile",+];++/// The UUIDv5 namespace every agentmemory ingest key is derived in.+fn key_namespace() -> Uuid {+ Uuid::new_v5(+ &Uuid::NAMESPACE_URL,+ b"https://github.com/akitaonrails/ai-memory/companions/ai-memory-importer/agentmemory",+ )+}++/// Stable ingest key for one agentmemory record: the same record always gets+/// the same key, whatever the destination or run.+pub(crate) fn ingest_key(record: &str, id: &str) -> String {+ Uuid::new_v5(&key_namespace(), format!("{record}:{id}").as_bytes()).to_string()+}++/// The line that ends every imported page version. Its key is what a rerun+/// looks for in the page the server already holds.+pub(crate) fn provenance_line(record: &str, id: &str, version: Option<u64>, key: &str) -> String {+ let version = version.map_or_else(String::new, |v| format!(", version {v}"));+ format!(+ "*Imported from agentmemory ({record} `{id}`{version}). {}*",+ key_marker(key)+ )+}++fn key_marker(key: &str) -> String {+ format!("Import key: `{key}`.")+}++/// Validate `--dest-prefix`: relative, portable, no traversal, and outside+/// every reserved family. Returns the normalized prefix (no trailing slash).+pub(crate) fn validate_dest_prefix(prefix: &str) -> Result<String> {+ let trimmed = prefix.trim().trim_end_matches('/');+ if trimmed.is_empty() {+ bail!("--dest-prefix must not be empty");+ }+ if trimmed.starts_with('/') || trimmed.contains('\\') || trimmed.contains(':') {+ bail!("unsafe --dest-prefix {trimmed:?}: it must be a relative wiki path");+ }+ for segment in trimmed.split('/') {+ if segment.is_empty() || segment == "." || segment == ".." {+ bail!("unsafe --dest-prefix {trimmed:?}: empty, '.' or '..' segment");+ }+ if segment.starts_with('.')+ || !segment.bytes().all(|b| {+ b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'-' | b'_' | b'.')+ })+ {+ bail!(+ "unsafe --dest-prefix segment {segment:?}: use lowercase letters, digits, '-', '_' or '.'"+ );+ }+ }+ let first = trimmed.split('/').next().unwrap_or_default();+ if first.starts_with('_') || RESERVED_FIRST_SEGMENTS.contains(&first) {+ bail!("reserved --dest-prefix {first:?}: ai-memory owns that path family");+ }+ Ok(trimmed.to_owned())+}++/// Validate one planned destination path against the prefix it must stay+/// under, on top of the importer's general destination checks.+pub(crate) fn validate_planned_path(path: &str, prefix: &str) -> Result<()> {+ crate::validate_destination_path(path)?;+ let inside = path+ .strip_prefix(prefix)+ .is_some_and(|rest| rest.starts_with('/') && rest.len() > 1);+ if !inside+ || path+ .split('/')+ .any(|s| s.is_empty() || s == "." || s == "..")+ {+ bail!("planned path {path:?} escapes the destination prefix {prefix:?}");+ }+ Ok(())+}++/// Validate an agentmemory record id before it names a path or a key.+pub(crate) fn validate_id<'a>(id: &'a str, what: &str) -> Result<&'a str> {+ if id.trim().is_empty() || id.len() > MAX_ID_BYTES || id.chars().any(char::is_control) {+ bail!(+ "{what} id must be non-empty, at most {MAX_ID_BYTES} bytes and free of control characters"+ );+ }+ ai_memory_client::reject_sensitive(&serde_json::json!(id))+ .with_context(|| format!("{what} id looks like a credential; refusing it"))?;+ Ok(id)+}++/// ai-memory `kind` and `tier` for an agentmemory memory `type`.+pub(crate) fn memory_kind_tier(memory_type: &str) -> (&'static str, &'static str) {+ match memory_type {+ "preference" => ("preference", "semantic"),+ "architecture" => ("decision", "semantic"),+ "workflow" => ("workflow", "procedural"),+ "pattern" => ("pattern", "procedural"),+ "bug" => ("bug", "semantic"),+ "fact" => ("fact", "semantic"),+ _ => ("memory", "semantic"),+ }+}++/// The first `max` characters of a one-line rendering of `text`.+pub(crate) fn one_line(text: &str, max: usize) -> String {+ let flat: String = text.split_whitespace().collect::<Vec<_>>().join(" ");+ flat.chars().take(max).collect()+}++/// Sanitized, bounded, deduplicated tags, led by the import tag.+pub(crate) fn tags_from<'a>(raw: impl IntoIterator<Item = &'a str>) -> Vec<String> {+ let mut seen = BTreeSet::new();+ let mut tags = vec![IMPORT_TAG.to_owned()];+ seen.insert(IMPORT_TAG.to_owned());+ for tag in raw {+ let tag = one_line(+ &ai_memory_client::sanitize_external_text(tag),+ MAX_TAG_CHARS,+ );+ if tag.is_empty() || !seen.insert(tag.clone()) {+ continue;+ }+ if tags.len() == MAX_TAGS {+ break;+ }+ tags.push(tag);+ }+ tags+}++/// Rewrite `[[target]]` and `[[target|alias]]` wikilinks. A target `resolve`+/// knows becomes a link to its imported path; any other target becomes plain+/// text (the alias when there is one), so no link points at a page the+/// import did not create.+pub(crate) fn rewrite_wikilinks(text: &str, resolve: impl Fn(&str) -> Option<String>) -> String {+ let mut out = String::with_capacity(text.len());+ let mut rest = text;+ while let Some(start) = rest.find("[[") {+ out.push_str(&rest[..start]);+ let after = &rest[start + 2..];+ let Some(end) = after.find("]]") else {+ out.push_str(&rest[start..]);+ return out;+ };+ let inner = &after[..end];+ if inner.contains('\n') || inner.contains("[[") {+ out.push_str("[[");+ rest = after;+ continue;+ }+ let (target, alias) = match inner.split_once('|') {+ Some((target, alias)) => (target.trim(), Some(alias.trim())),+ None => (inner.trim(), None),+ };+ match resolve(target) {+ Some(path) => {+ out.push_str("[[");+ out.push_str(&path);+ if let Some(alias) = alias.filter(|a| !a.is_empty()) {+ out.push('|');+ out.push_str(alias);+ }+ out.push_str("]]");+ }+ None => out.push_str(alias.filter(|a| !a.is_empty()).unwrap_or(target)),+ }+ rest = &after[end + 2..];+ }+ out.push_str(rest);+ out+}++/// A wikilink target for an imported page: the path without `.md`.+pub(crate) fn link_target(path: &str) -> String {+ path.strip_suffix(".md").unwrap_or(path).to_owned()+}++/// Where a planned page goes.+#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Serialize)]+pub(crate) struct Destination {+ pub(crate) workspace: String,+ pub(crate) project: String,+}++/// One version of one page, already sanitized and carrying its provenance.+#[derive(Debug, Clone)]+pub(crate) struct PlannedVersion {+ pub(crate) key: String,+ pub(crate) source_id: String,+ pub(crate) version: Option<u64>,+ pub(crate) title: String,+ pub(crate) body: String,+ pub(crate) kind: String,+ pub(crate) tier: &'static str,+ pub(crate) tags: Vec<String>,+ pub(crate) expires_at: Option<String>,+}++/// One destination page and its versions, oldest first.+#[derive(Debug, Clone)]+pub(crate) struct PlannedPage {+ pub(crate) record: &'static str,+ pub(crate) destination: Destination,+ pub(crate) path: String,+ pub(crate) versions: Vec<PlannedVersion>,+}++#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]+#[serde(rename_all = "kebab-case")]+pub(crate) enum RunStatus {+ Planned,+ Imported,+ Failed,+}++#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]+#[serde(rename_all = "kebab-case")]+pub(crate) enum VersionStatus {+ Planned,+ /// Written by this run.+ Written,+ /// Already on the server, written by an earlier run of this import.+ Present,+ Failed,+}++#[derive(Debug, Clone, Serialize)]+pub(crate) struct VersionEntry {+ pub(crate) key: String,+ pub(crate) source_id: String,+ #[serde(skip_serializing_if = "Option::is_none")]+ pub(crate) version: Option<u64>,+ pub(crate) status: VersionStatus,+ #[serde(skip_serializing_if = "Option::is_none")]+ pub(crate) page_id: Option<String>,+ #[serde(skip_serializing_if = "Option::is_none")]+ pub(crate) error: Option<String>,+}++#[derive(Debug, Clone, Serialize)]+pub(crate) struct PageEntry {+ pub(crate) record: &'static str,+ pub(crate) workspace: String,+ pub(crate) project: String,+ pub(crate) path: String,+ pub(crate) versions: Vec<VersionEntry>,+}++#[derive(Debug, Clone, Serialize)]+pub(crate) struct HookSessionEntry {+ pub(crate) source_session_id: String,+ pub(crate) stable_session_id: String,+ pub(crate) workspace: String,+ pub(crate) project: String,+ pub(crate) planned_events: usize,+ pub(crate) accepted_events: usize,+ pub(crate) status: RunStatus,+}++/// The durable record of one agentmemory import: what was planned, what was+/// left out and why, and the state of every page version.+#[derive(Debug, Clone, Serialize)]+pub(crate) struct Manifest {+ pub(crate) import_version: &'static str,+ pub(crate) source_path: String,+ pub(crate) source_sha256: String,+ pub(crate) status: RunStatus,+ /// Planned records per kind (`memories`, `memory_versions`, ...).+ pub(crate) planned: BTreeMap<String, usize>,+ /// Records left out, per reason. Nothing is dropped silently.+ pub(crate) dropped: BTreeMap<String, usize>,+ /// Source projects the mapping file does not cover, with record counts.+ #[serde(skip_serializing_if = "BTreeMap::is_empty")]+ pub(crate) unmapped_projects: BTreeMap<String, usize>,+ pub(crate) written_versions: usize,+ pub(crate) present_versions: usize,+ pub(crate) pages: Vec<PageEntry>,+ #[serde(skip_serializing_if = "Vec::is_empty")]+ pub(crate) hook_sessions: Vec<HookSessionEntry>,+ #[serde(skip_serializing_if = "Option::is_none")]+ pub(crate) error: Option<String>,+}++impl Manifest {+ pub(crate) fn new(+ import_version: &'static str,+ source_path: String,+ source_sha256: String,+ pages: &[PlannedPage],+ ) -> Self {+ let mut planned = BTreeMap::new();+ for page in pages {+ *planned.entry(format!("{}_pages", page.record)).or_default() += 1;+ *planned+ .entry(format!("{}_versions", page.record))+ .or_default() += page.versions.len();+ }+ Self {+ import_version,+ source_path,+ source_sha256,+ status: RunStatus::Planned,+ planned,+ dropped: BTreeMap::new(),+ unmapped_projects: BTreeMap::new(),+ written_versions: 0,+ present_versions: 0,+ pages: pages+ .iter()+ .map(|page| PageEntry {+ record: page.record,+ workspace: page.destination.workspace.clone(),+ project: page.destination.project.clone(),+ path: page.path.clone(),+ versions: page+ .versions+ .iter()+ .map(|version| VersionEntry {+ key: version.key.clone(),+ source_id: version.source_id.clone(),+ version: version.version,+ status: VersionStatus::Planned,+ page_id: None,+ error: None,+ })+ .collect(),+ })+ .collect(),+ hook_sessions: Vec::new(),+ error: None,+ }+ }++ pub(crate) fn write(&self, path: &Path) -> Result<()> {+ crate::atomic_write(path, serde_json::to_string_pretty(self)?.as_bytes())+ .with_context(|| format!("write manifest {}", path.display()))+ }++ /// Record a failure and persist it; returns the error to propagate.+ pub(crate) fn fail(&mut self, path: &Path, error: anyhow::Error) -> anyhow::Error {+ self.status = RunStatus::Failed;+ self.error = Some(crate::mcp::preview(&format!("{error:#}")));+ if let Err(write_error) = self.write(path) {+ return error.context(format!("also failed to update the manifest: {write_error}"));+ }+ error+ }+}++/// Print the dry-run summary every agentmemory route shares.+pub(crate) fn print_plan(pages: &[PlannedPage], manifest: &Manifest, show_body: bool) {+ let versions: usize = pages.iter().map(|page| page.versions.len()).sum();+ println!(+ "dry-run: planned {} pages ({versions} page versions); no HTTP writes performed",+ pages.len()+ );+ for (key, count) in &manifest.planned {+ println!(" planned {key}: {count}");+ }+ for (key, count) in &manifest.dropped {+ println!(" dropped {key}: {count}");+ }+ for (project, count) in &manifest.unmapped_projects {+ println!(" UNMAPPED project {project:?}: {count} record(s); --apply refuses to run");+ }+ for page in pages {+ println!(+ " {} -> {}/{}:{} ({} version{})",+ page.versions+ .last()+ .map_or("", |version| version.source_id.as_str()),+ page.destination.workspace,+ page.destination.project,+ page.path,+ page.versions.len(),+ if page.versions.len() == 1 { "" } else { "s" }+ );+ if show_body {+ for version in &page.versions {+ println!("--- {} ---\n{}", version.key, version.body);+ }+ }+ }+}++/// Check the server and every destination before the first write.+pub(crate) async fn preflight(+ client: &ImportClient,+ destinations: &BTreeSet<Destination>,+ create_destination: bool,+) -> Result<()> {+ client.ensure_conditional_writes().await?;+ if create_destination {+ return Ok(());+ }+ for destination in destinations {+ if !client+ .mcp_scope_exists(&destination.workspace, &destination.project)+ .await+ {+ bail!(+ "destination {}/{} must already exist (or pass --create-destination)",+ destination.workspace,+ destination.project+ );+ }+ }+ Ok(())+}++/// Write every planned page, versions oldest first, each conditional on the+/// version before it. Stops at the first failure; the manifest records every+/// version's state, and a rerun resumes from what the server holds.+pub(crate) async fn apply_pages(+ client: &ImportClient,+ pages: &[PlannedPage],+ manifest: &mut Manifest,+ manifest_path: &Path,+) -> Result<()> {+ for (page_index, page) in pages.iter().enumerate() {+ if let Err(error) = apply_page(client, page, page_index, manifest, manifest_path).await {+ return Err(manifest.fail(manifest_path, error));+ }+ }+ manifest.status = RunStatus::Imported;+ manifest.write(manifest_path)+}++async fn apply_page(+ client: &ImportClient,+ page: &PlannedPage,+ page_index: usize,+ manifest: &mut Manifest,+ manifest_path: &Path,+) -> Result<()> {+ let mut latest: Option<String> = None;+ let mut index = 0;+ while index < page.versions.len() {+ let version = &page.versions[index];+ let write = McpWrite {+ workspace: &page.destination.workspace,+ project: &page.destination.project,+ path: &page.path,+ body: &version.body,+ title: Some(&version.title),+ kind: &version.kind,+ tier: version.tier,+ tags: &version.tags,+ expires_at: version.expires_at.as_deref(),+ };+ let precondition = latest+ .as_deref()+ .map_or(Precondition::Absent, Precondition::Latest);+ match client.mcp_write_page(&write, precondition).await {+ Ok(page_id) => {+ let entry = &mut manifest.pages[page_index].versions[index];+ entry.status = VersionStatus::Written;+ entry.page_id = Some(page_id.clone());+ manifest.written_versions += 1;+ manifest.write(manifest_path)?;+ latest = Some(page_id);+ index += 1;+ }+ Err(error) if error.downcast_ref::<PreconditionFailed>().is_some() => {+ // The page exists or moved on. Resume only from a version+ // this import wrote; anything else is somebody else's page.+ let current = client+ .mcp_read_page(+ &page.destination.workspace,+ &page.destination.project,+ &page.path,+ )+ .await+ .with_context(|| format!("reading {} after a refused write", page.path))?;+ let Some(found) = page+ .versions+ .iter()+ .rposition(|v| current.body.contains(&key_marker(&v.key)))+ .filter(|found| *found >= index)+ else {+ let entry = &mut manifest.pages[page_index].versions[index];+ entry.status = VersionStatus::Failed;+ entry.error = Some("destination page was not written by this import".into());+ bail!(+ "{} already holds a page this import did not write; refusing to replace it",+ page.path+ );+ };+ for skipped in index..=found {+ let entry = &mut manifest.pages[page_index].versions[skipped];+ entry.status = VersionStatus::Present;+ if skipped == found {+ entry.page_id = current.page_id.clone();+ }+ manifest.present_versions += 1;+ }+ manifest.write(manifest_path)?;+ latest = current.page_id;+ index = found + 1;+ if latest.is_none() && index < page.versions.len() {+ bail!("{} has no version id; cannot continue its chain", page.path);+ }+ }+ Err(error) => {+ let entry = &mut manifest.pages[page_index].versions[index];+ entry.status = VersionStatus::Failed;+ entry.error = Some(crate::mcp::preview(&format!("{error:#}")));+ return Err(error.context(format!("writing {}", page.path)));+ }+ }+ }+ Ok(())+}++/// Print the closing line of a live run.+pub(crate) fn print_done(manifest: &Manifest) {+ println!(+ "import complete: wrote {} page version(s); {} were already present",+ manifest.written_versions, manifest.present_versions+ );+}++#[cfg(test)]+mod e2e;+#[cfg(test)]+pub(crate) mod mock;++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn dest_prefix_refuses_traversal_absolute_and_reserved_families() {+ assert_eq!(+ validate_dest_prefix("imported/agentmemory/").unwrap(),+ "imported/agentmemory"+ );+ assert_eq!(+ validate_dest_prefix("archive/am-2026").unwrap(),+ "archive/am-2026"+ );+ for bad in [+ "",+ "/abs/agentmemory",+ "../escape",+ "imported/../sessions",+ "imported/./x",+ "imported//x",+ "C:\\vault",+ "c:/vault",+ "sessions",+ "sessions/agentmemory",+ "_rules",+ "_prompts/x",+ "_global",+ ".git",+ "decisions/am",+ "gotchas",+ "procedures",+ "profile",+ "Imported/Upper",+ "imported/space here",+ "imported/.hidden",+ ] {+ assert!(+ validate_dest_prefix(bad).is_err(),+ "{bad:?} must be refused"+ );+ }+ }++ #[test]+ fn planned_paths_must_stay_under_the_prefix() {+ let prefix = "imported/agentmemory";+ validate_planned_path("imported/agentmemory/memories/a.md", prefix).unwrap();+ for bad in [+ "imported/agentmemory.md",+ "imported/agentmemory/../x.md",+ "imported/agentmemoryx/a.md",+ "sessions/a.md",+ "/imported/agentmemory/a.md",+ "imported/agentmemory/a",+ ] {+ assert!(+ validate_planned_path(bad, prefix).is_err(),+ "{bad:?} must be refused"+ );+ }+ }++ #[test]+ fn ingest_keys_are_stable_uuid_v5_and_distinct_per_record() {+ let key = ingest_key("memory", "mem_1");+ assert_eq!(key, ingest_key("memory", "mem_1"));+ let parsed = Uuid::parse_str(&key).unwrap();+ assert_eq!(parsed.get_version_num(), 5);+ assert_ne!(key, ingest_key("memory", "mem_2"));+ assert_ne!(key, ingest_key("lesson", "mem_1"));+ }++ #[test]+ fn ids_that_look_like_credentials_or_carry_controls_are_refused() {+ validate_id("mem_01HX", "memory").unwrap();+ assert!(validate_id("", "memory").is_err());+ assert!(validate_id("a\nb", "memory").is_err());+ assert!(validate_id(&"x".repeat(MAX_ID_BYTES + 1), "memory").is_err());+ assert!(validate_id("sk-1234567890abcdefghijklmnop", "memory").is_err());+ }++ #[test]+ fn wikilinks_resolve_or_fall_back_to_plain_text() {+ let resolve = |target: &str| {+ (target == "mem_a" || target == "memories/mem_a")+ .then(|| "imported/agentmemory/memories/mem-a".to_owned())+ };+ assert_eq!(+ rewrite_wikilinks("see [[mem_a]] and [[memories/mem_a|A title]]", resolve),+ "see [[imported/agentmemory/memories/mem-a]] and [[imported/agentmemory/memories/mem-a|A title]]"+ );+ assert_eq!(+ rewrite_wikilinks(+ "gone: [[mem_zz]], alias [[x|Shown]], evil [[../../_rules/x]]",+ resolve+ ),+ "gone: mem_zz, alias Shown, evil ../../_rules/x"+ );+ assert_eq!(+ rewrite_wikilinks("open [[ never closed", resolve),+ "open [[ never closed"+ );+ }++ #[test]+ fn tags_are_bounded_deduplicated_and_led_by_the_import_tag() {+ let raw: Vec<String> = (0..40).map(|i| format!("concept {}", i % 20)).collect();+ let tags = tags_from(raw.iter().map(String::as_str));+ assert_eq!(tags[0], IMPORT_TAG);+ assert_eq!(tags.len(), MAX_TAGS);+ let unique: BTreeSet<_> = tags.iter().collect();+ assert_eq!(unique.len(), tags.len());+ }++ #[test]+ fn memory_types_map_to_kind_and_tier() {+ assert_eq!(memory_kind_tier("architecture"), ("decision", "semantic"));+ assert_eq!(memory_kind_tier("workflow"), ("workflow", "procedural"));+ assert_eq!(memory_kind_tier("pattern"), ("pattern", "procedural"));+ assert_eq!(memory_kind_tier("something-new"), ("memory", "semantic"));+ }+}diff --git a/companions/ai-memory-importer/src/agentmemory/vault.rs b/companions/ai-memory-importer/src/agentmemory/vault.rsnew file mode 100644index 00000000..5464b814--- /dev/null+++ b/companions/ai-memory-importer/src/agentmemory/vault.rs@@ -0,0 +1,704 @@+//! `agentmemory-vault <dir>`: import the Obsidian vault agentmemory's+//! `obsidian-export` writes (`memories/`, `lessons/`, `crystals/`,+//! `sessions/`, each page with YAML frontmatter and `[[id]]` wikilinks).+//!+//! Pages land under `<dest-prefix>/{memories,lessons,crystals,sessions}/`,+//! never under ai-memory's own `sessions/`. A wikilink to another imported+//! page is rewritten to its new path; a link to anything else becomes plain+//! text. `MOC.md`, the vault's generated index, is not imported.++use std::collections::{BTreeMap, HashMap};+use std::fs;+use std::path::{Path, PathBuf};++use anyhow::{Context, Result, anyhow, bail};+use clap::Parser;+use walkdir::WalkDir;++use super::{+ DEFAULT_DEST_PREFIX, Destination, MAX_EXPLICIT_TITLE_CHARS, Manifest, PlannedPage,+ PlannedVersion, apply_pages, ingest_key, link_target, memory_kind_tier, one_line, preflight,+ print_done, print_plan, provenance_line, rewrite_wikilinks, tags_from, validate_dest_prefix,+ validate_id, validate_planned_path,+};+use crate::{+ DEFAULT_SERVER_URL, ImportClient, YamlMap, sha256_hex, sha256_parts, slugify,+ split_frontmatter, validate_label, validate_source_rel, yaml_string,+};++const IMPORT_VERSION: &str = "agentmemory-vault-v1";+/// Bound on one vault page.+const MAX_FILE_BYTES: u64 = 1024 * 1024;+/// Bound on the number of pages one vault import plans.+const MAX_FILES: usize = 100_000;++/// The four folders the export writes, with the record name each holds.+const FOLDERS: &[(&str, &str)] = &[+ ("memories", "memory"),+ ("lessons", "lesson"),+ ("crystals", "crystal"),+ ("sessions", "session"),+];++#[derive(Parser, Debug, Clone)]+pub(crate) struct VaultArgs {+ /// The vault directory agentmemory's Obsidian export wrote.+ #[arg(long)]+ pub(crate) dir: PathBuf,+ #[arg(long)]+ pub(crate) workspace: Option<String>,+ #[arg(long)]+ pub(crate) project: Option<String>,+ /// Wiki path prefix every imported page lands under.+ #[arg(long, default_value = DEFAULT_DEST_PREFIX)]+ pub(crate) dest_prefix: String,+ #[arg(long, env = "AI_MEMORY_SERVER_URL", default_value = DEFAULT_SERVER_URL)]+ pub(crate) server_url: String,+ /// Write the pages. Without this flag the command only prints a plan.+ #[arg(long)]+ pub(crate) apply: bool,+ /// Durable import manifest. Required with --apply.+ #[arg(long)]+ pub(crate) manifest_out: Option<PathBuf>,+ /// Let the first write create a missing destination workspace/project.+ #[arg(long)]+ pub(crate) create_destination: bool,+ /// Print every planned page body during the dry run.+ #[arg(long)]+ pub(crate) show_body: bool,+}++/// One vault page, parsed but not yet linked.+struct VaultPage {+ folder: &'static str,+ record: &'static str,+ rel: String,+ id: String,+ path: String,+ frontmatter: YamlMap,+ body: String,+}++pub(crate) async fn run(args: VaultArgs) -> Result<()> {+ if args.apply {+ if args+ .workspace+ .as_deref()+ .is_none_or(|s| s.trim().is_empty())+ || args.project.as_deref().is_none_or(|s| s.trim().is_empty())+ {+ bail!("--apply requires explicit --workspace and --project");+ }+ if args.manifest_out.is_none() {+ bail!("--apply requires --manifest-out <path>");+ }+ }+ let destination = Destination {+ workspace: validate_label(+ args.workspace.as_deref().unwrap_or("DRY-RUN-WORKSPACE"),+ "workspace",+ 128,+ )?+ .to_owned(),+ project: validate_label(+ args.project.as_deref().unwrap_or("DRY-RUN-PROJECT"),+ "project",+ 128,+ )?+ .to_owned(),+ };+ let prefix = validate_dest_prefix(&args.dest_prefix)?;+ let plan = plan(&args.dir, &destination, &prefix)?;+ let mut manifest = Manifest::new(+ IMPORT_VERSION,+ args.dir.to_string_lossy().into_owned(),+ plan.source_sha256,+ &plan.pages,+ );+ manifest.dropped = plan.dropped;++ if !args.apply {+ print_plan(&plan.pages, &manifest, args.show_body);+ if let Some(path) = &args.manifest_out {+ manifest.write(path)?;+ }+ return Ok(());+ }+ let manifest_path = args+ .manifest_out+ .as_deref()+ .ok_or_else(|| anyhow!("--apply requires --manifest-out <path>"))?;+ manifest.write(manifest_path)?;+ let client = ImportClient::new(&args.server_url)?;+ let destinations = plan.pages.iter().map(|p| p.destination.clone()).collect();+ if let Err(error) = preflight(&client, &destinations, args.create_destination).await {+ return Err(manifest.fail(manifest_path, error));+ }+ apply_pages(&client, &plan.pages, &mut manifest, manifest_path).await?;+ print_done(&manifest);+ Ok(())+}++#[derive(Debug)]+pub(crate) struct VaultPlan {+ pub(crate) pages: Vec<PlannedPage>,+ pub(crate) dropped: BTreeMap<String, usize>,+ pub(crate) source_sha256: String,+}++/// Read, validate and plan a whole vault. No network, no writes.+pub(crate) fn plan(dir: &Path, destination: &Destination, prefix: &str) -> Result<VaultPlan> {+ let root =+ fs::canonicalize(dir).with_context(|| format!("read vault dir {}", dir.display()))?;+ if !root.is_dir() {+ bail!("--dir must be a directory");+ }+ let mut dropped: BTreeMap<String, usize> = BTreeMap::new();+ count_unexported(&root, &mut dropped)?;++ let mut pages = Vec::new();+ let mut fingerprints = Vec::new();+ let mut destinations: HashMap<String, String> = HashMap::new();+ for &(folder, record) in FOLDERS {+ let folder_root = root.join(folder);+ let Ok(meta) = fs::symlink_metadata(&folder_root) else {+ continue;+ };+ if meta.file_type().is_symlink() {+ bail!("vault folder {folder}/ is a symlink; refusing to follow it");+ }+ if !meta.is_dir() {+ bail!("vault entry {folder} is not a directory");+ }+ let walker = WalkDir::new(&folder_root)+ .follow_links(false)+ .sort_by_file_name()+ .min_depth(1);+ for entry in walker {+ let entry = entry?;+ let name = entry.file_name().to_string_lossy().into_owned();+ if entry.file_type().is_symlink() {+ bail!(+ "vault entry {} is a symlink; refusing to follow it",+ entry.path().display()+ );+ }+ if entry.file_type().is_dir() {+ continue;+ }+ let rel = entry+ .path()+ .strip_prefix(&root)?+ .to_string_lossy()+ .replace('\\', "/");+ validate_source_rel(&rel)?;+ if rel.split('/').any(|segment| segment.starts_with('.')) {+ *dropped.entry("hidden entries".into()).or_default() += 1;+ continue;+ }+ if !name.ends_with(".md") {+ *dropped.entry("non-markdown files".into()).or_default() += 1;+ continue;+ }+ if entry.metadata()?.len() > MAX_FILE_BYTES {+ bail!("vault page {rel} is larger than {MAX_FILE_BYTES} bytes");+ }+ if pages.len() == MAX_FILES {+ bail!("vault has more than {MAX_FILES} pages");+ }+ let content =+ fs::read_to_string(entry.path()).with_context(|| format!("read {rel}"))?;+ fingerprints.push((rel.clone(), sha256_hex(content.as_bytes())));+ let (frontmatter, body) =+ split_frontmatter(&content).with_context(|| format!("parse {rel}"))?;+ let frontmatter = frontmatter.unwrap_or_default();+ let stem = name.trim_end_matches(".md");+ let id = yaml_string(&frontmatter, "id").unwrap_or_else(|| stem.to_owned());+ validate_id(&id, record).with_context(|| format!("in {rel}"))?;++ let inner = rel+ .strip_prefix(folder)+ .and_then(|r| r.strip_prefix('/'))+ .unwrap_or(&rel);+ let mut segments: Vec<String> = inner.split('/').map(str::to_owned).collect();+ let file = segments.pop().unwrap_or_default();+ let mut path = format!("{prefix}/{folder}");+ for segment in &segments {+ path.push('/');+ path.push_str(&slugify(segment));+ }+ path.push('/');+ path.push_str(&slugify(file.trim_end_matches(".md")));+ path.push_str(".md");+ validate_planned_path(&path, prefix)?;+ if let Some(other) = destinations.insert(path.clone(), rel.clone()) {+ bail!("duplicate destination path collision: {other} and {rel} both map to {path}");+ }+ pages.push(VaultPage {+ folder,+ record,+ rel,+ id,+ path,+ frontmatter,+ body,+ });+ }+ }++ // Every way the export spells a link to a page: its id, its file stem,+ // and the MOC's `<folder>/<stem>` form.+ let mut links: HashMap<String, String> = HashMap::new();+ for page in &pages {+ let target = link_target(&page.path);+ let stem = page+ .rel+ .rsplit('/')+ .next()+ .unwrap_or_default()+ .trim_end_matches(".md")+ .to_owned();+ let inner = page+ .rel+ .strip_prefix(page.folder)+ .and_then(|r| r.strip_prefix('/'))+ .unwrap_or(&page.rel)+ .trim_end_matches(".md")+ .to_owned();+ for alias in [+ page.id.clone(),+ stem.clone(),+ inner.clone(),+ format!("{}/{stem}", page.folder),+ format!("{}/{inner}", page.folder),+ format!("{}/{}", page.folder, page.id),+ ] {+ links.entry(alias).or_insert_with(|| target.clone());+ }+ }++ let mut planned = Vec::with_capacity(pages.len());+ for page in pages {+ let linked = rewrite_wikilinks(&page.body, |target| {+ links.get(target.trim_end_matches(".md")).cloned()+ });+ let body = ai_memory_client::sanitize_external_text(&linked);+ let title = body+ .lines()+ .find_map(|line| line.strip_prefix("# "))+ .map(|title| one_line(title, MAX_EXPLICIT_TITLE_CHARS))+ .filter(|title| !title.is_empty())+ .unwrap_or_else(|| one_line(&page.id, MAX_EXPLICIT_TITLE_CHARS));+ let (kind, tier, tags) = match page.record {+ "memory" => {+ let memory_type = yaml_string(&page.frontmatter, "type").unwrap_or_default();+ let (kind, tier) = memory_kind_tier(&memory_type);+ (kind, tier, string_list(&page.frontmatter, "concepts"))+ }+ "lesson" => ("lesson", "semantic", string_list(&page.frontmatter, "tags")),+ "crystal" => ("crystal", "episodic", Vec::new()),+ _ => ("session", "episodic", Vec::new()),+ };+ let version = page+ .frontmatter+ .get("version")+ .and_then(serde_json::Value::as_u64);+ let key = ingest_key(&format!("vault-{}", page.record), &page.id);+ let body = format!(+ "{}\n\n---\n{}\n",+ body.trim_end(),+ provenance_line(page.record, &page.id, version, &key)+ );+ planned.push(PlannedPage {+ record: page.record,+ destination: destination.clone(),+ path: page.path,+ versions: vec![PlannedVersion {+ key,+ source_id: page.id,+ version,+ title: ai_memory_client::sanitize_external_text(&title),+ body,+ kind: kind.to_owned(),+ tier,+ tags: tags_from(tags.iter().map(String::as_str)),+ expires_at: None,+ }],+ });+ }++ fingerprints.sort();+ let mut parts: Vec<&[u8]> = vec![IMPORT_VERSION.as_bytes()];+ for (rel, sha) in &fingerprints {+ parts.push(rel.as_bytes());+ parts.push(sha.as_bytes());+ }+ Ok(VaultPlan {+ pages: planned,+ dropped,+ source_sha256: sha256_parts(&parts),+ })+}++/// Count what the vault holds outside the four exported folders.+fn count_unexported(root: &Path, dropped: &mut BTreeMap<String, usize>) -> Result<()> {+ for entry in fs::read_dir(root)? {+ let entry = entry?;+ let name = entry.file_name().to_string_lossy().into_owned();+ if FOLDERS.iter().any(|(folder, _)| *folder == name) {+ continue;+ }+ let reason = if name == "MOC.md" {+ "index page (MOC.md)"+ } else if name.starts_with('.') {+ "hidden entries"+ } else {+ "entries outside memories/lessons/crystals/sessions"+ };+ *dropped.entry(reason.into()).or_default() += 1;+ }+ Ok(())+}++fn string_list(map: &YamlMap, key: &str) -> Vec<String> {+ match map.get(key) {+ Some(serde_json::Value::Array(items)) => items+ .iter()+ .filter_map(|item| item.as_str().map(str::to_owned))+ .collect(),+ Some(serde_json::Value::String(item)) => vec![item.clone()],+ _ => Vec::new(),+ }+}++#[cfg(test)]+pub(crate) mod tests {+ use super::*;+ use crate::agentmemory::mock::{Failure, MockServer};+ use tempfile::tempdir;++ fn put(root: &Path, rel: &str, body: &str) {+ let path = root.join(rel);+ fs::create_dir_all(path.parent().unwrap()).unwrap();+ fs::write(path, body).unwrap();+ }++ /// A synthetic vault in the shape `obsidian-export` writes: JSON-valued+ /// frontmatter, an H1, `[[id]]` links, and a generated `MOC.md`.+ pub(crate) fn fixture_vault(root: &Path) {+ put(+ root,+ "memories/mem_alpha.md",+ "---\nid: \"mem_alpha\"\ntype: \"architecture\"\ncreated: \"2026-01-02T03:04:05.000Z\"\nupdated: \"2026-01-03T03:04:05.000Z\"\nstrength: 0.9\nversion: 2\nconcepts: [\"writer actor\", \"sqlite\"]\nfiles: [\"src/store.rs\"]\n---\n\n# Single writer owns the store\n\nEvery mutation goes through one writer thread.\n\n## Concepts\n#writer-actor #sqlite\n\n## Related\n- [[mem_beta]]\n- [[mem_gone]]",+ );+ put(+ root,+ "memories/mem_beta.md",+ "---\nid: \"mem_beta\"\ntype: \"bug\"\ncreated: \"2026-01-04T00:00:00.000Z\"\nupdated: \"2026-01-04T00:00:00.000Z\"\nstrength: 0.5\nversion: 1\nconcepts: []\nfiles: []\n---\n\n# Retry storms on reconnect\n\nBack off exponentially; see [[lesson_one]].",+ );+ put(+ root,+ "lessons/lesson_one.md",+ "---\nid: \"lesson_one\"\ntype: \"lesson\"\nsource: \"manual\"\nconfidence: 0.8\nreinforcements: 3\ncreated: \"2026-01-05T00:00:00.000Z\"\nupdated: \"2026-01-05T00:00:00.000Z\"\nproject: \"demo\"\ntags: [\"networking\"]\ndecayRate: 0.01\n---\n\n# Lesson: Always cap retries\n\nAlways cap retries.\n\n## Sources\n- [[crys_one]]",+ );+ put(+ root,+ "crystals/crys_one.md",+ "---\nid: \"crys_one\"\ntype: \"crystal\"\ncreated: \"2026-01-06T00:00:00.000Z\"\nproject: \"demo\"\nsessionId: \"ses_one\"\nfilesAffected: [\"src/net.rs\"]\n---\n\n# Crystal: Reconnect work\n\nReconnect work.\n\n## Key Outcomes\n- capped retries\n\n## Source Actions\n- [[act_dropped]]",+ );+ put(+ root,+ "sessions/ses_one.md",+ "---\nid: \"ses_one\"\ntype: \"session\"\nproject: \"demo\"\nstatus: \"completed\"\nstarted: \"2026-01-06T00:00:00.000Z\"\nended: \"2026-01-06T01:00:00.000Z\"\nobservations: 4\n---\n\n# Session: demo\n\n**Status:** completed\n**Observations:** 4\n**CWD:** `/work/demo`",+ );+ put(+ root,+ "MOC.md",+ "---\ntype: moc\n---\n\n# agentmemory vault\n\n- [[memories/mem_alpha|Single writer owns the store]]",+ );+ put(root, "memories/attachment.png", "not markdown");+ }++ fn dest() -> Destination {+ Destination {+ workspace: "w".into(),+ project: "p".into(),+ }+ }++ #[test]+ fn plan_counts_equal_the_fixture_and_paths_stay_under_the_prefix() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let plan = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap();+ let manifest = Manifest::new(IMPORT_VERSION, String::new(), String::new(), &plan.pages);+ assert_eq!(manifest.planned["memory_pages"], 2);+ assert_eq!(manifest.planned["lesson_pages"], 1);+ assert_eq!(manifest.planned["crystal_pages"], 1);+ assert_eq!(manifest.planned["session_pages"], 1);+ assert_eq!(plan.dropped["index page (MOC.md)"], 1);+ assert_eq!(plan.dropped["non-markdown files"], 1);+ for page in &plan.pages {+ assert!(+ page.path.starts_with("imported/agentmemory/"),+ "{}",+ page.path+ );+ assert!(!page.path.starts_with("sessions/"));+ }+ let session = plan.pages.iter().find(|p| p.record == "session").unwrap();+ assert_eq!(session.path, "imported/agentmemory/sessions/ses-one.md");+ }++ #[test]+ fn links_point_at_imported_paths_or_become_plain_text() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let plan = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap();+ let alpha = plan+ .pages+ .iter()+ .find(|p| p.path.ends_with("mem-alpha.md"))+ .unwrap();+ let body = &alpha.versions[0].body;+ assert!(+ body.contains("[[imported/agentmemory/memories/mem-beta]]"),+ "{body}"+ );+ assert!(body.contains("- mem_gone"), "{body}");+ assert!(!body.contains("[[mem_gone]]"), "{body}");+ let crystal = plan.pages.iter().find(|p| p.record == "crystal").unwrap();+ assert!(crystal.versions[0].body.contains("- act_dropped"));+ assert_eq!(alpha.versions[0].kind, "decision");+ assert_eq!(alpha.versions[0].title, "Single writer owns the store");+ assert_eq!(+ alpha.versions[0].tags,+ ["agentmemory-import", "writer actor", "sqlite"]+ );+ }++ #[test]+ fn hostile_vaults_are_refused_before_any_request() {+ // A reserved, absolute or traversing prefix.+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ for prefix in ["sessions", "/abs", "../up", "_rules/x"] {+ assert!(validate_dest_prefix(prefix).is_err(), "{prefix}");+ }+ // Two files whose slugs collide.+ put(td.path(), "memories/Mem Alpha.md", "# Dup");+ let error = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap_err();+ assert!(error.to_string().contains("collision"), "{error}");++ // A credential-shaped id in frontmatter.+ let td = tempdir().unwrap();+ put(+ td.path(),+ "memories/m.md",+ "---\nid: \"ghp_1234567890abcdefghijklmnopqrstuvwxyzAB\"\n---\n# M",+ );+ assert!(plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).is_err());++ // A frontmatter id with traversal never names a path: the file name+ // does, slugged.+ let td = tempdir().unwrap();+ put(+ td.path(),+ "memories/m.md",+ "---\nid: \"../../_rules/evil\"\n---\n# M",+ );+ let planned = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap();+ assert_eq!(planned.pages[0].path, "imported/agentmemory/memories/m.md");+ }++ #[cfg(unix)]+ #[test]+ fn symlinks_inside_the_vault_are_refused() {+ let td = tempdir().unwrap();+ let outside = tempdir().unwrap();+ fs::write(outside.path().join("secret.md"), "# outside").unwrap();+ fs::create_dir_all(td.path().join("memories")).unwrap();+ std::os::unix::fs::symlink(+ outside.path().join("secret.md"),+ td.path().join("memories/link.md"),+ )+ .unwrap();+ let error = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap_err();+ assert!(error.to_string().contains("symlink"), "{error}");++ let td = tempdir().unwrap();+ std::os::unix::fs::symlink(outside.path(), td.path().join("lessons")).unwrap();+ let error = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap_err();+ assert!(error.to_string().contains("symlink"), "{error}");+ }++ #[test]+ fn page_bodies_are_redacted_before_planning() {+ let td = tempdir().unwrap();+ put(+ td.path(),+ "memories/m.md",+ "---\nid: \"m\"\n---\n# M\n\ntoken=github_pat_1234567890abcdefghijklmnop",+ );+ let planned = plan(td.path(), &dest(), DEFAULT_DEST_PREFIX).unwrap();+ let body = &planned.pages[0].versions[0].body;+ assert!(+ !body.contains("github_pat_1234567890abcdefghijklmnop"),+ "{body}"+ );+ assert!(body.contains("REDACTED"), "{body}");+ }++ fn args(dir: &Path, url: &str, manifest: &Path) -> VaultArgs {+ VaultArgs {+ dir: dir.to_path_buf(),+ workspace: Some("w".into()),+ project: Some("p".into()),+ dest_prefix: DEFAULT_DEST_PREFIX.into(),+ server_url: url.into(),+ apply: true,+ manifest_out: Some(manifest.to_path_buf()),+ create_destination: false,+ show_body: false,+ }+ }++ fn read_manifest(path: &Path) -> serde_json::Value {+ serde_json::from_slice(&fs::read(path).unwrap()).unwrap()+ }++ #[tokio::test]+ async fn apply_writes_create_only_and_a_rerun_writes_nothing() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let server = MockServer::start(&[("w", "p")]);+ let manifest = td.path().join("manifest.json");+ run(args(td.path(), &server.url, &manifest)).await.unwrap();+ assert_eq!(server.state().stored_versions(), 5);+ assert!(+ server+ .state()+ .pages+ .values()+ .flatten()+ .all(|v| v.arguments["create_only"] == true)+ );+ let first = read_manifest(&manifest);+ assert_eq!(first["status"], "imported");+ assert_eq!(first["written_versions"], 5);++ run(args(td.path(), &server.url, &manifest)).await.unwrap();+ assert_eq!(server.state().stored_versions(), 5, "a rerun writes 0");+ let second = read_manifest(&manifest);+ assert_eq!(second["written_versions"], 0);+ assert_eq!(second["present_versions"], 5);+ }++ #[tokio::test]+ async fn a_foreign_page_at_the_destination_is_never_replaced() {+ let td = tempdir().unwrap();+ put(td.path(), "memories/m.md", "---\nid: \"m\"\n---\n# M");+ let server = MockServer::start(&[("w", "p")]);+ server.state().pages.insert(+ (+ "w".into(),+ "p".into(),+ "imported/agentmemory/memories/m.md".into(),+ ),+ vec![crate::agentmemory::mock::StoredVersion {+ page_id: "someone-else".into(),+ body: "# Hand written".into(),+ arguments: serde_json::Value::Null,+ }],+ );+ let manifest = td.path().join("manifest.json");+ let error = run(args(td.path(), &server.url, &manifest))+ .await+ .unwrap_err();+ assert!(error.to_string().contains("did not write"), "{error:#}");+ let chain = server+ .state()+ .chain("w", "p", "imported/agentmemory/memories/m.md");+ assert_eq!(chain.len(), 1);+ assert_eq!(chain[0].page_id, "someone-else");+ assert_eq!(read_manifest(&manifest)["status"], "failed");+ }++ #[tokio::test]+ async fn a_server_older_than_2_7_is_refused_before_any_write() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let server = MockServer::start(&[("w", "p")]);+ server.state().conditional = false;+ let manifest = td.path().join("manifest.json");+ let error = run(args(td.path(), &server.url, &manifest))+ .await+ .unwrap_err();+ assert!(error.to_string().contains(">= 2.7"), "{error:#}");+ assert_eq!(server.state().write_calls, 0);+ }++ #[tokio::test]+ async fn a_missing_destination_is_refused_without_create_destination() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let server = MockServer::start(&[]);+ let manifest = td.path().join("manifest.json");+ let error = run(args(td.path(), &server.url, &manifest))+ .await+ .unwrap_err();+ assert!(+ error.to_string().contains("must already exist"),+ "{error:#}"+ );+ assert_eq!(server.state().write_calls, 0);++ let mut create = args(td.path(), &server.url, &manifest);+ create.create_destination = true;+ run(create).await.unwrap();+ assert_eq!(server.state().stored_versions(), 5);+ }++ #[tokio::test]+ async fn an_interrupted_import_resumes_without_duplicates() {+ for failure in [Failure::Before(3), Failure::After(3)] {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let server = MockServer::start(&[("w", "p")]);+ server.state().failure = Some(failure);+ let manifest = td.path().join("manifest.json");+ assert!(run(args(td.path(), &server.url, &manifest)).await.is_err());+ let failed = read_manifest(&manifest);+ assert_eq!(failed["status"], "failed");+ server.state().failure = None;+ run(args(td.path(), &server.url, &manifest)).await.unwrap();+ let state = server.state();+ assert_eq!(state.stored_versions(), 5, "{failure:?}");+ assert!(+ state.pages.values().all(|chain| chain.len() == 1),+ "{failure:?}"+ );+ }+ }++ #[tokio::test]+ async fn dry_run_makes_no_request() {+ let td = tempdir().unwrap();+ fixture_vault(td.path());+ let server = MockServer::start(&[("w", "p")]);+ let manifest = td.path().join("manifest.json");+ let mut dry = args(td.path(), &server.url, &manifest);+ dry.apply = false;+ run(dry).await.unwrap();+ assert!(server.state().requests.is_empty());+ let planned = read_manifest(&manifest);+ assert_eq!(planned["status"], "planned");+ assert!(+ planned["pages"]+ .as_array()+ .unwrap()+ .iter()+ .all(|page| page["versions"][0]["status"] == "planned")+ );+ }+}diff --git a/companions/ai-memory-importer/src/main.rs b/companions/ai-memory-importer/src/main.rsindex c3c7a2d8..24573696 100644--- a/companions/ai-memory-importer/src/main.rs+++ b/companions/ai-memory-importer/src/main.rs@@ -12,6 +12,11 @@ use sha2::{Digest, Sha256};use url::Url;use walkdir::WalkDir;+mod agentmemory;+mod mcp;+#[cfg(test)]+mod test_http;+const IMPORT_VERSION: &str = "omc-wiki-v1";const CONVERSATION_IMPORT_VERSION: &str = "external-conversation-v1";const DEFAULT_SERVER_URL: &str = "http://127.0.0.1:49374";@@ -36,6 +41,10 @@ enum Commands {OmcWiki(OmcArgs),/// Replay a generic external conversation through ai-memory's hook API.ExternalConversation(ConversationArgs),+ /// Import the Obsidian vault agentmemory's `obsidian-export` writes.+ AgentmemoryVault(agentmemory::vault::VaultArgs),+ /// Import an agentmemory `ExportData` JSON file.+ AgentmemoryExport(agentmemory::export::ExportArgs),}#[derive(Parser, Debug, Clone)]@@ -297,6 +306,8 @@ async fn main() -> Result<()> {match Cli::parse().command {Commands::OmcWiki(args) => run_omc(args).await,Commands::ExternalConversation(args) => run_conversation(args).await,+ Commands::AgentmemoryVault(args) => agentmemory::vault::run(args).await,+ Commands::AgentmemoryExport(args) => agentmemory::export::run(args).await,}}@@ -1058,13 +1069,31 @@ struct ParsedMarkdown {pinned: bool,}-/// Split an OMC page into frontmatter fields and body. The fence is found-/// the way `ai_memory_wiki::markdown::parse` finds it: past a leading UTF-8-/// BOM, and with either LF or CRLF fence lines, since a wiki checked out on-/// Windows with `core.autocrlf=true` has CRLF throughout. Missing it dropped-/// kind, tier, tags and pin, and imported the YAML block as body text.+/// Split an OMC page into frontmatter fields and body.fn parse_markdown(input: &str) -> Result<ParsedMarkdown> {let mut out = ParsedMarkdown::default();+ let (frontmatter, body) = split_frontmatter(input)?;+ if let Some(map) = frontmatter.as_ref() {+ out.title = yaml_string(map, "title");+ out.kind = yaml_string(map, "kind");+ out.tier = yaml_string(map, "tier");+ out.pinned = yaml_bool(map, "pinned").unwrap_or(false);+ out.tags = yaml_tags(map);+ }+ if out.title.is_none() {+ out.title = first_h1(&body);+ }+ out.body = body;+ Ok(out)+}++/// Split a Markdown page into its YAML frontmatter mapping (when the block+/// is a mapping) and its body. The fence is found the way+/// `ai_memory_wiki::markdown::parse` finds it: past a leading UTF-8 BOM, and+/// with either LF or CRLF fence lines, since a wiki checked out on Windows+/// with `core.autocrlf=true` has CRLF throughout. Missing it dropped kind,+/// tier, tags and pin, and imported the YAML block as body text.+fn split_frontmatter(input: &str) -> Result<(Option<YamlMap>, String)> {let input = input.strip_prefix('\u{FEFF}').unwrap_or(input);let (rest, newline) = if let Some(rest) = input.strip_prefix("---\r\n") {(Some(rest), "\r\n")@@ -1072,31 +1101,21 @@ fn parse_markdown(input: &str) -> Result<ParsedMarkdown> {(input.strip_prefix("---\n"), "\n")};let close = format!("\n---{newline}");- let body = if let Some(rest) = rest {- if let Some(end) = rest.find(&close) {- let yaml = rest[..end].trim_end_matches('\r');- let value: serde_json::Value =- serde_saphyr::from_str_with_options(yaml, frontmatter_parse_options())- .context("parse YAML frontmatter")?;- if let Some(map) = value.as_object() {- out.title = yaml_string(map, "title");- out.kind = yaml_string(map, "kind");- out.tier = yaml_string(map, "tier");- out.pinned = yaml_bool(map, "pinned").unwrap_or(false);- out.tags = yaml_tags(map);- }- rest[end + close.len()..].to_owned()- } else {- input.to_owned()- }- } else {- input.to_owned()+ let Some(rest) = rest else {+ return Ok((None, input.to_owned()));};- if out.title.is_none() {- out.title = first_h1(&body);- }- out.body = body;- Ok(out)+ let Some(end) = rest.find(&close) else {+ return Ok((None, input.to_owned()));+ };+ let yaml = rest[..end].trim_end_matches('\r');+ let value: serde_json::Value =+ serde_saphyr::from_str_with_options(yaml, frontmatter_parse_options())+ .context("parse YAML frontmatter")?;+ let map = match value {+ serde_json::Value::Object(map) => Some(map),+ _ => None,+ };+ Ok((map, rest[end + close.len()..].to_owned()))}/// Read frontmatter the way `serde_yaml` 0.9 did, matching the server's@@ -1374,8 +1393,8 @@ fn enc_path(s: &str) -> String {#[cfg(test)]mod tests {use super::*;- use std::io::{Read, Write};- use std::net::{TcpListener, TcpStream};+ use crate::test_http::{read_http_request, respond_json};+ use std::net::TcpListener;use std::thread;use tempfile::tempdir;@@ -1402,47 +1421,6 @@ mod tests {})}- fn read_http_request(stream: &mut TcpStream) -> (String, Vec<u8>) {- let mut bytes = Vec::new();- let mut buf = [0u8; 4096];- let header_end = loop {- let read = stream.read(&mut buf).unwrap();- assert!(read > 0, "client closed before sending complete headers");- bytes.extend_from_slice(&buf[..read]);- if let Some(pos) = bytes.windows(4).position(|window| window == b"\r\n\r\n") {- break pos + 4;- }- };- let headers = String::from_utf8(bytes[..header_end].to_vec()).unwrap();- let content_length = headers- .lines()- .find_map(|line| {- let (name, value) = line.split_once(':')?;- name.eq_ignore_ascii_case("content-length")- .then(|| value.trim().parse::<usize>().unwrap())- })- .unwrap_or(0);- while bytes.len() - header_end < content_length {- let read = stream.read(&mut buf).unwrap();- assert!(read > 0, "client closed before sending complete body");- bytes.extend_from_slice(&buf[..read]);- }- (- headers,- bytes[header_end..header_end + content_length].to_vec(),- )- }-- fn respond_json(stream: &mut TcpStream, status: &str, body: &str) {- write!(- stream,- "HTTP/1.1 {status}\r\ncontent-type: application/json\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{body}",- body.len()- )- .unwrap();- stream.flush().unwrap();- }-#[test]fn parses_omc_frontmatter() {let parsed = parse_markdown("---\ntitle: T\nkind: rule\ntier: procedural\ntags: [a, b]\npinned: true\nextra: ignored\n---\n# Body\ntext").unwrap();diff --git a/companions/ai-memory-importer/src/mcp.rs b/companions/ai-memory-importer/src/mcp.rsnew file mode 100644index 00000000..40f969f8--- /dev/null+++ b/companions/ai-memory-importer/src/mcp.rs@@ -0,0 +1,315 @@+//! The write seam for the agentmemory routes: the public `memory_write_page`+//! and `memory_read_page` MCP tools, called with explicit `workspace` and+//! `project`, the way `ai-memory-wikisync` calls them.+//!+//! Every write is conditional (`create_only` for a page's first version,+//! `expected_page_id` for each later one), so a page that appeared or changed+//! underneath the import is refused by the server instead of overwritten.+//! That closes the check-then-write race the `/admin/write-page` routes carry.++use std::fmt;++use anyhow::{Result, anyhow, bail};+use serde_json::{Value, json};++use crate::ImportClient;++/// Bound on server text echoed to the terminal or a manifest.+const ERROR_PREVIEW: usize = 300;+/// JSON-RPC `invalid_request`, the code a failed precondition carries.+const INVALID_REQUEST: i64 = -32600;+/// The server release that added conditional page writes.+pub(crate) const MIN_SERVER: &str = "2.7";++/// What a conditional write requires of the destination page.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub(crate) enum Precondition<'a> {+ /// No page may exist at the path.+ Absent,+ /// The page's latest version must still be this id.+ Latest(&'a str),+}++/// The server refused a conditional write; nothing was changed.+#[derive(Debug, Clone, PartialEq, Eq)]+pub(crate) struct PreconditionFailed {+ pub(crate) current_page_id: Option<String>,+}++impl fmt::Display for PreconditionFailed {+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {+ write!(+ f,+ "the destination page changed (current version {})",+ self.current_page_id+ .as_deref()+ .map_or_else(|| "none".to_owned(), preview)+ )+ }+}++impl std::error::Error for PreconditionFailed {}++/// One page write: the fields the MCP tool round-trips. The tool clears what+/// a call omits, so every field is always sent.+#[derive(Debug, Clone, PartialEq, Eq)]+pub(crate) struct McpWrite<'a> {+ pub(crate) workspace: &'a str,+ pub(crate) project: &'a str,+ pub(crate) path: &'a str,+ pub(crate) body: &'a str,+ pub(crate) title: Option<&'a str>,+ pub(crate) kind: &'a str,+ pub(crate) tier: &'a str,+ pub(crate) tags: &'a [String],+ pub(crate) expires_at: Option<&'a str>,+}++/// A page as `memory_read_page` returns it.+#[derive(Debug, Clone)]+pub(crate) struct ReadPage {+ pub(crate) page_id: Option<String>,+ pub(crate) body: String,+}++impl ImportClient {+ /// Refuse a server that predates conditional writes. It would accept+ /// `create_only` as an unknown argument and ignore it, turning every+ /// guarded write into an unconditional one.+ pub(crate) async fn ensure_conditional_writes(&self) -> Result<()> {+ let result = self.mcp_request("tools/list", json!({})).await?;+ let supported = result+ .get("tools")+ .and_then(Value::as_array)+ .into_iter()+ .flatten()+ .find(|tool| tool.get("name").and_then(Value::as_str) == Some("memory_write_page"))+ .and_then(|tool| tool.pointer("/inputSchema/properties/create_only"))+ .is_some();+ if !supported {+ bail!(+ "server too old for --apply; needs ai-memory >= {MIN_SERVER} (conditional page writes)"+ );+ }+ Ok(())+ }++ /// Write one page version only if `precondition` holds. Returns the new+ /// version id; a refused precondition comes back as [`PreconditionFailed`].+ pub(crate) async fn mcp_write_page(+ &self,+ write: &McpWrite<'_>,+ precondition: Precondition<'_>,+ ) -> Result<String> {+ let mut arguments = json!({+ "workspace": write.workspace,+ "project": write.project,+ "path": write.path,+ "body": write.body,+ "kind": write.kind,+ "tier": write.tier,+ "tags": write.tags,+ "pinned": false,+ });+ if let Some(title) = write.title {+ arguments["title"] = json!(title);+ }+ if let Some(expires_at) = write.expires_at {+ arguments["expires_at"] = json!(expires_at);+ }+ match precondition {+ Precondition::Absent => arguments["create_only"] = json!(true),+ Precondition::Latest(id) => arguments["expected_page_id"] = json!(id),+ }+ let result = self.call_tool("memory_write_page", arguments).await?;+ tool_json(&result)+ .and_then(|written| written.get("page_id")?.as_str().map(str::to_owned))+ .ok_or_else(|| anyhow!("memory_write_page answered without a page_id"))+ }++ /// Read one page by exact path.+ pub(crate) async fn mcp_read_page(+ &self,+ workspace: &str,+ project: &str,+ path: &str,+ ) -> Result<ReadPage> {+ let result = self+ .call_tool(+ "memory_read_page",+ json!({"workspace": workspace, "project": project, "path": path}),+ )+ .await?;+ let page = tool_json(&result)+ .ok_or_else(|| anyhow!("memory_read_page answered without a page"))?;+ Ok(ReadPage {+ page_id: page+ .get("page_id")+ .and_then(Value::as_str)+ .map(str::to_owned),+ body: page+ .get("body")+ .and_then(Value::as_str)+ .unwrap_or_default()+ .to_owned(),+ })+ }++ /// Ask for one page of the destination, the read every live run makes+ /// before writing: reads never create a scope, so an error means the+ /// destination does not exist (or cannot be read).+ pub(crate) async fn mcp_scope_exists(&self, workspace: &str, project: &str) -> bool {+ self.call_tool(+ "memory_recent",+ json!({"workspace": workspace, "project": project, "limit": 1}),+ )+ .await+ .is_ok()+ }++ /// Call one tool; a tool-level error becomes an `Err` carrying a bounded+ /// preview of the server's text.+ pub(crate) async fn call_tool(&self, name: &str, arguments: Value) -> Result<Value> {+ let result = self+ .mcp_request("tools/call", json!({"name": name, "arguments": arguments}))+ .await+ .map_err(|error| match error.downcast::<PreconditionFailed>() {+ Ok(failed) => anyhow::Error::new(failed),+ Err(error) => error.context(format!("MCP {name} failed")),+ })?;+ if result.get("isError").and_then(Value::as_bool) == Some(true) {+ let detail = result+ .pointer("/content/0/text")+ .and_then(Value::as_str)+ .unwrap_or("no detail");+ bail!("MCP {name} refused the call: {}", preview(detail));+ }+ Ok(result)+ }++ /// One JSON-RPC request; its `result`, or the error it answered with.+ async fn mcp_request(&self, method: &str, params: Value) -> Result<Value> {+ let request = json!({"jsonrpc": "2.0", "id": 1, "method": method, "params": params});+ let response = self+ .request(reqwest::Method::POST, "/mcp")+ // The streamable HTTP transport refuses a request that does not+ // accept both; it answers plain JSON in its default stateless+ // mode and SSE frames in stateful mode.+ .header(+ reqwest::header::ACCEPT,+ "application/json, text/event-stream",+ )+ .json(&request)+ .send()+ .await+ .map_err(|e| anyhow!("MCP {method} request failed: {e}"))?;+ let status = response.status();+ let text = response+ .text()+ .await+ .map_err(|e| anyhow!("reading the MCP {method} response failed: {e}"))?;+ if !status.is_success() {+ bail!("MCP {method} returned HTTP {status}: {}", preview(&text));+ }+ let mut message = json_rpc_message(&text)+ .ok_or_else(|| anyhow!("MCP {method} answered with no JSON-RPC message"))?;+ if let Some(error) = message.get("error") {+ if let Some(failed) = precondition_failure(error) {+ return Err(anyhow::Error::new(failed));+ }+ bail!("MCP {method} failed: {}", preview(&error.to_string()));+ }+ message+ .get_mut("result")+ .map(Value::take)+ .ok_or_else(|| anyhow!("MCP {method} answered with no result"))+ }+}++/// The typed precondition failure in a JSON-RPC error, if that is what it is.+fn precondition_failure(error: &Value) -> Option<PreconditionFailed> {+ let data = error.get("data")?;+ if error.get("code").and_then(Value::as_i64) != Some(INVALID_REQUEST)+ || data.get("reason").and_then(Value::as_str) != Some("precondition_failed")+ {+ return None;+ }+ Some(PreconditionFailed {+ current_page_id: data+ .get("current_page_id")+ .and_then(Value::as_str)+ .map(str::to_owned),+ })+}++/// The JSON a tool returned as its first text content, if any.+pub(crate) fn tool_json(result: &Value) -> Option<Value> {+ let text = result.pointer("/content/0/text")?.as_str()?;+ serde_json::from_str(text).ok()+}++/// The JSON-RPC message in a response body: the body itself, or the last+/// `data:` frame of an SSE stream.+fn json_rpc_message(text: &str) -> Option<Value> {+ if let Ok(value) = serde_json::from_str::<Value>(text) {+ return Some(value);+ }+ text.lines()+ .filter_map(|line| line.strip_prefix("data:"))+ .filter_map(|data| serde_json::from_str::<Value>(data.trim()).ok())+ .next_back()+}++/// Server text is untrusted: bound it and neutralise control characters so+/// it cannot drive the terminal.+pub(crate) fn preview(text: &str) -> String {+ text.chars()+ .take(ERROR_PREVIEW)+ .map(|c| if c.is_control() { '?' } else { c })+ .collect()+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn reads_plain_json_and_sse_frames() {+ let plain = r#"{"jsonrpc":"2.0","id":1,"result":{"isError":false}}"#;+ assert_eq!(json_rpc_message(plain).unwrap()["id"], 1);+ let sse = "event: message\ndata: {\"jsonrpc\":\"2.0\",\"id\":1,\"result\":{}}\n\n";+ assert!(json_rpc_message(sse).unwrap().get("result").is_some());+ assert!(json_rpc_message("event: ping\n\n").is_none());+ }++ #[test]+ fn only_a_typed_precondition_error_is_a_precondition_failure() {+ let error = json!({+ "code": -32600,+ "data": {"reason": "precondition_failed", "current_page_id": "v2"},+ });+ assert_eq!(+ precondition_failure(&error),+ Some(PreconditionFailed {+ current_page_id: Some("v2".into())+ })+ );+ assert_eq!(+ precondition_failure(&json!({"code": -32600, "data": {"reason": "nope"}})),+ None+ );+ assert_eq!(+ precondition_failure(+ &json!({"code": -32602, "data": {"reason": "precondition_failed"}})+ ),+ None+ );+ }++ #[test]+ fn previews_are_bounded_and_inert() {+ let shown = preview(&format!("\u{1b}[31m{}", "x".repeat(1_000)));+ assert!(!shown.contains('\u{1b}'));+ assert_eq!(shown.chars().count(), ERROR_PREVIEW);+ }+}diff --git a/companions/ai-memory-importer/src/test_http.rs b/companions/ai-memory-importer/src/test_http.rsnew file mode 100644index 00000000..319cd3a3--- /dev/null+++ b/companions/ai-memory-importer/src/test_http.rs@@ -0,0 +1,46 @@+//! Minimal HTTP/1.1 helpers for the hand-rolled test servers: read one+//! request, answer one JSON response and close the connection.++use std::io::{Read, Write};+use std::net::TcpStream;++pub(crate) fn read_http_request(stream: &mut TcpStream) -> (String, Vec<u8>) {+ let mut bytes = Vec::new();+ let mut buf = [0u8; 4096];+ let header_end = loop {+ let read = stream.read(&mut buf).unwrap();+ assert!(read > 0, "client closed before sending complete headers");+ bytes.extend_from_slice(&buf[..read]);+ if let Some(pos) = bytes.windows(4).position(|window| window == b"\r\n\r\n") {+ break pos + 4;+ }+ };+ let headers = String::from_utf8(bytes[..header_end].to_vec()).unwrap();+ let content_length = headers+ .lines()+ .find_map(|line| {+ let (name, value) = line.split_once(':')?;+ name.eq_ignore_ascii_case("content-length")+ .then(|| value.trim().parse::<usize>().unwrap())+ })+ .unwrap_or(0);+ while bytes.len() - header_end < content_length {+ let read = stream.read(&mut buf).unwrap();+ assert!(read > 0, "client closed before sending complete body");+ bytes.extend_from_slice(&buf[..read]);+ }+ (+ headers,+ bytes[header_end..header_end + content_length].to_vec(),+ )+}++pub(crate) fn respond_json(stream: &mut TcpStream, status: &str, body: &str) {+ write!(+ stream,+ "HTTP/1.1 {status}\r\ncontent-type: application/json\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{body}",+ body.len()+ )+ .unwrap();+ stream.flush().unwrap();+}diff --git a/crates/ai-memory-cli/src/cli.rs b/crates/ai-memory-cli/src/cli.rsindex 032bb6be..c113406f 100644--- a/crates/ai-memory-cli/src/cli.rs+++ b/crates/ai-memory-cli/src/cli.rs@@ -123,6 +123,8 @@ pub enum Command {/// Check capture coverage: compare local harness session stores for this/// project against what the server captured, and warn when a harness ran/// here recently but has no captured sessions (its hook is likely missing).+ /// Also reports agentmemory MCP entries and rules-file guideline blocks+ /// left after a migration; it only reads and prints the fix.Doctor(DoctorArgs),/// One-time import of this project's existing local harness session history/// into a brand-new (empty) ai-memory store, so installing hooksdiff --git a/crates/ai-memory-cli/src/commands/agentmemory_residue.rs b/crates/ai-memory-cli/src/commands/agentmemory_residue.rsnew file mode 100644index 00000000..32b748a5--- /dev/null+++ b/crates/ai-memory-cli/src/commands/agentmemory_residue.rs@@ -0,0 +1,634 @@+//! `ai-memory doctor`'s read-only check for configuration left behind by+//! agentmemory after a migration.+//!+//! Two leftovers keep confusing agents once agentmemory is gone: an MCP+//! server entry still registered in a harness config, and the memory-usage+//! guideline block `agentmemory connect` writes into native rules files. An+//! agent reading that block keeps calling tools that no longer exist, and one+//! of them, `memory_consolidate`, is also an ai-memory tool with different+//! arguments.+//!+//! This check only reads. It names each file and what it found there and+//! prints the line to fix by hand; it never edits or deletes another tool's+//! files.++use std::collections::BTreeSet;+use std::ffi::OsString;+use std::path::{Path, PathBuf};++use serde::Serialize;++/// agentmemory MCP tool names that ai-memory does not have. A rules file+/// naming one of these is following agentmemory's guidance.+pub(crate) const AGENTMEMORY_ONLY_TOOLS: &[&str] = &[+ "memory_action_create",+ "memory_action_update",+ "memory_audit",+ "memory_checkpoint",+ "memory_claude_bridge_sync",+ "memory_commit_lookup",+ "memory_commits",+ "memory_compress_file",+ "memory_crystallize",+ "memory_diagnose",+ "memory_export",+ "memory_facet_query",+ "memory_facet_tag",+ "memory_file_history",+ "memory_frontier",+ "memory_governance_delete",+ "memory_graph_query",+ "memory_heal",+ "memory_insight_list",+ "memory_lease",+ "memory_lesson_recall",+ "memory_lesson_save",+ "memory_mesh_sync",+ "memory_next",+ "memory_obsidian_export",+ "memory_patterns",+ "memory_profile",+ "memory_recall",+ "memory_reflect",+ "memory_relations",+ "memory_routine_run",+ "memory_save",+ "memory_sentinel_create",+ "memory_sentinel_trigger",+ "memory_sessions",+ "memory_signal_read",+ "memory_signal_send",+ "memory_sketch_create",+ "memory_sketch_promote",+ "memory_slot_append",+ "memory_slot_create",+ "memory_slot_delete",+ "memory_slot_get",+ "memory_slot_list",+ "memory_slot_replace",+ "memory_smart_search",+ "memory_snapshot_create",+ "memory_team_feed",+ "memory_team_share",+ "memory_timeline",+ "memory_verify",+ "memory_vision_search",+];++/// Tool names both servers expose, with different arguments and meaning.+pub(crate) const COLLIDING_TOOLS: &[&str] = &["memory_consolidate"];++/// The npm package agentmemory's MCP entries launch.+const MCP_PACKAGE: &str = "@agentmemory/";+/// The server name agentmemory registers itself under.+const SERVER_NAME: &str = "agentmemory";+/// A rules file larger than this is not a rules file; skip it.+const MAX_RULES_BYTES: u64 = 4 * 1024 * 1024;+/// Harness configs can be large (`~/.claude.json` keeps per-project history).+const MAX_CONFIG_BYTES: u64 = 64 * 1024 * 1024;+/// Bound on the files read from one rules directory.+const MAX_DIR_ENTRIES: usize = 256;++/// One leftover, with the fix line to show the operator.+#[derive(Debug, Clone, PartialEq, Eq, Serialize)]+pub(crate) struct ResidueFinding {+ /// `mcp_server` or `rules_block`.+ pub(crate) kind: &'static str,+ pub(crate) path: String,+ /// Where in the file: the config key path, or the line range.+ pub(crate) location: String,+ /// agentmemory tool names the block mentions that ai-memory also has.+ pub(crate) colliding_tools: Vec<String>,+ pub(crate) fix: String,+}++/// Scan `mcp_configs` and every known rules file. Read-only; a file that is+/// missing, unreadable, oversized or unparsable contributes nothing.+pub(crate) fn scan(home: &Path, repo_root: &Path, mcp_configs: &[PathBuf]) -> Vec<ResidueFinding> {+ let mut findings = Vec::new();+ for path in mcp_configs {+ if let Some(text) = read_bounded(path, MAX_CONFIG_BYTES) {+ findings.extend(scan_mcp_config(path, &text));+ }+ }+ for path in rules_file_candidates(home, repo_root) {+ if let Some(text) = read_bounded(&path, MAX_RULES_BYTES)+ && let Some(finding) = scan_rules_file(&path, &text)+ {+ findings.push(finding);+ }+ }+ findings+}++/// The MCP config files `install-mcp` knows for every client (resolved+/// through `env`, so harness home relocations apply), plus the project+/// files.+pub(crate) fn mcp_config_candidates(+ repo_root: &Path,+ env: &dyn Fn(&str) -> Option<OsString>,+) -> Vec<PathBuf> {+ use clap::ValueEnum as _;+ let mut paths: Vec<PathBuf> = crate::cli::McpClient::value_variants()+ .iter()+ .filter_map(|client| super::install_mcp::mcp_config_path_with(*client, env).ok())+ .collect();+ paths.extend(project_mcp_configs(repo_root));+ dedup(paths)+}++/// The project-scoped MCP files harnesses read from the repository root.+fn project_mcp_configs(repo_root: &Path) -> Vec<PathBuf> {+ [+ ".mcp.json",+ ".cursor/mcp.json",+ ".gemini/settings.json",+ ".vscode/mcp.json",+ ]+ .iter()+ .map(|name| repo_root.join(name))+ .collect()+}++/// Rules files where harnesses read standing instructions, globally and in+/// the repository: the places a memory-usage guideline block can sit.+pub(crate) fn rules_file_candidates(home: &Path, repo_root: &Path) -> Vec<PathBuf> {+ let mut files: Vec<PathBuf> = [+ "AGENTS.md",+ "CLAUDE.md",+ "GEMINI.md",+ "QWEN.md",+ ".cursorrules",+ ".windsurfrules",+ ".clinerules",+ ".rules",+ ".github/copilot-instructions.md",+ ]+ .iter()+ .map(|name| repo_root.join(name))+ .collect();+ files.extend(+ [+ ".claude/CLAUDE.md",+ ".codex/AGENTS.md",+ ".gemini/GEMINI.md",+ ".qwen/QWEN.md",+ ".factory/AGENTS.md",+ ".copilot/copilot-instructions.md",+ ".config/opencode/AGENTS.md",+ ]+ .iter()+ .map(|name| home.join(name)),+ );+ let dirs: Vec<PathBuf> = [+ ".cursor/rules",+ ".clinerules",+ ".kiro/steering",+ ".continue/rules",+ ".windsurf/rules",+ ]+ .iter()+ .map(|name| repo_root.join(name))+ .chain(+ [".kiro/steering", ".continue/rules", "Documents/Cline/Rules"]+ .iter()+ .map(|name| home.join(name)),+ )+ .collect();+ for dir in dirs {+ let Ok(entries) = std::fs::read_dir(&dir) else {+ continue;+ };+ let mut found: Vec<PathBuf> = entries+ .filter_map(Result::ok)+ .map(|entry| entry.path())+ .filter(|path| {+ path.is_file()+ && path+ .extension()+ .and_then(|ext| ext.to_str())+ .is_some_and(|ext| matches!(ext, "md" | "mdc" | "txt"))+ })+ .take(MAX_DIR_ENTRIES)+ .collect();+ found.sort();+ files.extend(found);+ }+ dedup(files)+}++fn dedup(paths: Vec<PathBuf>) -> Vec<PathBuf> {+ let mut seen = BTreeSet::new();+ paths+ .into_iter()+ .filter(|path| seen.insert(path.clone()))+ .collect()+}++fn read_bounded(path: &Path, max: u64) -> Option<String> {+ let meta = std::fs::metadata(path).ok()?;+ if !meta.is_file() || meta.len() > max {+ return None;+ }+ std::fs::read_to_string(path).ok()+}++/// Find agentmemory MCP server entries in one harness config (JSON, JSONC or+/// TOML): an entry named `agentmemory`, or any entry launching the+/// `@agentmemory/` package.+pub(crate) fn scan_mcp_config(path: &Path, text: &str) -> Vec<ResidueFinding> {+ let value = if path.extension().and_then(|e| e.to_str()) == Some("toml") {+ text.parse::<toml_edit::DocumentMut>()+ .ok()+ .map(|doc| toml_to_json(doc.as_item()))+ } else {+ jsonc_parser::cst::CstRootNode::parse(text, &jsonc_parser::ParseOptions::default())+ .ok()+ .and_then(|root| root.value())+ .and_then(|node| node.to_serde_value())+ };+ let Some(value) = value else {+ return Vec::new();+ };+ let mut locations = Vec::new();+ walk(&value, &mut Vec::new(), &mut locations);+ locations+ .into_iter()+ .map(|location| ResidueFinding {+ kind: "mcp_server",+ path: path.display().to_string(),+ fix: format!(+ "remove the `{location}` entry from {}; agentmemory is no longer running",+ path.display()+ ),+ location,+ colliding_tools: COLLIDING_TOOLS.iter().map(|t| (*t).to_owned()).collect(),+ })+ .collect()+}++/// A TOML document as JSON, enough for the entry walk: tables become objects,+/// arrays arrays, strings strings, and every other scalar its text.+fn toml_to_json(item: &toml_edit::Item) -> serde_json::Value {+ fn convert(value: &toml_edit::Value) -> serde_json::Value {+ match value {+ toml_edit::Value::String(s) => serde_json::Value::String(s.value().clone()),+ toml_edit::Value::Array(items) => items.iter().map(convert).collect(),+ toml_edit::Value::InlineTable(table) => table+ .iter()+ .map(|(k, v)| (k.to_owned(), convert(v)))+ .collect::<serde_json::Map<_, _>>()+ .into(),+ other => serde_json::Value::String(other.to_string().trim().to_owned()),+ }+ }+ match item {+ toml_edit::Item::Table(table) => table+ .iter()+ .map(|(k, v)| (k.to_owned(), toml_to_json(v)))+ .collect::<serde_json::Map<_, _>>()+ .into(),+ toml_edit::Item::ArrayOfTables(tables) => tables+ .iter()+ .map(|table| toml_to_json(&toml_edit::Item::Table(table.clone())))+ .collect(),+ toml_edit::Item::Value(v) => convert(v),+ toml_edit::Item::None => serde_json::Value::Null,+ }+}++/// Collect the key path of every object that is an agentmemory server entry.+fn walk(value: &serde_json::Value, trail: &mut Vec<String>, out: &mut Vec<String>) {+ match value {+ serde_json::Value::Object(map) => {+ for (key, child) in map {+ trail.push(key.clone());+ let named = key.eq_ignore_ascii_case(SERVER_NAME) && child.is_object();+ if named || (child.is_object() && launches_package(child)) {+ out.push(trail.join("."));+ } else {+ walk(child, trail, out);+ }+ trail.pop();+ }+ }+ serde_json::Value::Array(items) => {+ for (index, child) in items.iter().enumerate() {+ trail.push(index.to_string());+ walk(child, trail, out);+ trail.pop();+ }+ }+ _ => {}+ }+}++/// Whether a server entry's command, args or URL names the agentmemory+/// package, looking only at its own string fields.+fn launches_package(entry: &serde_json::Value) -> bool {+ let mentions = |value: &serde_json::Value| match value {+ serde_json::Value::String(s) => s.contains(MCP_PACKAGE),+ serde_json::Value::Array(items) => items+ .iter()+ .any(|item| item.as_str().is_some_and(|s| s.contains(MCP_PACKAGE))),+ _ => false,+ };+ ["command", "args", "url"]+ .iter()+ .any(|field| entry.get(field).is_some_and(mentions))+}++/// Find an agentmemory guideline block in one rules file: the lines that+/// name agentmemory-only tools, its package, or an `agentmemory` comment+/// marker. A file that only mentions agentmemory in prose is not flagged.+pub(crate) fn scan_rules_file(path: &Path, text: &str) -> Option<ResidueFinding> {+ let mut first = None;+ let mut last = 0usize;+ for (index, line) in text.lines().enumerate() {+ let lower = line.to_ascii_lowercase();+ let marker =+ (lower.contains("<!--") && lower.contains(SERVER_NAME)) || lower.contains(MCP_PACKAGE);+ if marker+ || AGENTMEMORY_ONLY_TOOLS+ .iter()+ .any(|tool| has_word(line, tool))+ {+ first.get_or_insert(index + 1);+ last = index + 1;+ }+ }+ let first = first?;+ let block: Vec<&str> = text+ .lines()+ .skip(first - 1)+ .take(last + 1 - first)+ .collect();+ let colliding: Vec<String> = COLLIDING_TOOLS+ .iter()+ .filter(|tool| block.iter().any(|line| has_word(line, tool)))+ .map(|tool| (*tool).to_owned())+ .collect();+ let location = if first == last {+ format!("line {first}")+ } else {+ format!("lines {first}-{last}")+ };+ Some(ResidueFinding {+ kind: "rules_block",+ path: path.display().to_string(),+ fix: format!(+ "delete the agentmemory guideline block ({location}) from {}; run `ai-memory install-instructions` if this file should carry ai-memory's routing instead",+ path.display()+ ),+ location,+ colliding_tools: colliding,+ })+}++/// `needle` as a whole identifier in `line`.+fn has_word(line: &str, needle: &str) -> bool {+ let is_ident = |c: char| c.is_ascii_alphanumeric() || c == '_';+ line.match_indices(needle).any(|(at, _)| {+ let before = line[..at].chars().next_back();+ let after = line[at + needle.len()..].chars().next();+ !before.is_some_and(is_ident) && !after.is_some_and(is_ident)+ })+}++/// Human lines for the doctor report; empty when nothing was found.+pub(crate) fn render(findings: &[ResidueFinding]) -> String {+ if findings.is_empty() {+ return String::new();+ }+ let mut out = format!(+ "⚠ agentmemory configuration is still present ({} place(s)). Agents reading it call \+ tools that no longer exist; `memory_consolidate` also exists in ai-memory with \+ different arguments. Nothing was changed; fix by hand:\n",+ findings.len()+ );+ for finding in findings {+ let what = match finding.kind {+ "mcp_server" => "MCP server entry",+ _ => "guideline block",+ };+ out.push_str(&format!(+ " - {what} in {} ({})\n fix: {}\n",+ finding.path, finding.location, finding.fix+ ));+ if !finding.colliding_tools.is_empty() && finding.kind == "rules_block" {+ out.push_str(&format!(+ " names {}, which ai-memory also has: an agent following this block would call ai-memory's tool with agentmemory's arguments\n",+ finding.colliding_tools.join(", ")+ ));+ }+ }+ out+}++#[cfg(test)]+mod tests {+ use super::*;++ /// A guideline block shaped like the ones `agentmemory connect` writes.+ const BLOCK: &str = "# Project rules\n\nUse tabs.\n\n<!-- agentmemory:start -->\n## Memory\nBefore a task, call `memory_smart_search`.\nSave decisions with memory_save.\nRun memory_consolidate at the end of a day.\n<!-- agentmemory:end -->\n\nMore rules.\n";++ #[test]+ fn a_guideline_block_is_found_with_its_line_range_and_collisions() {+ let finding = scan_rules_file(Path::new("/r/AGENTS.md"), BLOCK).unwrap();+ assert_eq!(finding.kind, "rules_block");+ assert_eq!(finding.location, "lines 5-10");+ assert_eq!(finding.colliding_tools, ["memory_consolidate"]);+ assert!(finding.fix.contains("/r/AGENTS.md"));+ }++ #[test]+ fn a_block_without_markers_is_still_found_by_tool_names() {+ let text = "Rules\n\nRecall with memory_recall before answering.\n";+ let finding = scan_rules_file(Path::new("GEMINI.md"), text).unwrap();+ assert_eq!(finding.location, "line 3");+ assert!(finding.colliding_tools.is_empty());+ }++ #[test]+ fn clean_files_and_prose_mentions_are_not_flagged() {+ for clean in [+ "# Rules\n\nUse memory_query to recall; memory_write_page to save.\n",+ "We migrated off agentmemory last month.\n",+ "Run memory_consolidate after a long session.\n",+ "my_memory_save_helper and memory_recall_count are our own names.\n",+ "",+ ] {+ assert_eq!(+ scan_rules_file(Path::new("AGENTS.md"), clean),+ None,+ "{clean:?}"+ );+ }+ }++ /// Everything ai-memory itself tells agents: the routing snippet and the+ /// managed skills.+ fn ai_memory_surface() -> Vec<&'static str> {+ std::iter::once(ai_memory_core::SNIPPET_BODY)+ .chain(+ ai_memory_core::routing_skills::MANAGED_SKILLS+ .iter()+ .map(|s| s.content),+ )+ .collect()+ }++ #[test]+ fn ai_memory_s_own_routing_is_clean() {+ for text in ai_memory_surface() {+ assert_eq!(scan_rules_file(Path::new("AGENTS.md"), text), None);+ }+ }++ #[test]+ fn the_tool_lists_agree_with_ai_memory_s_surface() {+ let surface = ai_memory_surface();+ for tool in COLLIDING_TOOLS {+ assert!(+ surface.iter().any(|text| has_word(text, tool)),+ "{tool} must be an ai-memory tool to collide"+ );+ }+ for tool in AGENTMEMORY_ONLY_TOOLS {+ assert!(+ !surface.iter().any(|text| has_word(text, tool)),+ "{tool} is an ai-memory tool; it must not mark agentmemory residue"+ );+ }+ }++ #[test]+ fn mcp_entries_are_found_by_name_or_package_in_json_jsonc_and_toml() {+ let cursor = r#"{"mcpServers": {"ai-memory": {"url": "http://127.0.0.1:49374/mcp"}, "agentmemory": {"command": "npx", "args": ["-y", "@agentmemory/mcp"]}}}"#;+ let found = scan_mcp_config(Path::new("/h/.cursor/mcp.json"), cursor);+ assert_eq!(found.len(), 1);+ assert_eq!(found[0].location, "mcpServers.agentmemory");+ assert_eq!(found[0].colliding_tools, ["memory_consolidate"]);++ // Renamed entry, JSONC with comments, nested per-project map.+ let claude = "{\n // user config\n \"projects\": {\"/w/api\": {\"mcpServers\": {\"mem\": {\"command\": \"npx\", \"args\": [\"@agentmemory/mcp\"]}}}}\n}";+ let found = scan_mcp_config(Path::new("/h/.claude.json"), claude);+ assert_eq!(found.len(), 1);+ assert_eq!(found[0].location, "projects./w/api.mcpServers.mem");++ let codex = "[mcp_servers.agentmemory]\ncommand = \"npx\"\nargs = [\"-y\", \"@agentmemory/mcp\"]\n\n[mcp_servers.agentmemory.env]\nAGENTMEMORY_URL = \"http://localhost:3111\"\n";+ let found = scan_mcp_config(Path::new("/h/.codex/config.toml"), codex);+ assert_eq!(found.len(), 1);+ assert_eq!(found[0].location, "mcp_servers.agentmemory");+ }++ #[test]+ fn clean_or_unparsable_configs_report_nothing() {+ for (name, text) in [+ (+ "mcp.json",+ r#"{"mcpServers": {"ai-memory": {"url": "http://127.0.0.1:49374/mcp"}}}"#,+ ),+ (+ "mcp.json",+ r#"{"note": "we used @agentmemory/mcp before", "agentmemory": true}"#,+ ),+ ("mcp.json", "{ not json"),+ (+ "config.toml",+ "[mcp_servers.ai-memory]\nurl = \"http://127.0.0.1:49374/mcp\"\n",+ ),+ ] {+ assert!(scan_mcp_config(Path::new(name), text).is_empty(), "{text}");+ }+ }++ #[test]+ fn scan_reads_planted_files_and_never_writes() {+ let home = tempfile::tempdir().unwrap();+ let repo = tempfile::tempdir().unwrap();+ std::fs::write(repo.path().join("AGENTS.md"), BLOCK).unwrap();+ std::fs::create_dir_all(repo.path().join(".cursor/rules")).unwrap();+ std::fs::write(+ repo.path().join(".cursor/rules/memory.mdc"),+ "Call memory_lesson_save.\n",+ )+ .unwrap();+ std::fs::write(repo.path().join("CLAUDE.md"), "@AGENTS.md\n").unwrap();+ std::fs::write(+ repo.path().join(".mcp.json"),+ r#"{"mcpServers": {"agentmemory": {"command": "npx", "args": ["@agentmemory/mcp"]}}}"#,+ )+ .unwrap();+ std::fs::create_dir_all(home.path().join(".gemini")).unwrap();+ std::fs::write(+ home.path().join(".gemini/GEMINI.md"),+ "Use memory_smart_search.\n",+ )+ .unwrap();+ let before: Vec<(PathBuf, Vec<u8>)> = walk_files(repo.path())+ .into_iter()+ .chain(walk_files(home.path()))+ .map(|p| {+ let bytes = std::fs::read(&p).unwrap();+ (p, bytes)+ })+ .collect();++ let findings = scan(home.path(), repo.path(), &project_mcp_configs(repo.path()));+ let mut paths: Vec<&str> = findings.iter().map(|f| f.path.as_str()).collect();+ paths.sort_unstable();+ let expect = |base: &Path, rel: &str| base.join(rel).display().to_string();+ let mut expected = [+ expect(repo.path(), "AGENTS.md"),+ expect(repo.path(), ".cursor/rules/memory.mdc"),+ expect(repo.path(), ".mcp.json"),+ expect(home.path(), ".gemini/GEMINI.md"),+ ];+ expected.sort();+ assert_eq!(+ paths,+ expected.iter().map(String::as_str).collect::<Vec<_>>()+ );+ for (path, bytes) in before {+ assert_eq!(+ std::fs::read(&path).unwrap(),+ bytes,+ "{} changed",+ path.display()+ );+ }+ let text = render(&findings);+ assert!(text.contains("Nothing was changed"), "{text}");+ assert!(text.contains("memory_consolidate"), "{text}");++ let clean_home = tempfile::tempdir().unwrap();+ let clean_repo = tempfile::tempdir().unwrap();+ std::fs::write(clean_repo.path().join("AGENTS.md"), "Use memory_query.\n").unwrap();+ assert!(+ scan(+ clean_home.path(),+ clean_repo.path(),+ &project_mcp_configs(clean_repo.path())+ )+ .is_empty()+ );+ assert_eq!(render(&[]), "");+ }++ fn walk_files(root: &Path) -> Vec<PathBuf> {+ let mut out = Vec::new();+ let mut stack = vec![root.to_path_buf()];+ while let Some(dir) = stack.pop() {+ for entry in std::fs::read_dir(&dir).unwrap().filter_map(Result::ok) {+ let path = entry.path();+ if path.is_dir() {+ stack.push(path);+ } else {+ out.push(path);+ }+ }+ }+ out+ }+}diff --git a/crates/ai-memory-cli/src/commands/doctor.rs b/crates/ai-memory-cli/src/commands/doctor.rsindex e4b9fac4..e35d557f 100644--- a/crates/ai-memory-cli/src/commands/doctor.rs+++ b/crates/ai-memory-cli/src/commands/doctor.rs@@ -18,6 +18,10 @@//!//! It never opens the store directly; the server remains the source of truth//! for the captured side, and the local side is read-only.+//!+//! It also reports configuration agentmemory left behind (an MCP server entry+//! or a guideline block in a rules file) with the line to fix by hand; see+//! [`super::agentmemory_residue`]. That check only reads.use std::collections::BTreeMap;use std::path::{Path, PathBuf};@@ -179,6 +183,9 @@ struct DoctorReport {/// itself. `None` when no ai-memory Codex hook is installed or Codex could/// not be asked.codex_hooks: Option<super::codex_hook_trust::CodexHookTrust>,+ /// agentmemory MCP entries and rules-file guideline blocks still present+ /// after a migration. Detected only; never changed.+ agentmemory_residue: Vec<super::agentmemory_residue::ResidueFinding>,}#[derive(Debug, Clone, Serialize)]@@ -574,6 +581,15 @@ pub async fn run(config: &Config, args: crate::cli::DoctorArgs) -> Result<()> {let native_memory = native_memory_reports(&home, &cwd);let codex_hooks = codex_hook_trust(&cwd).await;+ let repo_root =+ ai_memory_consolidate::discover_main_repo_root(&cwd).unwrap_or_else(|_| cwd.clone());+ let agentmemory_residue = super::agentmemory_residue::scan(+ &home,+ &repo_root,+ &super::agentmemory_residue::mcp_config_candidates(&repo_root, &|name| {+ std::env::var_os(name)+ }),+ );let report = DoctorReport {workspace,@@ -591,6 +607,7 @@ pub async fn run(config: &Config, args: crate::cli::DoctorArgs) -> Result<()> {spool,server_unreachable,codex_hooks,+ agentmemory_residue,};if args.json {@@ -755,6 +772,10 @@ fn render_human(report: &DoctorReport) {println!(" capture coverage: unavailable ({})", problem.reason);}print!("{}", render_capture_health(report));+ print!(+ "{}",+ super::agentmemory_residue::render(&report.agentmemory_residue)+ );if report.since_days == 0 {println!(" recent window: all on-disk sessions\n");} else {@@ -879,6 +900,7 @@ mod tests {spool,server_unreachable,codex_hooks,+ agentmemory_residue: Vec::new(),}}diff --git a/crates/ai-memory-cli/src/commands/mod.rs b/crates/ai-memory-cli/src/commands/mod.rsindex c0121420..ac9c2d99 100644--- a/crates/ai-memory-cli/src/commands/mod.rs+++ b/crates/ai-memory-cli/src/commands/mod.rs@@ -26,6 +26,7 @@ pub(crate) fn is_scope_ambiguous(error: &anyhow::Error) -> bool {})}+pub(crate) mod agentmemory_residue;pub mod api_key;pub mod apply_shared;pub mod audit_contamination;diff --git a/docs/companion-crates.md b/docs/companion-crates.mdindex 0ec5b62f..1ae4cd8f 100644--- a/docs/companion-crates.md+++ b/docs/companion-crates.md@@ -131,6 +131,16 @@ Initial source support is intentionally narrow:`session_id`, and bounded `role`/`content` messages), replayed through thepublic ordered hook-batch surface. Product-specific export adapters remainoutside this repository.+- agentmemory exports: its Obsidian vault (`agentmemory-vault`) and its+ `ExportData` JSON file (`agentmemory-export`, with a required project+ mapping file). Both write through MCP `memory_write_page` with explicit+ scope, every write conditional (`create_only`, then `expected_page_id` per+ later version), refuse servers older than 2.7, and land under+ `imported/agentmemory/`, outside every reserved family. Each page version+ carries a stable UUIDv5 ingest key, so a rerun skips what is already there.+ Observations are dropped unless replayed as `external-import` sessions+ through `/hook/batch`; records with no ai-memory counterpart are counted in+ the manifest. Details are in the importer README.Future sources can include:@@ -217,9 +227,11 @@ Re-home by kind:idempotency keys, and send the dedicated `external-import` wire identityrather than impersonating a live coding harness.- Surface all destructive actions in dry-run output before live mode.-- Treat non-overwrite checks as best-effort unless/until core exposes a generic- compare-and-write seam; companion v1 re-checks before each write but cannot make- `/admin/write-page` atomic with that read.+- Treat non-overwrite checks on `/admin/write-page` as best-effort: the OMC+ route re-checks before each write but cannot make that endpoint atomic with+ the read. The agentmemory routes use the conditional MCP write seam that+ shipped in 2.7 instead, so the server refuses a page that appeared or+ changed.- Keep LLM normalization optional; deterministic import should work with no LLM.- Keep provider-specific performance tweaks, such as model parameter changes, outof importer PRs. If ai-memory core needs a provider bugfix or optimization,diff --git a/docs/cookbook.md b/docs/cookbook.mdindex 255509f4..a2db039d 100644--- a/docs/cookbook.md+++ b/docs/cookbook.md@@ -121,6 +121,69 @@ Once the material is saved as pages in the right scope, any project can searchit (`memory_query`) and read a specific document in full (`memory_read_page`)before implementing against it.+## Coming from agentmemory++The data moves with the companion importer+([`ai-memory-importer`](../companions/ai-memory-importer), boundary in+[`companion-crates.md`](companion-crates.md)). Most of the work is re-wiring+each harness. Do it in this order:++1. **Export from agentmemory while it still runs.** Its export command writes+ one `ExportData` JSON file; its Obsidian export writes a vault directory.+ The JSON is the complete route (memory versions, TTLs, sessions,+ summaries, lessons, insights). The importer reads only that file and never+ contacts agentmemory.+2. **Stop agentmemory and unwire it.** Remove its MCP server entry, the+ plugins and skills it installed for each harness, and the memory guideline+ block its `connect` command writes into rules files (`AGENTS.md`,+ `GEMINI.md`, `.cursor/rules/`, `.kiro/steering/` and the like). Left in+ place, that block keeps agents calling tools that no longer exist, and one+ of them, `memory_consolidate`, is also an ai-memory tool with different+ arguments. `ai-memory doctor` lists every leftover it can see, with the file+ and lines and the fix, and changes nothing:++ ```text+ ⚠ agentmemory configuration is still present (2 place(s)). ...+ - MCP server entry in /home/me/.cursor/mcp.json (mcpServers.agentmemory)+ fix: remove the `mcpServers.agentmemory` entry from /home/me/.cursor/mcp.json; ...+ - guideline block in /home/me/work/api/AGENTS.md (lines 5-10)+ fix: delete the agentmemory guideline block (lines 5-10) from ...+ ```++ Rules kept outside files (Warp's, for example) need a look by hand.+3. **Wire ai-memory into each harness:** `ai-memory install-hooks --agent <name>+ --apply`, `ai-memory install-mcp --client <name> --apply` and `ai-memory+ install-skills --agent <name>`.+4. **Map projects, dry-run, then import.** agentmemory names a project by its+ directory basename, so the importer needs a mapping file to the ai-memory+ `workspace`/`project` (by basename, or by exact session `cwd` when two+ repositories share a name). The dry run lists every unmapped project and+ every record type it leaves out, with counts; `--apply` refuses to run+ while anything is unmapped:++ ```bash+ cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-export --file agentmemory-export.json --mapping mapping.json+ cargo run --manifest-path companions/ai-memory-importer/Cargo.toml -- \+ agentmemory-export --file agentmemory-export.json --mapping mapping.json \+ --apply --manifest-out agentmemory-manifest.json+ ```++ Pages land under `imported/agentmemory/` in the mapped project. A memory's+ versions become one page with its supersession history, `forgetAfter`+ becomes `expires_at`, and observations are dropped unless you pass+ `--replay-observations`. Rerunning is safe: pages already imported are+ skipped. For a vault, use `agentmemory-vault --dir <vault> --workspace W+ --project P` instead.+5. **Check.** Ask your agent something an old memory answered, or call+ `memory_query` for a known title in each project, and run `ai-memory+ doctor` again until it reports no leftovers.++What has no ai-memory counterpart (graph nodes and edges, actions, signals,+routines, sentinels, sketches, checkpoints) is counted in the manifest's+`dropped` section and not imported. ai-memory derives its own entity graph from+the imported pages.+## Recipe: mirror a team wiki into the repository (team wiki sync)To keep a team's shared memory reviewable next to the code it documents, use@@ -510,6 +573,10 @@ ai-memory serve # run the serverinstalled, it asks Codex whether it will run them. Codex skips a hook youhave not trusted, and skips it again after its command changes, withouttelling you: open Codex, run `/hooks`, and trust them.+- **Agents keep calling `memory_recall`, `memory_save` or other tools ai-memory+ does not have**: agentmemory's MCP entry or its rules-file guideline block is+ still there. `ai-memory doctor` names each one with the fix (see+ [Coming from agentmemory](#coming-from-agentmemory)).- **I just installed hooks in a project I've worked in for a while**: the firsttime you open the project after installing, ai-memory imports your existinglocal session history once (bounded, sanitized on the server, only into andiff --git a/docs/security-boundaries.md b/docs/security-boundaries.mdindex df234083..fa4656ea 100644--- a/docs/security-boundaries.md+++ b/docs/security-boundaries.md@@ -86,7 +86,7 @@ means the boundary is not built yet.| 13b | Managed-run lease exclusivity and explicit recovery (one active run per workstream, invariant #16) | `ai-memory-store/src/workstream.rs` `prepare_run` expires lapsed leases and otherwise refuses an `active` run inside the writer transaction (`StoreError::WorkstreamBusy`). The explicit `force_unlock` path may expire and replace that row in the same transaction only when its durable `owner_user` equals the incoming actor's attribution; different operators remain isolated. `heartbeat` renews only `active` rows. `ai-memory-cli/src/commands/run.rs` normally uses `wait_out_held_lease`; `--force-unlock` instead asks once and fails immediately when refused. Neither path treats the informational `lease_owner` label as authority | `multi_session.rs` `force_unlock_replaces_only_the_same_operators_active_run` (foreign operator refused with live control, same operator succeeds, old run loses heartbeat, unattributed single-user control); hooks `workstream.rs` `force_unlock_cannot_evict_another_operator` exercises the HTTP actor boundary; CLI `run.rs` `force_unlock_asks_once_instead_of_waiting_out_the_lease`; existing controls `managed_workstream_batches_are_idempotent_and_release_the_lease`, `a_renewed_lease_is_reported_as_a_live_owner_not_taken_over`, `interactive_launch_waits_out_a_lapsing_lease_then_proceeds`, and `ctrl_c_aborts_the_held_lease_wait_immediately` | STRONG for exclusivity. The `lease_owner` label (`host:pid`) is informational only and not unique inside ai-jail (every jailed launcher reports `ai-sandbox:<ns-pid>`), so it must never become an ownership key. A shared unattributed/root context is intentionally one operator bucket; deployments needing person-level refusal must authenticate DB users || 13c | A native store shared by every project yields only this checkout's records (Antigravity `history.jsonl`) | `ai-memory-workstream/src/transcript.rs` `export_antigravity` — the conversation must first pass `locate_session_file` (UUID file name + `.db` workspace == `cwd`); every `history.jsonl` line must match both that `conversationId` and, by `same_path`, this `cwd` (a conversation resumed in another directory keeps its id but records that directory, so the id alone is not enough); an id-less opening prompt is attributed only when exactly one same-workspace candidate falls within 5 s of the `.db` creation time and no other conversation of this checkout was created within 5 s of it; slash/shell entries and oversized lines are dropped, and only records naming this conversation are reported as its losses | `transcript.rs` `antigravity_history_exports_only_this_conversations_prompts_in_this_workspace` (other conversation in this and another workspace refused; own control passes), `antigravity_history_drops_prompts_from_a_resumed_conversation_in_another_workspace` (same id, foreign workspace refused), `antigravity_export_rejects_a_conversation_from_another_workspace`, `antigravity_export_rejects_unknown_or_traversal_ids`, `antigravity_opening_prompt_is_refused_when_another_conversation_started_with_it` (an opener another conversation of this checkout could own is refused), `antigravity_opening_prompt_covers_a_single_prompt_conversation` (control), `antigravity_history_reports_no_losses_for_foreign_records`; `backfill_e2e.rs` `backfill_imports_antigravity_prompts_from_history_and_nothing_foreign` (foreign prompts absent from wiki and db) | STRONG || 14 | Per-project authorization (#708) | `ai-memory-store/src/project_authz.rs` `authorize_project` / `ProjectAuthz::authorize` choke point (V68 `project_grants` + `projects.access_mode`, default `open`; V69 `projects.created_by` feeds `is_creator`); `scope.rs` `ScopeResolver::with_project_authz` (reader pool for reads, writer actor for writes) and its free forms `authorize_scope_for` / `*_guarded`, attached for every DB user by `ai-memory-mcp` `scope_resolver_as`, `ai-memory-web` `authorize_read` / `lookup_project`, and `ai-memory-hooks` (`grants.rs` `authorize_resolved` for run/workstream ids, the capture check in `router.rs`); read-shaped mutations resolve at `ProjectAccess::Write` (`resolve_existing_args(.., need)`); unscoped reads filtered before `LIMIT` by `reader.rs` `readable_repository_sql`; `WriterHandle::authorize_project` as defense in depth. Page ids are otherwise never taken from a caller (only derived from already-authorized hits); the one exception, the web page-history `from`/`to` version ids, is looked up only inside the scope `routes/mod.rs` `resolve_read_scope` (`lookup_existing_scope_guarded`, `ProjectAccess::Read`) has already authorized and the requested path (`page_version_body`), so a foreign id reads as not found; incremental `recent` resolves and authorizes the requested project before decoding its scope-bound cursor; batch outcomes do not bypass the capture grant or session owner check | `tests/suite/project_authz.rs` (decision matrix, ship-inert, resolver gate); `tests/suite/access_mode.rs` `every_caller_against_both_modes`, `a_restricted_project_admits_the_team_and_refuses_the_outsider`, `new_projects_follow_the_server_default_and_admit_their_creator`; `scope.rs` `the_argument_shape_does_not_decide_the_level`, `a_user_reaches_only_what_they_were_granted`, `creating_authorizes_against_a_project_that_already_exists`, `a_refused_scope_fails_the_search_instead_of_shortening_it`; `grants.rs` `search_finds_only_what_the_viewer_may_read`, `the_limit_counts_only_what_the_viewer_may_see`, `the_workspace_handoff_comes_only_from_readable_repositories`; `ai-memory-mcp` `server.rs` `a_reader_may_read_everything_and_change_nothing`, `bob_cannot_read_alices_page_in_a_restricted_project`, `bob_cannot_find_alices_page_by_searching_in_a_restricted_project`, `observation_stream_evidence_stays_inside_a_restricted_project` (with `[retrieval] observation_stream` on, a non-grantee naming the restricted project, listing it in `scopes`, or searching their own project never sees the session page or its observation excerpt; the writer and a read-grant holder do; fails with the read check in `ProjectAuthz::authorize` neutralized), `bob_cannot_consolidate_a_session_in_alices_repository`, `a_restricted_projects_queues_need_write`; `ai-memory-hooks` `a_capture_needs_writer_on_the_repository_it_lands_in`, `session_start_delivers_nothing_from_a_repository_the_viewer_cannot_read`, `run_and_workstream_ids_only_answer_someone_who_may_reach_the_repository` (foreign raw workstream id refuses populated provenance; granted reader receives the source record and tool correlation); `ai-memory-web` `web_reads_honour_grants_in_a_restricted_project`, `metadata_shows_only_what_the_viewer_may_read` — each with a granted or open-project control, and proven to fail with the choke point (19 tests) or the SQL filter (9 tests) neutralized; `routes.rs` `api_recent_incremental_rechecks_restricted_scope_on_cursor` (a valid cursor cannot bypass a revoked grant; granted control passes), `api_recent_incremental_paginates_over_120_pages_and_binds_cursor`; `mcp_stateless_http.rs` `generic_machine_client_writes_queries_and_claims_a_handoff` (real machine key, foreign scope and partial scope refused); `router.rs` `an_unauthorized_batch_item_is_consumed_rather_than_retried`, `batch_acknowledges_foreign_collision_then_commits_next_item` (refused foreign item followed by a legitimate stored item); `ai-memory-web` `history_replay.rs` `history_and_timeline_honour_grants_in_a_restricted_project` (page history in HTML and JSON and the session timeline: a non-grantee gets 403 without the body, a grantee and root get 200; bite: resolving through the unguarded `lookup_existing_scope` fails it) | STRONG — slice 3 closed both bypass classes (unscoped reads, raw-id entry points) and added the root-only management surface. Out of scope by design: per-project administrators (granting/restricting is root-only), and access modes, creators and grants live only in SQLite, so `reindex` resets them |-| 14aa | Companion capture privacy before persistence and delivery | `companions/ai-memory-client/src/lib.rs` `reject_sensitive`, `sanitize_external_value`, `check_body`; `src/scrub.rs` `sanitize_external_text`; relay `identity::validate`, `check_scope`, `relay::init` and `flush_locked`; importer `plan_external_conversation`, `planned_hook_event`, `truncate_utf8`, `ImportClient::post_hook_batch` | SDK secret-position, nested key, numeric metric (11 names plus numeric `*_tokens` counts; string values still redact), bounded schema and literal-marker tests; relay `sensitive_envelope_is_rejected_before_enqueue_with_clean_control`, `new_native_bodies_are_sanitized_before_hash_enqueue_and_exact_retry`, `unsafe_legacy_pending_is_dropped_locally_and_never_blocks_its_session`, `unsafe_legacy_native_identity_is_dropped_and_its_session_continues`, `legacy_binding_scope_spaces_are_preserved_on_delivery`; importer `sensitive_native_identity_is_refused_before_manifest_and_http_with_unicode_control`, `shared_key_sanitation_precedes_plan_manifest_transport_and_exact_retry`, `preview_and_manifest_use_scrubbed_bodies_and_digest`, `forged_body_authority_is_refused_before_transport`, `sanitized_redaction_markers_survive_utf8_byte_caps_in_bodies_and_titles`. Controls preserve opaque native Unicode IDs, metric representation and untrusted provenance. An unsafe item queued before these checks never reaches the wire (body or request line); it leaves the queue as a local `dropped_policy` receipt so the rest of its session ships. Guard-removal runtime probes target the final formatted sources and restore identical hashes. | Client-side privacy only; server scope, auth, admission and capture-policy guards remain authoritative |+| 14aa | Companion capture privacy before persistence and delivery | `companions/ai-memory-client/src/lib.rs` `reject_sensitive`, `sanitize_external_value`, `check_body`; `src/scrub.rs` `sanitize_external_text`; relay `identity::validate`, `check_scope`, `relay::init` and `flush_locked`; importer `plan_external_conversation`, `planned_hook_event`, `truncate_utf8`, `ImportClient::post_hook_batch`; importer agentmemory routes `vault::plan`, `export::plan`, `export::replay_session`, `agentmemory::validate_id` | SDK secret-position, nested key, numeric metric (11 names plus numeric `*_tokens` counts; string values still redact), bounded schema and literal-marker tests; relay `sensitive_envelope_is_rejected_before_enqueue_with_clean_control`, `new_native_bodies_are_sanitized_before_hash_enqueue_and_exact_retry`, `unsafe_legacy_pending_is_dropped_locally_and_never_blocks_its_session`, `unsafe_legacy_native_identity_is_dropped_and_its_session_continues`, `legacy_binding_scope_spaces_are_preserved_on_delivery`; importer `sensitive_native_identity_is_refused_before_manifest_and_http_with_unicode_control`, `shared_key_sanitation_precedes_plan_manifest_transport_and_exact_retry`, `preview_and_manifest_use_scrubbed_bodies_and_digest`, `forged_body_authority_is_refused_before_transport`, `sanitized_redaction_markers_survive_utf8_byte_caps_in_bodies_and_titles`, agentmemory `page_bodies_are_redacted_before_planning`, `text_is_redacted_before_the_plan_exists` (page bodies and replayed observations), `ids_that_look_like_credentials_or_carry_controls_are_refused`. Controls preserve opaque native Unicode IDs, metric representation and untrusted provenance. An unsafe item queued before these checks never reaches the wire (body or request line); it leaves the queue as a local `dropped_policy` receipt so the rest of its session ships. Guard-removal runtime probes target the final formatted sources and restore identical hashes. | Client-side privacy only; server scope, auth, admission and capture-policy guards remain authoritative || 14ap | Managed-run mutation owner and current project access (finish, cancel, heartbeat, native-session link, context acceptance) | `hooks/src/workstream.rs` `authorize_run_owner` derives `ManagedRunAuthority` only from authenticated middleware extensions and checks the real run before any mutation — for finish before status/payload/segment handling; cancel, heartbeat, link and context acceptance go through the same helper, so none is a side door around finish's rule; `ReaderPool::authorize_managed_run` and `store/src/workstream.rs` `authorize_run_mutation` use a read snapshot and the existing writer transaction, respectively. Strict resolution (`project_authz.rs` `resolve_project_authz_strict`, the canonical decision with a fail-closed resolution mode, so the reserved global-scope write gate applies here too) requires the actual workspace/project pair, current user existence and current Write decision for DB principals. The private trusted-proxy policy requires authenticated User capability, a canonical identity, no DB principal and server proxy configuration; it retains legacy project admission after strict scope validation, with ownership still enforced. Human-login disable does not revoke API keys. The legacy resolver and auth middleware remain unchanged. Both writer entry paths, including finished retries, require the same authority; no cross-owner bypass | MCP `auth.rs` `owner_finish_http_refuses_another_writer_before_payload_or_segment`, `owner_finish_http_trusted_proxy_preserves_restricted_lifecycle`, `owner_finish_http_active_api_key_survives_disabled_human_login`; store `multi_session.rs` `owner_finish_store_refuses_another_writer_including_finished_retry`, `owner_finish_store_rechecks_write_after_preflight`, `owner_finish_store_refuses_a_deleted_user_after_preflight`, `owner_finish_store_proxy_policy_requires_server_and_authenticated_identity`, `owner_finish_store_raw_ids_use_the_actual_project_and_workspace`, `owner_finish_store_topology_reload_and_phase_controls`, `owner_finish_store_strict_resolution_refuses_pairing_missing_and_sql_failure`, `owner_finish_store_keeps_qualified_identity_namespaces`; hooks `workstream.rs` `owner_finish_revocation_between_preflight_and_both_writer_paths`; MCP `auth.rs` `owner_finish_http_real_auth_rungs_shared_and_project_controls`, `owner_finish_http_preserves_late_active_terminal_and_missing_phases`; hooks `workstream.rs` `run_mutation_routes_refuse_another_operator` (another operator and root under another identity refused on heartbeat/link/accept/cancel, run left active; owner admitted); store `project_authz.rs` `strict_resolution_shares_the_global_gate_and_fails_closed_on_a_gap`. Guard-neutralization recipes and results accompany the local frozen patch | SQL refusal is atomic within finish's writer transaction; a revocation after preflight can leave an unindexed raw segment. Cancel, heartbeat, link and acceptance check one read snapshot immediately before a single-row update (a run's owner never changes). A later hook event carrying `AI_MEMORY_RUN_ID` links its native session only through `link_native_session_in_scope`, which applies the SessionStart boundary (`ManagedRunBoundaryRow::admits`: the run's own project and operator) and refuses a foreign run id while the event itself is still captured (`router.rs` `hook_event_links_a_managed_run_only_from_its_operator`, `hook_event_in_another_project_does_not_link_the_run`, control `hook_event_links_a_shared_run_on_a_single_operator_server`; #1082). Shared NULL ownership, trusted-proxy project admission and late still-active finishes remain intentional || 14au | Exact native managed identity and privacy (invariant #16) | `core/src/workstream.rs` `NativeSessionIdentity` validates original bytes and exact scrub equality; `workstream/src/harness.rs` and `transcript.rs` retain selector/path restrictions; CLI `run.rs` validates old-server prepare/status, planned selectors and finish transport; hooks `workstream.rs` checks payload and selected identity before raw segments, with the existing owner/current Write preflight first; `router.rs` validates both managed auto-links while preserving legacy shared capture; store `workstream.rs` carries validated identities into link/finish SQL after finish authority and before mutation. Store, HTTP search and CLI search project invalid historical identity to empty String without rewriting history | Core `native_identity_original_bytes_limits_privacy_and_collisions`; harness `native_identity_selector_rejects_original_before_resume_arguments`; transcript `native_identity_adapter_preserves_exact_and_rejects_privacy_changes`, `copilot_discovery_requires_uuid_directory_header_and_checkout` (another checkout's session, a `session.start` naming a different session than its directory, and non-UUID or `../` ids refused; exact-checkout control), `claude_transcript_is_located_by_the_real_folder_name` (a server-linked Claude id probed across project folders still resolves only to a transcript whose own header names this checkout; another checkout's transcript and a subagent sidechain with the same id are refused); store multi-session `native_identity_store_link_refuses_original_dirty_bytes`, `native_identity_store_adoption_refuses_before_link`, `native_identity_store_finish_refuses_without_sql_or_index_mutation`, `native_identity_store_history_projects_unknown_without_rewriting`; hooks `native_identity_http_link_refuses_before_status_or_sql`, `native_identity_http_finish_refuses_before_segment_or_sql`, `native_identity_http_event_guard_and_configured_history_projection`, `native_identity_http_stored_identity_refuses_before_raw_segment`, router `native_identity_managed_start_refuses_but_keeps_shared_legacy_capture`; CLI `native_identity_run_preflight_refuses_before_resume_or_private_path`, `native_identity_run_old_server_refusal_prevents_child_and_transport`, `native_identity_finish_transport_refuses_before_request_and_preserves_exact_control`, `native_identity_run_status_and_expected_refuse_before_discovery`, `native_identity_run_configured_selector_refuses_before_link_or_child`, `native_identity_run_configured_old_status_refuses_before_native_io_and_finish`, `native_identity_run_configured_discovery_refuses_before_export`, suite `native_identity_search_old_server_json_projects_unknown_and_keeps_history` | Representation/privacy only: no native authenticity, file-existence, run/event correlation or outcome proof. Original legacy rows and shared observation semantics remain intact; parent finish owner/current Write rules remain mandatory || 14b | Repository identity routing (#708) | `ai-memory-store/src/ops.rs` `resolve_project_by_identity` — one transaction; an identity already on a project is never overwritten; an unclaimed project is claimed only when the capturing user may write it (the choke point's `resolve_project_authz` on the same transaction), otherwise it is returned unclaimed; a different identity under the same name splits into a new project with no shared `repo_path`; when a Phase-5 client explicitly sends `IdentityStyle::Path` (#1033) a project it creates takes `path_style_name` only while no different full identity in the workspace claims that canonical key (checked in the same transaction), so the same path on another forge falls back instead of being merged; authorized captures can claim/promote compatible legacy rows in that transaction while preserving UUID, but handoff/static reads never rename; explicit `host_path` remains the opt-out; `ai-memory-hooks/src/workstream.rs` `resolve_run_scope` sends a managed run through the same transaction (identity holder, else the `$HOME`-guarded cwd-prefix candidate, inside the requested workspace only) only when the wire identity derives the requested name, then authorizes Write (#1182); `ai-memory-hooks/src/router.rs` `cache_key_for` keys the path cache by identity and style; `ai-memory-core/src/repository_identity.rs` strips credentials from remote URLs client-side, and the server accepts only the routing rungs (`explicit`, `git_remote`) from the wire | `tests/suite/identity_resolution.rs` `an_outsider_cannot_take_an_unclaimed_projects_identity` (control: `an_existing_project_is_claimed_in_place`), `two_unrelated_repositories_with_one_folder_name_stay_apart`, `a_created_or_split_project_admits_its_creator`, `the_same_path_on_another_forge_falls_back_instead_of_merging`, `an_unclaimed_project_holding_the_path_name_is_not_taken`, `old_client_omitted_style_stays_host_path_and_does_not_promote`, `explicit_path_capture_claims_and_promotes_an_identityless_legacy_name_in_one_transaction`, `unauthorized_capture_cannot_claim_or_promote_an_identityless_legacy_name`, and `explicit_path_capture_falls_back_when_another_identity_claims_the_canonical_key` (controls: `the_path_style_names_a_new_project_from_the_host_less_path`, `opting_in_leaves_existing_projects_where_they_are`), `multi_session.rs` `two_operators_on_two_clones_of_one_repository_share_its_path_named_project`; hooks `workstream.rs` `managed_run_by_remote_identity_refuses_a_viewer_without_write`, `managed_run_identity_never_claims_a_restricted_legacy_project`, `managed_run_identity_never_redirects_another_project_name`; `router.rs` `the_path_style_names_a_new_repository_and_never_merges_another_forge`, `one_path_with_two_remotes_is_two_projects`, `two_api_checkouts_with_different_remotes_get_two_projects` (a manifest rung on the wire is ignored); `repository_identity.rs` credential and normalisation tables — the claim guard and the cache key each proven to fail their test when removed | STRONG — first claimant wins: in a restricted workspace a user who creates a project under a remote identity first holds it until root grants others |