Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(migration): take the safety archive before the DB schema migration (#633)

AkitaOnRails681b1a860ce1
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a80f1671..52defbe1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -35,14 +35,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`dest_free_bytes`, so the same number is visible right before a migration.
### Fixed
-- Corrected `docs/MIGRATION-2.0.md`, which wrongly promised the automatic
- pre-migration archive could roll a store back to 1.x (#633). The archive is
- written *after* `Store::open` migrates the SQLite schema forward, so its
- `db/` is already at the 2.x schema and a 1.x binary refuses to open it. The
- docs now state the archive is a 2.x recovery point and that a genuine 1.x
- rollback requires a backup taken with the 1.x binary before upgrading.
- (The underlying ordering — snapshotting before the DB migration runs — is
- tracked separately.)
+- The pre-migration safety archive is now taken **before** the SQLite schema
+ is migrated, so it is a genuine pre-2.0 recovery point a 1.x binary can
+ reopen — restoring the documented "reversible upgrade" (#633). Previously the
+ archive was written inside the wiki migration, which runs *after*
+ `Store::open` has already advanced the DB schema, so the archived `db/` was
+ already at 2.x and a 1.x binary refused it — the documented rollback was
+ impossible. The snapshot now runs in the boot path before `Store::open`,
+ gated to the real 1.x→2.0 upgrade (a fresh install or already-migrated store
+ takes nothing, unchanged) and idempotent; the OKF migration reuses that
+ archive instead of taking a second one. An integration test asserts the
+ archived DB reflects the pre-migration state.
- Typed edges (`relations`) now actually work on OpenAI-family providers
(#630). `ConsolidatedPage.relations` was an open map
(`BTreeMap<String, Vec<String>>`), which the strict structured-output
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index c04ceade..91e2122d 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -719,6 +719,20 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {
// second active-project pointer (#563). Held until `run` returns.
let _serve_lock = acquire_serve_lock(&config.data_dir, args.force)?;
+ // #633: take the pre-migration safety archive BEFORE `Store::open` advances
+ // the SQLite schema, so the archive captures the true pre-2.0 state a 1.x
+ // binary can reopen. `Store::open` below runs the DB migrations; if the
+ // backup ran after it (as it used to, inside the wiki migration), the
+ // archived `db/` would already be at the 2.x schema and the documented
+ // rollback to 1.x would be impossible. A no-op for a fresh install or an
+ // already-migrated store; the OKF wiki migration reuses this archive instead
+ // of taking a second, post-DB-migration one.
+ let backup_dest_override = std::env::var("AI_MEMORY_BACKUP_DIR")
+ .ok()
+ .map(std::path::PathBuf::from);
+ migrations::snapshot_before_db_migration(&config.data_dir, backup_dest_override.as_deref())
+ .with_context(|| "taking the pre-migration safety backup")?;
+
let store = Store::open(&config.data_dir)
.with_context(|| format!("opening store at {}", config.data_dir.display()))?;
diff --git a/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs b/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs
index c477c5b0..eb252039 100644
--- a/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs
+++ b/crates/ai-memory-wiki/src/migrations/m2026_09_okf_conformance.rs
@@ -48,6 +48,50 @@ impl OkfConformance {
const BACKUP_LABEL: &str = "okf-v0.2";
+/// Take the pre-migration safety archive **before** the store's DB schema is
+/// migrated, so it captures the true pre-2.0 state a 1.x binary can reopen
+/// (#633). Call this from the boot path *before* `Store::open`, which advances
+/// the SQLite schema — otherwise the archive's `db/` is already at 2.x and the
+/// documented 1.x rollback is impossible.
+///
+/// Gated to the real 1.x→2.0 upgrade and idempotent:
+/// - if a usable backup already exists (`pre-migration-backup.json` pointing at
+/// a present archive), reuse it — never overwrite a good pre-migration snapshot;
+/// - if the wiki has no pre-OKF files, this is a fresh install or an already
+/// conformant store — nothing to protect, take nothing (unchanged behaviour);
+/// - otherwise archive the whole data dir now.
+///
+/// `dest_override` is the destination directory (the serve path passes
+/// `AI_MEMORY_BACKUP_DIR`); `None` uses the default resolution. A `None`
+/// *return* means "no backup was needed/taken here".
+///
+/// # Errors
+/// [`WikiError`] if the wiki scan or the archive write fails.
+pub fn snapshot_before_db_migration(
+ data_dir: &Path,
+ dest_override: Option<&Path>,
+) -> WikiResult<Option<crate::backup::BackupReceipt>> {
+ if crate::backup::BackupReceipt::load(data_dir).is_some_and(|r| r.archive_present()) {
+ // A prior boot in this same upgrade already captured the pre-migration
+ // state; do not overwrite it with a now-partially-migrated snapshot.
+ return Ok(None);
+ }
+ let wiki_root = data_dir.join("wiki");
+ if nonconformant_files(&wiki_root)?.is_empty() {
+ // No pre-OKF wiki files → not a 1.x store being upgraded.
+ return Ok(None);
+ }
+ let receipt =
+ crate::backup::create_pre_migration_backup(data_dir, BACKUP_LABEL, dest_override)?;
+ tracing::info!(
+ archive = %receipt.archive_path.display(),
+ size_bytes = receipt.size_bytes,
+ dest_free_bytes = ?receipt.dest_free_bytes,
+ "pre-migration backup taken before the DB schema migration (#633: 1.x-restorable)"
+ );
+ Ok(Some(receipt))
+}
+
#[async_trait::async_trait]
impl WikiMigration for OkfConformance {
fn name(&self) -> &'static str {
@@ -76,19 +120,36 @@ impl WikiMigration for OkfConformance {
"OKF migration: taking the pre-migration backup first"
);
- // 1. Backup gate.
- let receipt = crate::backup::create_pre_migration_backup(
- data_dir,
- BACKUP_LABEL,
- self.dest_override.as_deref(),
- )?;
- tracing::info!(
- archive = %receipt.archive_path.display(),
- size_bytes = receipt.size_bytes,
- entries = receipt.entries,
- dest_free_bytes = ?receipt.dest_free_bytes,
- "pre-migration backup verified"
- );
+ // 1. Backup gate. The boot path takes this archive BEFORE `Store::open`
+ // migrates the DB schema (#633), so a usable receipt normally already
+ // exists here — reuse it rather than archive the now-DB-migrated tree
+ // a second time. Only archive here for the rare path where the DB was
+ // already at 2.x but the wiki was not yet conformant (no pre-open
+ // snapshot was taken).
+ match crate::backup::BackupReceipt::load(data_dir)
+ .filter(crate::backup::BackupReceipt::archive_present)
+ {
+ Some(existing) => {
+ tracing::info!(
+ archive = %existing.archive_path.display(),
+ "reusing the pre-open pre-migration backup (#633)"
+ );
+ }
+ None => {
+ let receipt = crate::backup::create_pre_migration_backup(
+ data_dir,
+ BACKUP_LABEL,
+ self.dest_override.as_deref(),
+ )?;
+ tracing::info!(
+ archive = %receipt.archive_path.display(),
+ size_bytes = receipt.size_bytes,
+ entries = receipt.entries,
+ dest_free_bytes = ?receipt.dest_free_bytes,
+ "pre-migration backup verified"
+ );
+ }
+ }
// 2. Checkpoint whatever the tree holds before touching it.
let git = crate::git::GitAdapter::open_or_init(wiki_root)?;
@@ -481,6 +542,140 @@ mod tests {
assert!(crate::backup::BackupReceipt::load(tmp.path()).is_none());
}
+ // ---- #633: the safety archive must be taken BEFORE the DB migration ----
+
+ /// The core assertion for #633: the pre-open snapshot captures the DB as it
+ /// was at snapshot time, so a mutation applied afterwards — which is exactly
+ /// what `Store::open`'s schema migration is — is NOT in the archive. Prove
+ /// it end to end: write PRE, snapshot, overwrite to POST, and the extracted
+ /// archive must still read PRE. If it read POST, the archive would be
+ /// post-migration and a 1.x binary could not use it.
+ #[test]
+ fn snapshot_archive_captures_the_db_before_a_later_mutation() {
+ let tmp = TempDir::new().unwrap();
+ // A non-OKF wiki file makes this look like a 1.x store to the gate.
+ let proj_dir = tmp.path().join("wiki").join("w").join("p");
+ std::fs::create_dir_all(&proj_dir).unwrap();
+ std::fs::write(proj_dir.join("note.md"), "# Note\n\nlegacy body\n").unwrap();
+ // A DB carrying a sentinel row.
+ let db_path = tmp.path().join("db").join("memory.sqlite");
+ std::fs::create_dir_all(db_path.parent().unwrap()).unwrap();
+ {
+ let conn = rusqlite::Connection::open(&db_path).unwrap();
+ conn.execute_batch(
+ "CREATE TABLE snapshot_marker(v TEXT); INSERT INTO snapshot_marker VALUES('PRE');",
+ )
+ .unwrap();
+ }
+
+ let dest = TempDir::new().unwrap();
+ let receipt = snapshot_before_db_migration(tmp.path(), Some(dest.path()))
+ .unwrap()
+ .expect("a legacy store must be snapshotted");
+
+ // Mutate the on-disk DB AFTER the snapshot, as a schema migration would.
+ {
+ let conn = rusqlite::Connection::open(&db_path).unwrap();
+ conn.execute("UPDATE snapshot_marker SET v = 'POST'", [])
+ .unwrap();
+ }
+
+ // Extract the archive and read its DB: it must still say PRE.
+ let extract = TempDir::new().unwrap();
+ {
+ let file = std::fs::File::open(&receipt.archive_path).unwrap();
+ let dec = flate2::read::GzDecoder::new(file);
+ tar::Archive::new(dec).unpack(extract.path()).unwrap();
+ }
+ let archived_db = extract.path().join("db").join("memory.sqlite");
+ let conn = rusqlite::Connection::open(&archived_db).unwrap();
+ let marker: String = conn
+ .query_row("SELECT v FROM snapshot_marker", [], |r| r.get(0))
+ .unwrap();
+ assert_eq!(
+ marker, "PRE",
+ "the archive captured the DB after a later mutation — the backup is not pre-migration"
+ );
+ }
+
+ #[tokio::test]
+ async fn snapshot_backs_up_a_legacy_store() {
+ let (tmp, _store, _ws, _proj) = legacy_store().await;
+ let dest = TempDir::new().unwrap();
+ let receipt = snapshot_before_db_migration(tmp.path(), Some(dest.path())).unwrap();
+ assert!(
+ receipt.is_some(),
+ "a legacy store must be snapshotted pre-open"
+ );
+ assert_eq!(std::fs::read_dir(dest.path()).unwrap().count(), 1);
+ assert!(crate::backup::BackupReceipt::load(tmp.path()).is_some());
+ }
+
+ #[tokio::test]
+ async fn snapshot_skips_a_conformant_or_fresh_store() {
+ let tmp = TempDir::new().unwrap();
+ let _store = Store::open(tmp.path()).unwrap();
+ std::fs::create_dir_all(tmp.path().join("wiki")).unwrap();
+ let dest = TempDir::new().unwrap();
+ assert!(
+ snapshot_before_db_migration(tmp.path(), Some(dest.path()))
+ .unwrap()
+ .is_none()
+ );
+ assert_eq!(std::fs::read_dir(dest.path()).unwrap().count(), 0);
+ }
+
+ /// The pre-open snapshot and the OKF migration must not both archive: the
+ /// migration reuses the receipt the boot path already wrote.
+ #[tokio::test]
+ async fn okf_migration_reuses_the_pre_open_snapshot() {
+ let (tmp, store, _ws, _proj) = legacy_store().await;
+ let pre_dest = TempDir::new().unwrap();
+ let receipt = snapshot_before_db_migration(tmp.path(), Some(pre_dest.path()))
+ .unwrap()
+ .expect("legacy store snapshotted pre-open");
+ assert_eq!(std::fs::read_dir(pre_dest.path()).unwrap().count(), 1);
+
+ // A different dest: if the migration re-archived, an artifact would land
+ // here. It must reuse the pre-open one instead.
+ let okf_dest = TempDir::new().unwrap();
+ let migration = OkfConformance {
+ dest_override: Some(okf_dest.path().to_path_buf()),
+ };
+ migration
+ .up(&store.writer, &tmp.path().join("wiki"))
+ .await
+ .unwrap();
+ assert_eq!(
+ std::fs::read_dir(okf_dest.path()).unwrap().count(),
+ 0,
+ "the OKF migration took a second backup instead of reusing the pre-open one"
+ );
+ assert!(
+ receipt.archive_present(),
+ "the reused archive must still exist"
+ );
+ }
+
+ #[tokio::test]
+ async fn snapshot_is_idempotent_and_never_overwrites() {
+ let (tmp, _store, _ws, _proj) = legacy_store().await;
+ let dest = TempDir::new().unwrap();
+ assert!(
+ snapshot_before_db_migration(tmp.path(), Some(dest.path()))
+ .unwrap()
+ .is_some()
+ );
+ let after_first = std::fs::read_dir(dest.path()).unwrap().count();
+ assert!(
+ snapshot_before_db_migration(tmp.path(), Some(dest.path()))
+ .unwrap()
+ .is_none(),
+ "a second snapshot when a usable backup exists must be a no-op"
+ );
+ assert_eq!(std::fs::read_dir(dest.path()).unwrap().count(), after_first);
+ }
+
/// The gate itself: when the backup cannot be written, the migration
/// must fail and leave every row and file untouched.
#[tokio::test]
diff --git a/crates/ai-memory-wiki/src/migrations/mod.rs b/crates/ai-memory-wiki/src/migrations/mod.rs
index d713203a..7b8b4f40 100644
--- a/crates/ai-memory-wiki/src/migrations/mod.rs
+++ b/crates/ai-memory-wiki/src/migrations/mod.rs
@@ -46,6 +46,7 @@ use std::path::Path;
use ai_memory_store::WriterHandle;
+pub use m2026_09_okf_conformance::snapshot_before_db_migration;
pub use runner::run_pending;
use crate::error::WikiResult;
diff --git a/docs/MIGRATION-2.0.md b/docs/MIGRATION-2.0.md
index 1e6fff08..2a28511e 100644
--- a/docs/MIGRATION-2.0.md
+++ b/docs/MIGRATION-2.0.md
@@ -1,19 +1,14 @@
# Upgrading to 2.0
2.0 changes the wiki's on-disk format to the [Open Knowledge Format
-v0.2](okf.md). The upgrade is automatic and backup-gated. This page
-describes exactly what happens and what the automatic archive can and
-cannot restore.
-
-> **Rollback to 1.x needs your own 1.x-era backup.** The automatic
-> archive below is written **after** the SQLite schema has already been
-> migrated forward, so its `db/` is at the 2.x schema and a 1.x binary
-> will refuse to open it (`store schema … newer than this ai-memory
-> build`). The archive is a full **2.x** recovery point (wiki content and
-> business state intact); it is **not** a 1.x rollback point. If you want
-> the option to return to 1.x, take a backup with the **1.x** binary (or
-> `ai-memory backup` on 1.x, or a filesystem copy of the data dir) before
-> the first 2.x start. See [#633](https://github.com/akitaonrails/ai-memory/issues/633).
+v0.2](okf.md). The upgrade is automatic, backup-gated, and reversible.
+This page describes exactly what happens and how to go back.
+
+> **The safety archive is a true pre-migration recovery point.** It is
+> written **before** the SQLite schema is migrated forward (#633), so its
+> `db/` is still at the 1.x schema and restoring it lets you start the old
+> 1.x binary again. (Earlier 2.0.x builds took this archive *after* the DB
+> migration, which left it 2.x-only — fixed in 2.0.3.)
## What happens on the first 2.0 start
@@ -115,12 +110,11 @@ ai-memory status # server healthy, page counts unchanged
grep -L "^type:" <data_dir>/wiki/*/*/*/*.md # no output = all pages typed
```
-## Restoring the automatic archive
+## Restoring the backup
-Blunt and complete — returns the entire data directory to the state it
-had **the moment the OKF wiki migration began**. Note this is *after* the
-SQLite schema was migrated forward, so the restored `db/` is at the 2.x
-schema: reopen it with a **2.x** binary, not 1.x.
+Blunt and complete — returns the entire data directory to its exact
+pre-migration state (the archive is taken before both the DB schema and
+the wiki are migrated, #633), so you can start the old 1.x binary again:
```bash
# 1. stop the server (docker compose down / systemctl stop ai-memory)
@@ -129,13 +123,9 @@ mv <data_dir> <data_dir>.post-migration
# 3. unpack the archive as the new data dir
mkdir <data_dir>
tar -xzf ~/ai-memory-backup-okf-v0.2-<date>.tar.gz -C <data_dir>
-# 4. start ai-memory 2.x against it (a 1.x binary will refuse the DB)
+# 4. start the OLD (1.x) binary against it
```
-To actually roll **back to 1.x**, you need a backup taken with the 1.x
-binary before the upgrade (see the note at the top of this page); the
-automatic archive above cannot serve that purpose.
-
Surgical alternative (keeps post-migration work, reverts only the wiki
files): the `pre-okf-migration checkpoint` commit in the wiki's git
history, followed by `ai-memory reindex`.
@@ -146,10 +136,11 @@ A 2.0-migrated data directory records the migration in the
`wiki_migrations` table. A **newer** binary opening an **older** wiki
migrates it (as above). An **older 2.0+** binary opening a **newer**
wiki refuses to start with `NewerWikiFormat` instead of silently mixing
-formats. (1.x binaries predate the *wiki* guard, but the **SQLite** schema guard
-still stops them: a 1.x binary refuses a 2.x-migrated `db/` outright, so
-you cannot stay on 1.x by restoring the automatic archive — only by
-restoring a backup you took with the 1.x binary before upgrading.)
+formats. (1.x binaries predate the guard: they can open a migrated
+directory and will tolerate the extra frontmatter, but new writes from
+1.x will not carry the OKF keys — avoid mixing. To stay on 1.x cleanly,
+restore the pre-migration archive, which is captured before the DB
+migration and so reopens under 1.x.)
## Sharing bundles