YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(db): disable foreign keys during page recovery (#6963)
Ezra Ellette12b66b177ba6
diff --git a/crates/screenpipe-sqlite-recovery/VENDORING.md b/crates/screenpipe-sqlite-recovery/VENDORING.mdindex 50b22d6bc..d188f2f48 100644--- a/crates/screenpipe-sqlite-recovery/VENDORING.md+++ b/crates/screenpipe-sqlite-recovery/VENDORING.md@@ -11,9 +11,15 @@ Source archive:Vendored upstream files and SHA-256 digests:- `dbdata.c`: `0a57eae22b51507c28f6c647fba00ccacd39b5aaf3393dfe9435fb04810a6e0e`-- `sqlite3recover.c`: `bc56c1131dfdc979fd93a7e013856f4c2a98570c722c069600f6c66d2d47743f`+- `sqlite3recover.c`: `2b93163ec8af9a3667881b1aa1835a494ff6c6d91ba898790cade67225e203fd`- `sqlite3recover.h`: `c088088d15af055304e931842f7c197024d46ff1b7077912ec42149e9fa4f9bb`+`sqlite3recover.c` carries the required Screenpipe source header and one local+integration patch: the file-output connection disables foreign-key enforcement+before recovery writes, matching the SQL emitted by the extension's+callback-output mode. The unpatched upstream digest is+`bc56c1131dfdc979fd93a7e013856f4c2a98570c722c069600f6c66d2d47743f`.+When the bundled SQLite version changes, replace these files from the matchingofficial SQLite tag, update the digests above, and run the recovery tests onmacOS, Windows, and Linux. The extension requiresdiff --git a/crates/screenpipe-sqlite-recovery/src/lib.rs b/crates/screenpipe-sqlite-recovery/src/lib.rsindex 1531073bb..a2ca258b5 100644--- a/crates/screenpipe-sqlite-recovery/src/lib.rs+++ b/crates/screenpipe-sqlite-recovery/src/lib.rs@@ -277,6 +277,42 @@ mod tests {);}+ #[test]+ fn recovers_foreign_key_child_when_parent_row_is_unavailable() {+ // Page-level recovery must salvage every readable row. The higher-level+ // screenpipe-db verifier still rejects a candidate with unresolved+ // foreign-key violations before it can replace the quarantined source.+ let directory = tempfile::tempdir().expect("temporary recovery directory");+ let source = directory.path().join("source.sqlite");+ let destination = directory.path().join("destination.sqlite");+ let connection = Connection::open(&source).expect("open source");+ connection+ .execute_batch(+ "PRAGMA foreign_keys = OFF;\+ CREATE TABLE children (\+ id INTEGER PRIMARY KEY,\+ parent_id INTEGER NOT NULL REFERENCES parents(id)\+ DEFERRABLE INITIALLY DEFERRED\+ );\+ CREATE TABLE parents (id INTEGER PRIMARY KEY);\+ INSERT INTO children VALUES (1, 1);",+ )+ .expect("seed related rows");+ drop(connection);++ recover_database(&source, &destination).expect("recover foreign-key relationships");++ let recovered = Connection::open_with_flags(&destination, OpenFlags::SQLITE_OPEN_READ_ONLY)+ .expect("open recovered database");+ assert_eq!(+ recovered+ .query_row("SELECT COUNT(*) FROM children", [], |row| row+ .get::<_, i64>(0))+ .expect("count recovered children"),+ 1+ );+ }+#[test]fn salvages_rows_when_an_index_page_is_corrupt() {let directory = tempfile::tempdir().expect("temporary recovery directory");diff --git a/crates/screenpipe-sqlite-recovery/vendor/sqlite-3.51.3-recovery/sqlite3recover.c b/crates/screenpipe-sqlite-recovery/vendor/sqlite-3.51.3-recovery/sqlite3recover.cindex d50aa25ea..177703736 100644--- a/crates/screenpipe-sqlite-recovery/vendor/sqlite-3.51.3-recovery/sqlite3recover.c+++ b/crates/screenpipe-sqlite-recovery/vendor/sqlite-3.51.3-recovery/sqlite3recover.c@@ -1,3 +1,6 @@+// screenpipe — AI that knows everything you've seen, said, or heard+// https://screenpipe.com+/*** 2022-08-27**@@ -996,6 +999,8 @@ static int recoverOpenOutput(sqlite3_recover *p){recoverDbError(p, db);}+ recoverExec(p, db, "PRAGMA foreign_keys = off");+/* Register the sqlite_dbdata and sqlite_dbptr virtual table modules.** These two are registered with the output database handle - this** module depends on the input handle supporting the sqlite_dbpage