Skip to content

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

fix(db): keep recording running through WAL checkpoint backlog (#7194)

Ezra Ellette2978ef82da35
diff --git a/apps/screenpipe-app-tauri/e2e/COVERAGE.md b/apps/screenpipe-app-tauri/e2e/COVERAGE.md
index 479fe00a5..cd590f8cc 100644
--- a/apps/screenpipe-app-tauri/e2e/COVERAGE.md
+++ b/apps/screenpipe-app-tauri/e2e/COVERAGE.md
@@ -11,8 +11,8 @@ and layer declared in the manifest, weighted by confidence and criticality.
- Manifest: `e2e/coverage-map.json`
- Specs directory: `e2e/specs`
- Mapped specs: 141
-- Declared test blocks: 405
-- Weighted coverage points: 325.5
+- Declared test blocks: 406
+- Weighted coverage points: 326.5
Confidence weights: strong=1.0, partial=0.7, conditional=0.4, smoke=0.3.
Criticality weights: high=1.0, medium=0.7, low=0.4.
@@ -23,9 +23,9 @@ can execute more runtime cases than this number shows.
| Platform | Specs | Declared tests | Weighted points | Layers | Features | Critical score |
| --- | --- | --- | --- | --- | --- | --- |
-| windows | 107 | 342 | 285.1 | 15 | 122 | 85% |
-| macos | 137 | 367 | 295.3 | 17 | 132 | 88% |
-| linux | 95 | 300 | 254.5 | 14 | 119 | 80% |
+| windows | 107 | 343 | 286.1 | 15 | 122 | 85% |
+| macos | 137 | 368 | 296.3 | 17 | 132 | 88% |
+| linux | 95 | 301 | 255.5 | 14 | 119 | 80% |
## Runtime Results
@@ -43,13 +43,13 @@ pass/fail/skip counts.
| capture-ocr | 2 specs / 16 tests / 6.4 pts | 10 specs / 14 tests / 6.2 pts | 1 specs / 3 tests / 1.2 pts |
| chat-ai | 37 specs / 87 tests / 71.3 pts | 51 specs / 116 tests / 90.8 pts | 35 specs / 86 tests / 70.8 pts |
| entitlement | - | 1 specs / 1 tests / 1.0 pts | - |
-| local-api | 29 specs / 120 tests / 101.0 pts | 40 specs / 118 tests / 100.9 pts | 24 specs / 88 tests / 79.2 pts |
+| local-api | 29 specs / 121 tests / 102.0 pts | 40 specs / 119 tests / 101.9 pts | 24 specs / 89 tests / 80.2 pts |
| notifications | 4 specs / 26 tests / 17.3 pts | 3 specs / 5 tests / 3.4 pts | 2 specs / 4 tests / 3.1 pts |
| onboarding | 9 specs / 39 tests / 34.8 pts | 11 specs / 43 tests / 38.2 pts | 9 specs / 39 tests / 34.8 pts |
| os-integration | 7 specs / 32 tests / 26.9 pts | 16 specs / 31 tests / 18.8 pts | 2 specs / 15 tests / 10.8 pts |
| performance | 3 specs / 45 tests / 45.0 pts | 5 specs / 36 tests / 31.8 pts | 2 specs / 30 tests / 30.0 pts |
| pipes | 6 specs / 20 tests / 20.0 pts | 8 specs / 26 tests / 26.0 pts | 6 specs / 20 tests / 20.0 pts |
-| real-ui-e2e | 80 specs / 240 tests / 203.9 pts | 98 specs / 259 tests / 219.3 pts | 74 specs / 216 tests / 189.9 pts |
+| real-ui-e2e | 80 specs / 241 tests / 204.9 pts | 98 specs / 260 tests / 220.3 pts | 74 specs / 217 tests / 190.9 pts |
| settings | 15 specs / 42 tests / 39.0 pts | 17 specs / 36 tests / 31.7 pts | 14 specs / 33 tests / 30.0 pts |
| storage-privacy | 10 specs / 44 tests / 35.3 pts | 10 specs / 29 tests / 28.1 pts | 7 specs / 22 tests / 21.1 pts |
| tauri-command | 23 specs / 64 tests / 50.9 pts | 35 specs / 87 tests / 69.3 pts | 22 specs / 65 tests / 51.8 pts |
@@ -76,7 +76,7 @@ pass/fail/skip counts.
| Meetings-only audio device ownership | audio-device, local-api, real-ui-e2e | weak (conditional; meetings-only-audio-lifecycle) | weak (conditional; meetings-only-audio-lifecycle) | - |
| Window lifecycle, focus, and dedupe | window-lifecycle | covered (strong; windows-system-integration, window-lifecycle) | covered (strong; renderer-recovery, window-lifecycle) | covered (strong; window-lifecycle, chat-window) |
| macOS frozen WebKit renderer recovery | window-lifecycle, tauri-command, real-ui-e2e, local-api | - | covered (strong; renderer-recovery) | - |
-| Meeting note creation and editing | real-ui-e2e | covered (strong; windows-user-journey, meeting-note-bottom-click) | covered (strong; meeting-note-bottom-click, meeting-chat-panel) | covered (strong; meeting-note-bottom-click, meeting-replay-player) |
+| Meeting note creation and editing | real-ui-e2e | covered (strong; windows-user-journey, meeting-note-bottom-click) | covered (strong; meeting-note-bottom-click, meeting-chat-panel) | covered (strong; meeting-note-bottom-click, meeting-restart-live-transcription) |
| Pipes discover, install, and play | pipes | covered (strong; pipes, brain-overview) | covered (strong; pipes, brain-overview) | covered (strong; pipes, brain-overview) |
| Chat window, composer, and streaming state | chat-ai | covered (strong; acp-backend, chat-tool-activity) | covered (strong; acp-backend, chat-tool-activity) | covered (strong; acp-backend, chat-tool-activity) |
| Conversation-owned coding worktrees | chat-ai | gap | gap | gap |
@@ -195,7 +195,7 @@ pass/fail/skip counts.
| meeting-overlay-pin.spec.ts | macos | real-ui-e2e, local-api, tauri-command, os-integration | meeting-notes, meeting-overlay-stream, shortcut-overlay | medium | partial | real-user-flow | 1 | Pins the native macOS live-transcript card against a real started meeting: it opens on hover, stays visible after the pointer leaves, and the authoritative stop edge hides it and clears the pin, asserted on the AppKit panel's observed visibility. Hover and the pin toggle are driven through the controller entry points the tracking area and pin button use, because AppKit delivers no synthetic hover to a nonactivating panel. The Windows/Linux reminder webview renders the sign-in gate under E2E seeds, so that surface is covered by component tests instead. |
| meeting-overlay-stream.spec.ts | windows, macos, linux | local-api, tauri-command | meeting-notes, meeting-overlay-stream | high | strong | mixed | 1 | Starts a real isolated meeting, verifies bounded snapshot plus deterministic delta/final transcript frames, and proves the stream clears on the authoritative stop edge. |
| meeting-replay-player.spec.ts | windows, macos, linux | real-ui-e2e, local-api | meeting-notes, meeting-replay | high | strong | real-user-flow | 1 | Seeds four stored frames, wraps their exact timestamps in an ended meeting, reopens it from the previous-meetings list, and verifies deduped silent playback, pause stability, unobscured frame layout, and a review screenshot. |
-| meeting-restart-live-transcription.spec.ts | windows, macos, linux | real-ui-e2e, local-api | meeting-notes | high | strong | command | 1 | Opens an auto-detected meeting, runs the full stop_screenpipe/spawn_screenpipe restart the health overlay's RESTART button uses, and asserts the meeting is still open afterwards so live transcription has something to reattach to. |
+| meeting-restart-live-transcription.spec.ts | windows, macos, linux | real-ui-e2e, local-api | meeting-notes | high | strong | command | 2 | Opens an auto-detected meeting, runs the full stop_screenpipe/spawn_screenpipe restart the health overlay's RESTART button uses, and asserts the meeting is still open afterwards so live transcription has something to reattach to. |
| meeting-summary-recovery.spec.ts | windows, macos, linux | real-ui-e2e, local-api | meeting-notes, meeting-summary-recovery | high | strong | real-user-flow | 1 | Runs an ended meeting through real Stop -> Resume -> Stop transitions in the isolated desktop, holds both mutations to prove their pending receipts, reconciles a delayed stale started event, and captures saved-finalizing, transcript, resumed-live, the anchored Notes receipt, the Summary draft surface, ready, and failed states without launching hosted AI. |
| meeting-workspace-tabs.spec.ts | windows, macos, linux | real-ui-e2e, local-api | meeting-notes | high | strong | real-user-flow | 1 | Seeds a live meeting with notes and summary, proves the saved summary is the preferred outcome view, switches persistent tabs, resizes to 640x560, checks footer separation and horizontal overflow, captures a review screenshot, then stops from the transcript and verifies the Summary tab opens. |
| meetings-only-audio-lifecycle.spec.ts | windows, macos | audio-device, local-api, real-ui-e2e | meetings-only-audio-lifecycle, audio-device-health | high | conditional | real-user-flow | 1 | Opt-in real-audio lifecycle lane: configured devices stay closed outside meetings and open only across a manual meeting edge. |
diff --git a/apps/screenpipe-app-tauri/e2e/specs/meeting-restart-live-transcription.spec.ts b/apps/screenpipe-app-tauri/e2e/specs/meeting-restart-live-transcription.spec.ts
index 7cb423f5a..7fbf39046 100644
--- a/apps/screenpipe-app-tauri/e2e/specs/meeting-restart-live-transcription.spec.ts
+++ b/apps/screenpipe-app-tauri/e2e/specs/meeting-restart-live-transcription.spec.ts
@@ -37,7 +37,7 @@
* nothing closes the meeting and the assertion passes even against the bug.
*
* Run:
- * SCREENPIPE_E2E_SEED=onboarding,capture-restart-devices \
+ * SCREENPIPE_E2E_SEED=onboarding,capture-restart-devices,db-hard-fault \
* bun run wdio run e2e/wdio.conf.ts \
* --spec e2e/specs/meeting-restart-live-transcription.spec.ts
*/
@@ -151,4 +151,17 @@ describe("live transcription survives a recording restart", function () {
await saveScreenshot("meeting-restart-live-transcription");
});
+
+ it("reattaches live transcription after database write-wedge recovery", async function () {
+ if (!E2E_SEED_FLAGS.split(",").includes("db-hard-fault")) this.skip();
+ expect(await activeMeetingId()).toBe(meetingId);
+ await browser.setTimeout({ script: t(120_000) });
+ const event = await invokeOrThrow<{ meeting_id: number }>(
+ "plugin:e2e|recover_meeting_from_db_wedge",
+ );
+ expect(event.meeting_id).toBe(meetingId);
+ await waitForLocalApi(cfg, t(45_000));
+ expect(await activeMeetingId()).toBe(meetingId);
+ await saveScreenshot("meeting-db-recovery-reattached");
+ });
});
diff --git a/apps/screenpipe-app-tauri/src-tauri/build.rs b/apps/screenpipe-app-tauri/src-tauri/build.rs
index 9d7c67890..7eb632f8a 100644
--- a/apps/screenpipe-app-tauri/src-tauri/build.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/build.rs
@@ -561,6 +561,7 @@ const E2E_COMMANDS: &[&str] = &[
"owned_browser_tab_snapshot",
"inject_db_hard_fault",
"inject_db_transient_fault",
+ "recover_meeting_from_db_wedge",
"db_retry_write_probe",
"db_hard_fault_state",
"seed_flags",
diff --git a/apps/screenpipe-app-tauri/src-tauri/src/diagnostic_logs.rs b/apps/screenpipe-app-tauri/src-tauri/src/diagnostic_logs.rs
index 0468d5279..66ca0f5bb 100644
--- a/apps/screenpipe-app-tauri/src-tauri/src/diagnostic_logs.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/src/diagnostic_logs.rs
@@ -220,6 +220,56 @@ async fn build_bundle(files: &[LogFile]) -> String {
mod tests {
use super::*;
+ #[tokio::test]
+ #[cfg(feature = "e2e")]
+ async fn abandoned_read_recovery_survives_support_rotation_and_redaction() {
+ use tracing::instrument::WithSubscriber;
+ let root = tempfile::tempdir().unwrap();
+ let logs = root.path().join("logs");
+ std::fs::create_dir(&logs).unwrap();
+ let current = logs.join("screenpipe-app.2026-09-22.log");
+ let subscriber = tracing_subscriber::fmt()
+ .with_ansi(false)
+ .with_writer(std::fs::File::create(&current).unwrap())
+ .finish();
+ async {
+ let db = screenpipe_db::DatabaseManager::new(
+ root.path().join("db.sqlite").to_str().unwrap(),
+ Default::default(),
+ )
+ .await
+ .unwrap();
+ let mut read = db
+ .acquire_cancellable_read(
+ std::time::Instant::now() + Duration::from_secs(30),
+ Default::default(),
+ )
+ .await
+ .unwrap();
+ // Simulate the actual failure boundary: raw BEGIN has executed,
+ // but the owner disappears without rolling its transaction back.
+ sqlx::query("BEGIN").execute(&mut *read).await.unwrap();
+ sqlx::query("SELECT COUNT(*) FROM meetings")
+ .fetch_one(&mut *read)
+ .await
+ .unwrap();
+ drop(read);
+ db.close().await;
+ }
+ .with_subscriber(subscriber)
+ .await;
+ std::fs::rename(&current, logs.join("screenpipe-app.2026-09-22.1.log")).unwrap();
+ std::fs::write(
+ &current,
+ "recorder restarted\[email protected]\n",
+ )
+ .unwrap();
+ let report = collect_redacted_from_dirs(&[logs]).await.unwrap();
+ assert!(report.contains("abandoned SQLite read transaction rolled back before pool reuse"));
+ assert!(report.contains("released WAL snapshot without stopping recording"));
+ assert!(!report.contains("[email protected]"));
+ }
+
#[tokio::test]
async fn capture_pause_cause_and_resume_survive_support_collection() {
// The native focus-warm-pause E2E also asserts these actual messages.
diff --git a/apps/screenpipe-app-tauri/src-tauri/src/e2e/commands.rs b/apps/screenpipe-app-tauri/src-tauri/src/e2e/commands.rs
index a8363f7e9..d84e2a088 100644
--- a/apps/screenpipe-app-tauri/src-tauri/src/e2e/commands.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/src/e2e/commands.rs
@@ -839,6 +839,14 @@ async fn inject_db_transient_fault(
super::seeds::inject_db_transient_fault(state).await
}
+#[command]
+async fn recover_meeting_from_db_wedge(
+ app: tauri::AppHandle,
+ state: State<'_, RecordingState>,
+) -> Result<serde_json::Value, String> {
+ super::seeds::recover_meeting_from_db_wedge(app, state).await
+}
+
#[command]
async fn db_retry_write_probe(
state: State<'_, RecordingState>,
@@ -975,6 +983,7 @@ pub(super) fn plugin() -> TauriPlugin<Wry> {
owned_browser_tab_snapshot,
inject_db_hard_fault,
inject_db_transient_fault,
+ recover_meeting_from_db_wedge,
db_retry_write_probe,
db_hard_fault_state,
seed_flags,
diff --git a/apps/screenpipe-app-tauri/src-tauri/src/e2e/seeds.rs b/apps/screenpipe-app-tauri/src-tauri/src/e2e/seeds.rs
index 48cdb55e4..c9faef757 100644
--- a/apps/screenpipe-app-tauri/src-tauri/src/e2e/seeds.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/src/e2e/seeds.rs
@@ -764,6 +764,35 @@ pub(super) async fn inject_db_transient_fault(
}))
}
+/// Trigger the real write-wedge lifecycle, observing the same reattach event
+/// the live transcription coordinator consumes. Never available outside E2E.
+pub(super) async fn recover_meeting_from_db_wedge(
+ app: AppHandle,
+ state: State<'_, RecordingState>,
+) -> Result<serde_json::Value, String> {
+ use futures::StreamExt;
+ require_isolated_db_hard_fault_seed()?;
+ let health = state
+ .server
+ .lock()
+ .await
+ .as_ref()
+ .ok_or("embedded server is not running")?
+ .db
+ .write_queue_health();
+ let mut reattach = screenpipe_events::subscribe_to_event::<serde_json::Value>(
+ screenpipe_audio::meeting_streaming::MEETING_STREAMING_REATTACH_EVENT,
+ );
+ let hook =
+ crate::recording::make_database_restart_hook(app, state.db_wedge_breaker.clone(), health);
+ hook(screenpipe_db::DatabaseRestartReason::PersistentWriteFailure);
+ timeout(TokioDuration::from_secs(90), reattach.next())
+ .await
+ .map_err(|_| "database recovery never reattached the live meeting".to_string())?
+ .map(|event| event.data)
+ .ok_or_else(|| "meeting reattach subscription ended".to_string())
+}
+
/// A new production queue write after retry must be visible from a freshly
/// opened query-only SQLite connection, alongside the pre-fault sentinel.
pub(super) async fn db_retry_write_probe(
diff --git a/apps/screenpipe-app-tauri/src-tauri/src/recording.rs b/apps/screenpipe-app-tauri/src-tauri/src/recording.rs
index 9c29317d1..730fa551c 100644
--- a/apps/screenpipe-app-tauri/src-tauri/src/recording.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/src/recording.rs
@@ -579,16 +579,23 @@ async fn remember_active_meeting_for_capture_restart(state: &RecordingState) {
let Some(server) = server_guard.as_ref() else {
return;
};
+ remember_active_meeting_from_server(state, server).await;
+}
+/// Also used after DB recovery takes the server out of the shared slot, before
+/// stopping the watcher closes its meeting and disconnects live transcription.
+async fn remember_active_meeting_from_server(state: &RecordingState, server: &ServerCore) {
let manual_id = *server.manual_meeting.read().await;
let meeting = match manual_id {
- Some(id) => server.db.get_active_meeting_by_id(id).await.ok().flatten(),
- None => server
- .db
- .get_most_recent_active_meeting()
- .await
- .ok()
- .flatten(),
+ Some(id) => server.db.get_active_meeting_by_id(id).await,
+ None => server.db.get_most_recent_active_meeting().await,
+ };
+ let meeting = match meeting {
+ Ok(meeting) => meeting,
+ Err(error) => {
+ warn!(%error, "could not preserve active meeting for live transcription across recording recovery");
+ None
+ }
};
let Some(meeting) = meeting else {
diff --git a/apps/screenpipe-app-tauri/src-tauri/src/recording/db_wedge.rs b/apps/screenpipe-app-tauri/src-tauri/src/recording/db_wedge.rs
index 0b85ce099..62e03de78 100644
--- a/apps/screenpipe-app-tauri/src-tauri/src/recording/db_wedge.rs
+++ b/apps/screenpipe-app-tauri/src-tauri/src/recording/db_wedge.rs
@@ -234,7 +234,19 @@ async fn recover_from_db_wedge(
"database lifecycle recovery requested — restarting recording to rebuild all DB pools + the shared WAL-index"
);
- *recording_state.interrupted_meeting.lock().await = None;
+ if hard_faulted {
+ // A verified SQLite fault forbids further reads from this generation.
+ *recording_state.interrupted_meeting.lock().await = None;
+ } else if tokio::time::timeout(
+ DB_WEDGE_SERVER_SHUTDOWN_TIMEOUT,
+ super::remember_active_meeting_from_server(&recording_state, &server),
+ )
+ .await
+ .is_err()
+ {
+ *recording_state.interrupted_meeting.lock().await = None;
+ warn!("active meeting lookup timed out during database recovery; live transcription could not be preserved");
+ }
if let Some(session) = capture {
session.stop().await;
}
diff --git a/crates/screenpipe-db/src/cancellable_query.rs b/crates/screenpipe-db/src/cancellable_query.rs
index 6ccbd8271..7795512e3 100644
--- a/crates/screenpipe-db/src/cancellable_query.rs
+++ b/crates/screenpipe-db/src/cancellable_query.rs
@@ -18,7 +18,7 @@ use std::sync::atomic::{AtomicU8, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio_util::sync::CancellationToken;
-use tracing::warn;
+use tracing::{instrument::WithSubscriber, warn};
/// Number of SQLite virtual-machine instructions between cancellation checks.
///
@@ -166,7 +166,8 @@ impl CancellableReadConnection {
// The cleanup task owns the connection before this method awaits. If
// the caller is cancelled while awaiting the JoinHandle, cleanup keeps
// running and a stale progress handler still cannot reach the pool.
- let cleanup = tokio::spawn(clean_and_return_connection(connection));
+ let cleanup =
+ tokio::spawn(clean_and_return_connection(connection).with_current_subscriber());
cleanup
.await
.map_err(|error| sqlx::Error::Protocol(format!("query cleanup task failed: {error}")))?
@@ -208,14 +209,17 @@ impl Drop for CancellableReadConnection {
return;
};
- runtime.spawn(async move {
- if let Err(error) = clean_and_return_connection(connection).await {
- warn!(
- error = %error,
- "failed to clear SQLite progress handler; connection closed"
- );
+ runtime.spawn(
+ async move {
+ if let Err(error) = clean_and_return_connection(connection).await {
+ warn!(
+ error = %error,
+ "failed to clean up SQLite read connection; connection closed"
+ );
+ }
}
- });
+ .with_current_subscriber(),
+ );
}
}
@@ -251,18 +255,27 @@ pub fn is_sqlite_interrupt(error: &sqlx::Error) -> bool {
}
}
-async fn remove_progress_handler(
- connection: &mut PoolConnection<Sqlite>,
-) -> Result<(), sqlx::Error> {
- let mut handle = connection.lock_handle().await?;
- handle.remove_progress_handler();
+async fn clean_read_connection(connection: &mut PoolConnection<Sqlite>) -> Result<(), sqlx::Error> {
+ let in_transaction = {
+ let mut handle = connection.lock_handle().await?;
+ handle.remove_progress_handler();
+ // SAFETY: the handle guard exclusively owns this connection.
+ unsafe { libsqlite3_sys::sqlite3_get_autocommit(handle.as_raw_handle().as_ptr()) == 0 }
+ };
+ // Cancellation can land after SQLite executes a raw BEGIN but before
+ // the snapshot owner is constructed. SQLx does not track that transaction.
+ // Nobody owns this read now: release its snapshot before pool reuse.
+ if in_transaction {
+ sqlx::query("ROLLBACK").execute(&mut **connection).await?;
+ warn!("abandoned SQLite read transaction rolled back before pool reuse; released WAL snapshot without stopping recording");
+ }
Ok(())
}
async fn clean_and_return_connection(
mut connection: PoolConnection<Sqlite>,
) -> Result<(), sqlx::Error> {
- match remove_progress_handler(&mut connection).await {
+ match clean_read_connection(&mut connection).await {
Ok(()) => {
// Drive SQLx's pool-return path to completion instead of merely
// dropping the connection and spawning another detached task.
@@ -270,8 +283,8 @@ async fn clean_and_return_connection(
Ok(())
}
Err(error) => {
- // A connection whose handler could not be cleared must never be
- // reused. Closing lets the pool create a clean replacement.
+ // A stale handler or open transaction must never reach the next
+ // borrower. Closing lets the pool create a clean replacement.
let _ = connection.close().await;
Err(error)
}
@@ -287,6 +300,79 @@ mod tests {
const LONG_QUERY: &str = "WITH RECURSIVE counter(value) AS (VALUES(0) UNION ALL SELECT value + 1 FROM counter WHERE value < 100000000) SELECT sum(value) FROM counter";
const HANDLER_PROBE_QUERY: &str = "WITH RECURSIVE counter(value) AS (VALUES(0) UNION ALL SELECT value + 1 FROM counter WHERE value < 2000000) SELECT sum(value) FROM counter";
+ #[tokio::test]
+ async fn abandoned_read_transaction_does_not_pin_recording_wal() {
+ let dir = tempfile::tempdir().unwrap();
+ let mut config = screenpipe_config::DbConfig::default();
+ config.read_pool_max = 1;
+ config.read_pool_min = 1;
+ let db = DatabaseManager::new(dir.path().join("db.sqlite").to_str().unwrap(), config)
+ .await
+ .unwrap();
+ db.execute_raw_sql_write("CREATE TABLE capture_probe(id INTEGER PRIMARY KEY, value TEXT)")
+ .await
+ .unwrap();
+ db.wal_checkpoint().await.unwrap();
+
+ let (ready, began) = tokio::sync::oneshot::channel();
+ let pool = db.pool.clone();
+ let request = tokio::spawn(async move {
+ let mut connection = CancellableReadConnection::acquire(
+ &pool,
+ Instant::now() + Duration::from_secs(30),
+ CancellationToken::new(),
+ )
+ .await
+ .unwrap();
+ // The raw BEGIN used by hybrid snapshots is not a SQLx Transaction.
+ // Abort while the request owns it, before an explicit rollback.
+ sqlx::query("BEGIN")
+ .execute(&mut *connection)
+ .await
+ .unwrap();
+ sqlx::query("SELECT count(*) FROM capture_probe")
+ .fetch_one(&mut *connection)
+ .await
+ .unwrap();
+ ready.send(()).unwrap();
+ std::future::pending::<()>().await;
+ });
+ began.await.unwrap();
+ db.execute_raw_sql_write("INSERT INTO capture_probe(value) VALUES('during read')")
+ .await
+ .unwrap();
+ let (_, pages, checkpointed) = db.wal_checkpoint().await.unwrap();
+ assert!(
+ pages > checkpointed,
+ "the read must actually pin WAL frames"
+ );
+ request.abort();
+ let _ = request.await;
+
+ // Acquiring the sole read slot waits for cancellation cleanup. Keep
+ // the returned connection idle to expose a leaked transaction.
+ let idle_reader = db.pool.acquire().await.unwrap();
+ db.execute_raw_sql_write("INSERT INTO capture_probe(value) VALUES('after cancellation')")
+ .await
+ .unwrap();
+ let (busy, pages, checkpointed) = db.wal_checkpoint().await.unwrap();
+ assert_eq!(busy, 0);
+ assert_eq!(
+ pages, checkpointed,
+ "abandoned reader pinned the capture WAL"
+ );
+ drop(idle_reader);
+ let count: i64 = sqlx::query_scalar("SELECT count(*) FROM capture_probe")
+ .fetch_one(&db.pool)
+ .await
+ .unwrap();
+ assert_eq!(
+ count, 2,
+ "recording must persist through reader cancellation"
+ );
+ db.close().await;
+ }
+
#[tokio::test]
async fn aborting_release_cannot_return_a_stale_handler() {
let pool = SqlitePoolOptions::new()
diff --git a/crates/screenpipe-db/src/db/maintenance.rs b/crates/screenpipe-db/src/db/maintenance.rs
index 4028eb4f0..7b247302c 100644
--- a/crates/screenpipe-db/src/db/maintenance.rs
+++ b/crates/screenpipe-db/src/db/maintenance.rs
@@ -4,29 +4,44 @@
use super::*;
-const WAL_HARD_CAP_PAGES: i32 = 40_000;
+// Diagnostic threshold only: backlog does not establish a recording failure.
+const WAL_BACKLOG_WARNING_PAGES: i32 = 40_000;
-fn wal_backlog_restart_reason(
+fn report_wal_checkpoint(
+ busy: i32,
log_pages: i32,
- checkpointed_pages: i32,
-) -> Option<crate::write_queue::DatabaseRestartReason> {
- let pending_pages = log_pages.saturating_sub(checkpointed_pages);
- (pending_pages > WAL_HARD_CAP_PAGES).then_some(
- crate::write_queue::DatabaseRestartReason::WalBacklog {
+ checkpointed: i32,
+ backlog_since: &mut Option<std::time::Instant>,
+) {
+ let pending_pages = log_pages.saturating_sub(checkpointed);
+ if pending_pages > WAL_BACKLOG_WARNING_PAGES {
+ let since = backlog_since.get_or_insert_with(std::time::Instant::now);
+ warn!(
pending_pages,
log_pages,
- checkpointed_pages,
- },
- )
+ checkpointed,
+ busy,
+ backlog_seconds = since.elapsed().as_secs(),
+ "passive WAL checkpoint deferred; recording continues while reader snapshots drain"
+ );
+ } else if pending_pages == 0 && backlog_since.take().is_some() {
+ info!(
+ log_pages,
+ checkpointed, "WAL backlog cleared; recording continued without an engine restart"
+ );
+ } else {
+ debug!(
+ busy,
+ log_pages, checkpointed, pending_pages, "passive WAL checkpoint"
+ );
+ }
}
async fn run_routine_wal_checkpoint(pool: &SqlitePool) -> Result<(i32, i32, i32), sqlx::Error> {
- // Routine maintenance must never shorten the live WAL file. A TRUNCATE
- // checkpoint can make an already-open connection short-read the old WAL
- // extent if its wal-index generation is stale. PASSIVE still copies every
- // safe frame into the main database, but leaves WAL reuse/reset to SQLite's
- // normal writer path instead of physically truncating the file underneath
- // the app's many long-lived readers.
+ // PASSIVE copies available frames without waiting for readers. FULL,
+ // RESTART and TRUNCATE can hold SQLite's writer lock while waiting for
+ // readers, delaying recording. SQLite coordinates WAL reuse itself;
+ // an incomplete checkpoint does not require an engine restart.
let row = sqlx::query("PRAGMA wal_checkpoint(PASSIVE)")
.fetch_one(pool)
.await?;
@@ -1367,26 +1382,25 @@ impl DatabaseManager {
)
}
- /// Spawn the background task that owns ALL WAL checkpointing.
+ /// Spawn the background task that owns routine WAL checkpointing.
///
/// Since `wal_autocheckpoint = 0` (see [`WAL_SAFETY_PRAGMAS`]), committing
/// connections do not checkpoint inline. This task therefore owns routine
- /// checkpointing: it runs a non-truncating `PASSIVE` checkpoint often enough
- /// to copy safe frames into the main database without shortening the WAL
- /// underneath long-lived readers. No live path truncates the WAL; physical
- /// cleanup belongs exclusively to offline recovery after all owners close.
+ /// checkpointing with `PASSIVE`, allowing readers to defer checkpoint work
+ /// while recording continues. SQLite can reuse the WAL after those readers
+ /// finish; backlog alone must not interrupt capture.
pub fn start_wal_maintenance(&self) {
let pool = self.write_pool.clone();
let shutdown = self.close_token.clone();
let write_queue_health = self.write_queue_health.clone();
let write_semaphore = std::sync::Arc::clone(&self.write_semaphore);
- let persistent_failure_hook = self.persistent_failure_hook.clone();
tokio::spawn(async move {
// 60s (not 300s): with inline auto-checkpoint off, the WAL grows for
// the whole interval between ticks, so check more often to keep it
// small under sustained write load.
const INTERVAL: Duration = Duration::from_secs(60);
let mut interval = tokio::time::interval(INTERVAL);
+ let mut backlog_since = None;
// `interval()` yields its first tick immediately. Startup has just
// run a serialized checkpoint, so consume that tick and wait a
// full interval instead of racing callers' first transactions.
@@ -1403,10 +1417,8 @@ impl DatabaseManager {
}
}
- // The upstream WAL-reset race requires a checkpoint and write
- // to overlap on independent connections. Every routine pass,
- // including the common below-cap path, shares the same
- // process-wide coordinator as every capture writer.
+ // Share the process-wide writer coordinator so maintenance
+ // participates in the same ordering and shutdown as capture.
let _write_guard = tokio::select! {
permit = Arc::clone(&write_semaphore).acquire_owned() => {
match permit {
@@ -1424,30 +1436,9 @@ impl DatabaseManager {
};
match run_guarded_routine_wal_checkpoint(&pool, &write_queue_health).await {
Ok(Some((busy, log_pages, checkpointed))) => {
- let backlog_pages = log_pages.saturating_sub(checkpointed);
- if let Some(reason) = wal_backlog_restart_reason(log_pages, checkpointed) {
- warn!(
- "passive wal checkpoint left {} pages pending; requesting a full engine restart before WAL reuse (log={}, checkpointed={}, busy={})",
- backlog_pages, log_pages, checkpointed, busy
- );
- // The writer coordinator cannot quiesce query-pool
- // readers or an external opener, so a serialized
- // RESTART is not a complete lifecycle boundary. Ask
- // the owner to stop capture, drain readers, close
- // both pools, and respawn; startup checkpoints before
- // any traffic is admitted.
- if persistent_failure_hook.signal_wal_backlog_restart(reason) {
- return;
- }
- warn!(
- "wal hard cap exceeded but no lifecycle restart hook is installed; staying on safe PASSIVE checkpoints"
- );
- } else {
- debug!(
- "passive wal checkpoint: busy={}, checkpointed {}/{} pages ({} pending)",
- busy, checkpointed, log_pages, backlog_pages
- );
- }
+ // Backlog is deferred checkpoint work, not a failed
+ // capture writer. Keep recording and retry PASSIVE.
+ report_wal_checkpoint(busy, log_pages, checkpointed, &mut backlog_since);
}
Ok(None) => {
error!(
@@ -1493,6 +1484,7 @@ impl DatabaseManager {
pub(crate) fn spawn_startup_integrity_check(&self, database_path: Arc<str>) {
let pool = self.startup_integrity_pool();
+ let persistent_failure_hook = self.persistent_failure_hook.clone();
crate::recovery::register_database_pool(std::path::Path::new(&*database_path), &pool);
let timeout = Duration::from_secs(self.storage.as_ref().map_or_else(
|| crate::storage::StorageBudget::default().lifecycle_timeout_secs,
@@ -1500,7 +1492,6 @@ impl DatabaseManager {
));
let health = self.write_queue_health.clone();
let shutdown = self.close_token.clone();
- let persistent_failure_hook = self.persistent_failure_hook.clone();
tokio::spawn(async move {
// Let boot settle so the scan doesn't compete with migrations
// and the first capture writes for I/O.
@@ -1645,11 +1636,7 @@ impl DatabaseManager {
#[cfg(test)]
mod wal_maintenance_tests {
- use super::{
- run_guarded_routine_wal_checkpoint, run_routine_wal_checkpoint, wal_backlog_restart_reason,
- DatabaseManager, WAL_HARD_CAP_PAGES,
- };
- use crate::write_queue::{persistent_failure_slot, DatabaseRestartReason};
+ use super::{run_guarded_routine_wal_checkpoint, run_routine_wal_checkpoint, DatabaseManager};
use screenpipe_config::{DbConfig, DeviceTier};
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions};
use sqlx::Row;
@@ -1978,117 +1965,9 @@ mod wal_maintenance_tests {
pool.close().await;
}
- #[tokio::test]
- async fn reader_pinned_backlog_requests_one_lifecycle_restart_without_resetting_wal() {
- let dir = tempfile::tempdir().expect("temp dir");
- let db_path = dir.path().join("db.sqlite");
- let wal_path = dir.path().join("db.sqlite-wal");
- let options = SqliteConnectOptions::new()
- .filename(&db_path)
- .create_if_missing(true)
- .journal_mode(SqliteJournalMode::Wal)
- .pragma("wal_autocheckpoint", "0")
- .busy_timeout(std::time::Duration::from_secs(5));
- let pool = SqlitePoolOptions::new()
- .min_connections(3)
- .max_connections(3)
- .connect_with(options)
- .await
- .expect("open WAL database");
-
- sqlx::query("CREATE TABLE events (id INTEGER PRIMARY KEY, body TEXT NOT NULL)")
- .execute(&pool)
- .await
- .expect("create table");
- run_routine_wal_checkpoint(&pool)
- .await
- .expect("checkpoint setup frames");
-
- let mut reader = pool.begin().await.expect("begin reader");
- let _: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM events")
- .fetch_one(&mut *reader)
- .await
- .expect("establish reader snapshot");
-
- for id in 0..128 {
- sqlx::query("INSERT INTO events (body) VALUES (?1)")
- .bind(format!("event-{id}-{}", "x".repeat(1024)))
- .execute(&pool)
- .await
- .expect("insert event");
- }
- let wal_size_before = std::fs::metadata(&wal_path)
- .expect("WAL exists before checkpoint")
- .len();
-
- let (_, log_pages, checkpointed) = run_routine_wal_checkpoint(&pool)
- .await
- .expect("passive checkpoint with reader");
- assert!(
- log_pages > checkpointed,
- "reader must leave frames pending for the escalation test"
- );
-
- let observed = Arc::new(Mutex::new(Vec::new()));
- let observed_from_hook = Arc::clone(&observed);
- let restart_hook = persistent_failure_slot(Some(Arc::new(move |reason| {
- observed_from_hook.lock().unwrap().push(reason);
- })));
- let reason = DatabaseRestartReason::WalBacklog {
- pending_pages: log_pages.saturating_sub(checkpointed),
- log_pages,
- checkpointed_pages: checkpointed,
- };
- assert!(restart_hook.signal_wal_backlog_restart(reason));
- assert!(restart_hook.signal_wal_backlog_restart(reason));
- assert_eq!(
- observed.lock().unwrap().as_slice(),
- &[reason],
- "one manager generation must request at most one lifecycle restart"
- );
-
- let wal_size_after = std::fs::metadata(&wal_path)
- .expect("WAL remains allocated after restart request")
- .len();
- assert_eq!(
- wal_size_after, wal_size_before,
- "requesting lifecycle recovery must not reset the live WAL"
- );
-
- sqlx::query("INSERT INTO events (body) VALUES ('before-lifecycle-handoff')")
- .execute(&pool)
- .await
- .expect("restart request itself must not mutate or block SQLite");
- reader.rollback().await.expect("release reader snapshot");
- let integrity: String = sqlx::query_scalar("PRAGMA integrity_check")
- .fetch_one(&pool)
- .await
- .expect("integrity check");
- assert_eq!(integrity, "ok");
-
- pool.close().await;
- }
-
- #[test]
- fn wal_backlog_policy_preserves_the_existing_hard_cap() {
- assert_eq!(
- wal_backlog_restart_reason(WAL_HARD_CAP_PAGES, 0),
- None,
- "the ceiling is exclusive"
- );
- assert_eq!(
- wal_backlog_restart_reason(WAL_HARD_CAP_PAGES + 1, 0),
- Some(DatabaseRestartReason::WalBacklog {
- pending_pages: WAL_HARD_CAP_PAGES + 1,
- log_pages: WAL_HARD_CAP_PAGES + 1,
- checkpointed_pages: 0,
- })
- );
- }
-
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
#[ignore = "manual 60-second production scheduler chaos test with a ~170 MB WAL"]
- async fn production_scheduler_requests_lifecycle_for_oversized_reader_pinned_wal() {
+ async fn production_scheduler_preserves_recording_through_reader_pinned_wal() {
let dir = tempfile::tempdir().expect("temp dir");
let db_path = dir.path().join("db.sqlite");
let wal_path = dir.path().join("db.sqlite-wal");
@@ -2096,13 +1975,16 @@ mod wal_maintenance_tests {
let db = DatabaseManager::new(&db_path_string, DbConfig::for_tier(DeviceTier::Low))
.await
.expect("production database manager");
- sqlx::query("CREATE TABLE wal_cap_chaos(id INTEGER PRIMARY KEY, payload BLOB NOT NULL)")
- .execute(&db.pool)
- .await
- .expect("create chaos table");
- run_routine_wal_checkpoint(&db.pool)
+ db.execute_raw_sql_write(
+ "CREATE TABLE wal_cap_chaos(id INTEGER PRIMARY KEY, payload BLOB NOT NULL)",
+ )
+ .await
+ .expect("create chaos table");
+ let meeting_id = db
+ .insert_meeting("Teams", "audio_process", Some("WAL pressure test"), None)
.await
- .expect("checkpoint setup frames");
+ .unwrap();
+ db.wal_checkpoint().await.expect("checkpoint setup frames");
let mut reader = db.pool.begin().await.expect("begin pinned reader");
let _: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM wal_cap_chaos")
@@ -2110,24 +1992,24 @@ mod wal_maintenance_tests {
.await
.expect("establish reader snapshot");
- // One transaction creates more than WAL_HARD_CAP_PAGES real pages while
+ // One transaction creates more than WAL_BACKLOG_WARNING_PAGES real pages while
// the old reader pins the first frame. This exercises the exact
- // production timer + threshold + lifecycle-handoff branch after 60 seconds.
- sqlx::query(
+ // production timer and backlog branch after 60 seconds.
+ db.execute_raw_sql_write(
"WITH RECURSIVE rows(n) AS (\
VALUES(1) UNION ALL SELECT n + 1 FROM rows WHERE n < 42000\
) INSERT INTO wal_cap_chaos(payload) SELECT zeroblob(4096) FROM rows",
)
- .execute(&db.pool)
.await
.expect("create oversized WAL");
- let (_, log_pages, checkpointed) = run_routine_wal_checkpoint(&db.pool)
+ let (_, log_pages, checkpointed) = db
+ .wal_checkpoint()
.await
.expect("measure reader-pinned backlog");
let backlog = log_pages.saturating_sub(checkpointed);
assert!(
- backlog > super::WAL_HARD_CAP_PAGES,
- "test did not exceed hard cap: backlog={backlog}, log={log_pages}, checkpointed={checkpointed}"
+ backlog > super::WAL_BACKLOG_WARNING_PAGES,
+ "test did not exceed backlog warning threshold: backlog={backlog}, log={log_pages}, checkpointed={checkpointed}"
);
let wal_size_before = std::fs::metadata(&wal_path)
.expect("oversized WAL exists")
@@ -2137,31 +2019,65 @@ mod wal_maintenance_tests {
"WAL was not realistically large"
);
- let (restart_tx, restart_rx) = tokio::sync::oneshot::channel();
- let restart_tx = Arc::new(Mutex::new(Some(restart_tx)));
+ let restarts = Arc::new(Mutex::new(Vec::new()));
+ let observed = restarts.clone();
db.set_database_restart_hook(Arc::new(move |reason| {
- if let Some(tx) = restart_tx.lock().unwrap().take() {
- let _ = tx.send(reason);
- }
+ observed.lock().unwrap().push(reason)
}));
- let reason = tokio::time::timeout(Duration::from_secs(65), restart_rx)
+ // Cross a real production maintenance tick while captures keep using
+ // the coordinated writer. Backpressure must not trigger a lifecycle reset.
+ for index in 0..65 {
+ db.insert_meeting_transcript_segment(
+ meeting_id,
+ "e2e",
+ None,
+ &format!("segment-{index}"),
+ "microphone",
+ "input",
+ None,
+ "live transcript during WAL pressure",
+ chrono::Utc::now(),
+ )
.await
- .expect("scheduled lifecycle request timed out")
- .expect("scheduled lifecycle sender dropped");
- assert!(matches!(
- reason,
- DatabaseRestartReason::WalBacklog { pending_pages, .. }
- if pending_pages > WAL_HARD_CAP_PAGES
- ));
+ .expect("live transcript remains durable during checkpoint pressure");
+ tokio::time::sleep(Duration::from_secs(1)).await;
+ }
+ assert!(
+ restarts.lock().unwrap().is_empty(),
+ "backlog interrupted recording"
+ );
let wal_size_after = std::fs::metadata(&wal_path)
.expect("WAL remains allocated")
.len();
- assert_eq!(
- wal_size_after, wal_size_before,
- "scheduled hard-cap handling reset the live WAL"
+ assert!(
+ wal_size_after >= wal_size_before,
+ "maintenance truncated the reader-pinned WAL"
);
reader.rollback().await.expect("release pinned reader");
+ let (busy, log, checkpointed) = db.wal_checkpoint().await.unwrap();
+ assert_eq!(
+ (busy, log),
+ (0, checkpointed),
+ "backlog did not drain after reader release"
+ );
db.close().await;
+ let reopened = DatabaseManager::new(&db_path_string, DbConfig::for_tier(DeviceTier::Low))
+ .await
+ .unwrap();
+ let count: i64 = sqlx::query_scalar(
+ "SELECT COUNT(*) FROM meeting_transcript_segments WHERE meeting_id = ?1",
+ )
+ .bind(meeting_id)
+ .fetch_one(&reopened.pool)
+ .await
+ .unwrap();
+ assert_eq!(count, 65, "captures must survive close and reopen");
+ let integrity: String = sqlx::query_scalar("PRAGMA integrity_check")
+ .fetch_one(&reopened.pool)
+ .await
+ .unwrap();
+ assert_eq!(integrity, "ok");
+ reopened.close().await;
}
}
diff --git a/crates/screenpipe-db/src/db/setup.rs b/crates/screenpipe-db/src/db/setup.rs
index 3e1891a7f..7d2336883 100644
--- a/crates/screenpipe-db/src/db/setup.rs
+++ b/crates/screenpipe-db/src/db/setup.rs
@@ -585,14 +585,8 @@ impl DatabaseManager {
db_manager.spawn_startup_integrity_check(Arc::from(database_path));
}
- // Periodic WAL checkpoint so the write-ahead log can't grow unbounded
- // when passive auto-checkpoint is blocked by long-lived readers. An
- // oversized WAL (observed at 650MB on a heavy 24/7 install) is the main
- // driver of the WAL-index / `-shm` desync that corrupts the DB. Started
- // here in `new()` — next to the integrity check — so EVERY caller gets
- // it: the desktop app runs the engine in-process and previously never
- // started it (only the standalone `screenpipe-engine` CLI did), so app
- // users got no periodic checkpointing at all.
+ // Every recorder owns periodic PASSIVE checkpointing. Active readers
+ // can defer that work, but checkpoint pressure must not stop capture.
if background {
db_manager.start_wal_maintenance();
}
diff --git a/crates/screenpipe-db/src/write_queue.rs b/crates/screenpipe-db/src/write_queue.rs
index 538647e4c..b2574097c 100644
--- a/crates/screenpipe-db/src/write_queue.rs
+++ b/crates/screenpipe-db/src/write_queue.rs
@@ -480,19 +480,10 @@ impl WriteQueueHealth {
}
/// Why the database layer needs the owning process to rebuild every live pool.
-///
-/// A WAL backlog is intentionally handled at the same lifecycle boundary as a
-/// write wedge: only the owner can stop capture, drain readers, close every
-/// connection, and reopen SQLite against a fresh WAL-index generation.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum DatabaseRestartReason {
PersistentWriteFailure,
SqliteHardFault,
- WalBacklog {
- pending_pages: i32,
- log_pages: i32,
- checkpointed_pages: i32,
- },
}
/// Hook used by the database layer to ask its owner for a full lifecycle
@@ -509,7 +500,6 @@ pub type PersistentFailureHook = Arc<dyn Fn() + Send + Sync>;
pub(crate) struct PersistentFailureState {
hook: std::sync::Mutex<Option<DatabaseRestartHook>>,
hard_fault_signaled: AtomicBool,
- wal_backlog_signaled: AtomicBool,
}
pub(crate) type PersistentFailureSlot = Arc<PersistentFailureState>;
@@ -533,33 +523,12 @@ impl PersistentFailureState {
.ok()
.map(|_| hook)
}
-
- /// Deliver at most one WAL-backlog restart request for this manager.
- /// Leave the gate open until a hook is installed so embedders that wire the
- /// lifecycle after construction can still receive the request on a later
- /// maintenance pass.
- pub(crate) fn signal_wal_backlog_restart(&self, reason: DatabaseRestartReason) -> bool {
- debug_assert!(matches!(reason, DatabaseRestartReason::WalBacklog { .. }));
- let Some(hook) = self.hook() else {
- return false;
- };
- if self
- .wal_backlog_signaled
- .compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
- .is_err()
- {
- return true;
- }
- hook(reason);
- true
- }
}
pub(crate) fn persistent_failure_slot(hook: Option<DatabaseRestartHook>) -> PersistentFailureSlot {
Arc::new(PersistentFailureState {
hook: std::sync::Mutex::new(hook),
hard_fault_signaled: AtomicBool::new(false),
- wal_backlog_signaled: AtomicBool::new(false),
})
}
diff --git a/crates/screenpipe-db/tests/sqlite_architecture_invariants_test.rs b/crates/screenpipe-db/tests/sqlite_architecture_invariants_test.rs
index 89e9bb07e..61c6cd218 100644
--- a/crates/screenpipe-db/tests/sqlite_architecture_invariants_test.rs
+++ b/crates/screenpipe-db/tests/sqlite_architecture_invariants_test.rs
@@ -40,7 +40,6 @@ fn sqlite_lifecycle_has_one_owner_per_physical_database() {
rust_files(&repository.join("crates"), &mut files);
let mut live_truncation = Vec::new();
- let mut live_capture_restart = Vec::new();
let mut legacy_secret_open = Vec::new();
let mut capture_pool_secret_store = Vec::new();
for path in files {
@@ -48,12 +47,6 @@ fn sqlite_lifecycle_has_one_owner_per_physical_database() {
if source.contains("\"PRAGMA wal_checkpoint(TRUNCATE)") {
live_truncation.push(path.clone());
}
- if path.starts_with(crate_dir.join("src"))
- && !path.ends_with("screenpipe-db/src/db/setup.rs")
- && source.contains("wal_checkpoint(RESTART)")
- {
- live_capture_restart.push(path.clone());
- }
if !path.ends_with("screenpipe-secrets/src/store.rs")
&& source.contains("SecretStore::open(")
{
@@ -70,10 +63,6 @@ fn sqlite_lifecycle_has_one_owner_per_physical_database() {
live_truncation.is_empty(),
"live WAL truncation is forbidden; use non-truncating checkpoints or offline recovery: {live_truncation:?}"
);
- assert!(
- live_capture_restart.is_empty(),
- "live capture code must hand off to a full pool teardown before resetting WAL state: {live_capture_restart:?}"
- );
assert!(
legacy_secret_open.is_empty(),
"production credential callers must use open_for_data_dir and secrets.sqlite: {legacy_secret_open:?}"
@@ -88,7 +77,7 @@ fn sqlite_lifecycle_has_one_owner_per_physical_database() {
assert!(!maintenance.contains("PRAGMA synchronous = OFF"));
assert!(
!maintenance.contains("wal_checkpoint(RESTART)"),
- "live maintenance must hand off to a full pool teardown before resetting WAL state"
+ "routine checkpoints must not wait for readers while holding the recording writer lane"
);
assert!(maintenance.contains("online SQLite repair is disabled"));
@@ -99,7 +88,7 @@ fn sqlite_lifecycle_has_one_owner_per_physical_database() {
assert_eq!(
setup.matches("wal_checkpoint(RESTART)").count(),
1,
- "RESTART is reserved for the startup boundary before application traffic"
+ "startup checkpoints before admitting application traffic"
);
assert_eq!(
setup.matches("crate::storage::bulk::pool_options(").count(),
diff --git a/docs/COVERAGE.md b/docs/COVERAGE.md
index 4138823f2..5dec30fce 100644
--- a/docs/COVERAGE.md
+++ b/docs/COVERAGE.md
@@ -25,28 +25,28 @@ results and `cargo llvm-cov` data on top when judging release confidence.
### Tauri E2E
- Mapped specs: 141
-- Declared test blocks: 405
-- Weighted coverage points: 325.5
+- Declared test blocks: 406
+- Weighted coverage points: 326.5
| Platform | Specs | Declared tests | Weighted points | Layers | Features | Critical score |
| --- | --- | --- | --- | --- | --- | --- |
-| windows | 107 | 342 | 285.1 | 15 | 122 | 85% |
-| macos | 137 | 367 | 295.3 | 17 | 132 | 88% |
-| linux | 95 | 300 | 254.5 | 14 | 119 | 80% |
+| windows | 107 | 343 | 286.1 | 15 | 122 | 85% |
+| macos | 137 | 368 | 296.3 | 17 | 132 | 88% |
+| linux | 95 | 301 | 255.5 | 14 | 119 | 80% |
### Core Engine
- Mapped suites: 34
- Mapped Rust files: 367
-- Active test blocks: 3586
+- Active test blocks: 3585
- Ignored/manual test blocks: 150
-- Weighted coverage points: 2964.9
+- Weighted coverage points: 2963.6
| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score |
| --- | --- | --- | --- | --- | --- | --- | --- |
-| windows | 31 | 3442 | 144 | 2899.4 | 21 | 11 | 100% |
-| macos | 31 | 3502 | 124 | 2911.7 | 22 | 11 | 100% |
-| linux | 27 | 3061 | 116 | 2548.6 | 20 | 11 | 100% |
+| windows | 31 | 3441 | 144 | 2898.1 | 21 | 11 | 100% |
+| macos | 31 | 3501 | 124 | 2910.4 | 22 | 11 | 100% |
+| linux | 27 | 3060 | 116 | 2547.3 | 20 | 11 | 100% |
## Refresh
diff --git a/docs/coverage/CORE.md b/docs/coverage/CORE.md
index 52419c927..eee2a2d16 100644
--- a/docs/coverage/CORE.md
+++ b/docs/coverage/CORE.md
@@ -9,10 +9,10 @@ confidence, and criticality.
- Tracked crates: screenpipe-engine, screenpipe-db, screenpipe-sqlite-coordinator, screenpipe-audio, screenpipe-screen, screenpipe-a11y, screenpipe-fs
- Mapped suites: 34
- Mapped Rust files: 367
-- Active test blocks: 3586
+- Active test blocks: 3585
- Ignored/manual test blocks: 150
-- Declared test blocks: 3736
-- Weighted coverage points: 2964.9
+- Declared test blocks: 3735
+- Weighted coverage points: 2963.6
Confidence weights: strong=1.0, partial=0.7, conditional=0.4, smoke=0.3.
Criticality weights: high=1.0, medium=0.7, low=0.4.
@@ -23,16 +23,16 @@ are explicitly enabled in a runtime lane.
| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score |
| --- | --- | --- | --- | --- | --- | --- | --- |
-| windows | 31 | 3442 | 144 | 2899.4 | 21 | 11 | 100% |
-| macos | 31 | 3502 | 124 | 2911.7 | 22 | 11 | 100% |
-| linux | 27 | 3061 | 116 | 2548.6 | 20 | 11 | 100% |
+| windows | 31 | 3441 | 144 | 2898.1 | 21 | 11 | 100% |
+| macos | 31 | 3501 | 124 | 2910.4 | 22 | 11 | 100% |
+| linux | 27 | 3060 | 116 | 2547.3 | 20 | 11 | 100% |
## Crate Summary
| Crate | Suites | Integration files | Source unit files | Active tests | Ignored tests | Weighted points | Flows |
| --- | --- | --- | --- | --- | --- | --- | --- |
| screenpipe-engine | 11 | 21 | 121 | 1744 | 43 | 1344.3 | 10 |
-| screenpipe-db | 5 | 62 | 24 | 572 | 25 | 545.0 | 9 |
+| screenpipe-db | 5 | 62 | 24 | 571 | 25 | 543.7 | 9 |
| screenpipe-sqlite-coordinator | 1 | 0 | 3 | 27 | 0 | 27.0 | 2 |
| screenpipe-audio | 6 | 25 | 51 | 625 | 44 | 549.6 | 5 |
| screenpipe-screen | 6 | 9 | 18 | 257 | 9 | 232.1 | 4 |
@@ -66,21 +66,21 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su
| audio | 8 suites / 741 active / 45 ignored / 665.6 pts | 8 suites / 741 active / 45 ignored / 665.6 pts | 7 suites / 666 active / 44 ignored / 613.1 pts |
| audio-device | 2 suites / 222 active / 7 ignored / 199.5 pts | 2 suites / 222 active / 7 ignored / 199.5 pts | 1 suites / 147 active / 6 ignored / 147.0 pts |
| configuration | 2 suites / 143 active / 3 ignored / 129.2 pts | 2 suites / 143 active / 3 ignored / 129.2 pts | 2 suites / 143 active / 3 ignored / 129.2 pts |
-| database | 7 suites / 523 active / 21 ignored / 496.0 pts | 7 suites / 523 active / 21 ignored / 496.0 pts | 7 suites / 523 active / 21 ignored / 496.0 pts |
+| database | 7 suites / 522 active / 21 ignored / 494.7 pts | 7 suites / 522 active / 21 ignored / 494.7 pts | 7 suites / 522 active / 21 ignored / 494.7 pts |
| db-search | 2 suites / 113 active / 9 ignored / 113.0 pts | 2 suites / 113 active / 9 ignored / 113.0 pts | 2 suites / 113 active / 9 ignored / 113.0 pts |
| engine-lifecycle | 6 suites / 244 active / 1 ignored / 219.0 pts | 6 suites / 244 active / 1 ignored / 219.0 pts | 5 suites / 238 active / 1 ignored / 217.3 pts |
| local-api | 3 suites / 444 active / 10 ignored / 313.2 pts | 3 suites / 444 active / 10 ignored / 313.2 pts | 3 suites / 444 active / 10 ignored / 313.2 pts |
| meeting | 6 suites / 1638 active / 21 ignored / 1338.3 pts | 6 suites / 1638 active / 21 ignored / 1338.3 pts | 4 suites / 1293 active / 17 ignored / 1015.8 pts |
| ocr | 4 suites / 125 active / 7 ignored / 119.0 pts | 4 suites / 129 active / 7 ignored / 124.5 pts | 3 suites / 120 active / 6 ignored / 115.5 pts |
| os-integration | 1 suites / 6 active / 0 ignored / 1.7 pts | 1 suites / 6 active / 0 ignored / 1.7 pts | - |
-| performance | 13 suites / 1581 active / 76 ignored / 1400.3 pts | 14 suites / 1689 active / 80 ignored / 1443.5 pts | 13 suites / 1581 active / 76 ignored / 1400.3 pts |
+| performance | 13 suites / 1580 active / 76 ignored / 1399.0 pts | 14 suites / 1688 active / 80 ignored / 1442.2 pts | 13 suites / 1580 active / 76 ignored / 1399.0 pts |
| pipes | 1 suites / 512 active / 3 ignored / 358.4 pts | 1 suites / 512 active / 3 ignored / 358.4 pts | 1 suites / 512 active / 3 ignored / 358.4 pts |
| privacy | 5 suites / 946 active / 37 ignored / 766.6 pts | 5 suites / 1002 active / 17 ignored / 773.4 pts | 5 suites / 921 active / 14 ignored / 743.4 pts |
| real-app | - | 1 suites / 108 active / 4 ignored / 43.2 pts | - |
| speaker | 2 suites / 369 active / 9 ignored / 369.0 pts | 2 suites / 369 active / 9 ignored / 369.0 pts | 2 suites / 369 active / 9 ignored / 369.0 pts |
-| storage | 4 suites / 647 active / 38 ignored / 537.8 pts | 4 suites / 647 active / 38 ignored / 537.8 pts | 4 suites / 647 active / 38 ignored / 537.8 pts |
+| storage | 4 suites / 645 active / 38 ignored / 535.8 pts | 4 suites / 645 active / 38 ignored / 535.8 pts | 4 suites / 645 active / 38 ignored / 535.8 pts |
| sync | 1 suites / 512 active / 3 ignored / 358.4 pts | 1 suites / 512 active / 3 ignored / 358.4 pts | 1 suites / 512 active / 3 ignored / 358.4 pts |
-| timeline | 4 suites / 1187 active / 43 ignored / 973.1 pts | 4 suites / 1187 active / 43 ignored / 973.1 pts | 4 suites / 1187 active / 43 ignored / 973.1 pts |
+| timeline | 4 suites / 1185 active / 43 ignored / 971.1 pts | 4 suites / 1185 active / 43 ignored / 971.1 pts | 4 suites / 1185 active / 43 ignored / 971.1 pts |
| transcription | 5 suites / 815 active / 42 ignored / 638.5 pts | 5 suites / 815 active / 42 ignored / 638.5 pts | 5 suites / 815 active / 42 ignored / 638.5 pts |
| ui-events | 4 suites / 762 active / 29 ignored / 579.0 pts | 3 suites / 710 active / 5 ignored / 542.6 pts | 3 suites / 710 active / 5 ignored / 542.6 pts |
| vision-capture | 6 suites / 552 active / 32 ignored / 436.8 pts | 6 suites / 556 active / 32 ignored / 442.3 pts | 5 suites / 547 active / 31 ignored / 433.3 pts |
@@ -131,9 +131,9 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su
| audio-transcription-pipeline | screenpipe-audio | windows, macos, linux | audio, transcription, performance | audio-record-transcribe, meeting-live-notes, performance-liveness | high | partial | mixed | 15 | 105 | 7 | Batch deferral, cleanup, language detection, result normalization, and real recording/transcription tests. Hardware/model-heavy tests are ignored by default. Also covers ffmpeg encode child reaping on error paths. |
| db-accessibility-ui-events | screenpipe-db | windows, macos, linux | database, configuration, accessibility, ui-events, performance | settings-to-engine-config, accessibility-ui-events, performance-liveness | medium | partial | integration | 7 | 27 | 2 | Elements bulk insert, on-screen filtering, UI event batching, DB tier config, and ignored heavy-read real-DB probes. |
| db-audio-meetings-speakers | screenpipe-db | windows, macos, linux | database, audio, meeting, speaker | audio-record-transcribe, audio-device-health, meeting-live-notes | high | strong | integration | 15 | 113 | 1 | Audio transcript dedupe, live meeting mirroring, end-generation and open-meeting invariants, liveness, and speaker reassignment coverage. |
-| db-runtime-reliability | screenpipe-db | windows, macos, linux | database, performance | performance-liveness | high | partial | mixed | 16 | 44 | 6 | SQLite hard-fault classification, failpoint VFS injection, fresh-identity recovery verification with integrity/FK/write canaries, query cancellation, close-severs-connections regressions, multi-pool WAL parity, runtime version pinning, and WAL chaos plus memory-pressure probes, plus read-only quarantine self-heal verification for transient IOERR faults. |
+| db-runtime-reliability | screenpipe-db | windows, macos, linux | database, performance | performance-liveness | high | partial | mixed | 16 | 45 | 6 | SQLite hard-fault classification, failpoint VFS injection, fresh-identity recovery verification with integrity/FK/write canaries, query cancellation, close-severs-connections regressions, multi-pool WAL parity, runtime version pinning, and WAL chaos plus memory-pressure probes, plus read-only quarantine self-heal verification for transient IOERR faults. |
| db-search-indexing | screenpipe-db | windows, macos, linux | db-search, ocr, accessibility, performance | local-api-search, capture-ocr-pipeline, accessibility-ui-events, performance-liveness | high | strong | mixed | 13 | 105 | 4 | FTS, tokenizer, OCR snapshot search, query planning, ordering, accessibility search, and contention coverage. |
-| db-timeline-frames | screenpipe-db | windows, macos, linux | database, timeline, storage, performance | timeline-streaming, performance-liveness | high | strong | mixed | 35 | 283 | 12 | Frame/audio joins, timeline query shape, suggestions frames, write queue, DB primitives (src/db.rs split into src/db/ modules), feedback record upserts, media eviction anti-join regressions, SAF output registry, semantic storage, and timeline performance. |
+| db-timeline-frames | screenpipe-db | windows, macos, linux | database, timeline, storage, performance | timeline-streaming, performance-liveness | high | strong | mixed | 35 | 281 | 12 | Frame/audio joins, timeline query shape, suggestions frames, write queue, DB primitives (src/db.rs split into src/db/ modules), feedback record upserts, media eviction anti-join regressions, SAF output registry, semantic storage, and timeline performance. |
| engine-api-routes | screenpipe-engine | windows, macos, linux | local-api, timeline, meeting, transcription | local-api-search, timeline-streaming, meeting-live-notes, audio-record-transcribe | high | partial | mixed | 41 | 412 | 5 | Route/unit coverage for search, health, streaming, meetings, time/timezone, and transcription. Legacy endpoint/websocket tests require local data and remain ignored. |
| engine-capture-timeline | screenpipe-engine | windows, macos, linux | vision-capture, timeline, storage, performance | capture-ocr-pipeline, timeline-streaming, performance-liveness | high | partial | mixed | 25 | 301 | 26 | Covers capture trigger logic, frame/audio linking, hot cache, timeline refresh regressions, fragmented MP4 extraction, and HD-mode control. Several real-data tests are intentionally ignored by default. |
| engine-config-lifecycle | screenpipe-engine | windows, macos, linux | configuration, engine-lifecycle, performance | settings-to-engine-config, engine-health-lifecycle, performance-liveness | high | strong | mixed | 12 | 116 | 1 | Fast logic coverage for the config bridge, health-endpoint identity, tray health debounce, sleep/power policies, and queue backpressure. |
@@ -266,11 +266,11 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su
| audio-meetings-speakers-dedup | screenpipe-audio | tests/speaker_identification.rs | integration | 2 | 1 | 3 |
| audio-meetings-speakers-dedup | screenpipe-audio | tests/speaker_identity_gate_real_audio_test.rs | integration | 0 | 4 | 4 |
| audio-meetings-speakers-dedup | screenpipe-audio | tests/speaker_identity_gate_test.rs | integration | 11 | 0 | 11 |
-| db-runtime-reliability | screenpipe-db | src/cancellable_query.rs | source | 1 | 0 | 1 |
+| db-runtime-reliability | screenpipe-db | src/cancellable_query.rs | source | 2 | 0 | 2 |
| db-timeline-frames | screenpipe-db | src/db/activity_ledger.rs | source | 5 | 0 | 5 |
| db-accessibility-ui-events | screenpipe-db | src/db/elements.rs | source | 5 | 0 | 5 |
| db-timeline-frames | screenpipe-db | src/db/feedback.rs | source | 2 | 0 | 2 |
-| db-timeline-frames | screenpipe-db | src/db/maintenance.rs | source | 7 | 1 | 8 |
+| db-timeline-frames | screenpipe-db | src/db/maintenance.rs | source | 5 | 1 | 6 |
| db-timeline-frames | screenpipe-db | src/db/setup.rs | source | 7 | 0 | 7 |
| db-timeline-frames | screenpipe-db | src/db/source_identity.rs | source | 3 | 0 | 3 |
| db-timeline-frames | screenpipe-db | src/db/tests.rs | source | 26 | 1 | 27 |