YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(compaction): skip malformed snapshot inputs (#6728)
Louis Beaumont1c65b043f8b1
diff --git a/COVERAGE.md b/COVERAGE.mdindex ca01b8a1c..0dd1033ff 100644--- a/COVERAGE.md+++ b/COVERAGE.md@@ -36,15 +36,15 @@ results and `cargo llvm-cov` data on top when judging release confidence.- Mapped suites: 32- Mapped Rust files: 336-- Active test blocks: 3302+- Active test blocks: 3305- Ignored/manual test blocks: 137-- Weighted coverage points: 2705.3+- Weighted coverage points: 2707.4| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 29 | 3163 | 132 | 2641.9 | 21 | 11 | 100% |-| macos | 29 | 3221 | 112 | 2654.3 | 22 | 11 | 100% |-| linux | 25 | 2825 | 105 | 2333.2 | 20 | 11 | 100% |+| windows | 29 | 3166 | 132 | 2644.0 | 21 | 11 | 100% |+| macos | 29 | 3224 | 112 | 2656.4 | 22 | 11 | 100% |+| linux | 25 | 2828 | 105 | 2335.3 | 20 | 11 | 100% |## Refreshdiff --git a/crates/screenpipe-engine/src/snapshot_compaction.rs b/crates/screenpipe-engine/src/snapshot_compaction.rsindex d7a0c7bc0..c3a4dcaef 100644--- a/crates/screenpipe-engine/src/snapshot_compaction.rs+++ b/crates/screenpipe-engine/src/snapshot_compaction.rs@@ -17,7 +17,9 @@ use anyhow::Result;use chrono::{Duration, Utc};use screenpipe_db::DatabaseManager;use std::collections::BTreeMap;+use std::fs::File;use std::future::Future;+use std::io::{BufReader, Read, Seek, SeekFrom};use std::path::{Path, PathBuf};use std::sync::Arc;use tokio::sync::broadcast;@@ -85,6 +87,105 @@ fn next_poll_delay_secs(frames_found: usize, state: &PowerState) -> u64 {}type CacheUpdates = Vec<(i64, String, i64, f64)>;+type SnapshotFrame = (i64, String, String);++#[derive(Debug)]+struct PreparedSnapshotBatch {+ frames: Vec<SnapshotFrame>,+ invalid_ids: Vec<i64>,+ retryable_count: usize,+ first_dimensions: Option<(u32, u32)>,+}++#[derive(Debug, PartialEq, Eq)]+enum SnapshotJpegState {+ Ready((u32, u32)),+ Invalid,+ Retryable,+}++/// Validate enough of a JPEG to keep a malformed leading snapshot from+/// poisoning the whole image2pipe stream. `into_dimensions` alone only reads+/// the header, so explicitly require both the SOI and EOI markers as well.+fn inspect_snapshot_jpeg(path: &Path) -> SnapshotJpegState {+ let mut file = match File::open(path) {+ Ok(file) => file,+ Err(error) if error.kind() == std::io::ErrorKind::NotFound => {+ return SnapshotJpegState::Invalid;+ }+ Err(_) => return SnapshotJpegState::Retryable,+ };+ let file_len = match file.metadata() {+ Ok(metadata) => metadata.len(),+ Err(_) => return SnapshotJpegState::Retryable,+ };+ if file_len < 4 {+ return SnapshotJpegState::Invalid;+ }++ let mut marker = [0_u8; 2];+ if let Err(error) = file.read_exact(&mut marker) {+ return if error.kind() == std::io::ErrorKind::UnexpectedEof {+ SnapshotJpegState::Invalid+ } else {+ SnapshotJpegState::Retryable+ };+ }+ if marker != [0xff, 0xd8] {+ return SnapshotJpegState::Invalid;+ }+ if file.seek(SeekFrom::End(-2)).is_err() {+ return SnapshotJpegState::Retryable;+ }+ if let Err(error) = file.read_exact(&mut marker) {+ return if error.kind() == std::io::ErrorKind::UnexpectedEof {+ SnapshotJpegState::Invalid+ } else {+ SnapshotJpegState::Retryable+ };+ }+ if marker != [0xff, 0xd9] {+ return SnapshotJpegState::Invalid;+ }+ if file.seek(SeekFrom::Start(0)).is_err() {+ return SnapshotJpegState::Retryable;+ }++ match image::ImageReader::with_format(BufReader::new(file), image::ImageFormat::Jpeg)+ .into_dimensions()+ {+ Ok((width, height)) if width > 0 && height > 0 => SnapshotJpegState::Ready((width, height)),+ Err(image::ImageError::IoError(error))+ if error.kind() == std::io::ErrorKind::UnexpectedEof =>+ {+ SnapshotJpegState::Invalid+ }+ Err(image::ImageError::IoError(_)) => SnapshotJpegState::Retryable,+ _ => SnapshotJpegState::Invalid,+ }+}++fn prepare_snapshot_batch(frames: &[SnapshotFrame]) -> PreparedSnapshotBatch {+ let mut prepared = PreparedSnapshotBatch {+ frames: Vec::with_capacity(frames.len()),+ invalid_ids: Vec::new(),+ retryable_count: 0,+ first_dimensions: None,+ };++ for frame in frames {+ match inspect_snapshot_jpeg(Path::new(&frame.1)) {+ SnapshotJpegState::Ready(dimensions) => {+ prepared.first_dimensions.get_or_insert(dimensions);+ prepared.frames.push(frame.clone());+ }+ SnapshotJpegState::Invalid => prepared.invalid_ids.push(frame.0),+ SnapshotJpegState::Retryable => prepared.retryable_count += 1,+ }+ }++ prepared+}#[derive(Debug, Clone, Copy, PartialEq, Eq)]enum CompactionPauseReason {@@ -409,7 +510,7 @@ async fn run_compaction_cycle(async fn compact_chunk(db: &DatabaseManager,device_name: &str,- frames: &[(i64, String, String)], // (frame_id, snapshot_path, timestamp_str)+ frames: &[SnapshotFrame], // (frame_id, snapshot_path, timestamp_str)video_quality: &str,power_manager: &PowerManagerHandle,) -> Result<CompactChunkOutcome> {@@ -421,6 +522,32 @@ async fn compact_chunk(return Ok(CompactChunkOutcome::Paused(reason));}+ // image2pipe treats the batch as one byte stream. A malformed first JPEG+ // prevents ffmpeg from discovering dimensions even when later snapshots+ // are valid, so validate every candidate before spawning the encoder and+ // permanently clear pointers to files that cannot be displayed or encoded.+ let prepared = prepare_snapshot_batch(frames);+ if !prepared.invalid_ids.is_empty() {+ warn!(+ "snapshot compaction: clearing {} invalid snapshot pointers for {}",+ prepared.invalid_ids.len(),+ device_name+ );+ for batch in prepared.invalid_ids.chunks(100) {+ let _ = db.clear_snapshot_paths_queued(batch.to_vec()).await;+ }+ }+ if prepared.retryable_count > 0 {+ debug!(+ "snapshot compaction: deferring {} temporarily unreadable snapshots for {}",+ prepared.retryable_count, device_name+ );+ }+ if prepared.frames.is_empty() {+ return Ok(CompactChunkOutcome::Completed(None));+ }+ let frames = prepared.frames.as_slice();+let first_path = Path::new(&frames[0].1);let parent_dir = first_path.parent()@@ -442,36 +569,9 @@ async fn compact_chunk(.filter_map(|(_, p, _)| std::fs::metadata(p).ok().map(|m| m.len())).sum();- // Step 1: Confirm at least one JPEG is readable (and grab dimensions for- // the log line). Header-only read — ffmpeg does the actual decoding, so- // fully decoding a frame here would be pure waste.- let (frame_w, frame_h) = {- let mut dims = None;- for (_, snapshot_path, _) in frames {- dims = image::ImageReader::open(Path::new(snapshot_path))- .ok()- .and_then(|r| r.into_dimensions().ok());- if dims.is_some() {- break;- }- }- match dims {- Some(d) => d,- None => {- // All JPEGs gone/unreadable — clear stale DB pointers so we don't retry- let ids: Vec<i64> = frames.iter().map(|(id, _, _)| *id).collect();- debug!(- "snapshot compaction: clearing {} stale snapshot_path entries for {} (files missing)",- ids.len(),- device_name- );- for batch in ids.chunks(100) {- let _ = db.clear_snapshot_paths_queued(batch.to_vec()).await;- }- return Ok(CompactChunkOutcome::Completed(None));- }- }- };+ let (frame_w, frame_h) = prepared+ .first_dimensions+ .expect("a non-empty prepared batch has dimensions");debug!("compacting {} frames for {} into {} (fps={:.2}, {}x{}, source={:.1}MB)",@@ -556,6 +656,15 @@ async fn compact_chunk(}}+ // Every file can still disappear between preflight and the async read.+ // Do not ask ffmpeg to finalize an empty image2pipe stream: it produces the+ // noisy "unspecified size / output contains no stream" failure and turns a+ // stale snapshot into an endless five-minute retry loop.+ if encoded_frames.is_empty() {+ abort_ffmpeg_for_pause(child, stdin).await;+ return Ok(CompactChunkOutcome::Completed(None));+ }+let exit_status =match run_until_paused(power_manager, finish_ffmpeg_process(child, Some(stdin))).await {Ok(status) => status,@@ -874,6 +983,66 @@ mod tests {assert!(mp4.exists(), "DB-owned MP4 must survive guard cleanup");}+ #[test]+ fn snapshot_batch_skips_truncated_leading_jpeg() {+ let dir = tempfile::tempdir().expect("temp dir");+ let valid_path = dir.path().join("valid.jpg");+ let truncated_path = dir.path().join("truncated.jpg");+ image::DynamicImage::new_rgb8(4, 3)+ .save_with_format(&valid_path, image::ImageFormat::Jpeg)+ .expect("write valid JPEG");++ let mut truncated = std::fs::read(&valid_path).expect("read valid JPEG");+ truncated.truncate(truncated.len() - 2);+ std::fs::write(&truncated_path, truncated).expect("write truncated JPEG");++ let frames = vec![+ make_frame(+ 1,+ truncated_path.to_str().expect("UTF-8 path"),+ "2025-01-01T00:00:00Z",+ ),+ make_frame(+ 2,+ valid_path.to_str().expect("UTF-8 path"),+ "2025-01-01T00:00:01Z",+ ),+ ];++ let prepared = prepare_snapshot_batch(&frames);+ assert_eq!(prepared.invalid_ids, vec![1]);+ assert_eq!(prepared.retryable_count, 0);+ assert_eq!(prepared.frames, vec![frames[1].clone()]);+ assert_eq!(prepared.first_dimensions, Some((4, 3)));+ }++ #[test]+ fn snapshot_inspection_rejects_marker_only_and_missing_files() {+ let dir = tempfile::tempdir().expect("temp dir");+ let marker_only_path = dir.path().join("marker-only.jpg");+ std::fs::write(&marker_only_path, [0xff, 0xd8, 0xff, 0xd9])+ .expect("write marker-only JPEG");++ assert_eq!(+ inspect_snapshot_jpeg(&marker_only_path),+ SnapshotJpegState::Invalid+ );+ assert_eq!(+ inspect_snapshot_jpeg(&dir.path().join("missing.jpg")),+ SnapshotJpegState::Invalid+ );+ }++ #[test]+ fn snapshot_inspection_defers_non_file_io_failures() {+ let dir = tempfile::tempdir().expect("temp dir");++ assert_eq!(+ inspect_snapshot_jpeg(dir.path()),+ SnapshotJpegState::Retryable+ );+ }+#[test]fn test_calculate_fps_single_frame() {let frames = vec![make_frame(1, "/tmp/a.jpg", "2025-01-01T00:00:00Z")];diff --git a/docs/coverage/CORE.md b/docs/coverage/CORE.mdindex 96f144354..875f0d1d5 100644--- a/docs/coverage/CORE.md+++ b/docs/coverage/CORE.md@@ -9,10 +9,10 @@ confidence, and criticality.- Tracked crates: screenpipe-engine, screenpipe-db, screenpipe-sqlite-coordinator, screenpipe-audio, screenpipe-screen, screenpipe-a11y- Mapped suites: 32- Mapped Rust files: 336-- Active test blocks: 3302+- Active test blocks: 3305- Ignored/manual test blocks: 137-- Declared test blocks: 3439-- Weighted coverage points: 2705.3+- Declared test blocks: 3442+- Weighted coverage points: 2707.4Confidence weights: strong=1.0, partial=0.7, conditional=0.4, smoke=0.3.Criticality weights: high=1.0, medium=0.7, low=0.4.@@ -23,15 +23,15 @@ are explicitly enabled in a runtime lane.| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 29 | 3163 | 132 | 2641.9 | 21 | 11 | 100% |-| macos | 29 | 3221 | 112 | 2654.3 | 22 | 11 | 100% |-| linux | 25 | 2825 | 105 | 2333.2 | 20 | 11 | 100% |+| windows | 29 | 3166 | 132 | 2644.0 | 21 | 11 | 100% |+| macos | 29 | 3224 | 112 | 2656.4 | 22 | 11 | 100% |+| linux | 25 | 2828 | 105 | 2335.3 | 20 | 11 | 100% |## Crate Summary| Crate | Suites | Integration files | Source unit files | Active tests | Ignored tests | Weighted points | Flows || --- | --- | --- | --- | --- | --- | --- | --- |-| screenpipe-engine | 10 | 19 | 115 | 1628 | 42 | 1243.9 | 10 |+| screenpipe-engine | 10 | 19 | 115 | 1631 | 42 | 1246.0 | 10 || screenpipe-db | 5 | 52 | 15 | 454 | 16 | 431.2 | 9 || screenpipe-sqlite-coordinator | 1 | 0 | 2 | 17 | 0 | 17.0 | 2 || screenpipe-audio | 6 | 25 | 51 | 594 | 43 | 520.4 | 5 |@@ -72,17 +72,17 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| meeting | 6 suites / 1556 active / 19 ignored / 1263.5 pts | 6 suites / 1556 active / 19 ignored / 1263.5 pts | 4 suites / 1251 active / 15 ignored / 981.0 pts || ocr | 4 suites / 125 active / 7 ignored / 119.0 pts | 4 suites / 129 active / 7 ignored / 124.5 pts | 3 suites / 120 active / 6 ignored / 115.5 pts || os-integration | 1 suites / 6 active / 0 ignored / 1.7 pts | 1 suites / 6 active / 0 ignored / 1.7 pts | - |-| performance | 13 suites / 1441 active / 67 ignored / 1267.5 pts | 14 suites / 1544 active / 70 ignored / 1308.7 pts | 13 suites / 1441 active / 67 ignored / 1267.5 pts |+| performance | 13 suites / 1444 active / 67 ignored / 1269.6 pts | 14 suites / 1547 active / 70 ignored / 1310.8 pts | 13 suites / 1444 active / 67 ignored / 1269.6 pts || pipes | 1 suites / 495 active / 3 ignored / 346.5 pts | 1 suites / 495 active / 3 ignored / 346.5 pts | 1 suites / 495 active / 3 ignored / 346.5 pts || privacy | 5 suites / 926 active / 36 ignored / 752.6 pts | 5 suites / 980 active / 16 ignored / 759.5 pts | 5 suites / 904 active / 14 ignored / 731.5 pts || real-app | - | 1 suites / 103 active / 3 ignored / 41.2 pts | - || speaker | 2 suites / 351 active / 8 ignored / 351.0 pts | 2 suites / 351 active / 8 ignored / 351.0 pts | 2 suites / 351 active / 8 ignored / 351.0 pts |-| storage | 3 suites / 518 active / 29 ignored / 417.2 pts | 3 suites / 518 active / 29 ignored / 417.2 pts | 3 suites / 518 active / 29 ignored / 417.2 pts |+| storage | 3 suites / 521 active / 29 ignored / 419.3 pts | 3 suites / 521 active / 29 ignored / 419.3 pts | 3 suites / 521 active / 29 ignored / 419.3 pts || sync | 1 suites / 495 active / 3 ignored / 346.5 pts | 1 suites / 495 active / 3 ignored / 346.5 pts | 1 suites / 495 active / 3 ignored / 346.5 pts |-| timeline | 4 suites / 1076 active / 33 ignored / 865.1 pts | 4 suites / 1076 active / 33 ignored / 865.1 pts | 4 suites / 1076 active / 33 ignored / 865.1 pts |+| timeline | 4 suites / 1079 active / 33 ignored / 867.2 pts | 4 suites / 1079 active / 33 ignored / 867.2 pts | 4 suites / 1079 active / 33 ignored / 867.2 pts || transcription | 5 suites / 787 active / 40 ignored / 614.4 pts | 5 suites / 787 active / 40 ignored / 614.4 pts | 5 suites / 787 active / 40 ignored / 614.4 pts || ui-events | 4 suites / 742 active / 28 ignored / 565.0 pts | 3 suites / 693 active / 5 ignored / 530.7 pts | 3 suites / 693 active / 5 ignored / 530.7 pts |-| vision-capture | 5 suites / 548 active / 32 ignored / 432.7 pts | 5 suites / 552 active / 32 ignored / 438.2 pts | 4 suites / 543 active / 31 ignored / 429.2 pts |+| vision-capture | 5 suites / 551 active / 32 ignored / 434.8 pts | 5 suites / 555 active / 32 ignored / 440.3 pts | 4 suites / 546 active / 31 ignored / 431.3 pts |## Critical Flow Matrix@@ -134,7 +134,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| db-search-indexing | screenpipe-db | windows, macos, linux | db-search, ocr, accessibility, performance | local-api-search, capture-ocr-pipeline, accessibility-ui-events, performance-liveness | high | strong | mixed | 13 | 105 | 4 | FTS, tokenizer, OCR snapshot search, query planning, ordering, accessibility search, and contention coverage. || db-timeline-frames | screenpipe-db | windows, macos, linux | database, timeline, storage, performance | timeline-streaming, performance-liveness | high | strong | mixed | 19 | 182 | 3 | Frame/audio joins, timeline query shape, suggestions frames, write queue, DB primitives (src/db.rs split into src/db/ modules), feedback record upserts, media eviction anti-join regressions, SAF output registry, semantic storage, and timeline performance. || engine-api-routes | screenpipe-engine | windows, macos, linux | local-api, timeline, meeting, transcription | local-api-search, timeline-streaming, meeting-live-notes, audio-record-transcribe | high | partial | mixed | 40 | 405 | 4 | Route/unit coverage for search, health, streaming, meetings, time/timezone, and transcription. Legacy endpoint/websocket tests require local data and remain ignored. |-| engine-capture-timeline | screenpipe-engine | windows, macos, linux | vision-capture, timeline, storage, performance | capture-ocr-pipeline, timeline-streaming, performance-liveness | high | partial | mixed | 25 | 298 | 26 | Covers capture trigger logic, frame/audio linking, hot cache, timeline refresh regressions, fragmented MP4 extraction, and HD-mode control. Several real-data tests are intentionally ignored by default. |+| engine-capture-timeline | screenpipe-engine | windows, macos, linux | vision-capture, timeline, storage, performance | capture-ocr-pipeline, timeline-streaming, performance-liveness | high | partial | mixed | 25 | 301 | 26 | Covers capture trigger logic, frame/audio linking, hot cache, timeline refresh regressions, fragmented MP4 extraction, and HD-mode control. Several real-data tests are intentionally ignored by default. || engine-config-lifecycle | screenpipe-engine | windows, macos, linux | configuration, engine-lifecycle, performance | settings-to-engine-config, engine-health-lifecycle, performance-liveness | high | strong | mixed | 11 | 111 | 1 | Fast logic coverage for the config bridge, tray health debounce, sleep/power policies, and queue backpressure. || engine-db-recovery-cli | screenpipe-engine | windows, macos, linux | database, engine-lifecycle | engine-health-lifecycle, performance-liveness | high | strong | unit | 1 | 8 | 0 | Exact DB/WAL/SHM working-copy preservation, rollback on archive failure, and restart repair for crashes during the multi-file generation swap. || engine-focus-os | screenpipe-engine | windows, macos | engine-lifecycle, os-integration | engine-health-lifecycle, performance-liveness | medium | conditional | unit | 3 | 6 | 0 | Platform focus-tracker parsing/helpers. These files are cfg-gated and only execute on their target OS. |@@ -431,7 +431,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| engine-capture-timeline | screenpipe-engine | src/semantic_worker.rs | source | 12 | 0 | 12 || engine-api-routes | screenpipe-engine | src/server.rs | source | 13 | 0 | 13 || engine-config-lifecycle | screenpipe-engine | src/sleep_monitor.rs | source | 11 | 1 | 12 |-| engine-capture-timeline | screenpipe-engine | src/snapshot_compaction.rs | source | 18 | 0 | 18 |+| engine-capture-timeline | screenpipe-engine | src/snapshot_compaction.rs | source | 21 | 0 | 21 || engine-meeting-privacy-sync | screenpipe-engine | src/structured_outputs.rs | source | 8 | 0 | 8 || engine-meeting-privacy-sync | screenpipe-engine | src/sync_api.rs | source | 13 | 0 | 13 || engine-meeting-privacy-sync | screenpipe-engine | src/sync_provider.rs | source | 4 | 0 | 4 |