YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(vision): serve focus-tracker monitor reads from cache (#6170)
Louis Beaumont79668d912bda
diff --git a/COVERAGE.md b/COVERAGE.mdindex 854e49c42..39cdaee79 100644--- a/COVERAGE.md+++ b/COVERAGE.md@@ -32,15 +32,15 @@ results and `cargo llvm-cov` data on top when judging release confidence.- Mapped suites: 32- Mapped Rust files: 319-- Active test blocks: 3044+- Active test blocks: 3048- Ignored/manual test blocks: 137-- Weighted coverage points: 2506.1+- Weighted coverage points: 2508.1| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 29 | 2913 | 132 | 2446.2 | 21 | 11 | 100% |-| macos | 29 | 2967 | 112 | 2457.3 | 22 | 11 | 100% |-| linux | 25 | 2600 | 105 | 2163.7 | 20 | 11 | 100% |+| windows | 29 | 2917 | 132 | 2448.1 | 21 | 11 | 100% |+| macos | 29 | 2971 | 112 | 2459.2 | 22 | 11 | 100% |+| linux | 25 | 2604 | 105 | 2165.6 | 20 | 11 | 100% |## Refreshdiff --git a/crates/screenpipe-engine/src/focus_tracker/darwin.rs b/crates/screenpipe-engine/src/focus_tracker/darwin.rsindex de2a32596..e2eef5d3c 100644--- a/crates/screenpipe-engine/src/focus_tracker/darwin.rs+++ b/crates/screenpipe-engine/src/focus_tracker/darwin.rs@@ -266,7 +266,12 @@ impl DarwinFocusTracker {break;}tokio::time::sleep(poll_interval).await;- let monitors = screenpipe_screen::monitor::list_monitors().await;+ // Cached: this poll only needs display geometry to resolve the+ // cursor, and at 5s it is the single heaviest source of+ // `SCShareableContent` traffic in the process. The monitor+ // watcher owns connect/disconnect on its own fresh path and+ // keeps this cache warm.+ let monitors = screenpipe_screen::monitor::list_monitors_cached().await;poll_inner.resolve_and_emit(&monitors);}});@@ -276,7 +281,7 @@ impl DarwinFocusTracker {// spawn to avoid blocking start() on the async monitor list call.let seed_inner = Arc::clone(&inner);handle.spawn(async move {- let monitors = screenpipe_screen::monitor::list_monitors().await;+ let monitors = screenpipe_screen::monitor::list_monitors_cached().await;seed_inner.resolve_and_emit(&monitors);});@@ -311,12 +316,13 @@ fn run_workspace_observer(inner: Arc<Inner>) {// retains an NSNotification + any transient NSObjects from// `list_monitors` forever.cidre::objc::ar_pool(|| {- // Fetch the current monitor list synchronously. This call- // can block briefly (sck-rs query); the observer thread- // doesn't need to be snappy — tokio workers aren't starved- // by it.+ // Fetch the current monitor list synchronously. Cached: an app+ // switch does not change display topology, and resolving the+ // focused monitor only needs geometry. Uncached this fired a+ // full `SCShareableContent` round-trip on every activation,+ // each one able to strand a worker permanently.let monitors =- futures::executor::block_on(screenpipe_screen::monitor::list_monitors());+ futures::executor::block_on(screenpipe_screen::monitor::list_monitors_cached());inner_activate.resolve_and_emit(&monitors);});};@@ -331,8 +337,10 @@ fn run_workspace_observer(inner: Arc<Inner>) {let space_cb = move |_note: &ns::Notification| {// Same autorelease-pool wrap as activate_cb above.cidre::objc::ar_pool(|| {+ // Cached for the same reason as `activate_cb`: switching Space+ // rearranges windows, not displays.let monitors =- futures::executor::block_on(screenpipe_screen::monitor::list_monitors());+ futures::executor::block_on(screenpipe_screen::monitor::list_monitors_cached());inner_space.resolve_and_emit(&monitors);});};diff --git a/crates/screenpipe-engine/src/sleep_monitor.rs b/crates/screenpipe-engine/src/sleep_monitor.rsindex 0ed2f0467..a4f123e70 100644--- a/crates/screenpipe-engine/src/sleep_monitor.rs+++ b/crates/screenpipe-engine/src/sleep_monitor.rs@@ -246,6 +246,8 @@ fn check_screen_locked_cgsession() -> bool {#[cfg(target_os = "macos")]fn handle_screen_unlock_transition() {screenpipe_screen::stream_invalidation::request();+ // Displays are routinely attached, detached or rearranged while locked.+ screenpipe_screen::monitor::invalidate_monitor_lookup_cache();crate::permission_monitor::notify_wake();SCREEN_UNLOCK_NOTIFY.notify_one();}@@ -453,6 +455,10 @@ pub fn start_sleep_monitor() {// and audio streams (CoreAudio can go silent after display changes)#[cfg(target_os = "macos")]screenpipe_screen::stream_invalidation::request();+ // Authoritative topology change: drop the cached monitor list so+ // geometry reads re-enumerate instead of serving the old layout.+ #[cfg(target_os = "macos")]+ screenpipe_screen::monitor::invalidate_monitor_lookup_cache();screenpipe_audio::stream_invalidation::request();// SCStream re-creation after a reconfiguration transiently reports// `denied`; arm the permission-monitor grace so it isn't surfaced@@ -575,6 +581,9 @@ fn on_did_wake(handle: &tokio::runtime::Handle) {// recreates them with fresh frames after wake.#[cfg(target_os = "macos")]screenpipe_screen::stream_invalidation::request();+ // A Mac often wakes into a different display layout than it slept in.+ #[cfg(target_os = "macos")]+ screenpipe_screen::monitor::invalidate_monitor_lookup_cache();// Invalidate audio streams so the device monitor force-restarts all// audio devices. CoreAudio streams can go silent after sleep/wakediff --git a/crates/screenpipe-screen/src/monitor.rs b/crates/screenpipe-screen/src/monitor.rsindex 237ab6112..30b7a6ee0 100644--- a/crates/screenpipe-screen/src/monitor.rs+++ b/crates/screenpipe-screen/src/monitor.rs@@ -29,7 +29,8 @@ pub use linux::{#[cfg(target_os = "macos")]pub use macos::{e2e_arm_sck_lookup_hang_fault, get_capture_backend, get_default_monitor, get_monitor_by_id,- is_screen_capture_supported, list_monitors, list_monitors_detailed, macos_version,+ invalidate_monitor_lookup_cache, is_screen_capture_supported, list_monitors,+ list_monitors_cached, list_monitors_detailed, macos_version,secs_since_capture_backend_fallback, set_sck_capture_max_width, HdCapture,};diff --git a/crates/screenpipe-screen/src/monitor/macos.rs b/crates/screenpipe-screen/src/monitor/macos.rsindex 37c026f57..6084af40e 100644--- a/crates/screenpipe-screen/src/monitor/macos.rs+++ b/crates/screenpipe-screen/src/monitor/macos.rs@@ -640,29 +640,70 @@ fn store_monitor_lookup_cache(monitors: &[SafeMonitor]) {});}-/// Look `id` up in the cache when the entry is younger than the TTL.-fn cached_monitor_by_id(id: u32, now: Instant) -> Option<SafeMonitor> {+/// How long a successful enumeration may answer reads that only need the set+/// of displays and their geometry, rather than connect/disconnect detection.+///+/// The macOS focus tracker calls [`list_monitors`] from a 5s safety-net poll+/// and again from every `didActivateApplication` / `activeSpaceDidChange`+/// notification, purely to resolve which display the cursor sits on. That is+/// upwards of 720 `SCShareableContent` round-trips an hour before a single app+/// switch is counted, and every one of them can strand a worker: sck-rs+/// charges a live-call slot before the call and a hung completion handler+/// never releases it, so six unlucky calls across the whole process lifetime+/// saturate the cap and refuse capture until relaunch. Serving these reads+/// from the last enumeration removes the traffic.+///+/// Sized to the monitor watcher's own 60s backstop, which keeps calling+/// [`list_monitors_detailed`] and refreshing this cache. Connect/disconnect+/// detection is unchanged — it runs on that fresh path, never this one — and+/// in steady state the watcher keeps the entry warm so focus resolution costs+/// no SCK calls at all.+const MONITOR_TOPOLOGY_CACHE_TTL: Duration = Duration::from_secs(60);++/// Run `f` against the cached enumeration when it is younger than `ttl`.+///+/// Two read paths share one cache with different freshness needs: a lookup+/// wants one display's geometry, a topology read wants the whole set. Taking+/// the TTL as an argument keeps both explicit about how stale an answer they+/// accept, without a second cache to keep in sync. `f` runs under the read+/// lock so the by-id path still clones a single monitor rather than the list.+fn with_fresh_monitor_cache<T>(+ now: Instant,+ ttl: Duration,+ f: impl FnOnce(&[SafeMonitor]) -> T,+) -> Option<T> {let guard = MONITOR_LOOKUP_CACHE.read().unwrap_or_else(|e| e.into_inner());let cached = guard.as_ref()?;- if now.duration_since(cached.captured_at) >= MONITOR_LOOKUP_CACHE_TTL {+ if now.duration_since(cached.captured_at) >= ttl {return None;}- cached- .monitors- .iter()- .find(|monitor| monitor.id() == id)- .cloned()+ Some(f(&cached.monitors))}-/// Drop the lookup cache.+/// Look `id` up in the cache when the entry is younger than the TTL.+fn cached_monitor_by_id(id: u32, now: Instant) -> Option<SafeMonitor> {+ with_fresh_monitor_cache(now, MONITOR_LOOKUP_CACHE_TTL, |monitors| {+ monitors.iter().find(|monitor| monitor.id() == id).cloned()+ })+ .flatten()+}++/// Read the whole cached enumeration when it is younger than `ttl`.+fn cached_monitor_list(now: Instant, ttl: Duration) -> Option<Vec<SafeMonitor>> {+ with_fresh_monitor_cache(now, ttl, <[SafeMonitor]>::to_vec)+}++/// Drop the cached enumeration.///-/// Production has no caller by design: `list_monitors_detailed` overwrites the-/// cache on every successful enumeration, so the watcher's own polling is the-/// invalidation path.-#[cfg(test)]-fn invalidate_monitor_lookup_cache() {+/// Call this the moment the display topology is known to have changed, rather+/// than waiting for a TTL to lapse. `sleep_monitor` already owns the two+/// authoritative signals — the CoreGraphics display-reconfiguration callback,+/// and wake/unlock — and both are exactly when a cached list stops describing+/// reality. Wiring them here means the TTLs are only a backstop for changes+/// nobody told us about, not the primary correctness mechanism.+pub fn invalidate_monitor_lookup_cache() {*MONITOR_LOOKUP_CACHE.write().unwrap_or_else(|e| e.into_inner()) = None;@@ -821,6 +862,24 @@ pub async fn list_monitors() -> Vec<SafeMonitor> {list_monitors_detailed().await.unwrap_or_default()}+/// List monitors for callers that only need the current display geometry.+///+/// Answers from the last enumeration while it is younger than+/// [`MONITOR_TOPOLOGY_CACHE_TTL`], falling back to a fresh [`list_monitors`]+/// (which refreshes the cache) on a miss. Use this for cursor-to-display+/// resolution and other geometry reads on a hot path; use+/// [`list_monitors_detailed`] when the caller is responsible for noticing that+/// a display appeared or disappeared.+///+/// Only errors are uncached, so a miss after a failed enumeration re-attempts+/// rather than serving an empty list as truth.+pub async fn list_monitors_cached() -> Vec<SafeMonitor> {+ if let Some(monitors) = cached_monitor_list(Instant::now(), MONITOR_TOPOLOGY_CACHE_TTL) {+ return monitors;+ }+ list_monitors().await+}+pub async fn get_default_monitor() -> Option<SafeMonitor> {if use_sck_rs() {let sck_result = run_bounded_sck_enumeration(@@ -1018,6 +1077,104 @@ mod tests {invalidate_monitor_lookup_cache();}+ /// The production failure this path exists for: the focus tracker polls+ /// every 5s and resolves again on every app/Space switch, and uncached+ /// each of those was an `SCShareableContent` round-trip that could strand+ /// a worker for the life of the process.+ #[test]+ fn topology_reads_are_served_from_a_recent_enumeration() {+ let _guard = lock_lookup_cache_tests();+ invalidate_monitor_lookup_cache();+ assert!(cached_monitor_list(Instant::now(), MONITOR_TOPOLOGY_CACHE_TTL).is_none());++ store_monitor_lookup_cache(&[cache_test_monitor(1), cache_test_monitor(2)]);+ let stored_at = Instant::now();++ let served = cached_monitor_list(stored_at, MONITOR_TOPOLOGY_CACHE_TTL)+ .expect("a fresh enumeration must answer topology reads");+ assert_eq!(+ served.iter().map(|m| m.id()).collect::<Vec<_>>(),+ vec![1, 2],+ "the cached read must return the whole set, not just one display"+ );++ invalidate_monitor_lookup_cache();+ }++ /// Topology reads accept a staler answer than lookups, but both must+ /// eventually miss so a disconnected display stops being handed out.+ #[test]+ fn topology_cache_outlives_the_lookup_ttl_then_expires() {+ let _guard = lock_lookup_cache_tests();+ invalidate_monitor_lookup_cache();+ store_monitor_lookup_cache(&[cache_test_monitor(3)]);+ let stored_at = Instant::now();++ // Past the lookup TTL the by-id path re-enumerates while the topology+ // path is still happy — that difference is the whole point of the+ // separate TTL, so pin it.+ assert!(MONITOR_TOPOLOGY_CACHE_TTL > MONITOR_LOOKUP_CACHE_TTL);+ assert!(cached_monitor_by_id(3, stored_at + MONITOR_LOOKUP_CACHE_TTL).is_none());+ assert!(cached_monitor_list(+ stored_at + MONITOR_LOOKUP_CACHE_TTL,+ MONITOR_TOPOLOGY_CACHE_TTL+ )+ .is_some());++ assert!(cached_monitor_list(+ stored_at + MONITOR_TOPOLOGY_CACHE_TTL,+ MONITOR_TOPOLOGY_CACHE_TTL+ )+ .is_none());+ assert!(cached_monitor_list(+ stored_at + MONITOR_TOPOLOGY_CACHE_TTL * 2,+ MONITOR_TOPOLOGY_CACHE_TTL+ )+ .is_none());++ invalidate_monitor_lookup_cache();+ }++ /// A display reconfiguration or wake must drop the cache immediately+ /// rather than let a stale layout answer until the TTL lapses.+ /// `sleep_monitor` wires the CoreGraphics reconfiguration callback and the+ /// wake/unlock transitions to this, so the TTL is only a backstop.+ #[test]+ fn topology_reads_miss_after_an_explicit_invalidation() {+ let _guard = lock_lookup_cache_tests();+ invalidate_monitor_lookup_cache();+ store_monitor_lookup_cache(&[cache_test_monitor(1), cache_test_monitor(2)]);+ assert!(cached_monitor_list(Instant::now(), MONITOR_TOPOLOGY_CACHE_TTL).is_some());++ invalidate_monitor_lookup_cache();++ assert!(+ cached_monitor_list(Instant::now(), MONITOR_TOPOLOGY_CACHE_TTL).is_none(),+ "a display change must force the next read to re-enumerate"+ );+ assert!(cached_monitor_by_id(1, Instant::now()).is_none());+ }++ /// The watcher's fresh enumeration is what keeps the topology cache warm,+ /// so a newer set must immediately replace an older one on this path too.+ #[test]+ fn newer_enumeration_replaces_the_cached_topology() {+ let _guard = lock_lookup_cache_tests();+ invalidate_monitor_lookup_cache();+ store_monitor_lookup_cache(&[cache_test_monitor(1), cache_test_monitor(2)]);+ store_monitor_lookup_cache(&[cache_test_monitor(1)]);++ let served = cached_monitor_list(Instant::now(), MONITOR_TOPOLOGY_CACHE_TTL)+ .expect("the newer enumeration must answer");+ assert_eq!(+ served.iter().map(|m| m.id()).collect::<Vec<_>>(),+ vec![1],+ "a display that disappeared must stop being served"+ );++ invalidate_monitor_lookup_cache();+ }+#[test]fn monitor_lookup_cache_expires_and_can_be_invalidated() {let _guard = lock_lookup_cache_tests();diff --git a/docs/coverage/CORE.md b/docs/coverage/CORE.mdindex 4fa02a759..a64306572 100644--- a/docs/coverage/CORE.md+++ b/docs/coverage/CORE.md@@ -9,10 +9,10 @@ confidence, and criticality.- Tracked crates: screenpipe-engine, screenpipe-db, screenpipe-sqlite-coordinator, screenpipe-audio, screenpipe-screen, screenpipe-a11y- Mapped suites: 32- Mapped Rust files: 319-- Active test blocks: 3044+- Active test blocks: 3048- Ignored/manual test blocks: 137-- Declared test blocks: 3181-- Weighted coverage points: 2506.1+- Declared test blocks: 3185+- Weighted coverage points: 2508.1Confidence weights: strong=1.0, partial=0.7, conditional=0.4, smoke=0.3.Criticality weights: high=1.0, medium=0.7, low=0.4.@@ -23,9 +23,9 @@ are explicitly enabled in a runtime lane.| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 29 | 2913 | 132 | 2446.2 | 21 | 11 | 100% |-| macos | 29 | 2967 | 112 | 2457.3 | 22 | 11 | 100% |-| linux | 25 | 2600 | 105 | 2163.7 | 20 | 11 | 100% |+| windows | 29 | 2917 | 132 | 2448.1 | 21 | 11 | 100% |+| macos | 29 | 2971 | 112 | 2459.2 | 22 | 11 | 100% |+| linux | 25 | 2604 | 105 | 2165.6 | 20 | 11 | 100% |## Crate Summary@@ -35,7 +35,7 @@ are explicitly enabled in a runtime lane.| screenpipe-db | 5 | 51 | 14 | 444 | 16 | 422.1 | 9 || screenpipe-sqlite-coordinator | 1 | 0 | 2 | 11 | 0 | 11.0 | 2 || screenpipe-audio | 6 | 25 | 50 | 580 | 43 | 507.3 | 5 |-| screenpipe-screen | 6 | 9 | 18 | 237 | 9 | 215.0 | 4 |+| screenpipe-screen | 6 | 9 | 18 | 241 | 9 | 216.9 | 4 || screenpipe-a11y | 4 | 2 | 28 | 333 | 27 | 247.6 | 3 |## Line Coverage@@ -82,7 +82,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| timeline | 4 suites / 928 active / 33 ignored / 755.2 pts | 4 suites / 928 active / 33 ignored / 755.2 pts | 4 suites / 928 active / 33 ignored / 755.2 pts || transcription | 5 suites / 688 active / 40 ignored / 542.1 pts | 5 suites / 688 active / 40 ignored / 542.1 pts | 5 suites / 688 active / 40 ignored / 542.1 pts || ui-events | 4 suites / 700 active / 28 ignored / 533.2 pts | 3 suites / 652 active / 5 ignored / 499.6 pts | 3 suites / 652 active / 5 ignored / 499.6 pts |-| vision-capture | 5 suites / 485 active / 32 ignored / 385.9 pts | 5 suites / 489 active / 32 ignored / 391.4 pts | 4 suites / 480 active / 31 ignored / 382.4 pts |+| vision-capture | 5 suites / 489 active / 32 ignored / 387.8 pts | 5 suites / 493 active / 32 ignored / 393.3 pts | 4 suites / 484 active / 31 ignored / 384.3 pts |## Critical Flow Matrix@@ -147,7 +147,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| screen-capture-windowing | screenpipe-screen | windows, macos, linux | vision-capture, timeline, performance, privacy | capture-ocr-pipeline, timeline-streaming, privacy-and-redaction, performance-liveness | high | strong | mixed | 14 | 177 | 0 | Window filtering, empty-window regressions, retry policy, URL timing, monitor cache, OCR cache, snapshots, and image comparison. || screen-custom-ocr | screenpipe-screen | windows, macos, linux | ocr | capture-ocr-pipeline | medium | conditional | manual | 1 | 0 | 2 | Custom OCR tests are ignored by default and only contribute when explicitly run. || screen-macos-ocr | screenpipe-screen | macos | ocr, vision-capture | capture-ocr-pipeline | high | strong | mixed | 2 | 9 | 1 | Apple Vision OCR source/unit coverage and fixture OCR assertions. |-| screen-monitor-platform | screenpipe-screen | windows, macos, linux | vision-capture | capture-ocr-pipeline | medium | partial | unit | 5 | 32 | 5 | Per-OS monitor enumeration (Windows, macOS, Wayland/portal on Linux) and the persistent Windows.Graphics.Capture session. Each file is cfg-gated and only executes on its target OS. |+| screen-monitor-platform | screenpipe-screen | windows, macos, linux | vision-capture | capture-ocr-pipeline | medium | partial | unit | 5 | 36 | 5 | Per-OS monitor enumeration (Windows, macOS, Wayland/portal on Linux) and the persistent Windows.Graphics.Capture session. Each file is cfg-gated and only executes on its target OS. || screen-windows-ocr | screenpipe-screen | windows | ocr, vision-capture | capture-ocr-pipeline | high | partial | integration | 2 | 5 | 1 | Windows OCR fixture coverage plus an ignored continuous-capture probe that requires a live desktop. || sqlite-coordinator-durable-quarantine | screenpipe-sqlite-coordinator | windows, macos, linux | database, engine-lifecycle | engine-health-lifecycle, performance-liveness | high | strong | unit | 2 | 11 | 0 | Process-wide single-writer gates, SQLite runtime pinning, atomic hard-fault markers, OS file identity, fail-closed malformed metadata, fresh-identity resolution, and a real cross-process restart test. |@@ -455,7 +455,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| screen-capture-windowing | screenpipe-screen | src/monitor.rs | source | 5 | 0 | 5 || screen-monitor-platform | screenpipe-screen | src/monitor/linux_portal.rs | source | 5 | 0 | 5 || screen-monitor-platform | screenpipe-screen | src/monitor/linux_wayland.rs | source | 5 | 0 | 5 |-| screen-monitor-platform | screenpipe-screen | src/monitor/macos.rs | source | 12 | 2 | 14 |+| screen-monitor-platform | screenpipe-screen | src/monitor/macos.rs | source | 16 | 2 | 18 || screen-monitor-platform | screenpipe-screen | src/monitor/windows.rs | source | 2 | 1 | 3 || screen-capture-windowing | screenpipe-screen | src/ocr_cache.rs | source | 15 | 0 | 15 || screen-capture-windowing | screenpipe-screen | src/snapshot_writer.rs | source | 4 | 0 | 4 |