YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context
Rust agentsagiaiai-memoryaudio-recordingcomputer-visionhermeshermes-agentllmlocal-ailocal-firstmachine-learningmcpmultimodalopenclawprivacyrewindscreen-recordingspeech-to-textycombinator
fix(audio): refresh macOS device inventory on reconnect (#7132)
Louis Beaumont1d49aa58c283
diff --git a/crates/screenpipe-audio/src/core/device.rs b/crates/screenpipe-audio/src/core/device.rsindex 38ea6d4af..2574e751e 100644--- a/crates/screenpipe-audio/src/core/device.rs+++ b/crates/screenpipe-audio/src/core/device.rs@@ -610,46 +610,116 @@ fn ensure_usable_stream_config(channels: u16, sample_rate: u32, device_name: &stOk(())}-/// How long a cached device list is considered fresh. Audio devices change-/// rarely (hotplug, sleep/wake) — caching for 30s eliminates >99% of the-/// CoreAudio enumeration calls without making the UI feel stale.+/// Cache stable inventories for 30s, but never across a macOS topology change.const DEVICE_CACHE_TTL: Duration = Duration::from_secs(30);struct CachedDevices {devices: Vec<AudioDevice>,fetched_at: Instant,+ topology_generation: u64,}-/// Single source of truth for the device-list cache. The `AsyncMutex`-/// serializes refreshes — only one task at a time can hit cpal/CoreAudio.-/// On macOS this is critical: concurrent `cpal::Device::supports_input`-/// calls can race the CoreAudio IOThread on macOS 26.x and crash the-/// process with EXC_BAD_ACCESS at offset 0x410 (NULL deref of an-/// `AudioObjectID` whose listeners are being torn down).+/// Serializes enumeration: concurrent cpal/CoreAudio probes can race device+/// teardown on macOS 26.x. Notifications invalidate data, never this lock.fn device_cache() -> &'static AsyncMutex<Option<CachedDevices>> {static CACHE: OnceLock<AsyncMutex<Option<CachedDevices>>> = OnceLock::new();CACHE.get_or_init(|| AsyncMutex::new(None))}-/// List audio devices, served from a 30s cache when fresh.-///-/// This is the only public entry point — every caller (HTTP handler,-/// device manager, CLI) goes through here. Cached results avoid hammering-/// CoreAudio on every `/audio/list` request and serialize cold refreshes-/// behind a single mutex so two threads can't enumerate concurrently.+#[cfg(target_os = "macos")]+mod inventory_topology {+ use cidre::{core_audio as ca, os};+ use std::sync::{+ atomic::{AtomicU64, Ordering},+ OnceLock,+ };++ static GENERATION: AtomicU64 = AtomicU64::new(0);++ // CoreAudio owns this callback thread. No enumeration, allocation, locks,+ // or async work here. The next inventory read refreshes under its mutex.+ extern "C-unwind" fn changed(+ _object: ca::Obj,+ _count: u32,+ _addresses: *const ca::PropAddr,+ _context: *mut std::ffi::c_void,+ ) -> os::Status {+ GENERATION.fetch_add(1, Ordering::Release);+ os::Status(0)+ }++ pub(super) fn generation() -> u64 {+ // System object + static callback have process lifetime. Register once+ // even if many HTTP/monitor callers request inventories concurrently.+ static REGISTERED: OnceLock<()> = OnceLock::new();+ REGISTERED.get_or_init(|| {+ for selector in [+ ca::PropSelector::HW_DEVICES,+ ca::PropSelector::HW_DEFAULT_INPUT_DEVICE,+ ca::PropSelector::HW_DEFAULT_OUTPUT_DEVICE,+ ] {+ if let Err(error) = ca::System::OBJ.add_prop_listener::<std::ffi::c_void>(+ &selector.global_addr(),+ changed,+ std::ptr::null_mut(),+ ) {+ tracing::warn!(+ ?error,+ "audio inventory change listener unavailable; retaining TTL fallback"+ );+ }+ }+ });+ GENERATION.load(Ordering::Acquire)+ }+}++fn device_inventory_generation() -> u64 {+ #[cfg(target_os = "macos")]+ {+ inventory_topology::generation()+ }+ #[cfg(not(target_os = "macos"))]+ {+ 0+ }+}++/// Cached enumeration shared by HTTP, device startup, and the recovery monitor.+/// Hotplug/default-device notifications make a cached absence stale immediately.pub async fn list_audio_devices() -> Result<Vec<AudioDevice>> {- let mut cache = device_cache().lock().await;+ list_audio_devices_with_cache(+ device_cache(),+ device_inventory_generation,+ list_audio_devices_uncached,+ )+ .await+}++async fn list_audio_devices_with_cache<F, Fut>(+ cache: &AsyncMutex<Option<CachedDevices>>,+ generation: impl Fn() -> u64,+ enumerate: F,+) -> Result<Vec<AudioDevice>>+where+ F: FnOnce() -> Fut,+ Fut: std::future::Future<Output = Result<Vec<AudioDevice>>>,+{+ let mut cache = cache.lock().await;+ let topology_generation = generation();if let Some(c) = cache.as_ref() {- if c.fetched_at.elapsed() < DEVICE_CACHE_TTL {+ if c.topology_generation == topology_generation && c.fetched_at.elapsed() < DEVICE_CACHE_TTL+ {return Ok(c.devices.clone());}}- // Cache miss or stale — refresh under the lock so concurrent callers- // see exactly one underlying enumeration.- let devices = list_audio_devices_uncached().await?;+ let devices = enumerate().await?;*cache = Some(CachedDevices {devices: devices.clone(),fetched_at: Instant::now(),+ // Capture before enumeration: a change DURING the probe must make the+ // result stale on the next read, rather than being marked as handled.+ topology_generation,});Ok(devices)}@@ -1896,3 +1966,125 @@ mod stream_config_tests {assert!(ensure_usable_stream_config(1, 8_000, "Bluetooth HFP").is_ok());}}++#[cfg(test)]+mod inventory_cache_tests {+ use super::*;+ use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering};++ fn microphone() -> AudioDevice {+ AudioDevice {+ name: "Dock mic".to_string(),+ device_type: DeviceType::Input,+ }+ }++ async fn read(+ cache: &AsyncMutex<Option<CachedDevices>>,+ generation: &AtomicU64,+ calls: &AtomicUsize,+ devices: Vec<AudioDevice>,+ ) -> Vec<AudioDevice> {+ list_audio_devices_with_cache(+ cache,+ || generation.load(Ordering::Acquire),+ || async {+ calls.fetch_add(1, Ordering::Relaxed);+ tokio::task::yield_now().await;+ Ok(devices)+ },+ )+ .await+ .unwrap()+ }++ #[tokio::test]+ async fn inventory_cache_reconnect_replaces_cached_absence_before_ttl() {+ let cache = AsyncMutex::new(None);+ let generation = AtomicU64::new(0);+ let calls = AtomicUsize::new(0);+ assert!(read(&cache, &generation, &calls, vec![]).await.is_empty());+ generation.fetch_add(1, Ordering::Release);+ assert_eq!(+ read(&cache, &generation, &calls, vec![microphone()]).await,+ vec![microphone()]+ );+ assert_eq!(calls.load(Ordering::Relaxed), 2);+ // Removal and return of the same named microphone are separate edges.+ generation.fetch_add(1, Ordering::Release);+ assert!(read(&cache, &generation, &calls, vec![]).await.is_empty());+ generation.fetch_add(1, Ordering::Release);+ assert_eq!(+ read(&cache, &generation, &calls, vec![microphone()]).await,+ vec![microphone()]+ );+ }++ #[tokio::test]+ async fn inventory_cache_coalesces_event_burst_and_concurrent_readers() {+ let cache = AsyncMutex::new(None);+ let generation = AtomicU64::new(0);+ let calls = AtomicUsize::new(0);+ read(&cache, &generation, &calls, vec![]).await;+ generation.fetch_add(3, Ordering::Release);+ let reads = (0..16).map(|_| read(&cache, &generation, &calls, vec![microphone()]));+ let results = futures::future::join_all(reads).await;+ assert!(results.iter().all(|devices| devices == &vec![microphone()]));+ assert_eq!(+ calls.load(Ordering::Relaxed),+ 2,+ "a change burst needs one refresh, not one per reader"+ );+ }++ #[tokio::test]+ async fn inventory_cache_does_not_lose_change_during_enumeration() {+ let cache = AsyncMutex::new(None);+ let generation = AtomicU64::new(0);+ let calls = AtomicUsize::new(0);+ list_audio_devices_with_cache(+ &cache,+ || generation.load(Ordering::Acquire),+ || async {+ generation.fetch_add(1, Ordering::Release);+ Ok(vec![])+ },+ )+ .await+ .unwrap();+ assert_eq!(+ read(&cache, &generation, &calls, vec![microphone()]).await,+ vec![microphone()]+ );+ assert_eq!(calls.load(Ordering::Relaxed), 1);+ }++ #[tokio::test]+ async fn inventory_cache_retains_ttl_and_retries_failed_refresh() {+ let cache = AsyncMutex::new(Some(CachedDevices {+ devices: vec![],+ fetched_at: Instant::now() - DEVICE_CACHE_TTL,+ topology_generation: 0,+ }));+ let generation = AtomicU64::new(0);+ let calls = AtomicUsize::new(0);+ assert_eq!(+ read(&cache, &generation, &calls, vec![microphone()]).await,+ vec![microphone()]+ );+ generation.fetch_add(1, Ordering::Release);+ let failed = list_audio_devices_with_cache(+ &cache,+ || generation.load(Ordering::Acquire),+ || async { Err(anyhow!("transient enumeration failure")) },+ )+ .await;+ assert!(failed.is_err());+ assert!(read(&cache, &generation, &calls, vec![]).await.is_empty());+ // An absent device with no new event stays cached, avoiding retry storms.+ for _ in 0..8 {+ read(&cache, &generation, &calls, vec![]).await;+ }+ assert_eq!(calls.load(Ordering::Relaxed), 2);+ }+}diff --git a/docs/COVERAGE.md b/docs/COVERAGE.mdindex 82ab08822..176566bc1 100644--- a/docs/COVERAGE.md+++ b/docs/COVERAGE.md@@ -38,15 +38,15 @@ results and `cargo llvm-cov` data on top when judging release confidence.- Mapped suites: 34- Mapped Rust files: 362-- Active test blocks: 3552+- Active test blocks: 3556- Ignored/manual test blocks: 149-- Weighted coverage points: 2936.3+- Weighted coverage points: 2940.3| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 31 | 3409 | 143 | 2871.2 | 21 | 11 | 100% |-| macos | 31 | 3471 | 123 | 2885.2 | 22 | 11 | 100% |-| linux | 27 | 3031 | 115 | 2522.5 | 20 | 11 | 100% |+| windows | 31 | 3413 | 143 | 2875.2 | 21 | 11 | 100% |+| macos | 31 | 3475 | 123 | 2889.2 | 22 | 11 | 100% |+| linux | 27 | 3035 | 115 | 2526.5 | 20 | 11 | 100% |## Refreshdiff --git a/docs/coverage/CORE.md b/docs/coverage/CORE.mdindex 5aabd841d..df120119f 100644--- a/docs/coverage/CORE.md+++ b/docs/coverage/CORE.md@@ -9,10 +9,10 @@ confidence, and criticality.- Tracked crates: screenpipe-engine, screenpipe-db, screenpipe-sqlite-coordinator, screenpipe-audio, screenpipe-screen, screenpipe-a11y, screenpipe-fs- Mapped suites: 34- Mapped Rust files: 362-- Active test blocks: 3552+- Active test blocks: 3556- Ignored/manual test blocks: 149-- Declared test blocks: 3701-- Weighted coverage points: 2936.3+- Declared test blocks: 3705+- Weighted coverage points: 2940.3Confidence weights: strong=1.0, partial=0.7, conditional=0.4, smoke=0.3.Criticality weights: high=1.0, medium=0.7, low=0.4.@@ -23,9 +23,9 @@ are explicitly enabled in a runtime lane.| Platform | Suites | Active tests | Ignored tests | Weighted points | Layers | Flows | Critical score || --- | --- | --- | --- | --- | --- | --- | --- |-| windows | 31 | 3409 | 143 | 2871.2 | 21 | 11 | 100% |-| macos | 31 | 3471 | 123 | 2885.2 | 22 | 11 | 100% |-| linux | 27 | 3031 | 115 | 2522.5 | 20 | 11 | 100% |+| windows | 31 | 3413 | 143 | 2875.2 | 21 | 11 | 100% |+| macos | 31 | 3475 | 123 | 2889.2 | 22 | 11 | 100% |+| linux | 27 | 3035 | 115 | 2526.5 | 20 | 11 | 100% |## Crate Summary@@ -34,7 +34,7 @@ are explicitly enabled in a runtime lane.| screenpipe-engine | 11 | 20 | 120 | 1732 | 43 | 1335.9 | 10 || screenpipe-db | 5 | 60 | 23 | 558 | 24 | 531.3 | 9 || screenpipe-sqlite-coordinator | 1 | 0 | 3 | 27 | 0 | 27.0 | 2 |-| screenpipe-audio | 6 | 25 | 51 | 621 | 44 | 545.6 | 5 |+| screenpipe-audio | 6 | 25 | 51 | 625 | 44 | 549.6 | 5 || screenpipe-screen | 6 | 9 | 18 | 257 | 9 | 232.1 | 4 || screenpipe-a11y | 4 | 2 | 30 | 354 | 29 | 261.3 | 3 || screenpipe-fs | 1 | 0 | 1 | 3 | 0 | 3.0 | 3 |@@ -63,8 +63,8 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| Layer | windows | macos | linux || --- | --- | --- | --- || accessibility | 4 suites / 352 active / 30 ignored / 323.5 pts | 4 suites / 410 active / 10 ignored / 332.0 pts | 4 suites / 330 active / 7 ignored / 302.5 pts |-| audio | 8 suites / 737 active / 45 ignored / 661.6 pts | 8 suites / 737 active / 45 ignored / 661.6 pts | 7 suites / 662 active / 44 ignored / 609.1 pts |-| audio-device | 2 suites / 218 active / 7 ignored / 195.5 pts | 2 suites / 218 active / 7 ignored / 195.5 pts | 1 suites / 143 active / 6 ignored / 143.0 pts |+| audio | 8 suites / 741 active / 45 ignored / 665.6 pts | 8 suites / 741 active / 45 ignored / 665.6 pts | 7 suites / 666 active / 44 ignored / 613.1 pts |+| audio-device | 2 suites / 222 active / 7 ignored / 199.5 pts | 2 suites / 222 active / 7 ignored / 199.5 pts | 1 suites / 147 active / 6 ignored / 147.0 pts || configuration | 2 suites / 143 active / 3 ignored / 129.2 pts | 2 suites / 143 active / 3 ignored / 129.2 pts | 2 suites / 143 active / 3 ignored / 129.2 pts || database | 7 suites / 509 active / 20 ignored / 482.3 pts | 7 suites / 509 active / 20 ignored / 482.3 pts | 7 suites / 509 active / 20 ignored / 482.3 pts || db-search | 2 suites / 113 active / 9 ignored / 113.0 pts | 2 suites / 113 active / 9 ignored / 113.0 pts | 2 suites / 113 active / 9 ignored / 113.0 pts |@@ -73,7 +73,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| meeting | 6 suites / 1636 active / 21 ignored / 1336.9 pts | 6 suites / 1636 active / 21 ignored / 1336.9 pts | 4 suites / 1291 active / 17 ignored / 1014.4 pts || ocr | 4 suites / 125 active / 7 ignored / 119.0 pts | 4 suites / 129 active / 7 ignored / 124.5 pts | 3 suites / 120 active / 6 ignored / 115.5 pts || os-integration | 1 suites / 6 active / 0 ignored / 1.7 pts | 1 suites / 6 active / 0 ignored / 1.7 pts | - |-| performance | 13 suites / 1563 active / 75 ignored / 1382.6 pts | 14 suites / 1670 active / 79 ignored / 1425.4 pts | 13 suites / 1563 active / 75 ignored / 1382.6 pts |+| performance | 13 suites / 1567 active / 75 ignored / 1386.6 pts | 14 suites / 1674 active / 79 ignored / 1429.4 pts | 13 suites / 1567 active / 75 ignored / 1386.6 pts || pipes | 1 suites / 511 active / 3 ignored / 357.7 pts | 1 suites / 511 active / 3 ignored / 357.7 pts | 1 suites / 511 active / 3 ignored / 357.7 pts || privacy | 5 suites / 942 active / 37 ignored / 763.8 pts | 5 suites / 1000 active / 17 ignored / 772.3 pts | 5 suites / 920 active / 14 ignored / 742.7 pts || real-app | - | 1 suites / 107 active / 4 ignored / 42.8 pts | - |@@ -123,7 +123,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| a11y-linux-tree | screenpipe-a11y | linux | accessibility, privacy | accessibility-ui-events, privacy-and-redaction | medium | partial | unit | 4 | 27 | 1 | Linux-specific accessibility/incognito normalization tests. || a11y-macos-tree | screenpipe-a11y | macos | accessibility, privacy, real-app, performance | accessibility-ui-events, privacy-and-redaction, performance-liveness | high | conditional | mixed | 8 | 107 | 4 | macOS AX unit coverage, a scored 100-case click-attribution policy eval, and real TextEdit/Finder/Obsidian probes. Click attribution and Obsidian live tests are ignored by default when they require a logged-in desktop, app install, or AX permission. || a11y-windows-tree | screenpipe-a11y | windows | accessibility, privacy, ui-events | accessibility-ui-events, privacy-and-redaction | high | partial | unit | 6 | 49 | 24 | Windows UIA/accessibility parsing and privacy matching; some UIA tests are ignored where they require a live desktop. |-| audio-device-stream-health | screenpipe-audio | windows, macos, linux | audio-device, audio, performance | audio-device-health, audio-record-transcribe, performance-liveness | high | strong | mixed | 14 | 143 | 6 | Device monitor, device manager, stream buffering, source lag, audio metrics, Bluetooth gap/hallucination regressions, and cross-platform process-tap watchdog counters (process_tap.rs split into src/core/process_tap/ modules). |+| audio-device-stream-health | screenpipe-audio | windows, macos, linux | audio-device, audio, performance | audio-device-health, audio-record-transcribe, performance-liveness | high | strong | mixed | 14 | 147 | 6 | Device monitor, device manager, stream buffering, source lag, audio metrics, Bluetooth gap/hallucination regressions, and cross-platform process-tap watchdog counters (process_tap.rs split into src/core/process_tap/ modules). || audio-meetings-speakers-dedup | screenpipe-audio | windows, macos, linux | audio, meeting, speaker, transcription | audio-record-transcribe, meeting-live-notes, performance-liveness | high | strong | mixed | 26 | 256 | 8 | Meeting streaming config/controller logic, speaker embedding state, cross-device dedupe simulations, and overlap cleanup coverage. || audio-models-filtering | screenpipe-audio | windows, macos, linux | audio, transcription, privacy | audio-record-transcribe, privacy-and-redaction | medium | partial | mixed | 6 | 20 | 10 | Model-download/TLS guards, ONNX startup smoke, and music-versus-speech filtering. || audio-pipeline-benchmarks | screenpipe-audio | windows, macos, linux | audio, transcription, performance | audio-record-transcribe, meeting-live-notes, performance-liveness | medium | partial | benchmark | 8 | 22 | 12 | Benchmark-backed regression probes for VAD, smart mode, meeting audio, quality, cross-device, and end-to-end pipeline timing. |@@ -198,7 +198,7 @@ bun run coverage:core -- --llvm-cov-summary ../../docs/coverage/core-llvm-cov-su| audio-platform-output-capture | screenpipe-audio | src/audio_manager/windows_output_follow.rs | source | 15 | 0 | 15 || audio-meetings-speakers-dedup | screenpipe-audio | src/audio_manager/write_retry.rs | source | 2 | 0 | 2 || audio-device-stream-health | screenpipe-audio | src/core/device_detection.rs | source | 15 | 0 | 15 |-| audio-device-stream-health | screenpipe-audio | src/core/device.rs | source | 19 | 1 | 20 |+| audio-device-stream-health | screenpipe-audio | src/core/device.rs | source | 23 | 1 | 24 || audio-device-stream-health | screenpipe-audio | src/core/e2e_ghost_word_silent_room.rs | source | 0 | 2 | 2 || audio-transcription-pipeline | screenpipe-audio | src/core/engine.rs | source | 10 | 0 | 10 || audio-platform-output-capture | screenpipe-audio | src/core/meeting_audio/macos.rs | source | 6 | 0 | 6 |