Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.
TypeScript cross-platformelectronffmpegopen-sourceproduct-demorustscreen-capturescreen-recorderscreen-recordingscreen-studioscreencastspeech-to-texttypescriptvideo-editingvideo-editorwaylandwgpuwhisper
fix: honor recorded cursor visibility in compositor
NoahSeeger3a6e375f6d40
diff --git a/crates/compositor/src/cursor.rs b/crates/compositor/src/cursor.rsindex fb859ab34..2c8fb019a 100644--- a/crates/compositor/src/cursor.rs+++ b/crates/compositor/src/cursor.rs@@ -42,6 +42,9 @@ pub struct CursorTrack {/// réellement cliqué. `smoothed()` la garde telle quelle, alors que la piste lissée passe/// ailleurs à cet instant (cf. `pinned_at`).click_points: Vec<(f32, f32)>,+ /// CHANGEMENTS de visibilité : `(instant, visible)`, triés. Comme `types`, c'est une+ /// fonction en escalier. Une piste ancienne sans champ `visible` reste visible.+ visibility: Vec<(f32, bool)>,/// CHANGEMENTS d'état du curseur : (instant, `"arrow"` / `"text"` / `"pointer"` / …), triés./// Une fonction en escalier, pas une valeur par échantillon : l'état tient sur des secondes/// entières alors que la position est échantillonnée toutes les 33 ms (~30 Hz, cf.@@ -135,6 +138,15 @@ impl CursorTrack {samples: Vec<(f32, f32, f32)>,clicks: Vec<f32>,types: Vec<(f32, String)>,+ ) -> CursorTrack {+ CursorTrack::new_with_visibility(samples, clicks, types, Vec::new())+ }++ fn new_with_visibility(+ samples: Vec<(f32, f32, f32)>,+ clicks: Vec<f32>,+ types: Vec<(f32, String)>,+ visibility: Vec<(f32, bool)>,) -> CursorTrack {let follow_samples = smooth_follow_samples(&samples);let click_points = clicks@@ -146,11 +158,23 @@ impl CursorTrack {follow_samples,clicks,click_points,+ visibility,types,view_cache: Default::default(),}}+ /// Visibilité enregistrée au temps `t`. Les anciens sidecars qui n'ont pas ce champ sont+ /// visibles par défaut, comme les lecteurs TypeScript.+ pub fn visible_at(&self, t: f32) -> bool {+ let i = self.visibility.partition_point(|(tc, _)| *tc <= t);+ if i == 0 {+ true+ } else {+ self.visibility[i - 1].1+ }+ }+/// État du curseur au temps `t` : la dernière transition à `t` ou avant. `None` avant la/// première (enregistrement sans état tagué → l'appelant retombe sur la flèche).pub fn type_at(&self, t: f32) -> Option<&str> {@@ -171,6 +195,7 @@ impl CursorTrack {let mut samples = Vec::new();let mut clicks = Vec::new();let mut types: Vec<(f32, String)> = Vec::new();+ let mut visibility: Vec<(f32, bool)> = Vec::new();let end = offset_ms + dur_s * 1000.0;for s in arr {let tm = s["timeMs"].as_f64().unwrap_or(-1.0);@@ -181,7 +206,13 @@ impl CursorTrack {let cx = s["cx"].as_f64().unwrap_or(0.0) as f32;let cy = s["cy"].as_f64().unwrap_or(0.0) as f32;samples.push((t, cx, cy));- if s["interactionType"].as_str() == Some("click") {+ let visible = s.get("visible").and_then(|value| value.as_bool()).unwrap_or(true);+ if visibility.last().map(|(_, previous)| *previous) != Some(visible) {+ // La première valeur de la fenêtre vaut dès son début. Sans cela, un clip dont+ // le curseur était déjà masqué à `offset_ms` affichait une frame fugitive.+ visibility.push((if visibility.is_empty() { 0.0 } else { t }, visible));+ }+ if visible && s["interactionType"].as_str() == Some("click") {clicks.push(t);}// Seules les TRANSITIONS sont retenues — voir `types`. Le helper@@ -204,7 +235,8 @@ impl CursorTrack {samples.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());clicks.sort_by(|a, b| a.partial_cmp(b).unwrap());types.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());- Ok(CursorTrack::new(samples, clicks, types))+ visibility.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());+ Ok(CursorTrack::new_with_visibility(samples, clicks, types, visibility))}/// Position lissée au temps `t`, pour le suivi auto du zoom. La télémétrie brute est@@ -411,10 +443,11 @@ impl CursorTrack {/// bruts (le bounce est temporel, pas positionnel — ne doit pas suivre le lissage).pub fn smoothed(&self, factor: f32) -> CursorTrack {if self.samples.len() < 2 || factor <= 0.0 {- return CursorTrack::new(+ return CursorTrack::new_with_visibility(self.samples.clone(),self.clicks.clone(),self.types.clone(),+ self.visibility.clone(),);}const STEP_S: f32 = 1.0 / 240.0;@@ -437,8 +470,33 @@ impl CursorTrack {raw_y.push(cy);}let (stiffness, damping, mass) = cursor_spring_config(factor);- let xs = spring_smooth(&raw_x, stiffness, damping, mass, STEP_S);- let ys = spring_smooth(&raw_y, stiffness, damping, mass, STEP_S);+ // Une phase masquée coupe le ressort. Au retour, le curseur repart du premier point+ // réellement visible au lieu de traverser la trajectoire cachée avec du retard.+ let mut xs = Vec::with_capacity(n);+ let mut ys = Vec::with_capacity(n);+ let mut run_start = 0;+ while run_start < n {+ let visible = self.visible_at(times[run_start]);+ let mut run_end = run_start + 1;+ while run_end < n && self.visible_at(times[run_end]) == visible {+ run_end += 1;+ }+ xs.extend(spring_smooth(+ &raw_x[run_start..run_end],+ stiffness,+ damping,+ mass,+ STEP_S,+ ));+ ys.extend(spring_smooth(+ &raw_y[run_start..run_end],+ stiffness,+ damping,+ mass,+ STEP_S,+ ));+ run_start = run_end;+ }let samples = times.into_iter().zip(xs)@@ -450,7 +508,9 @@ impl CursorTrack {// cliqués restent ceux de la piste brute.CursorTrack {click_points: self.click_points.clone(),- ..CursorTrack::new(samples, self.clicks.clone(), self.types.clone())+ ..CursorTrack::new_with_visibility(+ samples, self.clicks.clone(), self.types.clone(), self.visibility.clone(),+ )}}@@ -755,6 +815,56 @@ mod tests {assert_eq!(smoothed.type_at(0.7), Some("text"));}+ #[test]+ fn load_preserves_hidden_intervals_and_ignores_hidden_clicks() {+ let unique = std::time::SystemTime::now()+ .duration_since(std::time::UNIX_EPOCH)+ .map(|d| d.as_nanos())+ .unwrap_or(0);+ let path = std::env::temp_dir().join(format!(+ "openscreen-cursor-visibility-{}-{}.json", std::process::id(), unique+ ));+ std::fs::write(+ &path,+ r#"{"samples":[+ {"timeMs":0,"cx":0.1,"cy":0.1,"visible":true},+ {"timeMs":100,"cx":0.2,"cy":0.2,"visible":false,"interactionType":"click"},+ {"timeMs":500,"cx":0.8,"cy":0.8,"visible":false},+ {"timeMs":600,"cx":0.9,"cy":0.9,"visible":true,"interactionType":"click"}+ ]}"#,+ )+ .expect("write temp sidecar");+ let path_str = path.to_str().expect("utf-8 temp path");+ let track = CursorTrack::load(path_str, 0.0, 1.0).expect("load sidecar");+ let clipped = CursorTrack::load(path_str, 200.0, 0.5).expect("load clipped sidecar");+ let _ = std::fs::remove_file(&path);++ assert!(track.visible_at(0.0));+ assert!(!track.visible_at(0.1));+ assert!(!track.visible_at(0.59));+ assert!(track.visible_at(0.6));+ assert_eq!(track.clicks, vec![0.6], "le clic masqué ne doit pas animer le curseur");+ assert!(!clipped.visible_at(0.0), "la fenêtre commence dans la phase masquée");+ assert!(clipped.visible_at(0.4), "la transition visible garde son temps relatif");++ let smoothed = track.smoothed(0.5);+ assert!(!smoothed.visible_at(0.3), "le lissage garde la phase masquée");+ assert!(smoothed.visible_at(0.6));+ let reappeared = smoothed.at(0.6).expect("position at reappearance");+ assert!(+ (reappeared.0 - 0.9).abs() < 0.02 && (reappeared.1 - 0.9).abs() < 0.02,+ "le ressort redémarre au point visible, sans traîne: {reappeared:?}"+ );+ }++ #[test]+ fn legacy_tracks_without_visibility_stay_visible() {+ let track = CursorTrack::new(vec![(0.0, 0.1, 0.1), (1.0, 0.9, 0.9)], vec![], vec![]);+ assert!(track.visible_at(0.0));+ assert!(track.visible_at(0.5));+ assert!(track.visible_at(10.0));+ }+/// JSON null et une clé `cursorType` absente resetent vers la flèche,/// au lieu de garder le dernier `pointer`/`text`.#[test]diff --git a/crates/compositor/src/frame_geometry.rs b/crates/compositor/src/frame_geometry.rsindex d098b2268..beda24f0f 100644--- a/crates/compositor/src/frame_geometry.rs+++ b/crates/compositor/src/frame_geometry.rs@@ -3451,6 +3451,9 @@ pub fn cursor_alpha(if !scene.map(|s| s.cursor.show).unwrap_or(cfg.cursor) {return 0.0;}+ if !track.visible_at(t) {+ return 0.0;+ }let idle_alpha = track.opacity_at(t, live.cursor_auto_hide);let zoom_alpha = match scene {Some(s) => {diff --git a/src/lib/ai-edition/timeline/zoom-suggestions.test.ts b/src/lib/ai-edition/timeline/zoom-suggestions.test.tsindex accf82f68..6569abb66 100644--- a/src/lib/ai-edition/timeline/zoom-suggestions.test.ts+++ b/src/lib/ai-edition/timeline/zoom-suggestions.test.ts@@ -107,6 +107,34 @@ describe("buildAutoZoomSuggestions", () => {expect(suggestions[0].focus).toEqual({ cx: 0.2, cy: 0.8 });});+ it("keeps a legacy click without mouseup when the cursor hides much later", () => {+ const suggestions = buildAutoZoomSuggestions({+ cursorTelemetry: [+ { ...click(1000, 0.2, 0.8), visible: true },+ { timeMs: 1050, cx: 0.3, cy: 0.7, visible: true, interactionType: "move" },+ { timeMs: 2000, cx: 0.7, cy: 0.3, visible: false, interactionType: "move" },+ ],+ totalMs: 4000,+ existingRegions: [],+ defaultDurationMs: 1000,+ });+ expect(suggestions).toHaveLength(1);+ expect(suggestions[0].focus).toEqual({ cx: 0.2, cy: 0.8 });+ });++ it("rejects a legacy click when the cursor hides on the next sampler tick", () => {+ const suggestions = buildAutoZoomSuggestions({+ cursorTelemetry: [+ { ...click(1000, 0.2, 0.8), visible: true },+ { timeMs: 1033, cx: 0.25, cy: 0.75, visible: false, interactionType: "move" },+ ],+ totalMs: 4000,+ existingRegions: [],+ defaultDurationMs: 1000,+ });+ expect(suggestions).toEqual([]);+ });+it("does not join visible samples across a hidden interval into a dwell", () => {const suggestions = buildAutoZoomSuggestions({cursorTelemetry: [diff --git a/src/lib/ai-edition/timeline/zoom-suggestions.ts b/src/lib/ai-edition/timeline/zoom-suggestions.tsindex 145b892cd..26c7f0946 100644--- a/src/lib/ai-edition/timeline/zoom-suggestions.ts+++ b/src/lib/ai-edition/timeline/zoom-suggestions.ts@@ -127,10 +127,20 @@ const CLICK_INTERACTION_TYPES: ReadonlySet<NonNullable<CursorTelemetryPoint["int*/function detectZoomClickCandidates(samples: CursorTelemetryPoint[]): ZoomDwellCandidate[] {const candidates: ZoomDwellCandidate[] = [];+ const hasMouseupTelemetry = samples.some((sample) => sample.interactionType === "mouseup");+ // PipeWire and older sidecars have clicks but no mouse-up events. A hidden sample on the next+ // sampler tick still identifies a slider press; a much later hidden interval must not erase an+ // already completed click merely because that format cannot mark its release.+ const LEGACY_PRESS_GRACE_MS = 150;+let pendingClick: CursorTelemetryPoint | null = null;- let hiddenDuringPress = false;+ let hiddenAtMs: number | null = null;const finishPress = () => {- if (pendingClick && !hiddenDuringPress) {+ if (!pendingClick) return;+ const hiddenDuringPress =+ hiddenAtMs !== null &&+ (hasMouseupTelemetry || hiddenAtMs - pendingClick.timeMs <= LEGACY_PRESS_GRACE_MS);+ if (!hiddenDuringPress) {candidates.push({centerTimeMs: pendingClick.timeMs,focus: { cx: pendingClick.cx, cy: pendingClick.cy },@@ -138,17 +148,19 @@ function detectZoomClickCandidates(samples: CursorTelemetryPoint[]): ZoomDwellCa});}pendingClick = null;- hiddenDuringPress = false;+ hiddenAtMs = null;};for (const sample of samples) {if (sample.interactionType && CLICK_INTERACTION_TYPES.has(sample.interactionType)) {finishPress();pendingClick = sample.visible === false ? null : sample;- } else {- if (sample.visible === false) hiddenDuringPress = true;- if (sample.interactionType === "mouseup") finishPress();+ continue;+ }+ if (pendingClick && sample.visible === false && hiddenAtMs === null) {+ hiddenAtMs = sample.timeMs;}+ if (sample.interactionType === "mouseup") finishPress();}finishPress();return candidates;diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.mdindex ea959a411..760959666 100644--- a/technical-documentation/testing/manual-e2e-checklist.md+++ b/technical-documentation/testing/manual-e2e-checklist.md@@ -610,6 +610,7 @@ The mask comes from the native compositor (ONNX Runtime + the vendored selfie-se| Date | Build / tag | Platform | Pass/fail | Notes ||------|-------------|----------|-----------|-------|+| 2026-09-23 | dev `fix/725-hidden-cursor-auto-zoom`; rebuilt compositor addon SHA256 `a0bcd61e17084c010141049a9978121e45be2985449ff1618ab6a3145f8381f6` | Windows 11 Home 26200 | Partial — issue #725 slider slice passed | Human-driven real recording in the Electron dev build: moving a slider whose application cursor becomes hidden kept the reconstructed cursor hidden in the Studio recording and did not create a click-driven automatic zoom for that press. A visible control click still produced the normal automatic zoom. Sidecar inspection independently confirmed the hidden interval (`visible: false`) and that no zoom was centred on the press immediately preceding it. Recording, stop and editor open passed. Not covered: export, packaged build, audio/webcam, tray/HUD, other editor features, macOS or Linux. || 2026-09-12 | dev `fix/643-gif-export-cancel`, based on `fbe461e9`; rebuilt compositor addon SHA256 `128c89fe65a459b3f17721d610ea9a78e0d3e980e9eb0b5fb2b0e5d018619e15` | Windows | Partial — export cancellation passed | Computer-use OS input, real Electron and native save dialog, imported synthetic 8 s source. GIF 15 FPS / Medium (720p) / loop on: observed 55/120 frames, clicked Cancel while rendering (pending snapshot 104/120), returned to the same options; no final or partial output remained. Retried in that dialog to the same path and received Saved to; resulting GIF is 1280×720, 120 frames, 8,200,202 bytes, fully decodes with ffmpeg (exit 0). Automated real-addon tests separately pass existing-destination preservation, GIF retry, ordinary GIF and MP4 export. Desktop adapter required screenshot-based coordinates and Ctrl+O to reach the editor; HUD reachability is not claimed. An unrelated process error window partly obscured the retry success screenshot; file verification independently passed. Skipped: recording, audio/mic, webcam, tray/HUD, effect/caption comparisons, second viewer, macOS/Linux and release promotion. Task-local profile and evidence retained; test app stopped. || 2026-07-31 | dev build, `claude/e2e-tests-v1-8-0-474894` (e9578f09) | macOS 26.5, M1 | Partial — 1 defect | Ran launch/HUD, media, modifier anchoring, and export. **Defect: a dangling asset blanks the preview.** Modifier anchoring across a reorder verified in preview and in the exported frames. macOS export produced 1280×720 h264 + AAC at ~2× realtime. Chat sections skipped: no AI provider configured. HUD drag not runnable under computer-use (drop point is the desktop). || 2026-08-13 | installed `v1.9.5-rc.1` | Windows 11 26200, 1920×1080 @ 100% | Partial — 1 defect | Ran launch/HUD, source selection, recording, stop, editor open. Fragmented MP4 confirmed on the shipped artifact: 48 `moof`+`mdat` pairs over 47.6s, `mvex` present, `mfra` on clean stop. **Defect: a recording that survives a helper kill is thrown away by the app** — killing `wgc-capture.exe` mid-recording leaves a fully decodable 41s file (2460 packets, `ffmpeg -f null -` exit 0) with no `.session.json` and no `.cursor.json`, and stop answers "The recording could not be saved". Fixed in #363, re-verified end to end. Truncation ablation at 60%: plain MP4 unreadable, fragmented plays 29s. **A dev build cannot test any of this** — the prebuilt worktree helper predated the change and silently ran the old path. Editor/export/chat sections not run. |