Skip to content

Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.

fix: honor recorded cursor visibility in compositor

NoahSeeger3a6e375f6d40
diff --git a/crates/compositor/src/cursor.rs b/crates/compositor/src/cursor.rs
index fb859ab34..2c8fb019a 100644
--- a/crates/compositor/src/cursor.rs
+++ b/crates/compositor/src/cursor.rs
@@ -42,6 +42,9 @@ pub struct CursorTrack {
/// réellement cliqué. `smoothed()` la garde telle quelle, alors que la piste lissée passe
/// ailleurs à cet instant (cf. `pinned_at`).
click_points: Vec<(f32, f32)>,
+ /// CHANGEMENTS de visibilité : `(instant, visible)`, triés. Comme `types`, c'est une
+ /// fonction en escalier. Une piste ancienne sans champ `visible` reste visible.
+ visibility: Vec<(f32, bool)>,
/// CHANGEMENTS d'état du curseur : (instant, `"arrow"` / `"text"` / `"pointer"` / …), triés.
/// Une fonction en escalier, pas une valeur par échantillon : l'état tient sur des secondes
/// entières alors que la position est échantillonnée toutes les 33 ms (~30 Hz, cf.
@@ -135,6 +138,15 @@ impl CursorTrack {
samples: Vec<(f32, f32, f32)>,
clicks: Vec<f32>,
types: Vec<(f32, String)>,
+ ) -> CursorTrack {
+ CursorTrack::new_with_visibility(samples, clicks, types, Vec::new())
+ }
+
+ fn new_with_visibility(
+ samples: Vec<(f32, f32, f32)>,
+ clicks: Vec<f32>,
+ types: Vec<(f32, String)>,
+ visibility: Vec<(f32, bool)>,
) -> CursorTrack {
let follow_samples = smooth_follow_samples(&samples);
let click_points = clicks
@@ -146,11 +158,23 @@ impl CursorTrack {
follow_samples,
clicks,
click_points,
+ visibility,
types,
view_cache: Default::default(),
}
}
+ /// Visibilité enregistrée au temps `t`. Les anciens sidecars qui n'ont pas ce champ sont
+ /// visibles par défaut, comme les lecteurs TypeScript.
+ pub fn visible_at(&self, t: f32) -> bool {
+ let i = self.visibility.partition_point(|(tc, _)| *tc <= t);
+ if i == 0 {
+ true
+ } else {
+ self.visibility[i - 1].1
+ }
+ }
+
/// État du curseur au temps `t` : la dernière transition à `t` ou avant. `None` avant la
/// première (enregistrement sans état tagué → l'appelant retombe sur la flèche).
pub fn type_at(&self, t: f32) -> Option<&str> {
@@ -171,6 +195,7 @@ impl CursorTrack {
let mut samples = Vec::new();
let mut clicks = Vec::new();
let mut types: Vec<(f32, String)> = Vec::new();
+ let mut visibility: Vec<(f32, bool)> = Vec::new();
let end = offset_ms + dur_s * 1000.0;
for s in arr {
let tm = s["timeMs"].as_f64().unwrap_or(-1.0);
@@ -181,7 +206,13 @@ impl CursorTrack {
let cx = s["cx"].as_f64().unwrap_or(0.0) as f32;
let cy = s["cy"].as_f64().unwrap_or(0.0) as f32;
samples.push((t, cx, cy));
- if s["interactionType"].as_str() == Some("click") {
+ let visible = s.get("visible").and_then(|value| value.as_bool()).unwrap_or(true);
+ if visibility.last().map(|(_, previous)| *previous) != Some(visible) {
+ // La première valeur de la fenêtre vaut dès son début. Sans cela, un clip dont
+ // le curseur était déjà masqué à `offset_ms` affichait une frame fugitive.
+ visibility.push((if visibility.is_empty() { 0.0 } else { t }, visible));
+ }
+ if visible && s["interactionType"].as_str() == Some("click") {
clicks.push(t);
}
// Seules les TRANSITIONS sont retenues — voir `types`. Le helper
@@ -204,7 +235,8 @@ impl CursorTrack {
samples.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());
clicks.sort_by(|a, b| a.partial_cmp(b).unwrap());
types.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());
- Ok(CursorTrack::new(samples, clicks, types))
+ visibility.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap());
+ Ok(CursorTrack::new_with_visibility(samples, clicks, types, visibility))
}
/// Position lissée au temps `t`, pour le suivi auto du zoom. La télémétrie brute est
@@ -411,10 +443,11 @@ impl CursorTrack {
/// bruts (le bounce est temporel, pas positionnel — ne doit pas suivre le lissage).
pub fn smoothed(&self, factor: f32) -> CursorTrack {
if self.samples.len() < 2 || factor <= 0.0 {
- return CursorTrack::new(
+ return CursorTrack::new_with_visibility(
self.samples.clone(),
self.clicks.clone(),
self.types.clone(),
+ self.visibility.clone(),
);
}
const STEP_S: f32 = 1.0 / 240.0;
@@ -437,8 +470,33 @@ impl CursorTrack {
raw_y.push(cy);
}
let (stiffness, damping, mass) = cursor_spring_config(factor);
- let xs = spring_smooth(&raw_x, stiffness, damping, mass, STEP_S);
- let ys = spring_smooth(&raw_y, stiffness, damping, mass, STEP_S);
+ // Une phase masquée coupe le ressort. Au retour, le curseur repart du premier point
+ // réellement visible au lieu de traverser la trajectoire cachée avec du retard.
+ let mut xs = Vec::with_capacity(n);
+ let mut ys = Vec::with_capacity(n);
+ let mut run_start = 0;
+ while run_start < n {
+ let visible = self.visible_at(times[run_start]);
+ let mut run_end = run_start + 1;
+ while run_end < n && self.visible_at(times[run_end]) == visible {
+ run_end += 1;
+ }
+ xs.extend(spring_smooth(
+ &raw_x[run_start..run_end],
+ stiffness,
+ damping,
+ mass,
+ STEP_S,
+ ));
+ ys.extend(spring_smooth(
+ &raw_y[run_start..run_end],
+ stiffness,
+ damping,
+ mass,
+ STEP_S,
+ ));
+ run_start = run_end;
+ }
let samples = times
.into_iter()
.zip(xs)
@@ -450,7 +508,9 @@ impl CursorTrack {
// cliqués restent ceux de la piste brute.
CursorTrack {
click_points: self.click_points.clone(),
- ..CursorTrack::new(samples, self.clicks.clone(), self.types.clone())
+ ..CursorTrack::new_with_visibility(
+ samples, self.clicks.clone(), self.types.clone(), self.visibility.clone(),
+ )
}
}
@@ -755,6 +815,56 @@ mod tests {
assert_eq!(smoothed.type_at(0.7), Some("text"));
}
+ #[test]
+ fn load_preserves_hidden_intervals_and_ignores_hidden_clicks() {
+ let unique = std::time::SystemTime::now()
+ .duration_since(std::time::UNIX_EPOCH)
+ .map(|d| d.as_nanos())
+ .unwrap_or(0);
+ let path = std::env::temp_dir().join(format!(
+ "openscreen-cursor-visibility-{}-{}.json", std::process::id(), unique
+ ));
+ std::fs::write(
+ &path,
+ r#"{"samples":[
+ {"timeMs":0,"cx":0.1,"cy":0.1,"visible":true},
+ {"timeMs":100,"cx":0.2,"cy":0.2,"visible":false,"interactionType":"click"},
+ {"timeMs":500,"cx":0.8,"cy":0.8,"visible":false},
+ {"timeMs":600,"cx":0.9,"cy":0.9,"visible":true,"interactionType":"click"}
+ ]}"#,
+ )
+ .expect("write temp sidecar");
+ let path_str = path.to_str().expect("utf-8 temp path");
+ let track = CursorTrack::load(path_str, 0.0, 1.0).expect("load sidecar");
+ let clipped = CursorTrack::load(path_str, 200.0, 0.5).expect("load clipped sidecar");
+ let _ = std::fs::remove_file(&path);
+
+ assert!(track.visible_at(0.0));
+ assert!(!track.visible_at(0.1));
+ assert!(!track.visible_at(0.59));
+ assert!(track.visible_at(0.6));
+ assert_eq!(track.clicks, vec![0.6], "le clic masqué ne doit pas animer le curseur");
+ assert!(!clipped.visible_at(0.0), "la fenêtre commence dans la phase masquée");
+ assert!(clipped.visible_at(0.4), "la transition visible garde son temps relatif");
+
+ let smoothed = track.smoothed(0.5);
+ assert!(!smoothed.visible_at(0.3), "le lissage garde la phase masquée");
+ assert!(smoothed.visible_at(0.6));
+ let reappeared = smoothed.at(0.6).expect("position at reappearance");
+ assert!(
+ (reappeared.0 - 0.9).abs() < 0.02 && (reappeared.1 - 0.9).abs() < 0.02,
+ "le ressort redémarre au point visible, sans traîne: {reappeared:?}"
+ );
+ }
+
+ #[test]
+ fn legacy_tracks_without_visibility_stay_visible() {
+ let track = CursorTrack::new(vec![(0.0, 0.1, 0.1), (1.0, 0.9, 0.9)], vec![], vec![]);
+ assert!(track.visible_at(0.0));
+ assert!(track.visible_at(0.5));
+ assert!(track.visible_at(10.0));
+ }
+
/// JSON null et une clé `cursorType` absente resetent vers la flèche,
/// au lieu de garder le dernier `pointer`/`text`.
#[test]
diff --git a/crates/compositor/src/frame_geometry.rs b/crates/compositor/src/frame_geometry.rs
index d098b2268..beda24f0f 100644
--- a/crates/compositor/src/frame_geometry.rs
+++ b/crates/compositor/src/frame_geometry.rs
@@ -3451,6 +3451,9 @@ pub fn cursor_alpha(
if !scene.map(|s| s.cursor.show).unwrap_or(cfg.cursor) {
return 0.0;
}
+ if !track.visible_at(t) {
+ return 0.0;
+ }
let idle_alpha = track.opacity_at(t, live.cursor_auto_hide);
let zoom_alpha = match scene {
Some(s) => {
diff --git a/src/lib/ai-edition/timeline/zoom-suggestions.test.ts b/src/lib/ai-edition/timeline/zoom-suggestions.test.ts
index accf82f68..6569abb66 100644
--- a/src/lib/ai-edition/timeline/zoom-suggestions.test.ts
+++ b/src/lib/ai-edition/timeline/zoom-suggestions.test.ts
@@ -107,6 +107,34 @@ describe("buildAutoZoomSuggestions", () => {
expect(suggestions[0].focus).toEqual({ cx: 0.2, cy: 0.8 });
});
+ it("keeps a legacy click without mouseup when the cursor hides much later", () => {
+ const suggestions = buildAutoZoomSuggestions({
+ cursorTelemetry: [
+ { ...click(1000, 0.2, 0.8), visible: true },
+ { timeMs: 1050, cx: 0.3, cy: 0.7, visible: true, interactionType: "move" },
+ { timeMs: 2000, cx: 0.7, cy: 0.3, visible: false, interactionType: "move" },
+ ],
+ totalMs: 4000,
+ existingRegions: [],
+ defaultDurationMs: 1000,
+ });
+ expect(suggestions).toHaveLength(1);
+ expect(suggestions[0].focus).toEqual({ cx: 0.2, cy: 0.8 });
+ });
+
+ it("rejects a legacy click when the cursor hides on the next sampler tick", () => {
+ const suggestions = buildAutoZoomSuggestions({
+ cursorTelemetry: [
+ { ...click(1000, 0.2, 0.8), visible: true },
+ { timeMs: 1033, cx: 0.25, cy: 0.75, visible: false, interactionType: "move" },
+ ],
+ totalMs: 4000,
+ existingRegions: [],
+ defaultDurationMs: 1000,
+ });
+ expect(suggestions).toEqual([]);
+ });
+
it("does not join visible samples across a hidden interval into a dwell", () => {
const suggestions = buildAutoZoomSuggestions({
cursorTelemetry: [
diff --git a/src/lib/ai-edition/timeline/zoom-suggestions.ts b/src/lib/ai-edition/timeline/zoom-suggestions.ts
index 145b892cd..26c7f0946 100644
--- a/src/lib/ai-edition/timeline/zoom-suggestions.ts
+++ b/src/lib/ai-edition/timeline/zoom-suggestions.ts
@@ -127,10 +127,20 @@ const CLICK_INTERACTION_TYPES: ReadonlySet<NonNullable<CursorTelemetryPoint["int
*/
function detectZoomClickCandidates(samples: CursorTelemetryPoint[]): ZoomDwellCandidate[] {
const candidates: ZoomDwellCandidate[] = [];
+ const hasMouseupTelemetry = samples.some((sample) => sample.interactionType === "mouseup");
+ // PipeWire and older sidecars have clicks but no mouse-up events. A hidden sample on the next
+ // sampler tick still identifies a slider press; a much later hidden interval must not erase an
+ // already completed click merely because that format cannot mark its release.
+ const LEGACY_PRESS_GRACE_MS = 150;
+
let pendingClick: CursorTelemetryPoint | null = null;
- let hiddenDuringPress = false;
+ let hiddenAtMs: number | null = null;
const finishPress = () => {
- if (pendingClick && !hiddenDuringPress) {
+ if (!pendingClick) return;
+ const hiddenDuringPress =
+ hiddenAtMs !== null &&
+ (hasMouseupTelemetry || hiddenAtMs - pendingClick.timeMs <= LEGACY_PRESS_GRACE_MS);
+ if (!hiddenDuringPress) {
candidates.push({
centerTimeMs: pendingClick.timeMs,
focus: { cx: pendingClick.cx, cy: pendingClick.cy },
@@ -138,17 +148,19 @@ function detectZoomClickCandidates(samples: CursorTelemetryPoint[]): ZoomDwellCa
});
}
pendingClick = null;
- hiddenDuringPress = false;
+ hiddenAtMs = null;
};
for (const sample of samples) {
if (sample.interactionType && CLICK_INTERACTION_TYPES.has(sample.interactionType)) {
finishPress();
pendingClick = sample.visible === false ? null : sample;
- } else {
- if (sample.visible === false) hiddenDuringPress = true;
- if (sample.interactionType === "mouseup") finishPress();
+ continue;
+ }
+ if (pendingClick && sample.visible === false && hiddenAtMs === null) {
+ hiddenAtMs = sample.timeMs;
}
+ if (sample.interactionType === "mouseup") finishPress();
}
finishPress();
return candidates;
diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.md
index ea959a411..760959666 100644
--- a/technical-documentation/testing/manual-e2e-checklist.md
+++ b/technical-documentation/testing/manual-e2e-checklist.md
@@ -610,6 +610,7 @@ The mask comes from the native compositor (ONNX Runtime + the vendored selfie-se
| Date | Build / tag | Platform | Pass/fail | Notes |
|------|-------------|----------|-----------|-------|
+| 2026-09-23 | dev `fix/725-hidden-cursor-auto-zoom`; rebuilt compositor addon SHA256 `a0bcd61e17084c010141049a9978121e45be2985449ff1618ab6a3145f8381f6` | Windows 11 Home 26200 | Partial — issue #725 slider slice passed | Human-driven real recording in the Electron dev build: moving a slider whose application cursor becomes hidden kept the reconstructed cursor hidden in the Studio recording and did not create a click-driven automatic zoom for that press. A visible control click still produced the normal automatic zoom. Sidecar inspection independently confirmed the hidden interval (`visible: false`) and that no zoom was centred on the press immediately preceding it. Recording, stop and editor open passed. Not covered: export, packaged build, audio/webcam, tray/HUD, other editor features, macOS or Linux. |
| 2026-09-12 | dev `fix/643-gif-export-cancel`, based on `fbe461e9`; rebuilt compositor addon SHA256 `128c89fe65a459b3f17721d610ea9a78e0d3e980e9eb0b5fb2b0e5d018619e15` | Windows | Partial — export cancellation passed | Computer-use OS input, real Electron and native save dialog, imported synthetic 8 s source. GIF 15 FPS / Medium (720p) / loop on: observed 55/120 frames, clicked Cancel while rendering (pending snapshot 104/120), returned to the same options; no final or partial output remained. Retried in that dialog to the same path and received Saved to; resulting GIF is 1280×720, 120 frames, 8,200,202 bytes, fully decodes with ffmpeg (exit 0). Automated real-addon tests separately pass existing-destination preservation, GIF retry, ordinary GIF and MP4 export. Desktop adapter required screenshot-based coordinates and Ctrl+O to reach the editor; HUD reachability is not claimed. An unrelated process error window partly obscured the retry success screenshot; file verification independently passed. Skipped: recording, audio/mic, webcam, tray/HUD, effect/caption comparisons, second viewer, macOS/Linux and release promotion. Task-local profile and evidence retained; test app stopped. |
| 2026-07-31 | dev build, `claude/e2e-tests-v1-8-0-474894` (e9578f09) | macOS 26.5, M1 | Partial — 1 defect | Ran launch/HUD, media, modifier anchoring, and export. **Defect: a dangling asset blanks the preview.** Modifier anchoring across a reorder verified in preview and in the exported frames. macOS export produced 1280×720 h264 + AAC at ~2× realtime. Chat sections skipped: no AI provider configured. HUD drag not runnable under computer-use (drop point is the desktop). |
| 2026-08-13 | installed `v1.9.5-rc.1` | Windows 11 26200, 1920×1080 @ 100% | Partial — 1 defect | Ran launch/HUD, source selection, recording, stop, editor open. Fragmented MP4 confirmed on the shipped artifact: 48 `moof`+`mdat` pairs over 47.6s, `mvex` present, `mfra` on clean stop. **Defect: a recording that survives a helper kill is thrown away by the app** — killing `wgc-capture.exe` mid-recording leaves a fully decodable 41s file (2460 packets, `ffmpeg -f null -` exit 0) with no `.session.json` and no `.cursor.json`, and stop answers "The recording could not be saved". Fixed in #363, re-verified end to end. Truncation ablation at 60%: plain MP4 unreadable, fragmented plays 29s. **A dev build cannot test any of this** — the prebuilt worktree helper predated the change and silently ran the old path. Editor/export/chat sections not run. |