Skip to content

Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.

fix(export): recover nearby frames before sparse clip starts

Julio Garcia260a428d6f52
diff --git a/crates/compositor/src/timeline_walk.rs b/crates/compositor/src/timeline_walk.rs
index e5fe15b44..7fb9fe180 100644
--- a/crates/compositor/src/timeline_walk.rs
+++ b/crates/compositor/src/timeline_walk.rs
@@ -298,9 +298,10 @@ fn accepts_clip_start(current: f64, target: f64, hold_tail_sec: f64) -> Option<b
None
}
-/// Seek a clip start within the tail margin. A future frame across an
-/// interior gap is invalid, but a request before the first frame of an offset
-/// stream retains the existing first-frame hold.
+/// Seek a clip start within the tail margin. A sparse recording can have a
+/// nearby preceding frame even when the next update is far ahead. Recover
+/// that frame within the same margin, while still rejecting larger gaps and
+/// retaining the first-frame hold for an offset stream.
unsafe fn seek_clip_start(decoder: &mut Decoder, target: f64, hold_tail_sec: f64) -> Result<bool> {
let frame = decoder.seek_to_or_last(target)?;
if frame.is_null() {
@@ -309,6 +310,13 @@ unsafe fn seek_clip_start(decoder: &mut Decoder, target: f64, hold_tail_sec: f64
if let Some(accepted) = accepts_clip_start(decoder.cur_time_sec(), target, hold_tail_sec) {
return Ok(accepted);
}
+ // `seek_to` rounds forward. Try the bounded preceding window before deciding
+ // that the clip starts in a hole; do not show the future update early.
+ let from = (target - CLIP_START_TAIL_TOLERANCE_SEC).max(0.0);
+ let previous = decoder.seek_to(from)?;
+ if !previous.is_null() && decoder.cur_time_sec() <= target {
+ return advance_decoder_to(decoder, target, 0.0);
+ }
// Only a distant future result needs this extra seek. Distinguish a late
// stream origin from a missing interior interval using the actual first
// frame, without changing decoder timestamps or the normal seek path.
diff --git a/crates/compositor/tests/export_timing.rs b/crates/compositor/tests/export_timing.rs
index 0ed7f77b0..fb1e48f5e 100644
--- a/crates/compositor/tests/export_timing.rs
+++ b/crates/compositor/tests/export_timing.rs
@@ -25,6 +25,8 @@
//! -fps_mode vfr -c:v libopenh264 -g 60 -pix_fmt yuv420p video_gap.mp4
//! ffmpeg -i screen_colors.mp4 -c copy -output_ts_offset 1 offset_start.mp4
//! ffmpeg -i video_gap.mp4 -c copy -output_ts_offset 1 offset_gap.mp4
+//! ffmpeg -i screen_colors.mp4 -vf "select='not(mod(n,60))'" \
+//! -fps_mode vfr -c:v libopenh264 -g 60 -pix_fmt yuv420p sparse_colors.mp4
//! The gap fixture must retain timestamps: video frames stop at 0.983 s and
//! resume at 3.000 s. An equivalent H.264 encoder is sufficient if libopenh264
//! is unavailable.
@@ -478,6 +480,98 @@ fn mp4_export_bounds_tail_recovery_and_reports_the_correct_source() {
std::fs::remove_dir_all(out_dir).expect("remove test outputs");
}
+/// A static screen may have a nearby preceding frame but no new frame for a second.
+/// Starting a cut between those updates must hold the preceding image, then adopt
+/// the next image when it is due, both in frame inspection and a real MP4 export.
+#[test]
+fn mp4_export_recovers_a_nearby_frame_before_a_sparse_clip_start() {
+ let Some(dir) = media_dir() else {
+ eprintln!("skipped: set OPENSCREEN_TEST_MEDIA");
+ return;
+ };
+ let sparse = dir.join("sparse_colors.mp4");
+ assert!(
+ sparse.exists(),
+ "generate sparse_colors.mp4 using the fixture header recipe"
+ );
+ let outputs = TestOutputDir::new("sparse-clip-start");
+ let gpu = Gpu::create(false).expect("gpu");
+ let comp = Compositor::new_sized(&gpu, 640, 360).expect("compositor");
+ let params = ExportParams {
+ width: 640,
+ height: 360,
+ fps: Some(30),
+ codec: ExportCodec::H264,
+ bit_rate: None,
+ };
+ for has_camera in [false, true] {
+ for after_valid_clip in [false, true] {
+ let mut clip = whole_clip(&dir);
+ clip.screen = sparse.to_string_lossy().into_owned();
+ if !has_camera {
+ clip.webcam.clear();
+ }
+ // Frames at 1.0 s (green) and 2.0 s (blue), with the cut at 1.25 s.
+ clip.source_start_sec = 1.25;
+ clip.source_end_sec = 2.5;
+ let mut clips = Vec::new();
+ if after_valid_clip {
+ let first = ClipSource {
+ screen: clip.screen.clone(),
+ webcam: clip.webcam.clone(),
+ source_start_sec: 0.0,
+ source_end_sec: 0.5,
+ ..whole_clip(&dir)
+ };
+ clips.push(first);
+ }
+ clips.push(clip);
+ let first_frame = if after_valid_clip { 15 } else { 0 };
+ let frames = unsafe {
+ openscreen_compositor::timeline_walk::render_frames(
+ &clips,
+ &gpu,
+ &comp,
+ &Cfg::c8(),
+ 30,
+ &[first_frame, first_frame + 22, first_frame + 23],
+ )
+ }
+ .expect("inspect a clip starting between sparse screen updates");
+ assert_eq!(frames.len(), 3);
+ for (i, frame) in frames.iter().enumerate() {
+ let center = ((frame.height / 2 * frame.width + frame.width / 2) * 4) as usize;
+ let rgb = &frame.rgba[center..center + 3];
+ let expected_channel = if i < 2 { 1 } else { 2 };
+ assert!(
+ rgb[expected_channel] > rgb[0].saturating_add(40),
+ "frame {i}: {rgb:?}"
+ );
+ assert!(
+ rgb[expected_channel] > rgb[3 - expected_channel].saturating_add(40),
+ "frame {i}: {rgb:?}"
+ );
+ }
+ let out = outputs.0.join("out.mp4");
+ let stats = pipeline::run_composited_multi(
+ &clips,
+ &out.to_string_lossy(),
+ &gpu,
+ &comp,
+ &Cfg::c8(),
+ &params,
+ &mut |_| {},
+ )
+ .expect("export a clip starting between sparse screen updates");
+ assert_eq!(stats.frames, first_frame + 38);
+ assert_eq!(
+ pipeline::probe_frame_count(&out.to_string_lossy()).expect("probe"),
+ stats.frames
+ );
+ }
+ }
+}
+
/// A seek inside a multi-second hole must not show a frame from the future.
/// Exercise both source roles and preservation after an earlier valid clip.
#[test]
diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.md
index 42091c84f..5ed62414e 100644
--- a/technical-documentation/testing/manual-e2e-checklist.md
+++ b/technical-documentation/testing/manual-e2e-checklist.md
@@ -1070,3 +1070,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov
| 2026-10-28 | `main` @ `5d82a0cb2` (prerelease candidate, #1422), artifact `openscreen-linux` of build run 40772284800 (artifacts only, no release; the package still says 2.1.2). `Openscreen-Linux-2.1.2.AppImage` run directly with `--no-sandbox` and a fresh `--user-data-dir`, `OPENSCREEN_DISABLE_CONTENT_PROTECTION=1` | Ubuntu 24.04, GNOME Shell 46 **Wayland**, AMD Radeon 610M (RADV, Mesa 25.2.8), 1920×1080, USB2.0 HD UVC webcam, built-in mic | **Partial: minimal pass + 5 diff items, no defect** | **Driver:** the `computer-use-linux` 0.7.16 binary behind a python stdio bridge over SSH, pointer input through the RemoteDesktop portal (remembered grant, real compositor events), screenshots through the screenshot portal; no keyboard was used. Portal screenshots taken during a take flash the screen and are baked into it, as in the rc.3 row. **Smoke.** The AppImage launches; HUD shows; mic and webcam toggled on by click. One HUD take: picker → *Entire Screen* → countdown → 22.8 s take. Screen `.mp4` 1920×1080 H.264, 1300 frames (≈57 fps), AAC; webcam `.webm` 1280×720, 229 frames at 10 fps (dark room). Capture encoder `vaapi`, compositor `backend Hardware` on RADV. Stop opened the editor with the webcam PiP. Playback runs; a 1.80× zoom added from the toolbar zooms the preview. Export MP4 1080p60: 1752×1080 H.264, 1367 frames over 22.78 s, decodes without error (`ffmpeg -f null`), AAC mean −29.1 dB / max −2.9 dB; frames at 3.8 s (zoomed) and 15 s (side-by-side) as edited. Quit from the editor's close button, relaunch, HUD *Open Studio* reopened the project with the zoom and the layout region. **Diff since v2.1.2.** (1) #1340 `1b861830c`: a second record press while the ScreenCast picker was up gave one picker and one take (one `prepared: true` in the log). (2) `3957984ee`: pause 4.8 s mid-take then resume; the cursor sidecar (631 samples) has no gap above 300 ms, the pause is compacted out. (3) `4f064c5b1`: playback to the end (0:22.8), then Play restarts from 0:00; pause holds. (4) `46bcb0c9d`: a *Side by side* layout change at 0:10 from the timeline layout tool; preview and export switch from PiP to side-by-side. (5) `0abf010c0`: a still GNOME terminal picked under *Application Window* from the editor's Record mode; the take starts, `.mp4` is 866×628 (the window, not the monitor), and the picker appeared again on this second take. A first window take came out 73.5 s for ≈12 s of intent because the harness's Stop click stalled ≈60 s in the portal call; the repeat, with the click timed at 0.6 s, gave 9.15 s for 9.2 s, so not an app defect. **Not covered:** system audio, the tray, GIF export, the HUD *Device settings* list, cancel during countdown, `RADV_EXPERIMENTAL` Vulkan encode (`3620cf806`: the helper picked `vaapi` with nothing rejected, so the Vulkan path was not exercised), the Intel iHD VPP export path (AMD machine), KDE, and everything else in this file. |
| 2026-10-28 | `main` @ `5d82a0cb2`, CI NSIS x64 from build run 40772284800 | Windows 11, 1920×1080 @ 125 %, USB2.0 HD UVC WebCam (VivoWindows / sarae) | Pass — no defect | Driven remotely with real OS input (SetCursorPos/mouse_event in the interactive session via a scheduled task), `OPENSCREEN_DISABLE_CONTENT_PROTECTION=1` logged. Screen source picked, 2:10 take with webcam + mic + system audio stopped from the HUD: 1920×1080 ~60 fps H.264 + AAC 48k (signal present), webcam 1280×720. Tray icon turns red while recording, tooltip `Recording: Tout l'écran`; second 0:51 take stopped from tray → Stop Recording, editor opens. Zoom (Z), trim (T), undo/redo pass. Export MP4 1080p60: 1636×1080, 7012 frames, 116.85 s = 130.19 s source video − 13.33 s trim, decodes clean, AAC present, zoom and webcam PiP visible in extracted frames. Note: export progress counts 7034 frames (container/audio duration 130.56 s, not the video stream). Tray "Open" focuses the editor when it is open (by design). NOT covered: window source, pause/restart/cancel, GIF, DPI, content protection ON, persistence/reopen, AI sections. |
| 2026-10-30 | `main` @ `09ed9a231` (after #1424), DMG `Openscreen-macOS-Apple-Silicon-2.1.2.dmg` from artifact `openscreen-mac-arm64` of build run 40874803001 (artifacts only, no release; the package still says 2.1.2), sha256 `79dca7a1…6a6cab`. `spctl -a -vv -t open`: accepted, Notarized Developer ID (M4LK7C6S84); `xcrun stapler validate` OK on the DMG. `Openscreen.app` copied out of the mounted DMG to a scratch folder, not `/Applications` (which holds `2.2.0-rc.1`): `spctl` accepted, `codesign --verify --deep --strict` OK; the app itself carries no stapled ticket, only the DMG does. Fresh `--user-data-dir` | macOS 27.0.1, Mac mini M1, OBS virtual camera, BlackHole 2ch as default input | **Partial: signed DMG minimal pass, no defect** | Closes the gap of the 2026-10-28 macOS row (no CI DMG then, #1423). Driven with real OS input (CGEvent clicks and AXPress from the TCC-granted app), no screenshot while Apple's picker was up. **Passed:** permissions window → *Get started*; HUD camera and microphone toggles; Apple's picker → *Share* display, source button shows it; 18.5 s take with three clicks: screen H.264 1920×1080 (ScreenCaptureKit only emits changed frames: 105 frames, max gap 1.0 s on a still desktop), webcam VP8 1080p30, 563 frames, max gap 68 ms; cursor sidecar has the three clicks at their exact points; stop opens the editor with the webcam PiP and a 1.80× auto-zoom from the clicks; playback in real time; Export MP4 1080p/60 in ~8 s: 1752×1080, 1101 frames, max gap 17 ms, decodes clean, zoom and PiP visible in extracted frames. **Not covered:** system audio (not granted to this app path, left alone to avoid a system prompt), microphone signal (BlackHole carried silence: track present at −91 dB), reopen and persistence, tray, window source, GIF, the x64 DMG functionally (`Openscreen-macOS-Intel-2.1.2.dmg` from the same run, after a rerun of its transient stapler failure, sha256 `07997c9c…ae22af`: `spctl` accepted, Notarized Developer ID, `stapler validate` OK; not installed, no Intel Mac), and everything else in this file. |
+| 2026-11-04 | `main` @ `290bd3a` + `fix/export-sparse-clip-start`, locally built compositor addon and app renderer (2.2.1) | macOS 27.0.1, arm64, Metal | **Partial — automated export regression; GUI not covered** | Synthetic one-update-per-second H.264 reproduces #1479 on installed 2.2.1 and in the new native test before the fix. Patched frame inspection holds green at 1.25 s and 1.983 s, then shows blue at 2.017 s; MP4 frame counts pass with/without camera and after a preceding clip. Patched app CLI exports the synthetic project and the affected private project with all original cuts unchanged; private media is not attached. Native unit suite: 492 passed; export timing: 15 passed, one existing bounded-tail test fails identically on unmodified main (container duration clips the requested tail before seeking). JS: 5192 passed, 5 skipped; both TypeScript checks and lint pass (existing warnings). **Skipped:** real OS GUI export, because computer-use initialization and app selection both timed out; capture, devices, preview interactions, signing/packaging, Windows and Linux were not exercised. This is not a release-candidate pass. |