Skip to content

Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.

fix(perf-ab): address the review of the bench scripts

EtienneLescot9f9dd6b7cd6d
diff --git a/scripts/perf-ab/export-bench.mjs b/scripts/perf-ab/export-bench.mjs
index 4675cd5b8..3d2056fe7 100644
--- a/scripts/perf-ab/export-bench.mjs
+++ b/scripts/perf-ab/export-bench.mjs
@@ -1,6 +1,6 @@
// CLI export benchmark: wall time, output frames/fps, CPU seconds and peak memory of the process tree.
// node export-bench.mjs <scratch> <reps> — runs interleaved v200/main for each case.
-import { execFileSync, spawn } from "node:child_process";
+import { execFile, execFileSync, spawn } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
@@ -36,9 +36,10 @@ const cases = [
},
];
+// Asynchronous, so the export's exit event is timed when it happens, not after a sample.
const procsnap = (root) =>
- JSON.parse(
- execFileSync(
+ new Promise((resolve) =>
+ execFile(
"powershell",
[
"-NoProfile",
@@ -50,7 +51,8 @@ const procsnap = (root) =>
root,
],
{ encoding: "utf8" },
- ) || "[]",
+ (error, out) => resolve(error ? [] : JSON.parse(out || "[]")),
+ ),
);
const probe = (file) => {
@@ -105,9 +107,13 @@ async function runOne(ver, c, rep) {
p.stdout.on("data", (b) => (stdout += b));
const seen = new Map();
let done = false;
- p.on("exit", () => (done = true));
+ let exitedAt = 0;
+ p.on("exit", () => {
+ exitedAt = performance.now();
+ done = true;
+ });
while (!done) {
- for (const s of procsnap(app)) {
+ for (const s of await procsnap(app)) {
const prev = seen.get(s.pid);
seen.set(s.pid, {
role: s.role,
@@ -118,7 +124,7 @@ async function runOne(ver, c, rep) {
}
await new Promise((r) => setTimeout(r, 500));
}
- const wallSec = (performance.now() - t0) / 1000;
+ const wallSec = (exitedAt - t0) / 1000;
const last = stdout
.trim()
.split("\n")
diff --git a/scripts/perf-ab/gen-projects.mjs b/scripts/perf-ab/gen-projects.mjs
index 6bd887eb7..d970819f3 100644
--- a/scripts/perf-ab/gen-projects.mjs
+++ b/scripts/perf-ab/gen-projects.mjs
@@ -21,6 +21,12 @@ const DUR = 60;
const screen = path.join(mediaDir, "screen.mp4");
const webcam = path.join(mediaDir, "webcam.mp4");
const music = path.join(mediaDir, "music.mp3");
+// All three before writing anything: a heavy project whose camera track points at nothing would
+// benchmark something else than PiP.
+for (const file of [screen, webcam, music]) {
+ if (!fs.existsSync(file))
+ throw new Error(`missing ${file}: generate the media first (see above)`);
+}
const now = "2026-10-15T15:00:00.000Z";
// Cursor: horizontal triangle sweep 0.15 -> 0.85 -> 0.15, 4 s period, y fixed, sampled every 8 ms.
diff --git a/scripts/perf-ab/gui-bench.mjs b/scripts/perf-ab/gui-bench.mjs
index f8067fc11..b29efb60e 100644
--- a/scripts/perf-ab/gui-bench.mjs
+++ b/scripts/perf-ab/gui-bench.mjs
@@ -148,7 +148,12 @@ try {
{ encoding: "utf8" },
).trim(),
);
+ // Unpinned, another window may cover the editor and the capture would film that window.
+ if (a.capture && !result.topmostWindows)
+ throw new Error("no editor window could be pinned on top");
await sleep(3000);
+ // A window Windows minimized sits at -32000: wait for topmost.ps1's restore to land.
+ for (let i = 0; i < 20 && (await ed.evaluate("screenX <= -30000")); i++) await sleep(250);
result.videos = await ed.evaluate(
"[...document.querySelectorAll('video')].map(v => ({paused: v.paused, t: v.currentTime, d: v.duration, w: v.videoWidth}))",
diff --git a/scripts/perf-ab/profile-hud.mjs b/scripts/perf-ab/profile-hud.mjs
index 500554ee6..8a7162f5c 100644
--- a/scripts/perf-ab/profile-hud.mjs
+++ b/scripts/perf-ab/profile-hud.mjs
@@ -75,6 +75,8 @@ try {
}
if (!ready) await sleep(5);
}
+ // A timeout is not a startup: write no profile the analysis would take for one.
+ if (!ready) throw new Error("the HUD never became ready");
const nav = (
await browser.send(
"Runtime.evaluate",
@@ -94,11 +96,14 @@ try {
console.log(
JSON.stringify({ tag, ready, dcl: Math.round(JSON.parse(nav).domContentLoadedEventEnd) }),
);
+} catch (e) {
+ console.error(e);
+ process.exitCode = 1;
} finally {
try {
execFileSync("taskkill", ["/T", "/F", "/PID", String(app.pid)], { stdio: "ignore" });
} catch {
// already gone
}
- process.exit(0);
+ process.exit();
}
diff --git a/scripts/perf-ab/profile-startup.mjs b/scripts/perf-ab/profile-startup.mjs
index 75050cc2f..7ddcd8d2a 100644
--- a/scripts/perf-ab/profile-startup.mjs
+++ b/scripts/perf-ab/profile-startup.mjs
@@ -55,15 +55,19 @@ try {
await waitTarget(port, (ts) => ts.find((t) => t.url.includes("hud-overlay")), 60000, 5),
);
const hudTargetAt = Date.now() - t0;
- for (let i = 0; i < 2000; i++) {
+ let ready = false;
+ for (let i = 0; i < 2000 && !ready; i++) {
try {
- if (await hud.evaluate(`!!document.querySelector('[data-testid="launch-record-button"]')`))
- break;
+ ready = await hud.evaluate(
+ `!!document.querySelector('[data-testid="launch-record-button"]')`,
+ );
} catch {
// document not there yet
}
- await sleep(5);
+ if (!ready) await sleep(5);
}
+ // A timeout is not a startup time: write nothing the analysis would take for one.
+ if (!ready) throw new Error("the HUD never became ready");
const hudReadyAt = Date.now() - t0;
await sleep(300);
const { profile } = await main.send("Profiler.stop");
@@ -73,7 +77,10 @@ try {
JSON.stringify({ tag, ver: a.ver, hudTargetAt, hudReadyAt }),
);
console.log(JSON.stringify({ tag, hudTargetAt, hudReadyAt }));
+} catch (e) {
+ console.error(e);
+ process.exitCode = 1;
} finally {
kill();
- process.exit(0);
+ process.exit();
}
diff --git a/scripts/perf-ab/summarize-record.mjs b/scripts/perf-ab/summarize-record.mjs
index aa7215a25..58139609d 100644
--- a/scripts/perf-ab/summarize-record.mjs
+++ b/scripts/perf-ab/summarize-record.mjs
@@ -3,7 +3,12 @@ import fs from "node:fs";
import path from "node:path";
const dir = process.argv[2];
-const q = (a) => [...a].sort((x, y) => x - y)[Math.floor(a.length / 2)];
+// The same median as summarize-gui.mjs and doc-tables.mjs: the two middle values averaged.
+const q = (a) => {
+ const s = [...a].sort((x, y) => x - y);
+ const h = s.length >> 1;
+ return s.length % 2 ? s[h] : (s[h - 1] + s[h]) / 2;
+};
const rows = {};
for (const f of fs.readdirSync(dir).filter((f) => /^record-.*-r\d+\.json$/.test(f))) {
const r = JSON.parse(fs.readFileSync(path.join(dir, f), "utf8"));
diff --git a/scripts/perf-ab/topmost.ps1 b/scripts/perf-ab/topmost.ps1
index cc9ee3fef..69cb2b9b1 100644
--- a/scripts/perf-ab/topmost.ps1
+++ b/scripts/perf-ab/topmost.ps1
@@ -13,6 +13,8 @@ public static class Win {
[DllImport("user32.dll")] public static extern bool IsWindowVisible(IntPtr h);
[DllImport("user32.dll")] public static extern int GetWindowTextLength(IntPtr h);
[DllImport("user32.dll")] public static extern bool SetWindowPos(IntPtr h, IntPtr after, int x, int y, int cx, int cy, uint flags);
+ [DllImport("user32.dll")] public static extern bool IsIconic(IntPtr h);
+ [DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr h, int cmd);
}
"@
$all = Get-CimInstance Win32_Process | Select-Object ProcessId, ParentProcessId
@@ -27,6 +29,8 @@ $done = 0
$p = 0
[void][Win]::GetWindowThreadProcessId($h, [ref]$p)
if ($pids.ContainsKey([int]$p) -and [Win]::IsWindowVisible($h) -and [Win]::GetWindowTextLength($h) -gt 0) {
+ # A minimized window measures nothing: bring it back maximized first (SW_MAXIMIZE).
+ if ([Win]::IsIconic($h)) { [void][Win]::ShowWindow($h, 3) }
# HWND_TOPMOST, SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE
[void][Win]::SetWindowPos($h, [IntPtr](-1), 0, 0, 0, 0, 0x0013)
$script:done++
diff --git a/technical-documentation/testing/performance-ab.md b/technical-documentation/testing/performance-ab.md
index 68c98458c..97d397455 100644
--- a/technical-documentation/testing/performance-ab.md
+++ b/technical-documentation/testing/performance-ab.md
@@ -182,6 +182,10 @@ of its own would skip most of it; trimming the dozen small costs above one by on
|---|---|---|---|---|
| Process start to HUD ready (ms) | 620 [595–678] | 651 [635–684] | +4.9% | under 5%, but significant |
+This is the first measure, taken with the editor sometimes behind other windows. The verdict uses
+the controlled rerun in [the bisection](#bisecting-the-startup-slowdown): 625 → 672 ms, +7.5%,
+p < 0.001.
+
#### Recording, 60 s full screen with the webcam (3 takes each)
| Metric | 2.0.0 | main | Delta | Verdict |
@@ -213,6 +217,10 @@ camera, a Logitech StreamCam, delivers 1080p at 30 fps under the default 2160p r
| GIF 720p 15 fps: wall time (s) | 488.7 [484.3–493.0] | 16.2 [15.9–16.3] | -96.7% | improvement |
| GIF 720p 15 fps: frames written | 900 [900–900] | 900 [900–900] | +0.0% | info |
+These wall times ran up to about 1.3 s long: the bench took a synchronous process sample
+(~0.8 s) and slept 500 ms before it noticed an export had ended. Both builds carry the same
+overshoot, so the comparison holds; `export-bench.mjs` now times the exit event itself.
+
2.0.0 exports MP4 at 60 fps only (its CLI has no `--fps`, its dialog no frame rate), so 1080p30 has
no baseline. Both builds write the same frames at nearly the same size (MP4 144.9 MB each, GIF 475.2
against 475.5 MB). 2.0.0's GIF runs on one core (492 CPU