Skip to content

A local video studio for your coding agent. Describe a video in one sentence: your agent directs, your machine renders. Claude Code, Codex, Cursor, Devin and more. No API keys, no uploads.

showtime 0.4.2: the chapter join on Windows and check on slow CPU-only machines (#14)

Favio Vázquez5c9a95ddeced
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cb81ed1..57faaad 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -118,6 +118,16 @@ All notable changes to showtime. Each entry says what changed and why, so this f
`--chapter N` fix re-renders exactly its frames. A piece cut short is refused with its `--chapter N` command, a
one-chapter film is refused, and the receipt counts each piece's time once. Pieces carry `sources_hash` (the project's sources, the same on a copy at another path): other
files warn at the join, another length or size is refused. Tests: `test_chapters.py`.
+- **The join's seam check compares each frame on Windows too.** On the Windows CI runner `render --join` stopped
+ at a random frame of the seam check with "could not compare frame 119 of mux.mp4: frame= 1 ... time=00:00:00.00". The PSNR
+ comparison gave ffmpeg's `psnr` the captured image at time 0 and the video's frame half a frame later (the seek
+ lands between frames), so the one frame `-frames:v 1` keeps went out uncompared, and the real comparison only ran
+ when the image's end of stream arrived, after the output had finished. Whether ffmpeg ran it before shutting down
+ was thread timing: it did on macOS and Linux and now and then did not on Windows (no PSNR line, exit 0). Both
+ inputs now start at time 0, so the first frame is the compared one on any OS, and the error quotes ffmpeg's own
+ message instead of its progress line and stream list (`settb=AVTB` also drops psnr's "not matching timebases" warning;
+ same values). Test: `test_chapters.py` (`test_frame_psnr_compares_the_frame_it_outputs` holds the image's end of
+ stream back 1.5 s, which fails the old graph on any OS, and checks the value; `test_ff_messages_drop_progress`).
- **`test_signatures.py` checks reading time once per motion feel and template, not twelve times.** On the Windows
nightly the file ran 868 s on its first night (6 Oct), then out of its 900 s on 7 and 8 Oct (680 s on 9 Oct); the
Intel Mac reached 900 s on 9 Oct. It ran a full `showtime check` per look signature, twelve of them, on 4-thread
@@ -130,16 +140,42 @@ All notable changes to showtime. Each entry says what changed and why, so this f
first check of each template (the only probe of the data template in the suite). The twelve checks without a GPU
on a 6-core Intel Mac, one after another, with no probe at all: 994 CPU seconds and 762 s before (load 16-53,
shared), 658 and 513 s after (load 21-44). Test: `test_signatures.py` `CheckPlan` (issue #13).
-- **`check --no-cuts` skips the cut walk, and the showreel's aspect test uses it.** On the macOS Intel nightly,
- `test_showreel.py`'s check of the showreel template at 16:9 ran out of its 300 s (7, 8 and 9 Oct; the file passed
- at 879 s on 6 Oct). Without a GPU the template's WebGL shots (the chrome and heat shaders, the three.js knot) are
- drawn on the CPU, and every 12th WebGL frame cost the GPU process 8-18 CPU seconds here, whatever the shader, the
+- **Without a GPU a slow WebGL frame no longer ends `check` at 60 s, and `check --no-cuts` skips the cut walk (issue
+ #14).** On the macOS Intel nightly, `test_showreel.py`'s check of the showreel template at 16:9 ran out of its
+ 300 s (7, 8 and 9 Oct; the file passed at 879 s on 6 Oct). Without a GPU the template's WebGL shots (the chrome
+ and heat shaders, the three.js knot) are drawn on the CPU, and every 12th WebGL frame cost the GPU process 8-18
+ CPU seconds here, whatever the shader, the
canvas size or the compositing flags; the cut walk (a second page that seeks every cut four times) paid for those
frames again, at every aspect, although the first frame after a cut does not change with the aspect and
`test_cut_frames.py` already walks the template's cuts. `--no-cuts` (next to `--no-timeline`) leaves out the walk,
report.json's `cuts` and `late_first_frame`; the aspect test passes it. One 16:9 check without a GPU on a 6-core
Intel Mac: 361 and 364 CPU seconds before, 242 and 244 after (160-183 s against 133-149 s wall at load 15-19).
- Test: `test_cut_frames.py` `test_no_cuts_skips_the_walk` (issue #14).
+ That was not enough. On 10 Oct the full tier still failed on the 4-core ubuntu-24.04 runner (the aspect test at
+ 9:16) and on macos-15-intel (that test, and `test_cut_frames.py`'s template check) with `timed out after 60s:
+ reading the page's size`. A clipped screenshot (check's timeline, thumbnails and cut walk; render's `--scale` below
+ 1, and `--preview` of a page above 720p) first reads the page's layout size (#49). That read is the first call on
+ the page's main thread after a seek, and without a GPU the main thread waits there while SwiftShader draws the
+ seek's WebGL frame in the GPU process. The seek itself returns in milliseconds, so the page's pace stayed 1 and the
+ read kept a flat 60 s. Reproduced on a 6-core Intel Mac with `--gpu off` and 8 busy loops: both runs ended this way,
+ after 128 s and 127 s (the read after the seek to 0.9 s took 39 s, the one after 3.0 s passed 60 s), the GPU process
+ working at about 0.45 of a core all the while and the renderers idle. The read is now guarded like the screenshot it
+ belongs to (the stuck-frame guard, #24, below): waited for while the GPU process and renderers work, failed once
+ they have done no work for 30 s (x the page's pace), sooner than before. The longer bound is the same as for
+ screenshots: where the browser cannot report its work, or any of its pages keeps working (the work is summed over
+ all of them), a check or a clipped render frame can now wait up to 10 minutes at that step (render: up to 3
+ attempts, each in a new browser) where it waited 60 s x pace; the read and the screenshot of one capture share those
+ 10 minutes. The page load (to DOMContentLoaded) also had a flat 60 s, and on the same Mac under heavier load (23,
+ another session's renders) it ended check's cut walk once after the read was fixed (`Timeout 60000ms exceeded`, no
+ cuts probed; seen once there, not on CI). It now gets 3/5 of the page-open deadline, 180 s: a load that never
+ finishes still fails as the page (check's `ready_failed` with a report; on check's second pages, the cut walk, the
+ determinism load and the no-GPU probe, the soft finding it was), not as a stuck browser killed and tried again after
+ the page-open deadline. With the read fix the two checks pass on that Mac with the busy loops and another session's
+ renders (595 s for the 9:16 check, 886 s for the one with the cut walk, 13 cuts and none late, at load 20-35; single
+ size reads of up to 290 s, the GPU process at work throughout), and with the load deadline the 9:16 check again
+ (435 s at load 11-16). Tests: `test_pace.py` `TestSizeReadWaitsForTheFrame` (a main thread held past the old
+ deadline is waited for, a stopped renderer still fails the read), `TestSlowPages`
+ `test_load_that_never_finishes_is_the_page` and `test_second_page_load_is_a_finding`, `test_cut_frames.py`
+ `test_no_cuts_skips_the_walk`.
- **`review-findings` accepts `t=a-b s` ranges as a timestamp, and Polish items get ids.** A finding carrying a range
such as `t=38.73-41.50s` was warned as citing no timestamp (the check only knew `t=3.00s`), and Polish bullets were
left out of the id scheme, so `showtime review-respond <job> --fixed r1-P1` only printed the list of valid ids
diff --git a/skills/showtime/scripts/lib/chapters.mjs b/skills/showtime/scripts/lib/chapters.mjs
index cdd0924..7274bb1 100644
--- a/skills/showtime/scripts/lib/chapters.mjs
+++ b/skills/showtime/scripts/lib/chapters.mjs
@@ -316,13 +316,42 @@ export async function seamAudio(final, masterFile, seams, { rate = 48000, maxLag
return out;
}
+/**
+ * The ffmpeg arguments that compare a captured frame image with frame k of a video (framePsnr).
+ * Both branches start at pts 0 (setpts=PTS-STARTPTS): psnr pairs its frames by timestamp and has no reference
+ * for a main frame that comes before the video's first one (framesync's EXT_NULL). Without that, the image
+ * (pts 0) went out uncompared as the one frame -frames:v 1 keeps, and the comparison only happened on the
+ * image's EOF, after the output had finished, so a Windows runner now and then stopped before it (no PSNR line).
+ * settb=AVTB gives both the same time base (no "not matching timebases" warning; the values are the same).
+ */
+export function framePsnrArgs(image, video, k, fps, vf) {
+ const at = Math.max(0, (k - 0.5) / fps);
+ return ['-nostats', '-i', image, '-ss', at.toFixed(6), '-i', video, '-filter_complex',
+ `[0:v]${vf},format=yuv420p,setpts=PTS-STARTPTS,settb=AVTB[a];[1:v]format=yuv420p,trim=end_frame=1,setpts=PTS-STARTPTS,settb=AVTB[b];[a][b]psnr`,
+ '-frames:v', '1', '-f', 'null', '-'];
+}
+
+/**
+ * ffmpeg's own messages from a stderr, the last n: without its progress and stats lines (frame= ... speed=,
+ * video:0KiB ...) and its input/output listing (Input #, Output #, Stream mapping, the indented stream lines).
+ */
+export function ffMessages(stderr, n = 3) {
+ return String(stderr || '').split(/\r\n|\r|\n/)
+ .filter((l) => l.trim() && !/^\s/.test(l) && !/^(Input|Output) #\d|^Stream (#|mapping:)|^Metadata:/.test(l))
+ .map((l) => l.trim())
+ .filter((l) => !/^(frame|size)=\s*\S/.test(l) && !/\bspeed=\s*\S/.test(l) && !/\bvideo:\S+ audio:\S+/.test(l))
+ .slice(-n);
+}
+
/** PSNR (dB, Y'CbCr 4:2:0 as encoded; Infinity when equal) of a captured frame image against frame k of a video. */
export async function framePsnr(image, video, k, fps, vf) {
- const at = Math.max(0, (k - 0.5) / fps);
- const r = await ffmpeg(['-i', image, '-ss', at.toFixed(6), '-i', video, '-filter_complex',
- `[0:v]${vf},format=yuv420p[a];[1:v]format=yuv420p,trim=end_frame=1[b];[a][b]psnr`, '-frames:v', '1', '-f', 'null', '-'],
- { loglevel: 'info', allowFail: true });
+ const r = await ffmpeg(framePsnrArgs(image, video, k, fps, vf), { loglevel: 'info', allowFail: true });
const m = /PSNR .*average:(inf|[\d.]+)/.exec(String(r.stderr || ''));
- if (r.code !== 0 || !m) throw new Error(`could not compare frame ${k} of ${path.basename(video)}: ${String(r.stderr || '').trim().split('\n').pop()}`);
+ if (r.code !== 0 || !m) {
+ const said = ffMessages(r.stderr).join(' | ') || '(nothing but its stream list and progress)';
+ const why = r.code !== 0 ? `ffmpeg failed (exit ${r.code}): ${said}`
+ : `ffmpeg exited 0 but printed no PSNR line (no frame of the video reached the comparison); its last lines: ${said}`;
+ throw new Error(`could not compare frame ${k} of ${path.basename(video)}: ${why}`);
+ }
return m[1] === 'inf' ? Infinity : Number(m[1]);
}
diff --git a/skills/showtime/scripts/lib/stagehost.mjs b/skills/showtime/scripts/lib/stagehost.mjs
index 77818d4..16f2354 100644
--- a/skills/showtime/scripts/lib/stagehost.mjs
+++ b/skills/showtime/scripts/lib/stagehost.mjs
@@ -109,6 +109,12 @@ function testStopNow(b, when) {
// no end.
// SHOWTIME_TEST_OPEN_TIMEOUT=<seconds> shortens it (tests).
export const PAGE_OPEN_MS = Number(process.env.SHOWTIME_TEST_OPEN_TIMEOUT) > 0 ? Number(process.env.SHOWTIME_TEST_OPEN_TIMEOUT) * 1000 : 300000;
+// The page's own load (to DOMContentLoaded) ends well inside that deadline, so a page whose load never finishes (a
+// script that never arrives or never ends) fails as the page: `could not open`, check's ready_failed finding or a
+// soft finding for its second pages, render's retry on the same browser. Past PAGE_OPEN_MS it would be taken for a
+// stuck browser instead: killed, tried again, and no report. 3/5 of it: 180 s (the context, the tab and the ready
+// wait share the rest); it was a flat 60 s, which a slow load on a busy machine without a GPU passed (#14).
+export const PAGE_LOAD_MS = Math.round(PAGE_OPEN_MS * 0.6);
/** close() of a browser or page that may not answer: never waits more than ms. */
export function closeSoon(p, ms = 10000) {
@@ -190,10 +196,12 @@ async function drawingWork(browser, cache) {
/**
* The screenshot promise p under the deadline above: browser is its Playwright browser, label names it in the error,
* cache keeps the browser CDP session. The drawing processes' work is read every SHOT_POLL_MS from the first
- * SHOT_POLL_MS on (a frame that comes in time costs nothing; pollMs is for tests, which shorten it). -> p's result
+ * SHOT_POLL_MS on (a frame that comes in time costs nothing; pollMs is for tests, which shorten it). since: when the
+ * SHOT_MAX_MS budget started (default now): one shot()'s size read and screenshot share one. -> p's result
*/
-export async function guardShot(p, browser, label, pace, cache, pollMs = SHOT_POLL_MS) {
+export async function guardShot(p, browser, label, pace, cache, pollMs = SHOT_POLL_MS, since = null) {
const t0 = Date.now();
+ const start = since || t0;
let timer = null, settled = false;
const watch = new Promise((_, rej) => {
const samples = []; // [ms since the screenshot was asked for, CPU seconds, or null: the browser did not answer]
@@ -216,7 +224,7 @@ export async function guardShot(p, browser, label, pace, cache, pollMs = SHOT_PO
if (settled) return;
const now = Date.now() - t0;
samples.push([now, cpu]);
- if (now >= SHOT_MAX_MS) return fail(`still not there after ${Math.round(SHOT_MAX_MS / 1000)} s, the longest a screenshot may take`);
+ if (Date.now() - start >= SHOT_MAX_MS) return fail(`still not there after ${Math.round(SHOT_MAX_MS / 1000)} s, the longest a screenshot may take`);
if (cache.noWork) { timer = setTimeout(tick, pollMs); return; }
// the work done since the newest sample at least idleMs old, from the samples the browser answered (a busy
// machine may leave one unanswered: that is not idleness); one that answered none for twice that is stuck
@@ -506,7 +514,10 @@ async function openStageOnce(browser, o) {
const target = `${o.url}/${String(o.page || 'index.html').replace(/^\/+/, '')}`;
track.step = 'loading the page';
try {
- await page.goto(target, { waitUntil: 'domcontentloaded', timeout: 60000 });
+ // PAGE_LOAD_MS (180 s), inside the page-open deadline every openStage caller runs under (PAGE_OPEN_MS x pace:
+ // openGuarded, or render's worker open), so a load that never finishes fails here, as the page. A flat 60 s
+ // ended check's cut walk once on a busy 6-core Mac without a GPU (#14).
+ await page.goto(target, { waitUntil: 'domcontentloaded', timeout: PAGE_LOAD_MS });
} catch (e) {
await context.close().catch(() => {});
throw new UserError(`could not open ${target}: ${e.message.split('\n')[0]}`);
@@ -627,10 +638,18 @@ async function openStageOnce(browser, o) {
if (cs && cs !== 1) params.clip = { x: 0, y: 0, width, height, scale: cs };
const want = [Math.round(width * dsf * (params.clip ? cs : 1)), Math.round(height * dsf * (params.clip ? cs : 1))];
// a clipped screenshot has the clip's size whatever the view's: the page's layout size says instead whether
- // the view lost the page's size (see `metrics`); the frame drawn at the wrong size is drawn again
+ // the view lost the page's size (see `metrics`); the frame drawn at the wrong size is drawn again.
+ // This read is the first call on the page's main thread after a seek, so it waits for the frame the seek
+ // drew: without a GPU the main thread waits there while SwiftShader draws a WebGL frame in the GPU process
+ // (a seek returns in milliseconds and the cost lands here; the page's pace never sees it). On a 4-core CI
+ // runner one such wait passed a flat 60 s (issue #14), so it is guarded like the screenshot it belongs to:
+ // waited for while the drawing processes work, failed once they do nothing (guardShot). The read and the
+ // screenshot share one SHOT_MAX_MS: a shot that keeps working waits 10 minutes at most, not 10 for each.
+ const shotT0 = Date.now();
for (let k = 1; params.clip; k++) {
- const v = await withPacedTimeout(page.evaluate(() => [window.innerWidth, window.innerHeight,
- window.ST && typeof window.ST.t === 'number' ? window.ST.t : null]), 60000, 'reading the page\'s size', pace);
+ const v = await guardShot(page.evaluate(() => [window.innerWidth, window.innerHeight,
+ window.ST && typeof window.ST.t === 'number' ? window.ST.t : null]), browser, 'reading the page\'s size', pace, work,
+ SHOT_POLL_MS, shotT0);
if (!v || (v[0] === width && v[1] === height)) break;
const msg = `the page was laid out at ${v[0]}x${v[1]}, not ${width}x${height}, at the ${label}`;
if (k >= 3) throw new Error(`${msg}, ${k} times in a row (the page's size was set again each time)`);
@@ -640,7 +659,7 @@ async function openStageOnce(browser, o) {
}
for (let k = 1; ; k++) {
if (TEST_GPU) await testStopGpu(browser, work);
- const r = await guardShot(cdp.send('Page.captureScreenshot', params), browser, label, pace, work);
+ const r = await guardShot(cdp.send('Page.captureScreenshot', params), browser, label, pace, work, SHOT_POLL_MS, shotT0);
const buf = Buffer.from(r.data, 'base64');
// the view lost the page's size (see `metrics` above): set it again and take the screenshot again (the
// page itself kept its layout size, which seek() checks, so the frame needs no new seek)
diff --git a/skills/showtime/tests/test_chapters.py b/skills/showtime/tests/test_chapters.py
index 0267189..dbe4de6 100644
--- a/skills/showtime/tests/test_chapters.py
+++ b/skills/showtime/tests/test_chapters.py
@@ -6,7 +6,8 @@
to add); the tiling rules (gap, overlap, missing piece, another `total`) with the exact `--chapter N` hint;
`joinPieces` writes the concatenation with an IDR at each seam and never holds two pieces; pieces with other
SPS/PPS are not byte-joined; `sourcesHash` is stable across paths and mtimes; the seam audio check (equal
- audio: step 0, lag 0; a 21 ms shift: caught); `withForcedKeys`
+ audio: step 0, lag 0; a 21 ms shift: caught); `withForcedKeys`; `framePsnr` compares the frame it
+ outputs even when the image's end of stream comes late (the Windows join failure)
* render (a 6 s fixture, 3 chapters of 2 s, a white frame and a 1 ms click at each seam): a whole render (a
keyframe at each chapter start); the plan (plan.json only, the one-machine line); `--chapter 1..3` (video-only
pieces in work/chapters/, never the latest final); `--join` (frames = total, IDR at the seams, captured JPEGs
@@ -25,6 +26,7 @@ import _isolate # noqa: F401 (run from a scratch folder: never write into the
import json
import os
+import re
import shutil
import subprocess
import sys
@@ -113,11 +115,10 @@ def mad(a, b):
def psnr_frame(a, b, k):
- """PSNR (dB, 100 = equal) of frame k of two videos."""
+ """PSNR (dB, 100 = equal) of frame k of two videos (both at pts 0: psnr compares the frame it outputs)."""
cp = subprocess.run([ff.ffmpeg_path(), "-hide_banner", "-nostdin", "-i", str(a), "-i", str(b), "-filter_complex",
- "[0:v]select=eq(n\\,%d)[x];[1:v]select=eq(n\\,%d)[y];[x][y]psnr" % (k, k), "-frames:v", "1", "-f", "null", "-"],
+ "[0:v]select=eq(n\\,%d),setpts=PTS-STARTPTS[x];[1:v]select=eq(n\\,%d),setpts=PTS-STARTPTS[y];[x][y]psnr" % (k, k), "-frames:v", "1", "-f", "null", "-"],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding="utf-8", errors="replace", timeout=120)
- import re
m = re.search(r"PSNR .*average:(inf|[\d.]+)", cp.stderr)
assert m, cp.stderr[-1500:]
return 100.0 if m.group(1) == "inf" else float(m.group(1))
@@ -304,6 +305,65 @@ class TestChapterUnits(unittest.TestCase):
self.assertEqual(r[0], ["-crf", "16", "-force_key_frames", "expr:eq(n,60)+eq(n,120)", "-forced-idr", "1"])
self.assertEqual(r[1], ["-crf", "16"])
+ def test_frame_psnr_compares_the_frame_it_outputs(self):
+ """The seam check's PSNR is computed on the one frame -frames:v 1 keeps, not on a later one.
+
+ The image's end of stream is held back 1.5 s (a pipe): the old graph gave psnr the image at pts 0 and
+ the video frame at pts 0.5/fps, so the first output frame went out uncompared and the comparison waited
+ for that end of stream, after the output had finished: no PSNR line, exit 0 (the Windows CI failure,
+ "could not compare frame 119 of mux.mp4: frame= 1 ... time=00:00:00.00"), on any OS.
+ """
+ v = self.tmp / "psnr.mp4"
+ run_ff("-f", "lavfi", "-i", "testsrc2=s=256x144:r=30:d=6", "-c:v", "libx264", "-preset", "veryfast", "-crf", "16",
+ "-pix_fmt", "yuv420p", "-g", "60", v)
+ png = self.tmp / "f119.png"
+ run_ff("-i", v, "-vf", "select=eq(n\\,119)", "-frames:v", "1", png)
+ raw = run_ff("-i", png, "-f", "rawvideo", "-pix_fmt", "rgb24", "-")
+ self.assertEqual(len(raw), 256 * 144 * 3)
+ vf = "crop=trunc(iw/2)*2:trunc(ih/2)*2,scale=out_color_matrix=bt709:out_range=tv:flags=accurate_rnd+full_chroma_int"
+ r = node("import { framePsnr, framePsnrArgs } from '%s';\nconst [p, v, vf] = %s;\n"
+ "console.log(JSON.stringify({ args: framePsnrArgs('IMAGE', v, 119, 30, vf), same: await framePsnr(p, v, 119, 30, vf), "
+ "other: await framePsnr(p, v, 60, 30, vf) }));" % (CHAPTERS_MJS, json.dumps([str(png), str(v), vf])))
+ self.assertGreater(r["same"], 40, "frame 119 against its own capture")
+ self.assertLess(r["other"], r["same"] - 10, "frame 60 is another picture")
+ args = r["args"]
+ i = args.index("IMAGE")
+ args[i - 1:i + 1] = ["-f", "rawvideo", "-pix_fmt", "rgb24", "-s", "256x144", "-i", "-"]
+ p = subprocess.Popen([ff.ffmpeg_path(), "-hide_banner", "-loglevel", "info", "-y"] + args,
+ stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+ try:
+ p.stdin.write(raw)
+ p.stdin.flush()
+ time.sleep(1.5)
+ _, err = p.communicate(timeout=120)
+ finally:
+ if p.poll() is None:
+ p.kill()
+ err = err.decode("utf-8", "replace")
+ self.assertEqual(p.returncode, 0, err[-2000:])
+ m = re.search(r"PSNR .*average:(inf|[\d.]+)", err)
+ self.assertTrue(m, "no comparison before the output finished:\n" + err[-1500:])
+ self.assertAlmostEqual(float(m.group(1)), r["same"], places=2, msg="the late-EOF run compared frame 119 too")
+
+ def test_ff_messages_drop_progress(self):
+ err = ("Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'mux.mp4':\n"
+ " Metadata:\n major_brand : isom\n"
+ " Stream #1:0[0x1](und): Video: h264 (High), yuv420p, 256x144, 30 fps\n"
+ "[h264 @ 0x0] first message\n"
+ "[in#1 @ 0x1] Error opening input: No such file or directory\r\n"
+ "Stream mapping:\n Stream #0:0 (png) -> crop:default\n"
+ "frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed= 0x elapsed=0:00:00.01 \r"
+ "Output #0, null, to 'pipe:':\n Metadata:\n encoder : Lavf63.1.102\n"
+ " Stream #0:0: Video: wrapped_avframe, yuv420p, 256x144\n encoder : Lavc63.1.102 wrapped_avframe\n"
+ "[out#0/null @ 0x2] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown\n"
+ "frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed= 0x elapsed=0:00:00.01 \n")
+ r = node("import { ffMessages } from '%s';\nconst e = %s;\nconsole.log(JSON.stringify([ffMessages(e), ffMessages(e, 1), ffMessages('')]));"
+ % (CHAPTERS_MJS, json.dumps(err)))
+ self.assertEqual(r[0], ["[h264 @ 0x0] first message", "[in#1 @ 0x1] Error opening input: No such file or directory"],
+ "messages only: no progress, stats or stream listing")
+ self.assertEqual(r[1], ["[in#1 @ 0x1] Error opening input: No such file or directory"], "the last n, not the first")
+ self.assertEqual(r[2], [])
+
# --------------------------------------------------------------------------- real renders
diff --git a/skills/showtime/tests/test_pace.py b/skills/showtime/tests/test_pace.py
index 510e1b1..ad630b7 100644
--- a/skills/showtime/tests/test_pace.py
+++ b/skills/showtime/tests/test_pace.py
@@ -23,6 +23,12 @@ seeks, and every wait is its fixed value times a factor of 1-5 (never shorter th
processes have done no work for 30 s (it waited a flat 10 minutes, longer than any caller) and the frame is
drawn again in a new browser; check, whose screenshots had no deadline, ends with an error naming its step;
SHOWTIME_TIMEOUT ends a render at the caller's limit with an error naming each part's step
+ * a clipped screenshot's read of the page's size, which waits for the frame the seek drew (issue #14): a main
+ thread held past the old flat 60 s (3 s at the test scale) while its renderer works is waited for; a stopped
+ renderer still fails the read with the stuck-frame guard's error
+ * a page load that never reaches DOMContentLoaded fails as the page (check's ready_failed, with a report) at the
+ load deadline inside the page-open one, never as a stuck browser; on check's second pages it stays a soft
+ finding (issue #14 review)
* the stuck-frame guard on a browser that cannot report its work (CDP replies faked, node only): one whose
SystemInfo.getProcessInfo answers with an error is asked once and then only the screenshot cap applies, a late
frame still comes through; one that does not answer at all is stuck after twice the idle time
@@ -38,9 +44,11 @@ import _isolate # noqa: F401 (run from a scratch folder: never write into the
import json
import re
import shutil
+import socket
import subprocess
import sys
import tempfile
+import threading
import time
import unittest
from pathlib import Path
@@ -79,6 +87,12 @@ def showtime(*args, env=None, timeout=600):
timeout=timeout)
+# the stuck-frame guard's two verdicts (stagehost guardShot): the drawing processes did no work, or the browser stopped
+# saying what they do. With the GPU process stopped Chrome now and then stops answering SystemInfo.getProcessInfo too
+# (2 of 8 stopped-GPU checks on a 6-core Mac, 11 Oct), so either one is the guard ending a frame that never comes.
+STUCK_FRAME = (r"\((its GPU process and renderers did no work for \d+ s: the frame is not coming|"
+ r"the browser did not say what it was doing for \d+ s)\)")
+
STUCK_AT = (r"stuck at: (creating the browser context|opening a tab|setting up the tab|loading the page|"
r"waiting for the page to be ready)")
@@ -300,6 +314,50 @@ process.exit(0);
self.assertRegex(cp.stderr, STUCK_AT)
self.assertIn("in a new browser too", cp.stderr)
+ def test_load_that_never_finishes_is_the_page(self):
+ """A page whose load never reaches DOMContentLoaded (a parser-blocking script from a local server that never
+ answers) fails as the page: check's ready_failed finding with a report, at the page's load deadline
+ (PAGE_LOAD_MS, 3/5 of the page-open deadline: 18 s here), never as a stuck browser killed and tried again
+ with no report (a load with no deadline of its own ran into the page-open deadline twice)."""
+ with hanging_server() as port:
+ proj = self.tmp / "neverloads"
+ write(proj / "showtime.json", json.dumps({"width": 320, "height": 180, "fps": 10, "duration": 1, "background": "#202830"}))
+ write(proj / "index.html", HEAD + '<script src="http://127.0.0.1:%d/never.js"></script></head><body>'
+ "<script>ST.onSeek(() => {});</script></body></html>" % port)
+ t0 = time.time()
+ cp = showtime("check", proj, "--json", "--no-determinism", "--no-timeline", "--no-history", "--samples", "2",
+ env=dict(ENV, SHOWTIME_TEST_OPEN_TIMEOUT="30"), timeout=300)
+ took = time.time() - t0
+ self.assertTrue(cp.stdout.strip(), "check printed no report (rc=%d):\n%s" % (cp.returncode, cp.stderr[-2000:]))
+ rep = json.loads(cp.stdout)
+ ready = [f for f in rep["findings"] if f["code"] == "ready_failed"]
+ self.assertEqual(len(ready), 1, rep["findings"])
+ self.assertIn("could not open", ready[0]["message"])
+ self.assertNotIn("the browser stopped answering", cp.stderr)
+ self.assertNotIn("trying once more", cp.stderr)
+ self.assertLess(took, 120, cp.stderr[-1500:])
+
+ def test_second_page_load_is_a_finding(self):
+ """check's second pages (the determinism load, the cut walk, the no-GPU probe) whose load never finishes stay
+ soft findings: the first page loads its script, every later load waits for ever, and check reports it
+ (cut_probe_failed, determinism_incomplete) and finishes instead of ending with a stuck browser."""
+ with hanging_server(answer_first=True) as port:
+ proj = self.tmp / "secondhangs"
+ write(proj / "showtime.json", json.dumps({"width": 320, "height": 180, "fps": 10, "duration": 1, "background": "#202830"}))
+ write(proj / "index.html", HEAD + '<script src="http://127.0.0.1:%d/once.js"></script></head><body style="margin:0;'
+ 'background:#234"><h1 style="color:#fff;font:40px sans-serif">Hi</h1>'
+ "<script>ST.onSeek(() => {});</script></body></html>" % port)
+ cp = showtime("check", proj, "--json", "--no-timeline", "--no-history", "--samples", "2",
+ env=dict(ENV, SHOWTIME_TEST_OPEN_TIMEOUT="30"), timeout=400)
+ self.assertTrue(cp.stdout.strip(), "check printed no report (rc=%d):\n%s" % (cp.returncode, cp.stderr[-2000:]))
+ rep = json.loads(cp.stdout)
+ soft = [f for f in rep["findings"] if f["code"] in ("cut_probe_failed", "determinism_incomplete")]
+ self.assertTrue(soft, [(f["code"], f["message"][:120]) for f in rep["findings"]])
+ for f in soft:
+ self.assertIn(f["severity"], ("info", "warning"), f)
+ self.assertIn("could not open", f["message"])
+ self.assertNotIn("the browser stopped answering", cp.stderr)
+
@unittest.skipIf(sys.platform == "win32", "SIGSTOP is POSIX")
def test_check_stopped_browser(self):
"""A browser that really stops answering (SIGSTOP, SHOWTIME_TEST_STOP_BROWSER): stopped before the first
@@ -323,11 +381,145 @@ process.exit(0);
self.assertEqual(cp.returncode, 1, cp.stderr[-2000:])
self.assertIn("the browser stopped answering", cp.stderr)
self.assertIn("check's last step: opened the image lab page", cp.stderr)
- ps = subprocess.run(["ps", "-axo", "stat=,command="], stdout=subprocess.PIPE, encoding="utf-8", errors="replace").stdout
# the run's browsers hold a temp profile (--user-data-dir) of their own; the command line never names the
- # project, so a stopped (state T) Chrome after the run is any stopped Chrome that was not stopped before
- stopped = [l for l in ps.splitlines() if l.strip().startswith("T") and "--user-data-dir" in l and "chrom" in l.lower()]
- self.assertEqual([l for l in stopped if l not in before], [], "a stopped browser was left behind")
+ # project, so a stopped (state T) Chrome after the run is any stopped Chrome that was not stopped before.
+ # The stuck browser is killed (SIGKILL) as check ends; on a busy machine ps can still list it for a moment,
+ # so it is asked again for up to 10 s before that counts as left behind.
+ for _ in range(20):
+ ps = subprocess.run(["ps", "-axo", "stat=,command="], stdout=subprocess.PIPE, encoding="utf-8", errors="replace").stdout
+ stopped = [l for l in ps.splitlines() if l.strip().startswith("T") and "--user-data-dir" in l and "chrom" in l.lower()]
+ left = [l for l in stopped if l not in before]
+ if not left:
+ break
+ time.sleep(0.5)
+ self.assertEqual(left, [], "a stopped browser was left behind")
+
+
+class TestSizeReadWaitsForTheFrame(unittest.TestCase):
+ """A clipped screenshot (check's timeline and thumbnails, render's --preview and --scale below 1) first reads the
+ page's layout size (issue #49). That read is the first call on the page's main thread after a seek, so it waits
+ for the frame the seek drew: without a GPU, while SwiftShader draws a WebGL frame in the GPU process (the seek
+ itself returns in milliseconds, so the page's pace stays 1). It had a flat 60 s and ended check on a 4-core CI
+ runner without a GPU ("timed out after 60s: reading the page's size", issue #14). It is now guarded like the
+ screenshot it belongs to (guardShot): waited for while the browser's drawing processes work, failed once they do
+ nothing. Here a renderer kept busy past the old deadline (SHOWTIME_TEST_WAIT_SCALE=0.05: 3 s for the old 60 s)
+ stands in for the GPU process at work, and a stopped renderer for a page that is truly stuck."""
+
+ @classmethod
+ def setUpClass(cls):
+ cls.tmp = Path(tempfile.mkdtemp(prefix="st-sizeread-"))
+ cls.proj = cls.tmp / "p"
+ write(cls.proj / "showtime.json", json.dumps({"width": 320, "height": 180, "fps": 10, "duration": 1, "background": "#202830"}))
+ write(cls.proj / "index.html", HEAD + "</head><body style=\"margin:0;background:#202830\"><div id=\"box\" style=\""
+ "position:absolute;left:40px;top:40px;width:120px;height:80px;background:#f4f4f4\"></div><script>" + BUSY_JS +
+ "const box = document.getElementById('box');\n"
+ "ST.onSeek((t) => { box.style.transform = 'translateX(' + Math.round(t * 100) + 'px)'; });"
+ "</script></body></html>")
+
+ @classmethod
+ def tearDownClass(cls):
+ shutil.rmtree(str(cls.tmp), ignore_errors=True)
+
+ def run_shot(self, stop):
+ # the main thread is held right after the seek returns (a task queued by a call that returns at once, not
+ # onSeek, which would hold the seek itself and raise the page's pace), then a clipped screenshot is taken
+ code = """
+const S = await import(%s);
+const { startServer } = await import(%s);
+const server = await startServer({ root: %s, port: 0 });
+const b = await S.openBrowser({});
+const stop = %s;
+let out, stopped = [];
+try {
+ const sess = await S.openStage(b.browser, { url: server.url, page: 'index.html', config: { width: 320, height: 180, fps: 10, duration: 1 } });
+ await sess.seek(0.5);
+ await sess.page.evaluate(() => { setTimeout(() => busy(20000), 0); return true; });
+ if (stop) {
+ const cdp = await b.browser.newBrowserCDPSession();
+ const info = await cdp.send('SystemInfo.getProcessInfo');
+ stopped = info.processInfo.filter((p) => p.type === 'renderer').map((p) => p.id);
+ for (const pid of stopped) process.kill(pid, 'SIGSTOP');
+ }
+ const t0 = Date.now();
+ try {
+ const img = await sess.shot({ format: 'jpeg', scale: 0.5 });
+ out = { ok: true, ms: Date.now() - t0, size: S.imageSize(img), pace: sess.pace.factor };
+ } catch (e) { out = { ok: false, ms: Date.now() - t0, error: String(e.message || e) }; }
+ for (const pid of stopped) { try { process.kill(pid, 'SIGCONT'); } catch {} }
+ await sess.close();
+} catch (e) { out = { ok: false, error: 'setup: ' + String(e.message || e) }; }
+for (const pid of stopped) { try { process.kill(pid, 'SIGCONT'); } catch {} }
+await S.closeSoon(b.browser.close());
+await server.close().catch(() => {});
+console.log(JSON.stringify(out));
+process.exit(0);
+""" % (json.dumps(STAGEHOST), json.dumps(SERVER), json.dumps(str(self.proj)), "true" if stop else "false")
+ return node(code, env=dict(ENV, SHOWTIME_TEST_WAIT_SCALE="0.05"), timeout=180)
+
+ def test_busy_page_is_waited_for(self):
+ """The read waits out a main thread held 20 s, past the old deadline (3 s x the page's pace, 15 s at most;
+ 60 s x pace on a real machine), because the renderer keeps working. On the CI runner the pace stayed 1 (the
+ seek was fast); a slow runner's page may measure more, and the old deadline grows with it."""
+ r = self.run_shot(stop=False)
+ self.assertTrue(r["ok"], r)
+ self.assertEqual(r["size"], [160, 90], r)
+ self.assertGreater(r["ms"], 3000 * r["pace"], r)
+ self.assertGreaterEqual(r["ms"], 15000, r)
+
+ @unittest.skipIf(sys.platform == "win32", "SIGSTOP is POSIX")
+ def test_stopped_page_still_fails(self):
+ """A renderer that does no work at all (stopped) still fails the read, as a stuck screenshot does (#24), in
+ seconds here (the idle window is 1.5 s at this scale, read every 5 s), naming the step."""
+ r = self.run_shot(stop=True)
+ self.assertFalse(r["ok"], r)
+ self.assertRegex(r["error"], r"timed out after \d+s: reading the page's size " + STUCK_FRAME)
+ self.assertLess(r["ms"], 60000, r)
+
+
+class hanging_server:
+ """A local HTTP server whose requests never get an answer (answer_first: the first one gets a small script, every
+ later one waits): a page load that never reaches DOMContentLoaded. -> its port"""
+
+ def __init__(self, answer_first=False):
+ self.answer_first = answer_first
+ self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.sock.bind(("127.0.0.1", 0))
+ self.sock.listen(16)
+ self.held = []
+ self.done = False
+
+ def serve(self):
+ n = 0
+ while not self.done:
+ try:
+ c, _ = self.sock.accept()
+ except OSError:
+ return
+ self.held.append(c)
+ try:
+ c.settimeout(5)
+ c.recv(65536)
+ except OSError:
+ continue
+ n += 1
+ if self.answer_first and n == 1:
+ body = b"window.__once = 1;"
+ c.sendall(b"HTTP/1.1 200 OK\r\nContent-Type: text/javascript\r\nCache-Control: no-store\r\n"
+ b"Connection: close\r\nContent-Length: %d\r\n\r\n%s" % (len(body), body))
+ c.close()
+
+ def __enter__(self):
+ threading.Thread(target=self.serve, daemon=True).start()
+ return self.sock.getsockname()[1]
+
+ def __exit__(self, *exc):
+ self.done = True
+ for c in self.held:
+ try:
+ c.close()
+ except OSError:
+ pass
+ self.sock.close()
def stopped_gpu_processes():
@@ -366,8 +558,14 @@ class TestFrameNeverComes(unittest.TestCase):
self.skipTest("this browser has no GPU process to stop: " + cp.stderr[-300:])
def assertNoStoppedGpu(self):
- time.sleep(1)
- self.assertEqual(stopped_gpu_processes() - self.stopped_before, set(), "a stopped GPU process was left behind")
+ # the stuck browser is killed as the command ends; on a busy machine ps can still list its stopped GPU process
+ # for a moment, so it is asked again for up to 10 s before that counts as left behind
+ for _ in range(20):
+ time.sleep(0.5)
+ left = stopped_gpu_processes() - self.stopped_before
+ if not left:
+ break
+ self.assertEqual(left, set(), "a stopped GPU process was left behind")
def test_render_draws_the_frame_again(self):
"""The screenshot fails once the GPU process and the renderers have done no work for 30 s (x the page's
@@ -382,8 +580,7 @@ class TestFrameNeverComes(unittest.TestCase):
rep = json.loads(cp.stdout)
self.assertEqual(rep["frames"], 20)
self.assertEqual(rep["capture"]["retries"], 1, rep["capture"])
- self.assertRegex(cp.stderr, r"timed out after \d+s: screenshot at 0\.500s \(its GPU process and renderers did no work "
- r"for \d+ s: the frame is not coming\); retrying")
+ self.assertRegex(cp.stderr, r"timed out after \d+s: screenshot at 0\.500s " + STUCK_FRAME + r"; retrying")
# 30 s x the pace (5 at most) and a new browser: never the 10 minutes it was
self.assertLess(took, 300, cp.stderr[-1500:])
self.assertNoStoppedGpu()
@@ -395,8 +592,8 @@ class TestFrameNeverComes(unittest.TestCase):
env=dict(ENV, SHOWTIME_TEST_STOP_GPU="shot:1"), timeout=400)
self.injected(cp)
self.assertEqual(cp.returncode, 1, cp.stderr[-2000:])
- self.assertRegex(cp.stderr, r"the browser stopped drawing: timed out after \d+s: screenshot \(its GPU process and "
- r"renderers did no work for \d+ s: the frame is not coming\); check's last step: \S")
+ self.assertRegex(cp.stderr, r"the browser stopped drawing: timed out after \d+s: screenshot " + STUCK_FRAME +
+ r"; check's last step: \S")
self.assertNoStoppedGpu()
def test_command_timeout_names_the_step(self):