An open source design system that's fully customizable and agent ready
TypeScript
test(interaction-harness): a busy-window proof can judge a component against a compositor-only control (#8906)
Christopher Chedeau8bb30b553de6
diff --git a/internal/interaction-harness/README.md b/internal/interaction-harness/README.mdindex dc881a675a..9f7fb05cdf 100644--- a/internal/interaction-harness/README.md+++ b/internal/interaction-harness/README.md@@ -148,8 +148,15 @@ later in WebKit on a loaded machine). When the cast saw the element at restbefore the block (wait for `cast.frameCount()` to pass 0 before starting theanimation), judging starts at its first presented change: the judged part stillends with the block and still needs four intervals, and `onScreenFromMs` and the-summary report when it came. An element already moving is judged from the-block's start. WebKit's casts are 2 px quantized: the frames come at the+summary report when it came. So is an element that moved, then held from+before the block until its first presented change inside it: WebKit presents+a just-started compositor animation late (a sheet's release held on screen for+130 to 300 ms while its timeline ran), and its cast reaches Node about 0.7 s+late on a loaded runner, too late to wait on before blocking. Only a hold that+starts before the block, on two frames at least, is excused, and every frame+after its first change is judged, so a freeze later in the block still fails.+Any other element already moving is judged from the block's start. WebKit's+casts are 2 px quantized: the frames come at theviewport's size, but the page in them is captured at half resolution, sopositions move in 2 CSS px steps (Chromium's in 1). With under two steps ofmotion per frame, a frame delivered a few ms early shows no step and reads as@@ -157,6 +164,47 @@ not moved. The proof reports `stepPx` and `pxPerFrame` and notes motion undertwo steps per frame, which never fails it. Move a proof's element at least twosteps a frame: the self-tests move 560 px in 1,600 ms, about 4.8 px a frame.+On a two-core runner the software compositor and the screencast share the core+the block spins on, so every compositor animation presents fewer frames there:+Lightbox's double-tap zoom, pinned to two cores, missed a display-rate bound in+7 of 10 runs while it moved on every frame the compositor presented. Judge such+a proof against a control instead of a rate: `startBusyControl(page)` shows a+small `CONTROL_RGB` box in the top layer (a manual popover, so above a modal+dialog opened before it; `raise()` after one opened later) whose transform+moves it along the top edge for as long as the page runs. A proof of a+dialog's backdrop mounts the control inside the dialog with+`startBusyControl(page, {within: dialog})`, a plain fixed box with no top-layer+entry: in WebKit, with the popover above BottomSheet's open modal dialog, the+sheet's dim held for 155 to 225 ms into a block while the control moved, and+inside the dialog it moves in step with the sheet. `waitForAnimation` and+`blockMainThread`'s `during` never count the control's own animation, so on+a dialog they still wait for the backdrop's. Track it from the+same screencast (`trackMarker(page, frames, {rgb: CONTROL_RGB})`) and pass it+as `control`: the component passes when it moved on every frame the control+moved and its longest stall is no longer than the control's. Both are read from+the same screencast frames, so a compositor animation misses none. A window in+which the control moved on fewer than four frames fails, since the machine+presented too little to judge. A run of frames with no new position counts as+moved when the change that ends it is a single cast step: the component moves+under one step a frame there, as on an easing tail, which the cast cannot show+on every frame. The run must also read as a slowing curve, at most one frame+longer than the component's run before it. A freeze ends in a catch-up of more+than one step, a stall in a frame-counted animation is longer than its runs+before it, and a run that no change ends stays a miss. The component still has+to show a new position on at least four of the control's frames. The cast step (`stepPx`)+is read from the value's changes in the judged part, so a position the page+commits off the grid once the block ends does not shrink it.+A main-thread animation still fails: the control moves and it does not.++An animation that lasts a fraction of a second, such as a 0.2 s fade, leaves a+block only a few frames to judge: in WebKit, or on a loaded machine, the control+can present fewer than four, and one run cannot decide. Play it several times+and judge the runs pooled (`pooledBusyProof(runs)`, each run with its own+tracked frames, control and window): the control must move on at least four+frames over all the runs, the component on every one of them, and in each run+its longest stall may be no longer than that run's control's, since a stall+belongs to its run.+On a loaded machine Chromium's screencast can skip frames the compositorstill presents. `startCompositorTrace(page)`, started with the screencast andstopped after the block, records when the compositor presented each framediff --git a/internal/interaction-harness/src/busy.control.test.ts b/internal/interaction-harness/src/busy.control.test.tsnew file mode 100644index 0000000000..f18b732b53--- /dev/null+++ b/internal/interaction-harness/src/busy.control.test.ts@@ -0,0 +1,486 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * @file busy.control.test.ts+ * @input Synthetic tracked frames (no browser)+ * @output The busy-main-thread verdict against a control: a component that+ * moves on the control's frames passes through a stall the compositor itself+ * makes, and a frozen one, or one the control outpaces by a single frame,+ * still fails. Unchanged frames under one cast step a frame count as+ * moved; a freeze, whose catch-up is more than one step, and a stall+ * longer than the component's runs before it do not. Runs+ * of a block too short to judge alone are judged pooled: the control over+ * all of them, each run's stall against that run's control.+ * @position Unit test of ./busy's control judgment; the frames are+ * busy.test.ts's fixtures, and the browser proof with a real compositor+ * control is in harness.interaction.browser.spec.ts.+ */++import {describe, expect, it} from 'vitest';+import {busyWindowProof, pooledBusyProof} from './busy';+import type {TrackedFrame} from './frames';++const FRAME = 1000 / 60;+const BLOCK = {from: 1000, to: 1700};++/** A marker moving 3 px per frame from t = 800 to 1900, minus `dropped` frame indices. */+function moving(dropped: readonly number[] = []): TrackedFrame[] {+ const frames: TrackedFrame[] = [];+ for (let i = 0; 800 + i * FRAME <= 1900; i += 1) {+ if (dropped.includes(i)) {+ continue;+ }+ frames.push({+ t: 800 + i * FRAME,+ box: {x: 0, y: 3 * i, width: 10, height: 10},+ } as TrackedFrame);+ }+ return frames;+}++/** Index of the first frame at or after `t`. */+const at = (t: number) => Math.ceil((t - 800) / FRAME);++/** `moving()` with each frame in `repeats` presenting the previous frame's position. */+function repeating(repeats: readonly number[]): TrackedFrame[] {+ return moving().map((f, i) =>+ repeats.includes(i)+ ? ({...f, box: {...f.box!, y: 3 * (i - 1)}} as TrackedFrame)+ : f,+ );+}++/** `frames` as the control's: the same frames and times, moving along x. */+function asControl(frames: readonly TrackedFrame[]): TrackedFrame[] {+ return frames.map(+ f =>+ ({+ ...f,+ box: f.box == null ? null : {x: f.box.y, y: 0, width: 12, height: 12},+ }) as TrackedFrame,+ );+}++/**+ * A frame-counted loop: `step` px a frame, standing still from `lag` ms into+ * each window to its end, then going on without catching up.+ */+function frameCounted(+ windows: readonly {from: number; to: number}[],+ lag: number,+ step = 1,+): TrackedFrame[] {+ let y = 0;+ return moving().map((f, i) => {+ if (i > 0 && !windows.some(w => f.t >= w.from + lag && f.t < w.to)) {+ y += step;+ }+ return {...f, box: {...f.box!, y}} as TrackedFrame;+ });+}++describe('busyWindowProof against a control', () => {+ it('passes a component that moves on every frame the control does, through a stall the compositor makes for both, which fails without the control', () => {+ // Three frames in a row presented nothing, for the control too: the+ // compositor shared the core the block spins on (a two-core runner).+ const gap = [at(1300), at(1300) + 1, at(1300) + 2];+ const component = moving(gap);+ const control = asControl(moving(gap));+ const alone = busyWindowProof(component, BLOCK);+ expect(alone.verdict.failures.join('\n')).toMatch(/longest stall/);+ const against = busyWindowProof(component, BLOCK, {control});+ expect(against.verdict.failures, against.summary).toEqual([]);+ expect(against.control).toMatchObject({moved: against.control?.movedWith});+ expect(against.summary).toMatch(+ /against the control: moved on \d+ of its \d+ frames/,+ );+ });++ it('still fails a frozen component while the control moves', () => {+ const frozen = moving().map(+ f =>+ ({+ ...f,+ box: {+ ...f.box!,+ y: f.t >= BLOCK.from ? 3 * at(BLOCK.from) : f.box!.y,+ },+ }) as TrackedFrame,+ );+ const proof = busyWindowProof(frozen, BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.verdict.pass).toBe(false);+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on 0 of the \d+ frames the control moved/,+ );+ expect(proof.verdict.failures.join('\n')).toMatch(+ /longer than the control's/,+ );+ });++ it('fails a component the control outpaces: it repeats positions on frames the control moves', () => {+ const repeats = [0, 1, 2, 3].map(k => at(1100) + 6 * k);+ const proof = busyWindowProof(repeating(repeats), BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ });++ it('fails a component that misses a single frame the control moved', () => {+ // One repeated position: a frame the control moved on and the component+ // did not, which a compositor animation never shows, since both are read+ // from the same screencast frames.+ const proof = busyWindowProof(repeating([at(1300)]), BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ expect(proof.control!.moved - proof.control!.movedWith).toBe(1);+ });++ it('counts unchanged frames on a tail under one cast step a frame as moved: each run of them ends in a single step', () => {+ // 3 px a frame, then half a pixel a frame from 1400 ms: the whole-pixel+ // positions change every other frame, one step at a time.+ const k = at(1400);+ const tail = moving().map((f, i) =>+ i < k+ ? f+ : ({+ ...f,+ box: {...f.box!, y: 3 * k + Math.floor((i - k) / 2)},+ } as TrackedFrame),+ );+ const proof = busyWindowProof(tail, BLOCK, {control: asControl(moving())});+ expect(proof.verdict.failures, proof.summary).toEqual([]);+ expect(proof.control?.belowStep).toBeGreaterThan(0);+ expect(proof.summary).toMatch(/under one cast step a frame/);+ });++ it('still fails a slow component frozen on frames the control moved on: the change that ends the run catches up more than one step', () => {+ // Half a pixel a frame throughout, frozen for 12 frames from 1200 ms, then+ // back where its clock says: the change that ends the run is 6 steps.+ const k = at(1200);+ const slow = moving().map(+ (f, i) =>+ ({+ ...f,+ box: {+ ...f.box!,+ y: Math.floor((i >= k && i < k + 12 ? k - 1 : i) / 2),+ },+ }) as TrackedFrame,+ );+ const proof = busyWindowProof(slow, BLOCK, {control: asControl(moving())});+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ });++ it('still fails a component that stops for good while the control moves: no change ends its run', () => {+ const k = at(1300);+ const stopped = moving().map((f, i) =>+ i < k ? f : ({...f, box: {...f.box!, y: 3 * k}} as TrackedFrame),+ );+ const proof = busyWindowProof(stopped, BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ });++ it('still fails a frame-counted component that stands still through the block and resumes one step a frame: it shows too little motion to judge', () => {+ // 1 px a frame, counted per frame rather than by the clock: it stands+ // still from the block's start to its end, then goes on one step a+ // frame, so the change that ends the run is a single step.+ const s0 = at(BLOCK.from);+ const s1 = at(BLOCK.to);+ const counted = moving().map(+ (f, i) =>+ ({+ ...f,+ box: {+ ...f.box!,+ y: i < s0 ? i : i <= s1 ? s0 - 1 : s0 - 1 + (i - s1),+ },+ }) as TrackedFrame,+ );+ const proof = busyWindowProof(counted, BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.verdict.failures.join('\n')).toMatch(+ /too little motion in this window to judge/,+ );+ });++ it('still fails a frame-counted loop that stalls for a few frames inside the block: its run is longer than the runs before it', () => {+ // 1 px a frame, so each step is one cast step, standing still for three+ // frames from 1300 ms and going on from there without catching up.+ const proof = busyWindowProof(+ frameCounted([{from: 1300, to: 1350}], 0),+ BLOCK,+ {control: asControl(moving())},+ );+ expect(proof.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ expect(proof.control?.belowStep).toBe(0);+ });++ it('reads the finest cast step when the judged part shows no change, so a larger jump never passes for one step', () => {+ const frozen = moving().map(+ f =>+ ({...f, box: {...f.box!, y: f.t < BLOCK.to ? 0 : 2}}) as TrackedFrame,+ );+ const proof = busyWindowProof(frozen, BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.stepPx).toBe(0.5);+ expect(proof.control?.belowStep).toBe(0);+ });++ it('reads the cast step from changes, so a position committed off the grid once the block ends does not shrink it', () => {+ // An ease-out on a 2 px grid, as WebKit casts one, that ends on 401 once+ // the block is over: its tail shows runs of unchanged frames that end in+ // a single 2 px step.+ const eased = moving().map(+ (f, i) =>+ ({+ ...f,+ box: {+ ...f.box!,+ y:+ f.t > BLOCK.to + 20+ ? 401+ : 2 * Math.round((400 - 400 * 0.93 ** i) / 2),+ },+ }) as TrackedFrame,+ );+ const proof = busyWindowProof(eased, BLOCK, {+ control: asControl(moving()),+ });+ expect(proof.stepPx).toBe(2);+ expect(proof.verdict.failures, proof.summary).toEqual([]);+ expect(proof.control?.belowStep).toBeGreaterThan(0);+ });++ it('fails a window in which the control itself presented too little to judge', () => {+ const still = asControl(+ moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame),+ );+ const proof = busyWindowProof(moving(), BLOCK, {control: still});+ expect(proof.verdict.failures.join('\n')).toMatch(+ /the control moved on 0 frames in the judged part/,+ );+ });+});++/** A 90 ms block from `from`: its judged part holds three 60 Hz frames. */+const short = (from: number) => ({from, to: from + 90});++describe('pooledBusyProof', () => {+ it('judges a block too short for one run over pooled runs: one run cannot decide, three runs of a component that moves with the control pass', () => {+ const control = asControl(moving());+ const single = busyWindowProof(moving(), short(1000), {control});+ expect(single.verdict.failures.join('\n')).toMatch(+ /the control moved on 3 frames in the judged part, under 4/,+ );+ const runs = [1000, 1300, 1600].map(from => ({+ tracked: moving(),+ control,+ window: short(from),+ }));+ const pooled = pooledBusyProof(runs);+ expect(pooled.verdict.failures, pooled.summary).toEqual([]);+ expect(pooled.control).toEqual({moved: 9, movedWith: 9});+ expect(pooled.runs).toHaveLength(3);+ expect(pooled.summary).toMatch(+ /^3 runs pooled: moved on 9 of the control's 9 frames; run 1: /,+ );+ });++ it('still fails a component that is frozen in every run while the control moves', () => {+ const frozen = moving().map(+ f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame,+ );+ const pooled = pooledBusyProof(+ [1000, 1300, 1600].map(from => ({+ tracked: frozen,+ control: asControl(moving()),+ window: short(from),+ })),+ );+ expect(pooled.verdict.pass).toBe(false);+ expect(pooled.verdict.failures.join('\n')).toMatch(+ /moved on 0 of the 9 frames the control moved over 3 runs/,+ );+ expect(pooled.verdict.failures.join('\n')).toMatch(+ /run 1: longest stall .* longer than the control's/,+ );+ });++ it("judges each run's longest stall against that run's control, never pooled", () => {+ // Two frames in a row present no new position, inside run 2 only.+ const k = at(1320);+ const stalled = moving().map((f, i) =>+ i === k || i === k + 1+ ? ({...f, box: {...f.box!, y: 3 * (k - 1)}} as TrackedFrame)+ : f,+ );+ const pooled = pooledBusyProof(+ [1000, 1300, 1600].map(from => ({+ tracked: stalled,+ control: asControl(moving()),+ window: short(from),+ })),+ );+ const failures = pooled.verdict.failures.join('\n');+ expect(failures).toMatch(+ /run 2: longest stall .* longer than the control's/,+ );+ expect(failures).not.toMatch(/run [13]: longest stall/);+ });++ it('fails a pool in which the component misses a single frame the control moved', () => {+ const pooled = pooledBusyProof(+ [1000, 1300, 1600].map(from => ({+ tracked: repeating([at(1320)]),+ control: asControl(moving()),+ window: short(from),+ })),+ );+ expect(pooled.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved over 3 runs/,+ );+ expect(pooled.control.moved - pooled.control.movedWith).toBe(1);+ });++ it("counts a pooled run's unchanged frames under one cast step a frame as moved, as one window does", () => {+ const k = at(1000);+ const tail = moving().map((f, i) =>+ i < k+ ? f+ : ({+ ...f,+ box: {...f.box!, y: 3 * k + Math.floor((i - k) / 2)},+ } as TrackedFrame),+ );+ const pooled = pooledBusyProof(+ [1000, 1300, 1600].map(from => ({+ tracked: tail,+ control: asControl(moving()),+ window: short(from),+ })),+ );+ expect(pooled.verdict.failures, pooled.summary).toEqual([]);+ });++ it('still fails a pooled frame-counted loop that the cast shows moving early in each judged part', () => {+ // Six 70 ms blocks; the cast shows the loop's last frames up to 35 ms into+ // each, then it stands still to the block's end and goes on one step a+ // frame: a 1 px loop's steps are one cast step, and a 2 px loop's read+ // as one when its changes are all 2 px.+ const windows = [1000, 1150, 1300, 1450, 1600, 1750].map(from => ({+ from,+ to: from + 70,+ }));+ for (const step of [1, 2]) {+ const pooled = pooledBusyProof(+ windows.map(window => ({+ tracked: frameCounted(windows, 35, step),+ control: asControl(moving()),+ window,+ })),+ );+ expect(pooled.verdict.failures.join('\n'), `${step} px`).toMatch(+ /moved on \d+ of the \d+ frames the control moved over 6 runs/,+ );+ }+ });++ it('judges the control over the pool too: runs in which it presented too little fail', () => {+ const still = asControl(+ moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame),+ );+ const pooled = pooledBusyProof(+ [1000, 1300].map(from => ({+ tracked: moving(),+ control: still,+ window: short(from),+ })),+ );+ expect(pooled.verdict.failures.join('\n')).toMatch(+ /the control moved on 0 frames over 2 runs, under 4/,+ );+ });++ it('judges one long run as busyWindowProof judges it against the control', () => {+ const control = asControl(moving());+ const pooled = pooledBusyProof([+ {tracked: moving(), control, window: BLOCK},+ ]);+ const single = busyWindowProof(moving(), BLOCK, {control});+ expect(pooled.verdict).toEqual(single.verdict);+ expect(pooled.control).toEqual({+ moved: single.control?.moved,+ movedWith: single.control?.movedWith,+ });+ });++ it('throws on no runs', () => {+ expect(() => pooledBusyProof([])).toThrow(/no runs/);+ });+});++describe('a hold that starts before the block', () => {+ // A sheet's release in WebKit: moving until 120 ms before the block, held+ // (the just-started animation not presented yet) until 100 ms into it,+ // then moving on every frame.+ const holdFrom = BLOCK.from - 120;+ const holdTo = BLOCK.from + 100;+ const sheet = ({freezeAfterFirst = false} = {}) => {+ let y = 0;+ let changed = false;+ return moving().map(f => {+ if (f.t < holdFrom) {+ y += 3;+ } else if (f.t >= holdTo && (!freezeAfterFirst || !changed)) {+ y += 3;+ changed = true;+ }+ return {...f, box: {...f.box!, y}} as TrackedFrame;+ });+ };+ const control = asControl(moving());++ it('is judged from its first presented change inside the block', () => {+ const proof = busyWindowProof(sheet(), BLOCK, {axis: 'y', control});+ expect(proof.verdict.failures, proof.summary).toEqual([]);+ expect(proof.onScreenFromMs).toBeGreaterThanOrEqual(100);+ });++ it('still fails when the element changes once inside the block and then freezes', () => {+ const proof = busyWindowProof(sheet({freezeAfterFirst: true}), BLOCK, {+ axis: 'y',+ control,+ });+ expect(proof.verdict.pass, proof.summary).toBe(false);+ });++ it('is not excused when it starts inside the block', () => {+ let y = 0;+ const frames = moving().map(f => {+ if (f.t < BLOCK.from + 50 || f.t >= BLOCK.from + 250) {+ y += 3;+ }+ return {...f, box: {...f.box!, y}} as TrackedFrame;+ });+ const proof = busyWindowProof(frames, BLOCK, {axis: 'y', control});+ expect(proof.verdict.pass, proof.summary).toBe(false);+ expect(proof.onScreenFromMs).toBeNull();+ });+});diff --git a/internal/interaction-harness/src/busy.ts b/internal/interaction-harness/src/busy.tsindex 55fd9d670d..918834f7cd 100644--- a/internal/interaction-harness/src/busy.ts+++ b/internal/interaction-harness/src/busy.ts@@ -4,8 +4,11 @@* @file busy.ts* @input Uses a Playwright `Page`* @output `blockMainThread` — a synchronous busy loop on the page's main thread- * that reports exactly when it ran — and `busyWindowProof`, which reads frame- * evidence for that window.+ * that reports exactly when it ran —, `startBusyControl` — a plain+ * compositor-only control animation recorded in the same window —,+ * `busyWindowProof`, which reads frame evidence for that window, against+ * the control when it has one, and `pooledBusyProof`, which judges several+ * short blocks as one.* @position The busy-main-thread proof: an animation that runs on the* compositor keeps advancing at display rate while JavaScript holds the main* thread; one driven from the main thread (rAF, style writes, layout@@ -18,6 +21,138 @@import type {Locator, Page} from '@playwright/test';import {frameStats, type FrameStats, type TrackedFrame} from './frames';+/** The control's colour, apart from the component's marker (`MARKER_RGB`). */+export const CONTROL_RGB: readonly [number, number, number] = [0, 255, 255];++/** The control box's side, and how far it travels along the top edge, CSS px. */+const CONTROL_SIDE = 12;+const CONTROL_TRAVEL = 240;++/**+ * Starts a plain compositor-only control animation: a `CONTROL_RGB` box in the+ * page's top-left corner whose transform moves it back and forth along the top+ * edge (`CONTROL_TRAVEL` px every 400 ms, linear) for as long as the page+ * runs. It shows what the compositor presented in a busy window on this+ * machine, so a proof can judge a component against it rather than against a+ * frame rate: track it from the same screencast (`trackMarker(page, frames,+ * {rgb: CONTROL_RGB})`) and pass it to `busyWindowProof` as `control`.+ *+ * The box is a manual popover, so it shows in the top layer, above a modal+ * dialog the page opened before it; `raise()` shows it again on top of one+ * opened after. With `within`, it is mounted inside that element instead (an+ * open modal dialog), as a plain fixed box with no top-layer entry, for a+ * proof of the dialog's `::backdrop`: in WebKit, with the popover above+ * BottomSheet's open modal dialog, the sheet's dim held for 155 to 225 ms+ * into a block while the control moved. `waitForAnimation` and+ * `blockMainThread`'s `during` never count the control's own animation, so+ * they still wait for the dialog's. Start it before the block: it+ * resolves once the animation is running and past its first frame (on the+ * compositor; see `blockMainThread`). Keep the component's marker out of the+ * strip it crosses (the top `CONTROL_SIDE` px, `CONTROL_SIDE ++ * CONTROL_TRAVEL` px wide).+ */+export async function startBusyControl(+ page: Page,+ {+ rgb = CONTROL_RGB,+ within,+ }: {rgb?: readonly [number, number, number]; within?: Locator} = {},+): Promise<{+ readonly raise: () => Promise<void>;+ readonly remove: () => Promise<void>;+}> {+ if (within != null) {+ await within.evaluate(host =>+ host.setAttribute('data-harness-control-host', ''),+ );+ }+ await page.evaluate(+ ({color, side, travel}) =>+ new Promise<void>((resolve, reject) => {+ const host = document.querySelector('[data-harness-control-host]');+ host?.removeAttribute('data-harness-control-host');+ const box = document.createElement('div');+ box.setAttribute('data-harness-control', '');+ box.style.cssText = `position:fixed;inset:auto;left:0;top:0;margin:0;padding:0;border:0;overflow:visible;width:${side}px;height:${side}px;background:rgb(${color.join(',')});z-index:2147483647;pointer-events:none;`;+ if (host == null) {+ box.setAttribute('popover', 'manual');+ document.body.appendChild(box);+ box.showPopover();+ } else {+ host.appendChild(box);+ }+ const animation = box.animate(+ [+ {transform: 'translateX(0px)'},+ {transform: `translateX(${travel}px)`},+ ],+ {+ duration: 400,+ iterations: Infinity,+ direction: 'alternate',+ easing: 'linear',+ },+ );+ // The harness's own waits run on the real timers (see blockMainThread).+ const gate = (+ window as unknown as {+ __harnessTimerGate?: {frame?: typeof requestAnimationFrame};+ }+ ).__harnessTimerGate;+ const frame = gate?.frame ?? window.requestAnimationFrame.bind(window);+ const deadline = performance.now() + 5_000;+ const tick = () => {+ if (!animation.pending && Number(animation.currentTime ?? 0) >= 16) {+ resolve();+ } else if (performance.now() > deadline) {+ reject(new Error('startBusyControl: the control did not start'));+ } else {+ frame(tick);+ }+ };+ frame(tick);+ }),+ {color: [...rgb], side: CONTROL_SIDE, travel: CONTROL_TRAVEL},+ );+ return {+ raise: () =>+ page.evaluate(() => {+ for (const box of document.querySelectorAll<HTMLElement>(+ '[data-harness-control][popover]',+ )) {+ box.hidePopover();+ box.showPopover();+ }+ }),+ remove: () =>+ page.evaluate(() => {+ for (const box of document.querySelectorAll('[data-harness-control]')) {+ box.remove();+ }+ }),+ };+}++/** What the control presented in the judged part (see `busyWindowProof`). */+export interface BusyControlRead {+ /** Frames in the judged part that presented a new control position. */+ readonly moved: number;+ /** Of those, the frames that also presented a new component position. */+ readonly movedWith: number;+ /** The control's longest time without a new position, the edges included. */+ readonly longestStallMs: number;+ /**+ * Of `movedWith`, the frames counted as moved because the component's motion+ * there was under one cast step per frame (see `busyWindowProof`).+ */+ readonly belowStep: number;+ /**+ * The component's longest time without a new position, read the same way:+ * a run of unchanged frames under one cast step per frame is motion.+ */+ readonly componentStallMs: number;+}+export interface BusyWindow {/** Epoch ms when the loop started and ended, on the page's own clock. */readonly from: number;@@ -43,8 +178,9 @@ export interface BusyWindow {* holds that hand-off, and even a compositor animation freezes. To catch a* settle near its start, call it before the release that starts the settle and* await it after: the page is already waiting when the animation begins. Any- * animation inside `during` counts, so pick an element that runs nothing- * else meanwhile.+ * animation inside `during` counts but the busy-window control's own+ * (`startBusyControl`, mounted inside a dialog with `within`), so pick an+ * element that runs nothing else meanwhile.** An animation counts once it is no longer pending and has run at least one* frame (`minProgressMs` below 16 ms is raised to 16). One created in the@@ -94,11 +230,21 @@ export async function blockMainThread(const frame = gate?.frame ?? window.requestAnimationFrame.bind(window);const later = gate?.timeout ?? window.setTimeout.bind(window);const deadline = performance.now() + wait;+ // The busy-window control animates on its own, inside a dialog with+ // `within`: it never stands for the animation under test.+ const isControl = (a: Animation) => {+ const target = (a.effect as KeyframeEffect | null)?.target ?? null;+ return (+ target instanceof Element &&+ target.closest('[data-harness-control]') != null+ );+ };const tick = () => {const running = element.getAnimations({subtree: true}).some(a =>+ !isControl(a) &&a.playState === 'running' &&!a.pending &&Number(a.currentTime ?? 0) >= minProgress,@@ -149,7 +295,8 @@ export interface BusyProof extends FrameStats {readonly onScreenFromMs: number | null;/*** The cast's position step, CSS px: the largest of 0.5, 1 and 2 that every- * tracked position is a multiple of (WebKit's casts are 2: see the README).+ * change of the tracked value in the judged part is a multiple of, or 0.5+ * when it does not change there (WebKit's casts step 2: see the README).*/readonly stepPx: number;/** The tracked value's mean motion per screencast interval in the judged part, px. */@@ -206,6 +353,8 @@ export interface BusyProof extends FrameStats {* frames were delivered.*/readonly movedFraction: number;+ /** With a `control`: what it presented in the judged part; null without one. */+ readonly control: BusyControlRead | null;/** The judgment against the limits (see `busyWindowProof`). */readonly verdict: BusyVerdict;/** One line with the raw numbers, for an annotation or a failure message. */@@ -244,10 +393,17 @@ export interface BusyProof extends FrameStats {* tracked value was seen at rest before the judged part (a frame there, and no* change in the 250 ms before it), judging starts at its first presented* change inside the block, which the- * proof reports as `onScreenFromMs`. The judged part still ends with the- * block and the `minJudgedFrames` floor applies to it, so a start the block- * itself delays fails; an animation that was already moving is judged from- * the block's start, and a stall there fails as before.+ * proof reports as `onScreenFromMs`. So does a hold that starts before the+ * block (two frames at least at one position, the first before it) and ends+ * at the first presented change inside it: WebKit presents a just-started+ * compositor animation late (a sheet's release held on screen for 130 to+ * 300 ms while its timeline ran, and the screencast reaches Node about 0.7 s+ * late on a loaded runner, too late to wait on before blocking). Every frame+ * after that first change is judged, so a freeze later in the block fails.+ * The judged part still ends with the block and the `minJudgedFrames` floor+ * applies to it, so a start the block itself delays fails; any other+ * animation that was already moving is judged from the block's start, and a+ * stall there fails as before.** `stepPx` and `pxPerFrame` say whether a frame's motion is resolvable at* all: under two position steps per frame, a frame delivered early can show@@ -262,6 +418,30 @@ export interface BusyProof extends FrameStats {* that opens or closes the judged part, or holds a delivered frame that did* not move, is judged from the screencast as before, so a frozen element* fails whatever else the compositor draws.+ *+ * With a `control` (`startBusyControl`'s box, tracked from the same+ * screencast), the window is judged against what the compositor presented in+ * it rather than against a frame rate: on a two-core runner the software+ * compositor and the screencast share the core the block spins on, so every+ * compositor animation presents fewer frames there, the control too. The+ * component passes when:+ * - the control moved on at least `minJudgedFrames` frames of the judged part+ * (fewer: the machine presented too little to judge, which fails);+ * - the component moved on every frame the control moved: both are read from+ * the same screencast frames, so a compositor animation misses none;+ * - its longest stall is no longer than the control's.+ * A run of frames with no new component position counts as moved when the+ * change that ends it is a single cast step (`stepPx`): the component moves+ * under one step per frame there, as on an easing tail, which the cast cannot+ * show on every frame. The run also has to read as a slowing curve, at most+ * one frame longer than the component's run before it. A freeze ends in a+ * catch-up of more than one step, a stall in a frame-counted animation is+ * longer than its runs before it, and a run that no change ends stays a miss.+ * The component still has to show a new position on at least+ * `minJudgedFrames` of the control's frames.+ * A main-thread animation still fails: the control moves and it does not.+ * `maxMedianFrameMs`, when given, still applies; `minMovedFraction` and the+ * absolute stall, frame and motion limits do not.*/export function busyWindowProof(tracked: readonly TrackedFrame[],@@ -276,6 +456,7 @@ export function busyWindowProof(minMovedFraction = 0.9,minJudgedFrames = 4,presented,+ control,}: {axis?: 'x' | 'y' | 'width' | 'height';marginMs?: number;@@ -316,6 +497,11 @@ export function busyWindowProof(* judged alone.*/presented?: readonly number[];+ /**+ * The control's frames (`startBusyControl`, tracked from the same+ * screencast with `CONTROL_RGB`): judge the component against them.+ */+ control?: readonly TrackedFrame[];} = {},): BusyProof {let end = window.to - marginMs;@@ -369,8 +555,40 @@ export function busyWindowProof(const movedBefore = lead.some((value, i) => i > leadFrom && Math.abs(value - lead[i - 1]) > 0.25,);+ // Held from before the block to its first presented change inside it: in+ // WebKit a just-started compositor animation is presented late (a sheet's+ // release held on screen for 130 to 300 ms while its timeline ran, and the+ // screencast reaches Node about 0.7 s late on a loaded runner, too late to+ // wait on before blocking), so a hold that STARTS before the block is the+ // engine's, and judging starts at that first change. Only such a hold: from+ // that change on every frame is judged, so a freeze later in the block+ // still fails. The hold shows on two frames at least, the first before the+ // block.+ let holdStart = lead.length - 1;+ while (+ holdStart > 0 &&+ Math.abs(lead[holdStart - 1] - lead[lead.length - 1]) <= 0.25+ ) {+ holdStart -= 1;+ }+ const heldFrames =+ lead.length -+ holdStart ++ values.filter(+ v =>+ v.value != null &&+ v.t < (inWindow[0] ?? -Infinity) &&+ Math.abs(v.value - lead[lead.length - 1]) <= 0.25,+ ).length;+ const heldFromBeforeBlock =+ lead.length > 0 &&+ holdStart > 0 &&+ prior[holdStart].t < window.from &&+ heldFrames >= 2;const onScreenFrom =- prior.length > 0 && !movedBefore && inWindow.length > 0+ prior.length > 0 &&+ (!movedBefore || heldFromBeforeBlock) &&+ inWindow.length > 0? inWindow[0]: null;const judged = {from: onScreenFrom ?? inner.from, to: inner.to};@@ -479,12 +697,29 @@ export function busyWindowProof(? expectedFrames: deliveredFrames;const movedFraction = (moved + recorderSkipped) / Math.max(1, judgedFrames);- // The cast's position step, and the motion per interval it has to resolve.- const positions = tracked.flatMap(f => (f.box == null ? [] : [f.box[axis]]));+ // The cast's position step for the tracked value: the largest of 2, 1 and+ // 0.5 CSS px that every change between its consecutive positions in the+ // judged part is a multiple of. Changes rather than positions: a position+ // the page commits once the block ends can sit off that grid.+ const partValues = tracked.flatMap(f =>+ f.box != null && f.t >= judged.from && f.t <= judged.to+ ? [f.box[axis]]+ : [],+ );+ const stepChanges = partValues+ .slice(1)+ .map((p, i) => Math.abs(p - partValues[i]))+ .filter(d => d > 0.25);+ // With no change to read, the finest step: a coarser one would let a larger+ // jump pass for a single step.const stepPx =- [2, 1, 0.5].find(step =>- positions.every(p => Math.abs(p / step - Math.round(p / step)) < 0.01),- ) ?? 0.5;+ stepChanges.length === 0+ ? 0.5+ : ([2, 1, 0.5].find(step =>+ stepChanges.every(+ d => Math.abs(d / step - Math.round(d / step)) < 0.01,+ ),+ ) ?? 0.5);const judgedValues = tracked.flatMap(f =>f.box != null && f.t >= judged.from && f.t <= judged.to? [{t: f.t, value: f.box[axis]}]@@ -512,20 +747,123 @@ export function busyWindowProof(`${pxPerFrame.toFixed(1)} px/frame is under two of the cast's ${stepPx} px position steps: a frame delivered early can show no step and read as not moved`,);}+ // The control's own new positions in the judged part, against its frame+ // before the part, and of those the frames the component moved on too:+ // both are tracked from the same screencast frames, so their times match.+ let controlRead: BusyControlRead | null = null;+ if (control != null) {+ const baseline = [...control]+ .reverse()+ .find(f => f.box != null && f.t < inner.from);+ let last: number | null = baseline?.box == null ? null : baseline.box.x;+ const controlChanges: number[] = [];+ for (const f of control) {+ if (f.box == null || f.t < inner.from || f.t > judged.to) {+ continue;+ }+ if (last != null && Math.abs(f.box.x - last) > 0.25) {+ if (f.t > judged.from || onScreenFrom == null) {+ controlChanges.push(f.t);+ }+ }+ last = f.box.x;+ }+ const controlEdges = [judged.from, ...controlChanges, judged.to];+ // A frame with no new component position, in a run of such frames whose+ // next change is a single cast step: the component moves under one step+ // per frame there, which the cast cannot show on every frame, so the+ // frame counts as moved. A freeze ends in a catch-up of more than one+ // step, and a run that no change ends stays a miss.+ const series = tracked.flatMap(f =>+ f.box == null ? [] : [{t: f.t, value: f.box[axis]}],+ );+ const seriesIndex = new Map(series.map((f, i) => [f.t, i]));+ const same = (a: number, b: number) =>+ Math.abs(series[a].value - series[b].value) <= 0.25;+ // The first frame of the run of equal positions that holds frame i.+ const runStart = (i: number): number => {+ let s = i;+ while (s > 0 && same(s - 1, i)) {+ s -= 1;+ }+ return s;+ };+ const underStep = (t: number): boolean => {+ const i = seriesIndex.get(t);+ if (i == null || changed.has(t)) {+ return false;+ }+ const start = runStart(i);+ let k = i + 1;+ while (k < series.length && same(k, i)) {+ k += 1;+ }+ if (+ k >= series.length ||+ Math.abs(series[k].value - series[i].value) >= 1.5 * stepPx+ ) {+ return false;+ }+ // A slowing curve lengthens its runs a frame at a time: a run more than+ // one frame longer than the one before it is a stall.+ const unchanged = k - 1 - start;+ const before = start > 0 ? start - 1 - runStart(start - 1) : 0;+ return unchanged <= before + 1;+ };+ const componentEdges = [+ judged.from,+ ...delivered.filter(t => changed.has(t) || underStep(t)),+ judged.to,+ ];+ controlRead = {+ moved: controlChanges.length,+ movedWith: controlChanges.filter(t => changed.has(t) || underStep(t))+ .length,+ belowStep: controlChanges.filter(underStep).length,+ longestStallMs: Math.max(+ ...controlEdges.slice(1).map((t, i) => t - controlEdges[i]),+ ),+ componentStallMs: Math.max(+ ...componentEdges.slice(1).map((t, i) => t - componentEdges[i]),+ ),+ };+ }const failures: string[] = [];- if (stats.frames === 0) {+ if (stats.frames === 0 && controlRead == null) {failures.push('no frame presented in the judged part');} else if (stats.medianFrameMs > maxMedianFrameMs) {failures.push(`median frame interval ${stats.medianFrameMs.toFixed(1)} ms > ${maxMedianFrameMs} ms (${stats.frames} frames presented)`,);}- if (longestStallFrames > maxStallFrames) {+ if (controlRead != null) {+ if (controlRead.moved < minJudgedFrames) {+ failures.push(+ `the control moved on ${controlRead.moved} frames in the judged part, under ${minJudgedFrames}: the compositor presented too little in this window to judge`,+ );+ }+ const visible = controlRead.movedWith - controlRead.belowStep;+ if (visible < minJudgedFrames) {+ failures.push(+ `the component showed a new position on ${visible} of the control's frames, under ${minJudgedFrames}: too little motion in this window to judge`,+ );+ }+ if (controlRead.movedWith < controlRead.moved) {+ failures.push(+ `moved on ${controlRead.movedWith} of the ${controlRead.moved} frames the control moved (${((controlRead.movedWith / Math.max(1, controlRead.moved)) * 100).toFixed(0)}%)`,+ );+ }+ if (controlRead.componentStallMs > controlRead.longestStallMs) {+ failures.push(+ `longest stall ${controlRead.componentStallMs.toFixed(1)} ms, longer than the control's ${controlRead.longestStallMs.toFixed(1)} ms`,+ );+ }+ } else if (longestStallFrames > maxStallFrames) {failures.push(`longest stall ${longestStallMs.toFixed(1)} ms = ${longestStallFrames} intervals > ${maxStallFrames}`,);}- if (expectedFrames < minJudgedFrames) {+ if (controlRead == null && expectedFrames < minJudgedFrames) {failures.push(`${(judged.to - judged.from).toFixed(0)} ms judged = ${expectedFrames} frames < ${minJudgedFrames}: the animation under test was not moving during the block`,);@@ -534,7 +872,10 @@ export function busyWindowProof(1,Math.floor((1 - minMovedFraction) * judgedFrames + 1e-9),);- if (judgedFrames - moved - recorderSkipped > allowedMisses) {+ if (+ controlRead == null &&+ judgedFrames - moved - recorderSkipped > allowedMisses+ ) {failures.push(`${moved} of ${judgedFrames} expected frames moved${recorderSkipped > 0 ? ` (and ${recorderSkipped} the screencast skipped)` : ''} (${(movedFraction * 100).toFixed(0)}% < ${(minMovedFraction * 100).toFixed(0)}%)`,);@@ -554,6 +895,9 @@ export function busyWindowProof((onScreenFromMs == null? '': `, on screen from +${onScreenFromMs.toFixed(0)} ms (nothing moved before the block)`) ++ (controlRead == null+ ? ''+ : `; against the control: moved on ${controlRead.movedWith} of its ${controlRead.moved} frames${controlRead.belowStep > 0 ? ` (${controlRead.belowStep} under one cast step a frame)` : ''}, its longest stall ${controlRead.longestStallMs.toFixed(1)} ms`) +notes.map(note => `; note: ${note}`).join('');return {...stats,@@ -573,6 +917,109 @@ export function busyWindowProof(longestStallFrames,moved,movedFraction,+ control: controlRead,+ verdict: {pass: failures.length === 0, failures},+ summary,+ };+}++/** One run of a pooled busy proof (see `pooledBusyProof`). */+export interface BusyRun {+ /** The component's frames, tracked from the run's screencast. */+ readonly tracked: readonly TrackedFrame[];+ /** The control's frames (`startBusyControl`), from the same screencast. */+ readonly control: readonly TrackedFrame[];+ /** The run's block (`blockMainThread`). */+ readonly window: BusyWindow;+}++export interface PooledBusyProof {+ /**+ * Each run judged on its own against its control: read its numbers, not its+ * verdict (one short run can hold too few frames to judge).+ */+ readonly runs: readonly BusyProof[];+ /**+ * The control's new positions over every run's judged part, and those the+ * component moved on too.+ */+ readonly control: {readonly moved: number; readonly movedWith: number};+ /** The judgment over the pool (see `pooledBusyProof`). */+ readonly verdict: BusyVerdict;+ /** One line with the pooled numbers, then each run's summary. */+ readonly summary: string;+}++/**+ * A busy proof over several runs of the same short animation, against the+ * control. An animation that lasts a fraction of a second leaves a block only+ * a few frames to judge: in WebKit, or on a loaded machine, the control can+ * present fewer than `minJudgedFrames` in one run, and one run cannot decide.+ * Pooled, the runs are judged as one long window is:+ * - the control moved on at least `minJudgedFrames` frames over all the runs;+ * - the component moved on every one of those frames;+ * - in each run, its longest stall is no longer than that run's control's: a+ * stall belongs to its run and is never pooled.+ * `maxMedianFrameMs`, when given, applies to each run. A main-thread+ * animation still fails: in every run the control moves and it does not.+ */+export function pooledBusyProof(+ runs: readonly BusyRun[],+ options: Omit<+ NonNullable<Parameters<typeof busyWindowProof>[2]>,+ 'control' | 'minMovedFraction'+ > = {},+): PooledBusyProof {+ if (runs.length === 0) {+ throw new Error('pooledBusyProof: no runs to judge');+ }+ const {minJudgedFrames = 4, maxMedianFrameMs = Infinity} = options;+ const proofs = runs.map(run =>+ busyWindowProof(run.tracked, run.window, {+ ...options,+ control: run.control,+ }),+ );+ const reads = proofs.map(proof => proof.control as BusyControlRead);+ const moved = reads.reduce((sum, read) => sum + read.moved, 0);+ const movedWith = reads.reduce((sum, read) => sum + read.movedWith, 0);+ const failures: string[] = [];+ if (moved < minJudgedFrames) {+ failures.push(+ `the control moved on ${moved} frames over ${runs.length} runs, under ${minJudgedFrames}: the compositor presented too little in these windows to judge`,+ );+ }+ const visible =+ movedWith - reads.reduce((sum, read) => sum + read.belowStep, 0);+ if (visible < minJudgedFrames) {+ failures.push(+ `the component showed a new position on ${visible} of the control's frames over ${runs.length} runs, under ${minJudgedFrames}: too little motion in these windows to judge`,+ );+ }+ if (movedWith < moved) {+ failures.push(+ `moved on ${movedWith} of the ${moved} frames the control moved over ${runs.length} runs (${((movedWith / Math.max(1, moved)) * 100).toFixed(0)}%)`,+ );+ }+ proofs.forEach((proof, i) => {+ if (proof.frames > 0 && proof.medianFrameMs > maxMedianFrameMs) {+ failures.push(+ `run ${i + 1}: median frame interval ${proof.medianFrameMs.toFixed(1)} ms > ${maxMedianFrameMs} ms`,+ );+ }+ const read = reads[i];+ if (read.componentStallMs > read.longestStallMs) {+ failures.push(+ `run ${i + 1}: longest stall ${read.componentStallMs.toFixed(1)} ms, longer than the control's ${read.longestStallMs.toFixed(1)} ms`,+ );+ }+ });+ const summary =+ `${runs.length} runs pooled: moved on ${movedWith} of the control's ${moved} frames` ++ proofs.map((proof, i) => `; run ${i + 1}: ${proof.summary}`).join('');+ return {+ runs: proofs,+ control: {moved, movedWith},verdict: {pass: failures.length === 0, failures},summary,};diff --git a/internal/interaction-harness/src/curves.ts b/internal/interaction-harness/src/curves.tsindex 52d187ea0a..e4dd987203 100644--- a/internal/interaction-harness/src/curves.ts+++ b/internal/interaction-harness/src/curves.ts@@ -981,12 +981,23 @@ export async function waitForAnimation((target === element || element.contains(target)));};+ // The busy-window control (busy.ts) animates on its own, inside a+ // dialog with `within`: it never answers for the component.+ const isControl = (a: Animation) => {+ const target =+ (a.effect as KeyframeEffect | null)?.target ?? null;+ return (+ target instanceof Element &&+ target.closest('[data-harness-control]') != null+ );+ };const after = watchedFrom ?? recorder.armedAt();// Document-timeline animations only, as for holdAnimations: a scroll-// or view-timeline one that reached its end is not a finished curve.const recorded = recorder.log.filter(e =>e.seenAt >= after &&+ !isControl(e.animation) &&(inside(e.animation) || e.ancestors.includes(element)) &&e.animation.timeline instanceof DocumentTimeline,);@@ -1004,6 +1015,7 @@ export async function waitForAnimation(.map(e => e.animation)).filter(a =>+ !isControl(a) &&a.playState === 'running' &&a.timeline instanceof DocumentTimeline,);diff --git a/internal/interaction-harness/src/harness.interaction.browser.spec.ts b/internal/interaction-harness/src/harness.interaction.browser.spec.tsindex 1004c6b6bf..2c5bd9f7ad 100644--- a/internal/interaction-harness/src/harness.interaction.browser.spec.ts+++ b/internal/interaction-harness/src/harness.interaction.browser.spec.ts@@ -35,6 +35,7 @@*/import {expect, test, type Page} from '@playwright/test';+import type {CastFrame, TrackedFrame} from './frames';import {animationStoryboard,blockMainThread,@@ -42,6 +43,7 @@ import {compareCurve,countBetween,cubicBezier,+ CONTROL_RGB,curveOf,drag,flick,@@ -56,6 +58,7 @@ import {parseTransform,PARTIAL_LIFT_SKIP,pinch,+ pooledBusyProof,readInputLog,readLongTasks,releaseAnimations,@@ -65,6 +68,7 @@ import {springStep,startCompositorTrace,startScreencast,+ startBusyControl,Storyboard,tap,thawTimers,@@ -107,6 +111,73 @@ const BUSY_FIXTURE = `<!doctype html><meta name=viewport content="width=device-w</style><div id=box></div>`;+/**+ * The red channel at a CSS-px point in each cast frame (a 16 px square's mean),+ * as a tracked value on x: what a backdrop's dim shows there. Decoded in a+ * scratch page, as `trackMarker` does.+ */+async function brightnessAt(+ page: Page,+ castFrames: readonly CastFrame[],+ {x, y}: {x: number; y: number},+): Promise<TrackedFrame[]> {+ const scratch = await page.context().newPage();+ try {+ const values: number[] = [];+ for (let i = 0; i < castFrames.length; i += 12) {+ const batch = castFrames.slice(i, i + 12);+ values.push(+ ...(await scratch.evaluate(+ async ({items, px, py}) => {+ const out: number[] = [];+ for (const item of items) {+ const blob = await (+ await fetch(`data:image/${item.format};base64,${item.data}`)+ ).blob();+ const bitmap = await createImageBitmap(blob);+ const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);+ const context = canvas.getContext('2d');+ if (context == null) {+ throw new Error('no 2d context');+ }+ context.drawImage(bitmap, 0, 0);+ const scale = bitmap.width / item.viewportWidth;+ const {data} = context.getImageData(+ Math.round(px * scale),+ Math.round(py * scale),+ 16,+ 16,+ );+ let sum = 0;+ for (let o = 0; o < data.length; o += 4) {+ sum += data[o];+ }+ out.push(sum / (data.length / 4));+ }+ return out;+ },+ {+ items: batch.map(f => ({+ data: f.data,+ format: f.format,+ viewportWidth: f.viewportWidth,+ })),+ px: x,+ py: y,+ },+ )),+ );+ }+ return castFrames.map((f, i) => ({+ t: f.t,+ box: {x: values[i], y: 0, width: 1, height: 1},+ pixels: 1,+ }));+ } finally {+ await scratch.close();+ }+}+async function busyFixture(page: Page) {await page.route('http://busy.test/', route =>route.fulfill({contentType: 'text/html', body: BUSY_FIXTURE}),@@ -292,8 +363,19 @@ test.describe('frame evidence', () => {await busyFixture(page);const box = page.locator('#box');await markElement(box, {mode: 'outline'});-- const run = async (driver: 'compositor' | 'raf', everyNthFrame = 1) => {+ // A loaded runner's compositor skips frames the display-rate bound+ // counts: the component is judged against what the compositor presented+ // in the same window, and the bound's own verdict is recorded beside it.+ const control = await startBusyControl(page);++ // The README's order: the cast, then the animation, then the block once+ // it runs (on the compositor, for the compositor's run).+ const run = async (+ driver: 'compositor' | 'raf',+ everyNthFrame = 1,+ blockMs = 700,+ ) => {+ const cast = await startScreencast(page, {everyNthFrame});await box.evaluate((el, how) => {el.getAnimations().forEach(a => a.cancel());el.style.transform = 'none';@@ -314,48 +396,73 @@ test.describe('frame evidence', () => {requestAnimationFrame(step);}}, driver);- const cast = await startScreencast(page, {everyNthFrame});- await page.waitForTimeout(200);- const window = await blockMainThread(page, 700);+ // A rendering update first: until one commits the cancel, the+ // compositor keeps playing the previous run's animation into the block.+ await page.evaluate(+ () =>+ new Promise<void>(resolve =>+ requestAnimationFrame(() => requestAnimationFrame(() => resolve())),+ ),+ );+ const window = await blockMainThread(+ page,+ blockMs,+ driver === 'compositor' ? {during: box} : {},+ );await page.waitForTimeout(200);- const tracked = await trackMarker(page, await cast.stop());- return busyWindowProof(tracked, window, {axis: 'y'});+ const frames = await cast.stop();+ const tracked = await trackMarker(page, frames);+ const controlFrames = await trackMarker(page, frames, {rgb: CONTROL_RGB});+ const alone = busyWindowProof(tracked, window, {axis: 'y'});+ return {+ proof: busyWindowProof(tracked, window, {+ axis: 'y',+ control: controlFrames,+ }),+ alone,+ };};- const compositor = await run('compositor');+ const {proof: compositor, alone: compositorAlone} = await run('compositor');testInfo.annotations.push({type: 'busy-main-thread compositor',- description: compositor.summary,+ description: `${compositor.summary} | without the control: ${compositorAlone.verdict.failures.join('; ') || 'pass'}`,});expect(compositor.verdict.failures, compositor.summary).toEqual([]);expect(compositor.monotonic).toBe(true);// Resolvable on the cast's grid: two position steps or more per frame.- expect(compositor.notes, compositor.summary).toEqual([]);expect(compositor.pxPerFrame).toBeGreaterThanOrEqual(2 * compositor.stepPx);- const raf = await run('raf');+ const {proof: raf, alone: rafAlone} = await run('raf');testInfo.annotations.push({type: 'busy-main-thread rAF control',description: raf.summary,});- // Frozen for the whole block: no presented frame moved (the compositor- // sends none at all when nothing on screen changes), and the verdict that- // passes the compositor animation fails it on every criterion.- expect(raf.changed).toBe(0);+ // Frozen for the block: the compositor sends no frame while nothing on+ // screen changes, so at most the frame presented as the block starts+ // moved, and the bound's verdict fails it as the control's does.+ expect(raf.changed).toBeLessThanOrEqual(1);expect(raf.longestStallMs).toBeGreaterThan(600);- expect(raf.verdict.pass).toBe(false);- expect(raf.verdict.failures.join('\n')).toMatch(/longest stall/);- expect(raf.verdict.failures.join('\n')).toMatch(/expected frames moved/);+ expect(raf.verdict.failures.join('\n')).toMatch(+ /moved on [01] of the \d+ frames the control moved/,+ );+ expect(raf.verdict.failures.join('\n')).toMatch(+ /longest stall .* longer than the control's/,+ );+ expect(rafAlone.verdict.pass).toBe(false);+ expect(rafAlone.verdict.failures.join('\n')).toMatch(/longest stall/);// The verdict judges the frames the screencast delivered, not its rate:// at one frame in six (a loaded machine's ~100 ms), the compositor- // animation still passes and the rAF one still fails.- const slow = await run('compositor', 6);+ // animation still passes and the rAF one still fails. A 1.4 s block keeps+ // the control's four frames on a runner that presents half the rate.+ const {proof: slow} = await run('compositor', 6, 1400);expect(slow.cadenceMs, slow.summary).toBeGreaterThan(60);expect(slow.verdict.failures, slow.summary).toEqual([]);- const slowRaf = await run('raf', 6);+ const {proof: slowRaf} = await run('raf', 6, 1400);expect(slowRaf.verdict.pass, slowRaf.summary).toBe(false);expect(slowRaf.verdict.failures.join('\n')).toMatch(/longest stall/);+ await control.remove();});test('frames the screencast skips while the compositor presents them are not a stall; a frozen animation still fails with the compositor record', async ({@@ -450,6 +557,9 @@ test.describe('frame evidence', () => {await busyFixture(page);const box = page.locator('#box');await markElement(box, {mode: 'outline'});+ // Judged against what the compositor presented in the same window: a+ // loaded runner's compositor skips frames a display-rate bound counts.+ const control = await startBusyControl(page);const cast = await startScreencast(page);await expect.poll(() => cast.frameCount()).toBeGreaterThan(0);await box.evaluate(el => {@@ -462,8 +572,13 @@ test.describe('frame evidence', () => {minProgressMs: 16,});await waitForRest(box);- const tracked = await trackMarker(page, await cast.stop());- const proof = busyWindowProof(tracked, window, {axis: 'y'});+ const frames = await cast.stop();+ await control.remove();+ const tracked = await trackMarker(page, frames);+ const proof = busyWindowProof(tracked, window, {+ axis: 'y',+ control: await trackMarker(page, frames, {rgb: CONTROL_RGB}),+ });testInfo.annotations.push({type: 'busy-main-thread transition',description: proof.summary,@@ -472,6 +587,316 @@ test.describe('frame evidence', () => {expect(proof.monotonic).toBe(true);});+ test('against a compositor-only control in the same window, a compositor animation passes and a rAF one fails, whatever the screencast rate', async ({+ page,+ }, testInfo) => {+ await busyFixture(page);+ const box = page.locator('#box');+ await markElement(box, {mode: 'outline'});+ const control = await startBusyControl(page);++ const run = async (+ driver: 'compositor' | 'raf',+ everyNthFrame = 1,+ blockMs = 700,+ ) => {+ // Endless, so the animation still runs when the block starts however+ // long a loaded runner takes to start the cast, and wrapping from 560 px+ // back to 0 rather than turning, so every frame changes: two frames+ // straddling a turnaround evenly would read one position.+ await box.evaluate((el, how) => {+ el.getAnimations().forEach(a => a.cancel());+ el.style.transform = 'none';+ if (how === 'compositor') {+ el.animate(+ [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}],+ {duration: 1600, easing: 'linear', iterations: Infinity},+ );+ } else {+ const start = performance.now();+ const step = (now: number) => {+ const p = ((now - start) / 1600) % 1;+ el.style.transform = `translateY(${560 * p}px)`;+ requestAnimationFrame(step);+ };+ requestAnimationFrame(step);+ }+ }, driver);+ const cast = await startScreencast(page, {everyNthFrame});+ await page.waitForTimeout(200);+ // The compositor's run is blocked only once it is on the compositor.+ const window = await blockMainThread(+ page,+ blockMs,+ driver === 'compositor' ? {during: box} : {},+ );+ await page.waitForTimeout(200);+ const frames = await cast.stop();+ const tracked = await trackMarker(page, frames);+ const controlFrames = await trackMarker(page, frames, {rgb: CONTROL_RGB});+ const proof = busyWindowProof(tracked, window, {+ axis: 'y',+ control: controlFrames,+ });+ // The judgment without the control, for the record only: on a+ // two-core runner it can fail where the control shows the compositor+ // itself presented no more.+ const alone = busyWindowProof(tracked, window, {axis: 'y'});+ testInfo.annotations.push({+ type: `busy-main-thread ${driver} 1/${everyNthFrame} against the control`,+ description: `${proof.summary} | without the control: ${alone.verdict.failures.join('; ') || 'pass'}`,+ });+ return proof;+ };++ const compositor = await run('compositor');+ expect(compositor.verdict.failures, compositor.summary).toEqual([]);+ expect(compositor.control?.moved).toBeGreaterThanOrEqual(4);++ // The mutant: a main-thread animation freezes while the control moves. A+ // frame presented as the block starts can still show its last step, so+ // it can read as moved on one of the control's frames; never on the rest.+ const raf = await run('raf');+ expect(raf.verdict.pass, raf.summary).toBe(false);+ const rafFailures = raf.verdict.failures.join('\n');+ expect(rafFailures).toMatch(+ /moved on [01] of the \d+ frames the control moved/,+ );+ expect(rafFailures).toMatch(/longest stall .* longer than the control's/);++ // A slower screencast samples the control and the component alike; a+ // 1.4 s block keeps the control's four frames on a loaded runner.+ const slow = await run('compositor', 6, 1400);+ expect(slow.verdict.failures, slow.summary).toEqual([]);+ const slowRaf = await run('raf', 6, 1400);+ expect(slowRaf.verdict.pass, slowRaf.summary).toBe(false);+ await control.remove();+ });++ test("mounted within an open modal dialog, the control leaves the dialog's backdrop animation on the compositor", async ({+ page,+ }) => {+ // A dimming backdrop over a black page: its brightness at a point is the+ // value judged, read from the same frames as the control.+ await page.route('http://backdrop.test/', route =>+ route.fulfill({+ contentType: 'text/html',+ body: `<!doctype html><meta name="viewport" content="width=device-width,initial-scale=1"><style>body{margin:0;background:#000;height:100vh}dialog{width:120px;height:80px;padding:0;border:0;background:#fff}dialog::backdrop{background:rgb(255,255,0);opacity:.2}</style><dialog id="sheet"></dialog>`,+ }),+ );+ await page.goto('http://backdrop.test/');+ const dialog = page.locator('#sheet');+ // The dim runs from the dialog's open, endlessly and wrapping rather than+ // turning, so it is on screen and changing on every frame by the block.+ await dialog.evaluate(el => {+ (el as HTMLDialogElement).showModal();+ el.animate([{opacity: 0.2}, {opacity: 1}], {+ duration: 900,+ easing: 'linear',+ iterations: Infinity,+ pseudoElement: '::backdrop',+ });+ });+ const control = await startBusyControl(page, {within: dialog});+ const cast = await startScreencast(page);+ await page.waitForTimeout(200);+ const window = await blockMainThread(page, 600, {during: dialog});+ await page.waitForTimeout(200);+ const castFrames = await cast.stop();+ await control.remove();+ const controlFrames = await trackMarker(page, castFrames, {+ rgb: CONTROL_RGB,+ });+ const dim = await brightnessAt(page, castFrames, {x: 40, y: 600});+ const proof = busyWindowProof(dim, window, {+ axis: 'x',+ control: controlFrames,+ });+ expect(proof.verdict.failures, proof.summary).toEqual([]);+ });++ test("waitForAnimation and blockMainThread's during never count the control's own animation inside a dialog", async ({+ page,+ }) => {+ await page.setContent(+ '<dialog id="d" style="width:200px;height:200px">still</dialog>',+ );+ await page.evaluate(() =>+ (document.getElementById('d') as HTMLDialogElement).showModal(),+ );+ const dialog = page.locator('#d');+ const control = await startBusyControl(page, {within: dialog});+ // The dialog animates nothing of its own: both gates keep waiting.+ await expect(+ blockMainThread(page, 100, {during: dialog, timeout: 600}),+ ).rejects.toThrow(/no animation started/);+ await expect(waitForAnimation(dialog, {timeout: 600})).rejects.toThrow();+ // Its own animation answers both.+ await dialog.evaluate(el =>+ el.animate([{opacity: 1}, {opacity: 0.5}], {duration: 2000}),+ );+ await waitForAnimation(dialog);+ await blockMainThread(page, 100, {during: dialog});+ await control.remove();+ });++ test('a hold that starts before the block is judged from its first presented change inside it; a single change and then a freeze still fails', async ({+ page,+ }) => {+ await busyFixture(page);+ const box = page.locator('#box');+ await markElement(box, {mode: 'outline'});+ const control = await startBusyControl(page);+ // On the compositor: a move, a 300 ms hold (as WebKit holds a sheet's+ // just-started release on screen), then a move; or one jump inside the+ // block and nothing after it.+ const run = async (+ keyframes: Keyframe[],+ options: KeyframeAnimationOptions,+ ) => {+ const cast = await startScreencast(page);+ await page.waitForTimeout(150);+ await box.evaluate(+ (el, [frames, timing]) => {+ el.getAnimations().forEach(a => a.cancel());+ el.animate(frames, timing);+ },+ [keyframes, options] as const,+ );+ // The block starts inside the hold, which began before it.+ const window = await blockMainThread(page, 600, {+ during: box,+ minProgressMs: 700,+ });+ await page.waitForTimeout(150);+ const frames = await cast.stop();+ return busyWindowProof(await trackMarker(page, frames), window, {+ axis: 'y',+ control: await trackMarker(page, frames, {rgb: CONTROL_RGB}),+ });+ };+ const held = await run(+ [+ {transform: 'translateY(0px)', offset: 0},+ {transform: 'translateY(100px)', offset: 0.3},+ {transform: 'translateY(100px)', offset: 0.45},+ {transform: 'translateY(560px)', offset: 1},+ ],+ {duration: 2000, easing: 'linear'},+ );+ expect(held.verdict.failures, held.summary).toEqual([]);+ const jumpThenFreeze = await run(+ [{transform: 'translateY(0px)'}, {transform: 'translateY(50px)'}],+ {duration: 800, easing: 'steps(1, end)', fill: 'forwards'},+ );+ expect(jumpThenFreeze.verdict.pass, jumpThenFreeze.summary).toBe(false);+ await control.remove();+ });++ test('a block too short to judge alone is judged over pooled runs: a compositor animation passes, and a rAF one fails, timed or frame-counted', async ({+ page,+ }, testInfo) => {+ await busyFixture(page);+ const box = page.locator('#box');+ await markElement(box, {mode: 'outline'});+ const control = await startBusyControl(page);++ // Six 70 ms blocks: each judged part (20 ms margins, 30 ms) holds at most+ // three frames even at WebKit's 14 ms cast, under the four one window+ // needs, as a short fade's does.+ const runs = async (+ driver: 'compositor' | 'raf' | 'counted 1' | 'counted 2',+ ) => {+ await box.evaluate((el, how) => {+ el.getAnimations().forEach(a => a.cancel());+ el.style.transform = 'none';+ if (how === 'compositor') {+ // Endless, wrapping from 560 px back to 0 rather than turning, so+ // every frame changes: two frames straddling a turnaround evenly+ // would read one position.+ el.animate(+ [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}],+ {duration: 1600, easing: 'linear', iterations: Infinity},+ );+ } else if (how === 'raf') {+ const start = performance.now();+ // The same endless run on the main thread.+ const step = (now: number) => {+ const p = ((now - start) / 1600) % 1;+ el.style.transform = `translateY(${560 * p}px)`;+ requestAnimationFrame(step);+ };+ requestAnimationFrame(step);+ } else {+ // A fixed step a frame, counted rather than timed: after a block it+ // goes on from where it stood instead of catching up.+ const px = how === 'counted 1' ? 1 : 2;+ let y = 0;+ let direction = 1;+ const step = () => {+ if (y + direction * px > 560 || y + direction * px < 0) {+ direction = -direction;+ }+ y += direction * px;+ el.style.transform = `translateY(${y}px)`;+ requestAnimationFrame(step);+ };+ requestAnimationFrame(step);+ }+ }, driver);+ const cast = await startScreencast(page);+ await page.waitForTimeout(200);+ const windows = [];+ for (let i = 0; i < 6; i += 1) {+ windows.push(await blockMainThread(page, 70));+ await page.waitForTimeout(150);+ }+ const frames = await cast.stop();+ const tracked = await trackMarker(page, frames);+ const controlFrames = await trackMarker(page, frames, {rgb: CONTROL_RGB});+ return windows.map(window => ({tracked, control: controlFrames, window}));+ };++ const compositor = await runs('compositor');+ const single = busyWindowProof(+ compositor[0].tracked,+ compositor[0].window,+ {axis: 'y', control: compositor[0].control},+ );+ expect(single.verdict.failures.join('\n'), single.summary).toMatch(+ /too little in this window to judge/,+ );+ const pooled = pooledBusyProof(compositor, {axis: 'y'});+ testInfo.annotations.push({+ type: 'busy-pooled compositor',+ description: pooled.summary,+ });+ expect(pooled.verdict.failures, pooled.summary).toEqual([]);++ // The mutant: a main-thread animation freezes in every block.+ const raf = pooledBusyProof(await runs('raf'), {axis: 'y'});+ testInfo.annotations.push({+ type: 'busy-pooled raf',+ description: raf.summary,+ });+ expect(raf.verdict.pass, raf.summary).toBe(false);+ expect(raf.verdict.failures.join('\n')).toMatch(+ /moved on \d+ of the \d+ frames the control moved over 6 runs/,+ );++ // A frame-counted loop stalls in every block too and then steps on, one+ // cast step a frame at 1 px: its stall is longer than its runs before it.+ for (const counted of ['counted 1', 'counted 2'] as const) {+ const proof = pooledBusyProof(await runs(counted), {axis: 'y'});+ testInfo.annotations.push({+ type: `busy-pooled ${counted}`,+ description: proof.summary,+ });+ expect(proof.verdict.pass, `${counted}: ${proof.summary}`).toBe(false);+ }+ await control.remove();+ });+test('a block never starts on an animation that is not on the compositor yet', async ({page,}, testInfo) => {@@ -484,6 +909,9 @@ test.describe('frame evidence', () => {await box.evaluate(el => {el.style.willChange = 'transform';});+ // Judged against what the compositor presented in the same window: a+ // loaded runner's compositor skips frames a display-rate bound counts.+ const control = await startBusyControl(page);const cast = await startScreencast(page);// A rAF loop registered before the helper's tick runs ahead of it in every// frame; on its 12th frame it creates the animation, and the helper's tick@@ -508,8 +936,13 @@ test.describe('frame evidence', () => {minProgressMs: 0,});await waitForRest(box);- const tracked = await trackMarker(page, await cast.stop());- const proof = busyWindowProof(tracked, window, {axis: 'y'});+ const frames = await cast.stop();+ await control.remove();+ const tracked = await trackMarker(page, frames);+ const proof = busyWindowProof(tracked, window, {+ axis: 'y',+ control: await trackMarker(page, frames, {rgb: CONTROL_RGB}),+ });testInfo.annotations.push({type: 'busy-main-thread pending animation',description: proof.summary,