An open source design system that's fully customizable and agent ready
TypeScript
test(interaction-harness): a busy proof stops where its curve falls back (#10766)
Christopher Chedeau3e27971f3a1a
diff --git a/internal/interaction-harness/README.md b/internal/interaction-harness/README.mdindex 987c5e5fae..58af5df52d 100644--- a/internal/interaction-harness/README.md+++ b/internal/interaction-harness/README.md@@ -202,7 +202,14 @@ A judged part shorter than four of thescreencast's intervals fails: the animation under test was not moving duringthe block. One that had already settled when the judged part began (nothing inthe window to judge) throws `BusyWindowTooLateError`, with the numbers: the-block ran too late, which is no reading of the component. With+block ran too late, which is no reading of the component. The held animation+can also end inside the block. When it keeps no end value (`holds` false: no+held tail, and no fill the compositor honors; WebKit on Linux honors none)+and no settle bound applies, the judged part ends at its curve's end. Past it,+WebKit presents the value the main thread last committed, not the animation.+If that leaves the control fewer than four frames, where the uncut part held+enough, the block also began too late: `BusyWindowTooLateError`, with+`curveEndMs`. A starved run still throws `FrameStarvationError`. With`during`, `blockMainThread` starts the loop from a task queued in the framethat saw the animation, after that frame's rendering update: WebKit hands anewly started animation to its compositor there, and a loop inside the rAF@@ -258,7 +265,8 @@ A just-started animation can reach the screen after the block began (50–100 mslater in WebKit on a loaded machine). When the cast saw the element at restbefore the block (wait for `cast.frameCount()` to pass 0 before starting theanimation), judging starts at its first presented change: the judged part still-ends with the block and still needs four intervals, and `onScreenFromMs` and the+ends with the block (or the held curve, above) and still needs four intervals,+and `onScreenFromMs` and thesummary report when it came. So is an element that moved, then held frombefore the block until its first presented change inside it: WebKit presentsa just-started compositor animation late (a sheet's release held on screen for@@ -359,7 +367,7 @@ its longest stall may be no longer than that run's control's, since a stallbelongs to its run. A run whose control moved on fewer frames than`minRunFrames` (1 by default) is too little to judge: the summary names it,and the pool leaves it out, so it never counts as a pass. A run whose block-began after its motion had settled is left out the same way: its `runs` entry+began too late for its motion is left out the same way: its `runs` entryis null, and `late` holds its `BusyWindowTooLateError`. The pool reads eachrun without the starvation throw. The pool fails withfewer than `minJudgedRuns` judged runs (half of all the runs, rounded up). In each judged run,diff --git a/internal/interaction-harness/src/busy.control.test.ts b/internal/interaction-harness/src/busy.control.test.tsindex 84f1daef31..e36ca8ebc5 100644--- a/internal/interaction-harness/src/busy.control.test.ts+++ b/internal/interaction-harness/src/busy.control.test.ts@@ -10,7 +10,9 @@* moved; a freeze, whose catch-up is more than one step, and a stall* longer than the component's runs before it do not. Runs* of a block too short to judge alone are judged pooled: the control over- * all of them, each run's stall against that run's control.+ * all of them, each run's stall against that run's control. A held+ * animation is judged up to its curve's end. A stall before that end still+ * fails, and a curve that ends too soon is named.* @position Unit test of ./busy's control judgment; the frames are* busy.test.ts's fixtures, and the browser proof with a real compositor* control is in harness.interaction.browser.spec.ts.@@ -321,8 +323,120 @@ describe('busyWindowProof against a control', () => {/the control moved on 0 frames in the judged part, under 4: the compositor presented too little in this window to judge \(the cast delivered \d+ frames in the \d+ ms block, none for 1[67] ms\)/,);});++ it("judges a held animation up to its curve's end: after it, the frames show the compositor's fallback to the committed value", () => {+ // The curve ends 520 ms into the block. WebKit then presents the value+ // the main thread last committed, the box at rest, until the block ends.+ const curveEnd = 1520;+ const fallback = moving().map(f =>+ f.t > curveEnd ? ({...f, box: {...f.box!, y: 0}} as TrackedFrame) : f,+ );+ const proof = busyWindowProof(fallback, held(curveEnd), {+ control: asControl(moving()),+ });+ expect(proof.verdict.failures, proof.summary).toEqual([]);+ expect(proof.monotonic).toBe(true);+ expect(proof.judged.to).toBe(curveEnd);+ expect(proof.summary).toMatch(/judged to the held curve's end at \+520 ms/);+ });++ it('still names a starved run as starvation when its held curve also ends inside the block', () => {+ // Three frames in the whole block: too few even before the curve's end.+ const run = () =>+ busyWindowProof(starvedCast(), held(1600), {+ control: asControl(starvedCast()),+ });+ expect(run).toThrow(FrameStarvationError);+ });++ it("keeps a settle bound's end: the held curve's end does not cut a proof that bounds its settle", () => {+ // Arrives at 1530, just after its curve's end at 1520.+ const arrives = moving().map(f =>+ f.t > 1530+ ? ({...f, box: {...f.box!, y: 3 * at(1530)}} as TrackedFrame)+ : f,+ );+ const proof = busyWindowProof(arrives, held(1520), {+ control: asControl(moving()),+ settledWithin: 2,+ });+ expect(proof.judged.to).toBeGreaterThan(1520);+ expect(proof.summary).not.toMatch(/held curve's end/);+ });++ it('judges the whole block when the held animation keeps its end value: a stop at its curve end still fails', () => {+ // A fill that holds the end, or a held tail: the frames after the curve+ // still show the animation, at rest.+ const curveEnd = 1520;+ const stops = moving().map(f =>+ f.t > curveEnd+ ? ({...f, box: {...f.box!, y: 3 * at(curveEnd)}} as TrackedFrame)+ : f,+ );+ const proof = busyWindowProof(stops, held(curveEnd, true), {+ control: asControl(moving()),+ });+ expect(proof.judged.to).toBe(BLOCK.to - 20);+ expect(proof.verdict.failures.join('\n')).toMatch(+ /longest stall [\d.]+ ms, longer than the control's/,+ );+ });++ it('still fails a held animation that stops advancing before its curve ends', () => {+ // Frozen from 300 ms into the block, while its curve runs to 520 ms.+ const frozen = moving().map(f =>+ f.t > 1300+ ? ({...f, box: {...f.box!, y: 3 * (at(1300) - 1)}} as TrackedFrame)+ : f,+ );+ const proof = busyWindowProof(frozen, held(1520), {+ control: asControl(moving()),+ });+ const failures = proof.verdict.failures.join('\n');+ expect(failures).toMatch(+ /moved on \d+ of the \d+ frames the control moved/,+ );+ expect(failures).toMatch(+ /longest stall [\d.]+ ms, longer than the control's/,+ );+ });++ it('throws BusyWindowTooLateError when the held curve ends too soon after the block began for the control to move on enough frames', () => {+ const run = (curveEnd: number) => () =>+ busyWindowProof(moving(), held(curveEnd), {+ control: asControl(moving()),+ });+ expect(run(1060)).toThrow(BusyWindowTooLateError);+ expect(run(1060)).toThrow(+ /busyWindowProof: the held animation's curve ended 60 ms after the block began, so the part judged from 20 ms held 2 of the frames it needs, 4 \(\d+ frames presented in the block\): the block started late for the motion\./,+ );+ // A curve that ended before the judged part began leaves nothing.+ expect(run(1010)).toThrow(+ /curve ended 10 ms after the block began, so the part judged from 20 ms held 0 of the frames it needs/,+ );+ let caught: unknown = null;+ try {+ run(1060)();+ } catch (error) {+ caught = error;+ }+ expect(caught).toBeInstanceOf(BusyWindowTooLateError);+ expect((caught as BusyWindowTooLateError).numbers).toMatchObject({+ curveEndMs: 60,+ judgedFromMs: 20,+ });+ });});+/**+ * `BLOCK` as `blockMainThread` holds it with `during`: the held animation+ * started before the block and its curve ends at `curveEnd`. It keeps its end+ * value after that only with `holds`.+ */+function held(curveEnd: number, holds = false) {+ return {...BLOCK, spanMs: 700, animation: {from: 800, to: curveEnd, holds}};+}+/*** `moving()` with three frames in `BLOCK`, about 200 ms apart, as a cast the* machine starved delivers.@@ -404,7 +518,7 @@ describe('pooledBusyProof', () => {);expect(pooled.verdict.pass).toBe(false);expect(pooled.verdict.failures).toEqual([- '1 of 3 runs judged, under 2: 2 began after the motion had settled, and the compositor presented too little in any other to judge',+ '1 of 3 runs judged, under 2: 2 began too late for their motion, and the compositor presented too little in any other to judge',]);expect(pooled.summary).toMatch(/run 1: too late to judge/);expect(pooled.summary).toMatch(/run 2: too late to judge/);diff --git a/internal/interaction-harness/src/busy.ts b/internal/interaction-harness/src/busy.tsindex 79e8f89612..196fdad585 100644--- a/internal/interaction-harness/src/busy.ts+++ b/internal/interaction-harness/src/busy.ts@@ -3,14 +3,14 @@/*** @file busy.ts* @input Uses a Playwright `Page`- * @output `blockMainThread` — a synchronous busy loop on the page's main thread- * that reports exactly when it ran —, `startBusyControl` — a plain- * compositor-only control animation recorded in the same window —,+ * @output `blockMainThread`, a synchronous busy loop on the page's main+ * thread that reports exactly when it ran. `startBusyControl`, a plain+ * compositor-only control animation recorded in the same window.* `busyWindowProof`, which reads frame evidence for that window, against- * the control when it has one, `castStep`, the screencast's position step- * read from its frames, `pooledBusyProof`, which judges several short- * blocks as one, and `BusyWindowTooLateError`, for a block that began- * after the motion it was meant to catch had ended.+ * the control when it has one. `castStep`, the screencast's position step+ * read from its frames. `pooledBusyProof`, which judges several short+ * blocks as one. `BusyWindowTooLateError`, for a block that began too late+ * for the motion it was meant to catch.* `BUSY_PROOF_LOOKAROUND_MS` is how far around its judged part a proof* reads frames.* `trackControl` and `controlClock` read the control's box and its own@@ -383,9 +383,16 @@ export interface BlockedWindow extends BusyWindow {* With `during`: the animation the block caught, epoch ms on the page's* clock. `from` is its start, and `to` is where its curve ends, its held* tail excluded. It is absent when the animation's `playbackRate` is 0 or- * less, since its curve then has no forward end.+ * less, since its curve then has no forward end. `holds` says whether the+ * element keeps the curve's end value after `to`. A held tail keeps it, and+ * so does a fill the engine's compositor honors (not WebKit on Linux). When+ * it does not, `busyWindowProof` judges the block up to `to`.*/- readonly animation?: {readonly from: number; readonly to: number};+ readonly animation?: {+ readonly from: number;+ readonly to: number;+ readonly holds: boolean;+ };}/**@@ -438,19 +445,22 @@ export interface BlockedWindow extends BusyWindow {* that rejects with `BusyWindowTooLateError`, with the numbers.** The window it resolves with names the animation it caught (`animation`):- * when it started and where its curve ends. These times are on the real- * clock, as the window's own are, after a page freeze too.+ * when it started and where its curve ends. `holds` says whether the element+ * keeps the curve's end value after it. These times are on the real clock,+ * as the window's own are, after a page freeze too.*//** One display frame, ms: the least progress `blockMainThread` waits for. */const ONE_FRAME_MS = 16;/**- * The block began after the motion it was meant to catch had ended, so the- * window holds nothing to judge: when the block ran, not how the component+ * The block began too late for the motion it was meant to catch. So the+ * window holds too little to judge: when the block ran, not how the component* moved. Thrown by `blockMainThread` (with `during`) when the animation under- * test is already in its curve's held tail where the page first looks, and by- * `busyWindowProof` when the tracked value had settled before the judged part- * of the window began. Always a red, with the numbers; never a pass. Start the+ * test is already in its curve's held tail where the page first looks. Thrown+ * by `busyWindowProof` when the tracked value had settled before the judged+ * part of the window began. Thrown too when a held curve that keeps no end+ * value ends so soon in the block that too few frames are left+ * (`curveEndMs`). Always a red, with the numbers; never a pass. Start the* block earlier: call `blockMainThread` before the action that starts the* motion, and await it after.*/@@ -529,8 +539,16 @@ export async function blockMainThread(const defined = page.evaluate(`(${defineCurveEnd.toString()})();(${defineAnimatedProperties.toString()})();`,);+ // WebKit on Linux ignores `fill` on its compositor: a filled move that+ // ends during a main-thread block snaps back (motion/animator.ts,+ // END_HOLD_MS). Elsewhere a fill holds the end value. Node's platform is+ // the browser's: no config connects to a remote browser.+ const fillHolds = !(+ page.context().browser()?.browserType().name() === 'webkit' &&+ process.platform === 'linux'+ );const waited = during.evaluate(- (element, {duration, minProgress, prop, pseudo, wait}) =>+ (element, {duration, minProgress, prop, pseudo, wait, fillHolds}) =>new Promise<BlockedWindow>((resolve, reject) => {// The harness's own waits run on the real timers: a frozen timer gate// (story.ts) holds only the page's.@@ -641,9 +659,23 @@ export async function blockMainThread((window as unknown as {__harnessFreeze?: {offset: number}}).__harnessFreeze?.offset ?? 0;const from = performance.timeOrigin + Number(startTime) + frozen;+ // Whether the element keeps the curve's end value once it ends. A+ // held tail past the curve keeps it, and so does a fill the+ // compositor honors. An end delay shows nothing.+ const timing = caught.effect?.getComputedTiming();+ const keyframesEnd =+ Number(timing?.delay ?? 0) + Number(timing?.activeDuration ?? 0);+ const holds =+ curveEnd(caught) < keyframesEnd ||+ (fillHolds &&+ (timing?.fill === 'forwards' || timing?.fill === 'both'));const animation =startTime != null && caught.playbackRate > 0- ? {from, to: from + curveEnd(caught) / caught.playbackRate}+ ? {+ from,+ to: from + curveEnd(caught) / caught.playbackRate,+ holds,+ }: undefined;// The same loop as above, from a task queued in the frame that saw// the animation: it runs once that frame's rendering update has@@ -690,6 +722,7 @@ export async function blockMainThread(prop: property ?? null,pseudo: pseudoElement ?? null,wait: timeout,+ fillHolds,},);try {@@ -904,6 +937,18 @@ function motionStepPx(values: readonly number[]): number {* the block, or the block caught another animation), so there is nothing to* judge, and that fails rather than passes.*+ * With `window.animation` (`blockMainThread`'s `during`), the judged part+ * can end at the held curve's end, when that comes before the block's. It+ * does unless the element keeps the end value (`holds`), or a settle bound+ * (`settledWithin`, `settledWithinSteps`) already ends judging. Once a+ * compositor animation that keeps no end value ends inside a block, WebKit+ * presents the value the main thread last committed, not the animation. If+ * the part left gives the control fewer than `minJudgedFrames` frames, the+ * block started late for the motion. Without a control, the part's span+ * counts. That throws `BusyWindowTooLateError`, as a motion that settled+ * before the block does. A stall before the curve's end is still judged, and+ * fails.+ ** A just-started animation can reach the screen after the block began (its* hand-off to the compositor lags under load: 50–100 ms in WebKit). The* tracked value may be seen at rest before the judged part: a frame there, and@@ -922,10 +967,10 @@ function motionStepPx(values: readonly number[]): number {* position inside the part, as a card that enters from an edge. The cast's* frames before the part must show the screen without it. A main-thread* entrance shows no frame during the block, so it still fails.- * The judged part still ends with the block and the `minJudgedFrames` floor- * applies to it, so a start the block itself delays fails; any other- * animation that was already moving is judged from the block's start, and a- * stall there fails as before.+ * The judged part still ends with the block, or with the held curve. The+ * `minJudgedFrames` floor applies to it, so a start the block itself delays+ * fails. Any other animation that was already moving is judged from the+ * block's start, and a stall there fails as before.** `stepPx` and `pxPerFrame` say whether a frame's motion is resolvable at* all: under two position steps per frame, a frame delivered early can show@@ -1169,6 +1214,34 @@ function readBusyWindow();}}+ // A held curve that keeps no end value can end inside the block. WebKit's+ // compositor then presents the value the main thread last committed, not+ // the animation. So the judged part ends at the curve's end, unless a+ // settle bound already ends it where the motion arrives.+ const heldCurve = (window as Partial<BlockedWindow>).animation;+ const curveEnd =+ heldCurve?.holds === false && settledWithinPx == null+ ? heldCurve.to+ : undefined;+ const endsInBlock = curveEnd != null && curveEnd < end;+ const curveTooLate = (framesLeft: number) => {+ const presented = tracked.filter(+ f => f.t >= window.from && f.t <= window.to,+ ).length;+ const curveEndMs = (curveEnd ?? end) - window.from;+ return new BusyWindowTooLateError(+ `busyWindowProof: the held animation's curve ended ${curveEndMs.toFixed(0)} ms after the block began, so the part judged from ${marginMs} ms held ${framesLeft} of the frames it needs, ${minJudgedFrames} (${presented} frames presented in the block): the block started late for the motion. Start the block earlier, or play a longer curve.`,+ {curveEndMs, judgedFromMs: marginMs, presented},+ );+ };+ // Where judging would end without the curve's end.+ const endWithoutCurve = end;+ if (curveEnd != null && endsInBlock) {+ if (curveEnd <= window.from + marginMs) {+ throw curveTooLate(0);+ }+ end = curveEnd;+ }const inner = {from: window.from + marginMs, to: end};const values = tracked.filter(f => f.t >= inner.from && f.t <= inner.to)@@ -1421,24 +1494,32 @@ function readBusyWindow(// The control's own new positions in the judged part, against its frame// before the part, and of those the frames the component moved on too:// both are tracked from the same screencast frames, so their times match.- let controlRead: BusyControlRead | null = null;- if (control != null) {- const baseline = [...control]+ // The times of the control's new positions in the judged part, up to `to`.+ const controlChangesUpTo = (+ frames: readonly TrackedFrame[],+ to: number,+ ): number[] => {+ const baseline = [...frames].reverse().find(f => f.box != null && f.t < inner.from);let last: number | null = baseline?.box == null ? null : baseline.box.x;- const controlChanges: number[] = [];- for (const f of control) {- if (f.box == null || f.t < inner.from || f.t > judged.to) {+ const changes: number[] = [];+ for (const f of frames) {+ if (f.box == null || f.t < inner.from || f.t > to) {continue;}if (last != null && Math.abs(f.box.x - last) > 0.25) {if (f.t > judged.from || onScreenFrom == null) {- controlChanges.push(f.t);+ changes.push(f.t);}}last = f.box.x;}+ return changes;+ };+ let controlRead: BusyControlRead | null = null;+ if (control != null) {+ const controlChanges = controlChangesUpTo(control, judged.to);const controlEdges = [judged.from, ...controlChanges, judged.to];// A frame with no new component position, in a run of such frames whose// next change the cast's step or an early frame explains (`underStep`):@@ -1586,6 +1667,21 @@ function readBusyWindow(),};}+ // The curve's end left too little to judge only when the part up to the+ // block's end would have held enough. Otherwise the run fails as before,+ // as starvation when the cast was starved.+ if (endsInBlock) {+ const framesLeft = controlRead?.moved ?? expectedFrames;+ const framesWithoutCurve =+ control == null+ ? Math.floor(+ (endWithoutCurve - judged.from) / (compositorFrameMs ?? cadence),+ )+ : controlChangesUpTo(control, endWithoutCurve).length;+ if (framesLeft < minJudgedFrames && framesWithoutCurve >= minJudgedFrames) {+ throw curveTooLate(framesLeft);+ }+ }const failures: string[] = [];let starvation: FrameShortfall | null = null;if (stats.frames === 0 && controlRead == null) {@@ -1675,6 +1771,9 @@ function readBusyWindow((onScreenFromMs == null? '': `, on screen from +${onScreenFromMs.toFixed(0)} ms (${entersAt == null ? 'nothing moved before the block' : 'off screen before the block'})`) ++ (endsInBlock+ ? `, judged to the held curve's end at +${(end - window.from).toFixed(0)} ms`+ : '') +(controlRead == null? '': `; against the control: moved on ${controlRead.movedWith} of its ${controlRead.moved} frames${controlRead.belowStep > 0 ? ` (${controlRead.belowStep} where the cast could not show the step)` : ''}, its longest stall ${controlRead.longestStallMs.toFixed(1)} ms`) +@@ -1830,7 +1929,7 @@ export function pooledBusyProof(failures.push(lateRuns === 0? `${judgedReads.length} of ${runs.length} runs judged, under ${minJudgedRuns}: the compositor presented too little in the others to judge`- : `${judgedReads.length} of ${runs.length} runs judged, under ${minJudgedRuns}: ${lateRuns} began after the motion had settled, and the compositor presented too little in any other to judge`,+ : `${judgedReads.length} of ${runs.length} runs judged, under ${minJudgedRuns}: ${lateRuns} began too late for their motion, and the compositor presented too little in any other to judge`,);}if (judgedReads.length > 0 && moved < minJudgedFrames) {diff --git a/internal/interaction-harness/src/harness.interaction.browser.spec.ts b/internal/interaction-harness/src/harness.interaction.browser.spec.tsindex 819b789e3c..c94d17cdd7 100644--- a/internal/interaction-harness/src/harness.interaction.browser.spec.ts+++ b/internal/interaction-harness/src/harness.interaction.browser.spec.ts@@ -1946,14 +1946,63 @@ test.describe('frame evidence', () => {100.5,);expect(held.verdict.failures, held.summary).toEqual([]);+ // The freeze is the curve's held tail, so every engine shows it: WebKit+ // on Linux ignores a fill on its compositor.const jumpThenFreeze = await run(- [{transform: 'translateY(0px)'}, {transform: 'translateY(50px)'}],- {duration: 800, easing: 'steps(1, end)', fill: 'forwards'},+ [+ {transform: 'translateY(0px)', offset: 0},+ {transform: 'translateY(0px)', offset: 0.5},+ {transform: 'translateY(50px)', offset: 0.5},+ {transform: 'translateY(50px)', offset: 1},+ ],+ {duration: 1600, easing: 'linear'},);expect(jumpThenFreeze.verdict.pass, jumpThenFreeze.summary).toBe(false);await control.remove();});+ test('blockMainThread says a held curve keeps its end value only through a held tail or a fill the compositor honors', async ({+ page,+ browserName,+ }) => {+ await busyFixture(page);+ const box = page.locator('#box');+ const holds = async (+ timing: KeyframeAnimationOptions,+ keyframes: Keyframe[] = [+ {transform: 'translateY(0px)'},+ {transform: 'translateY(200px)'},+ ],+ ) => {+ await box.evaluate(+ (el, [frames, options]) => {+ el.getAnimations().forEach(a => a.cancel());+ el.animate(frames, options);+ },+ [keyframes, timing] as const,+ );+ const window = await blockMainThread(page, 50, {during: box});+ return window.animation?.holds;+ };+ // A held tail: the last keyframes repeat the end value. So the curve ends+ // halfway, and the element keeps that value after it with no fill.+ expect(+ await holds({duration: 1000}, [+ {transform: 'translateY(0px)', offset: 0},+ {transform: 'translateY(200px)', offset: 0.5},+ {transform: 'translateY(200px)', offset: 1},+ ]),+ ).toBe(true);+ // WebKit on Linux ignores `fill` on its compositor (motion/animator.ts,+ // END_HOLD_MS): there a filled move that ends inside a block snaps back.+ expect(await holds({duration: 1000, fill: 'forwards'})).toBe(+ browserName !== 'webkit' || process.platform !== 'linux',+ );+ expect(await holds({duration: 1000})).toBe(false);+ // An end delay shows nothing with no fill: it is no held tail.+ expect(await holds({duration: 1000, endDelay: 500})).toBe(false);+ });+test('a block too short to judge alone is judged over pooled runs: a compositor animation passes, and a rAF one fails, timed or frame-counted', async ({page,}, testInfo) => {