Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
docs(snapshots): clarify that threshold defines which pixels are different (#42998)
Dmitry Gozman0bd86d523fef
diff --git a/docs/src/api/params.md b/docs/src/api/params.mdindex 3a88e0127a..e8e4080e1e 100644--- a/docs/src/api/params.md+++ b/docs/src/api/params.md@@ -1099,13 +1099,13 @@ Whether to perform case-insensitive match. [`option: ignoreCase`] option takes p* langs: js- `maxDiffPixels` <[int]>-An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by default.+An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that [`option: threshold`] defines which pixels are considered different.## assertions-max-diff-pixel-ratio* langs: js- `maxDiffPixelRatio` <[float]>-An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is configurable with `TestConfig.expect`. Unset by default.+An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is configurable with `TestConfig.expect`. Note that [`option: threshold`] defines which pixels are considered different.## assertions-threshold* langs: jsdiff --git a/docs/src/test-api/class-testconfig.md b/docs/src/test-api/class-testconfig.mdindex 53bb0b23d0..39b0f1f53a 100644--- a/docs/src/test-api/class-testconfig.md+++ b/docs/src/test-api/class-testconfig.md@@ -75,8 +75,8 @@ The structure of the git commit metadata is subject to change.- `toHaveScreenshot` ?<[Object]> Configuration for the [`method: PageAssertions.toHaveScreenshot#1`] method.- `animations` ?<[ScreenshotAnimations]<"allow"|"disabled">> See [`option: Page.screenshot.animations`] in [`method: Page.screenshot`]. Defaults to `"disabled"`.- `caret` ?<[ScreenshotCaret]<"hide"|"initial">> See [`option: Page.screenshot.caret`] in [`method: Page.screenshot`]. Defaults to `"hide"`.- - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default.- - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.+ - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.+ - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.- `stylePath` ?<[string]|[Array]<[string]>> See [`option: Page.screenshot.style`] in [`method: Page.screenshot`].- `threshold` ?<[float]> An acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.@@ -86,8 +86,8 @@ The structure of the git commit metadata is subject to change.- `pathTemplate` ?<[string]> A template controlling location of the aria snapshots. See [`property: TestConfig.snapshotPathTemplate`] for details.- `children` ?<["contain" | "equal" | "deep-equal"]> Controls how children of the snapshot root are matched against the actual accessibility tree. This is equivalent to adding a `/children` property at the top of every aria snapshot template. Individual snapshots can override this by including an explicit `/children` property.- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.- - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default.- - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.+ - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.+ - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.- `threshold` ?<[float]> An acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.- `toPass` ?<[Object]> Configuration for the [expect(value).toPass()](../test-assertions.md#expecttopass) method.- `intervals` ?<[Array]<[int]>> Probe intervals for toPass method in milliseconds.diff --git a/docs/src/test-api/class-testproject.md b/docs/src/test-api/class-testproject.mdindex dc75000c0b..5a05879516 100644--- a/docs/src/test-api/class-testproject.md+++ b/docs/src/test-api/class-testproject.md@@ -124,8 +124,8 @@ Now `npx playwright test` only runs `chromium`, while `npx playwright test --pro- `timeout` ?<[int]> Default timeout for async expect matchers in milliseconds, defaults to 5000ms.- `toHaveScreenshot` ?<[Object]> Configuration for the [`method: PageAssertions.toHaveScreenshot#1`] method.- `threshold` ?<[float]> an acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.- - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default.- - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.+ - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.+ - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.- `animations` ?<[ScreenshotAnimations]<"allow"|"disabled">> See [`option: Page.screenshot.animations`] in [`method: Page.screenshot`]. Defaults to `"disabled"`.- `caret` ?<[ScreenshotCaret]<"hide"|"initial">> See [`option: Page.screenshot.caret`] in [`method: Page.screenshot`]. Defaults to `"hide"`.- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.@@ -138,8 +138,8 @@ Now `npx playwright test` only runs `chromium`, while `npx playwright test --pro- `children` ?<["contain" | "equal" | "deep-equal"]> Controls how children of the snapshot root are matched against the actual accessibility tree. This is equivalent to adding a `/children` property at the top of every aria snapshot template. Individual snapshots can override this by including an explicit `/children` property.- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.- `threshold` ?<[float]> an acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.- - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default.- - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.+ - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.+ - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.- `toPass` ?<[Object]> Configuration for the [expect(value).toPass()](../test-assertions.md) method.- `timeout` ?<[int]> timeout for toPass method in milliseconds.- `intervals` ?<[Array]<[int]>> probe intervals for toPass method in milliseconds.diff --git a/docs/src/test-snapshots-js.md b/docs/src/test-snapshots-js.mdindex 95d8fbc4b4..40b5a343e4 100644--- a/docs/src/test-snapshots-js.md+++ b/docs/src/test-snapshots-js.md@@ -92,26 +92,29 @@ await expect(page).toHaveScreenshot();## Options-### maxDiffPixels+### maxDiffPixels and thresholdPlaywright Test uses the [pixelmatch](https://github.com/mapbox/pixelmatch) library. You can [pass various options](./api/class-pageassertions.md#page-assertions-to-have-screenshot-1) to modify its behavior:+- `threshold` defines which pixels are considered different. It is an acceptable perceived color difference between the same pixel in compared images, between `0` (strict) and `1` (lax). Defaults to `0.2`.+- `maxDiffPixels` is an acceptable amount of pixels that could be different. Unset by default.+```js title="example.spec.ts"import { test, expect } from '@playwright/test';test('example test', async ({ page }) => {await page.goto('https://playwright.dev');- await expect(page).toHaveScreenshot({ maxDiffPixels: 100 });+ await expect(page).toHaveScreenshot({ maxDiffPixels: 100, threshold: 0.1 });});```-If you'd like to share the default value among all the tests in the project, you can specify it in the playwright config, either globally or per project:+If you'd like to share the default values among all the tests in the project, you can specify them in the playwright config, either globally or per project:```js title="playwright.config.ts"import { defineConfig } from '@playwright/test';export default defineConfig({expect: {- toHaveScreenshot: { maxDiffPixels: 100 },+ toHaveScreenshot: { maxDiffPixels: 100, threshold: 0.1 },},});```diff --git a/packages/playwright/types/test.d.ts b/packages/playwright/types/test.d.tsindex ce07a47c6d..30ac1fbc30 100644--- a/packages/playwright/types/test.d.ts+++ b/packages/playwright/types/test.d.ts@@ -237,13 +237,14 @@ interface TestProject<TestArgs = {}, WorkerArgs = {}> {threshold?: number;/**- * an acceptable amount of pixels that could be different, unset by default.+ * an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which+ * pixels are considered different.*/maxDiffPixels?: number;/*** an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by- * default.+ * default. Note that `threshold` defines which pixels are considered different.*/maxDiffPixelRatio?: number;@@ -327,13 +328,14 @@ interface TestProject<TestArgs = {}, WorkerArgs = {}> {threshold?: number;/**- * an acceptable amount of pixels that could be different, unset by default.+ * an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which+ * pixels are considered different.*/maxDiffPixels?: number;/*** an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by- * default.+ * default. Note that `threshold` defines which pixels are considered different.*/maxDiffPixelRatio?: number;};@@ -1195,13 +1197,14 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {caret?: "hide"|"initial";/**- * An acceptable amount of pixels that could be different, unset by default.+ * An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which+ * pixels are considered different.*/maxDiffPixels?: number;/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by- * default.+ * default. Note that `threshold` defines which pixels are considered different.*/maxDiffPixelRatio?: number;@@ -1266,13 +1269,14 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {*/toMatchSnapshot?: {/**- * An acceptable amount of pixels that could be different, unset by default.+ * An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which+ * pixels are considered different.*/maxDiffPixels?: number;/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by- * default.+ * default. Note that `threshold` defines which pixels are considered different.*/maxDiffPixelRatio?: number;@@ -9992,13 +9996,16 @@ interface LocatorAssertions {/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is- * configurable with `TestConfig.expect`. Unset by default.+ * configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixelRatio?: number;/**- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by- * default.+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixels?: number;@@ -10095,13 +10102,16 @@ interface LocatorAssertions {/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is- * configurable with `TestConfig.expect`. Unset by default.+ * configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-2-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixelRatio?: number;/**- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by- * default.+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-2-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixels?: number;@@ -10629,13 +10639,16 @@ interface SnapshotAssertions {toMatchSnapshot(name: string|ReadonlyArray<string>, options?: {/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is- * configurable with `TestConfig.expect`. Unset by default.+ * configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixelRatio?: number;/**- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by- * default.+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixels?: number;@@ -10681,13 +10694,16 @@ interface SnapshotAssertions {toMatchSnapshot(options?: {/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is- * configurable with `TestConfig.expect`. Unset by default.+ * configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-2-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixelRatio?: number;/**- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by- * default.+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-2-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixels?: number;@@ -11013,13 +11029,16 @@ export interface PageAssertionsToHaveScreenshotOptions {/*** An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is- * configurable with `TestConfig.expect`. Unset by default.+ * configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixelRatio?: number;/**- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by- * default.+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that+ * [`threshold`](https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1-option-threshold)+ * defines which pixels are considered different.*/maxDiffPixels?: number;