Skip to content

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

docs(snapshots): clarify that threshold defines which pixels are different (#42998)

Dmitry Gozman0bd86d523fef
diff --git a/docs/src/api/params.md b/docs/src/api/params.md
index 3a88e0127a..e8e4080e1e 100644
--- a/docs/src/api/params.md
+++ b/docs/src/api/params.md
@@ -1099,13 +1099,13 @@ Whether to perform case-insensitive match. [`option: ignoreCase`] option takes p
* langs: js
- `maxDiffPixels` <[int]>
-An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by default.
+An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that [`option: threshold`] defines which pixels are considered different.
## assertions-max-diff-pixel-ratio
* langs: js
- `maxDiffPixelRatio` <[float]>
-An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is configurable with `TestConfig.expect`. Unset by default.
+An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is configurable with `TestConfig.expect`. Note that [`option: threshold`] defines which pixels are considered different.
## assertions-threshold
* langs: js
diff --git a/docs/src/test-api/class-testconfig.md b/docs/src/test-api/class-testconfig.md
index 53bb0b23d0..39b0f1f53a 100644
--- a/docs/src/test-api/class-testconfig.md
+++ b/docs/src/test-api/class-testconfig.md
@@ -75,8 +75,8 @@ The structure of the git commit metadata is subject to change.
- `toHaveScreenshot` ?<[Object]> Configuration for the [`method: PageAssertions.toHaveScreenshot#1`] method.
- `animations` ?<[ScreenshotAnimations]<"allow"|"disabled">> See [`option: Page.screenshot.animations`] in [`method: Page.screenshot`]. Defaults to `"disabled"`.
- `caret` ?<[ScreenshotCaret]<"hide"|"initial">> See [`option: Page.screenshot.caret`] in [`method: Page.screenshot`]. Defaults to `"hide"`.
- - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default.
- - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.
+ - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.
+ - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.
- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.
- `stylePath` ?<[string]|[Array]<[string]>> See [`option: Page.screenshot.style`] in [`method: Page.screenshot`].
- `threshold` ?<[float]> An acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
@@ -86,8 +86,8 @@ The structure of the git commit metadata is subject to change.
- `pathTemplate` ?<[string]> A template controlling location of the aria snapshots. See [`property: TestConfig.snapshotPathTemplate`] for details.
- `children` ?<["contain" | "equal" | "deep-equal"]> Controls how children of the snapshot root are matched against the actual accessibility tree. This is equivalent to adding a `/children` property at the top of every aria snapshot template. Individual snapshots can override this by including an explicit `/children` property.
- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.
- - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default.
- - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.
+ - `maxDiffPixels` ?<[int]> An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.
+ - `maxDiffPixelRatio` ?<[float]> An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.
- `threshold` ?<[float]> An acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
- `toPass` ?<[Object]> Configuration for the [expect(value).toPass()](../test-assertions.md#expecttopass) method.
- `intervals` ?<[Array]<[int]>> Probe intervals for toPass method in milliseconds.
diff --git a/docs/src/test-api/class-testproject.md b/docs/src/test-api/class-testproject.md
index dc75000c0b..5a05879516 100644
--- a/docs/src/test-api/class-testproject.md
+++ b/docs/src/test-api/class-testproject.md
@@ -124,8 +124,8 @@ Now `npx playwright test` only runs `chromium`, while `npx playwright test --pro
- `timeout` ?<[int]> Default timeout for async expect matchers in milliseconds, defaults to 5000ms.
- `toHaveScreenshot` ?<[Object]> Configuration for the [`method: PageAssertions.toHaveScreenshot#1`] method.
- `threshold` ?<[float]> an acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
- - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default.
- - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.
+ - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.
+ - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.
- `animations` ?<[ScreenshotAnimations]<"allow"|"disabled">> See [`option: Page.screenshot.animations`] in [`method: Page.screenshot`]. Defaults to `"disabled"`.
- `caret` ?<[ScreenshotCaret]<"hide"|"initial">> See [`option: Page.screenshot.caret`] in [`method: Page.screenshot`]. Defaults to `"hide"`.
- `scale` ?<[ScreenshotScale]<"css"|"device">> See [`option: Page.screenshot.scale`] in [`method: Page.screenshot`]. Defaults to `"css"`.
@@ -138,8 +138,8 @@ Now `npx playwright test` only runs `chromium`, while `npx playwright test --pro
- `children` ?<["contain" | "equal" | "deep-equal"]> Controls how children of the snapshot root are matched against the actual accessibility tree. This is equivalent to adding a `/children` property at the top of every aria snapshot template. Individual snapshots can override this by including an explicit `/children` property.
- `toMatchSnapshot` ?<[Object]> Configuration for the [`method: SnapshotAssertions.toMatchSnapshot#1`] method.
- `threshold` ?<[float]> an acceptable perceived color difference between the same pixel in compared images, ranging from `0` (strict) and `1` (lax). `"pixelmatch"` comparator computes color difference in [YIQ color space](https://en.wikipedia.org/wiki/YIQ) and defaults `threshold` value to `0.2`.
- - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default.
- - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default.
+ - `maxDiffPixels` ?<[int]> an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which pixels are considered different.
+ - `maxDiffPixelRatio` ?<[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by default. Note that `threshold` defines which pixels are considered different.
- `toPass` ?<[Object]> Configuration for the [expect(value).toPass()](../test-assertions.md) method.
- `timeout` ?<[int]> timeout for toPass method in milliseconds.
- `intervals` ?<[Array]<[int]>> probe intervals for toPass method in milliseconds.
diff --git a/docs/src/test-snapshots-js.md b/docs/src/test-snapshots-js.md
index 95d8fbc4b4..40b5a343e4 100644
--- a/docs/src/test-snapshots-js.md
+++ b/docs/src/test-snapshots-js.md
@@ -92,26 +92,29 @@ await expect(page).toHaveScreenshot();
## Options
-### maxDiffPixels
+### maxDiffPixels and threshold
Playwright Test uses the [pixelmatch](https://github.com/mapbox/pixelmatch) library. You can [pass various options](./api/class-pageassertions.md#page-assertions-to-have-screenshot-1) to modify its behavior:
+- `threshold` defines which pixels are considered different. It is an acceptable perceived color difference between the same pixel in compared images, between `0` (strict) and `1` (lax). Defaults to `0.2`.
+- `maxDiffPixels` is an acceptable amount of pixels that could be different. Unset by default.
+
```js title="example.spec.ts"
import { test, expect } from '@playwright/test';
test('example test', async ({ page }) => {
await page.goto('https://playwright.dev');
- await expect(page).toHaveScreenshot({ maxDiffPixels: 100 });
+ await expect(page).toHaveScreenshot({ maxDiffPixels: 100, threshold: 0.1 });
});
```
-If you'd like to share the default value among all the tests in the project, you can specify it in the playwright config, either globally or per project:
+If you'd like to share the default values among all the tests in the project, you can specify them in the playwright config, either globally or per project:
```js title="playwright.config.ts"
import { defineConfig } from '@playwright/test';
export default defineConfig({
expect: {
- toHaveScreenshot: { maxDiffPixels: 100 },
+ toHaveScreenshot: { maxDiffPixels: 100, threshold: 0.1 },
},
});
```
diff --git a/packages/playwright/types/test.d.ts b/packages/playwright/types/test.d.ts
index ce07a47c6d..30ac1fbc30 100644
--- a/packages/playwright/types/test.d.ts
+++ b/packages/playwright/types/test.d.ts
@@ -237,13 +237,14 @@ interface TestProject<TestArgs = {}, WorkerArgs = {}> {
threshold?: number;
/**
- * an acceptable amount of pixels that could be different, unset by default.
+ * an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which
+ * pixels are considered different.
*/
maxDiffPixels?: number;
/**
* an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by
- * default.
+ * default. Note that `threshold` defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
@@ -327,13 +328,14 @@ interface TestProject<TestArgs = {}, WorkerArgs = {}> {
threshold?: number;
/**
- * an acceptable amount of pixels that could be different, unset by default.
+ * an acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which
+ * pixels are considered different.
*/
maxDiffPixels?: number;
/**
* an acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by
- * default.
+ * default. Note that `threshold` defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
};
@@ -1195,13 +1197,14 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {
caret?: "hide"|"initial";
/**
- * An acceptable amount of pixels that could be different, unset by default.
+ * An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which
+ * pixels are considered different.
*/
maxDiffPixels?: number;
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by
- * default.
+ * default. Note that `threshold` defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
@@ -1266,13 +1269,14 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {
*/
toMatchSnapshot?: {
/**
- * An acceptable amount of pixels that could be different, unset by default.
+ * An acceptable amount of pixels that could be different, unset by default. Note that `threshold` defines which
+ * pixels are considered different.
*/
maxDiffPixels?: number;
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1` , unset by
- * default.
+ * default. Note that `threshold` defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
@@ -9992,13 +9996,16 @@ interface LocatorAssertions {
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is
- * configurable with `TestConfig.expect`. Unset by default.
+ * configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
/**
- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by
- * default.
+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixels?: number;
@@ -10095,13 +10102,16 @@ interface LocatorAssertions {
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is
- * configurable with `TestConfig.expect`. Unset by default.
+ * configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-2-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
/**
- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by
- * default.
+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-2-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixels?: number;
@@ -10629,13 +10639,16 @@ interface SnapshotAssertions {
toMatchSnapshot(name: string|ReadonlyArray<string>, options?: {
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is
- * configurable with `TestConfig.expect`. Unset by default.
+ * configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
/**
- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by
- * default.
+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixels?: number;
@@ -10681,13 +10694,16 @@ interface SnapshotAssertions {
toMatchSnapshot(options?: {
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is
- * configurable with `TestConfig.expect`. Unset by default.
+ * configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-2-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
/**
- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by
- * default.
+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-snapshotassertions#snapshot-assertions-to-match-snapshot-2-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixels?: number;
@@ -11013,13 +11029,16 @@ export interface PageAssertionsToHaveScreenshotOptions {
/**
* An acceptable ratio of pixels that are different to the total amount of pixels, between `0` and `1`. Default is
- * configurable with `TestConfig.expect`. Unset by default.
+ * configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixelRatio?: number;
/**
- * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Unset by
- * default.
+ * An acceptable amount of pixels that could be different. Default is configurable with `TestConfig.expect`. Note that
+ * [`threshold`](https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1-option-threshold)
+ * defines which pixels are considered different.
*/
maxDiffPixels?: number;