Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
docs(skill): add bisect-published-versions playwright-dev guide (#40169)
Pavel Feldman1f0097422e4e
diff --git a/.claude/skills/playwright-dev/SKILL.md b/.claude/skills/playwright-dev/SKILL.mdindex d131f376de..0c693d382e 100644--- a/.claude/skills/playwright-dev/SKILL.md+++ b/.claude/skills/playwright-dev/SKILL.md@@ -14,3 +14,4 @@ See [CLAUDE.md](../../../CLAUDE.md) for monorepo structure, build/test/lint comm- [MCP Tools and CLI Commands](tools.md) — add MCP tools, CLI commands, config options- [Vendor Dependencies & Bundling](vendor.md) — utilsBundle, coreBundle, babelBundle; adding vendored npm packages; DEPS.list; `check_deps`- [Updating WebKit Safari Version](webkit-safari-version.md) — update the Safari version string in the WebKit user-agent+- [Bisecting Across Published Versions](bisect-published-versions.md) — reproduce regressions side-by-side from npm and diff `node_modules/playwright/lib/` between versionsdiff --git a/.claude/skills/playwright-dev/bisect-published-versions.md b/.claude/skills/playwright-dev/bisect-published-versions.mdnew file mode 100644index 0000000000..f41106d92c--- /dev/null+++ b/.claude/skills/playwright-dev/bisect-published-versions.md@@ -0,0 +1,71 @@+# Bisecting a Regression Across Published Playwright Versions++When a user reports a regression between two published Playwright versions (e.g. "works in 1.58, broken in 1.59.1"), reproduce both side by side from npm — do **not** try to bisect against the monorepo source. Reading the compiled JS in `node_modules/playwright/lib/**` is faster and avoids build/branch confusion.++## Setup (two side-by-side installs)++Use `~/tmp/<version-tag>/` (NOT `/tmp/`) — the user's shell sessions live in `~/tmp`.++```bash+mkdir -p ~/tmp/<good>/tests ~/tmp/<bad>/tests++# Skip `npm init playwright@latest` — it's interactive and the scaffold+# pulls in 3 projects (chromium/firefox/webkit) which produces 6 test runs+# from a single spec and is confusing. Do this instead:+( cd ~/tmp/<good> && npm init -y && npm install @playwright/test@<good-ver> && npx playwright install chromium)+( cd ~/tmp/<bad> && npm init -y && npm install @playwright/test@<bad-ver> && npx playwright install chromium )+```++Write a **minimal** `playwright.config.ts` with a single chromium project — the default scaffold's 3-project config will run the same spec 6 times and obscure output:++```ts+import { defineConfig, devices } from '@playwright/test';+export default defineConfig({+ testDir: './tests',+ projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],+});+```++Drop the repro spec (and any helper files) into both folders identically. Run:++```bash+( cd ~/tmp/<good> && npx playwright test )+( cd ~/tmp/<bad> && npx playwright test )+```++Confirm the difference is real before investigating.++## Investigating the diff in node_modules++The compiled JS in `node_modules/playwright-core/lib/` and `node_modules/playwright/lib/` is the source of truth for what shipped.++In recent versions of Playwright these are bundled, so you can't compare+on per-file basis. You can extract files from bundles via grep though and compare.++Once you've found a candidate function, diff it across the two versions. Patches are usually 1–3 lines.++## Verifying the hypothesis++Edit the compiled JS in `~/tmp/<bad>/node_modules/playwright/lib/...` directly and re-run the test. No build step is needed — Node loads the JS as-is. Revert when done (or just delete the folder).++For stack-trace bugs in particular, a `console.log(new Error().stack)` inserted at the capture site (e.g. inside `expect.js`'s `captureRawStack`) instantly shows whether the issue is microtask-boundary related vs. a stack-filter regression vs. something else.++## Reporting++When the root cause is confirmed:++1. Quote the offending lines from `node_modules/.../lib/...` of the **bad** version, with file path.+2. Show the equivalent code from the **good** version for contrast.+3. Explain *why* the change breaks the user's case (don't just point at the diff).+4. Propose and verify a minimal fix by patching the bad install in place.++Post the writeup as a comment on the original issue with `gh issue comment <number> --repo microsoft/playwright --body "$(cat <<'EOF' ... EOF)"`.++## Pitfalls++- **Don't run `npm init playwright@latest`** — it's interactive and `--quiet` does not skip the prompts. `npm init -y` + `npm install @playwright/test@<ver>` is faster and deterministic.+- **Don't use the scaffold's default config** — the 3 browser projects multiply test runs by 3 and confuse the output. One chromium project is enough for 99% of repros.+- **Don't `cd` between commands in a single Bash call without `&&`** — the shell cwd resets between tool invocations.+- **`/tmp/` is not `~/tmp/`** — pick one and stay consistent. The user's interactive shells default to `~/tmp/`, so prefer that.+- **Don't `rm -rf` an existing `~/tmp/<ver>/`** without checking — it may be the user's prior work. Edit in place instead.+- **Don't try to map the bug to monorepo source first.** The shipped JS is what the user is running; source may have already been refactored or fixed on `main`. Investigate `node_modules/` first, then map the fix back to source only when proposing the upstream patch.