Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
test(firefox): fixme flaky "should create userDataDir" on macOS (#41811)
microsoft-playwright-automation[bot]d333a3aca550
diff --git a/.github/workflows/fix-flakes-prompt.md b/.github/workflows/fix-flakes-prompt.mdindex ba43719927..29acc7016b 100644--- a/.github/workflows/fix-flakes-prompt.md+++ b/.github/workflows/fix-flakes-prompt.md@@ -20,7 +20,8 @@ bots/PRs it disrupts — **not** by "has a tidy error message".Pick a candidate whose failing `bot_name` OS matches yours so you can reproduce it.Keep the ranked list — you fall back down it in step 2. Say why you picked the top one.Once you have the target, use the "Generate a linked emoji run history" recipe in-`.claude/skills/playwright-test-results/SKILL.md`. Keep the Markdown output for the PR body.+`.claude/skills/playwright-test-results/SKILL.md`. Include the complete output in the PR body:+never trim it or remove green runs, and check the square count matches the per-run row count.## 2. Check nobody's on it — and that it isn't already fixed@@ -33,9 +34,8 @@ down your step-1 ranking, re-checking each; only stop once the whole shortlist ibefore a fix landed, so a test fixed mid-window looks maximally bimodal (old fails + newpasses), which is exactly what floats it up step 1. Check whether a fix has landed since itlast flaked: find the commit its most recent failing run ran on, and look at what's changed- in that test's file since. A commit that plausibly targets the test or its feature — a- browser roll, or a fix to the test or the code it exercises — means the remaining DB fails- are from before the fix, so move on.+ in that test's file since. Only move on if enough subsequent runs show the failures stopped;+ a browser roll alone is not evidence of a fix.## 3. Reproduce on this OS@@ -59,15 +59,17 @@ are project-locked, so use the one matching the failing bot:Other suites (electron `etest`, etc.): see `package.json` scripts.Add `--repeat-each=N` to force a flake's flip.-**You can only reproduce what this OS reaches.** A failure you can't reach here is itself-evidence it's environment-specific — a skip candidate (step 6), not a blind-fix.-Keep any OS handling keyed on the *current* OS, never hardcoded.+**You can only reproduce what this OS and architecture reaches.** Match both from `bot_name`;+non-reproduction under lighter local load or on another architecture is not itself a skip candidate.+Keep any OS/architecture handling keyed on the *current* environment, never hardcoded.Broader OS coverage comes from different agent runs on other OSes, not from you.## 4. Fix — root cause or scoped skip- **Tractable → fix the source.** Usually a test-side race: a missing `await`, waiting on thewrong signal, an under-specified locator, leaked state. Fix the test (or the product bug).+- **Timeout → check the budget before skipping.** Compare passing durations and neighboring tests;+ if working tests share an expensive operation near the timeout, prefer a scoped `it.slow()`.- **Engine/OS-specific, unreachable here, or genuinely hard → scope a skip**, narrowed to theexact failing condition — never a whole file or browser:@@ -81,9 +83,9 @@ Broader OS coverage comes from different agent runs on other OSes, not from you.flake-fighter rarely is). Never `skip` just to turn a red green.**Verify locally — your PR gets no CI (step 6), so this is the only proof:** flake → re-run with-`--repeat-each` and confirm it's stable; skip → confirm it's skipped on the target config and-**still runs elsewhere**. Then run `npm run flint`. Record exactly what you ran — it goes in the-PR body.+`--repeat-each` and confirm it's stable; slow → confirm it still runs and passes; skip → confirm+it's skipped on the target config and **still runs elsewhere**. Then run `npm run flint`. Record+exactly what you ran — it goes in the PR body.## 5. Pick a reviewer@@ -121,4 +123,4 @@ Report what you committed, the reviewer, and why.- **One target, one commit.** Don't batch.- **Scoped skips only** — narrow to the failing condition, never a whole file or browser matrix.-- **No unverified PRs** — if you couldn't run it locally, prefer a documented fixme.+- **No unverified PRs** — if you cannot validate a scoped mitigation, make no commit.diff --git a/.github/workflows/fix-flakes.yml b/.github/workflows/fix-flakes.ymlindex 346f735937..c2fcb65e2d 100644--- a/.github/workflows/fix-flakes.yml+++ b/.github/workflows/fix-flakes.yml@@ -73,11 +73,12 @@ jobs:via the playwright-test-results skill. \`bot_name\` encodes the OS.Write your answer to output/runner.txt, and nothing else there: exactly one of these- six runner labels, or the single word \`none\` if no OS is worth a run.+ seven runner labels, or the single word \`none\` if no platform is worth a run.ubuntu-22.04 ubuntu-22.04-arm ubuntu-24.04- macos-14-xlarge macos-15-xlarge windows-latest- Map a candidate's \`bot_name\` to the closest of those (e.g. \`*-macos-15-*\` ->- macos-15-xlarge). Explain your reasoning briefly in your final message.+ macos-14-xlarge macos-15-large macos-15-xlarge windows-latest+ Map \`*-macos-15-large\` to \`macos-15-large\` and \`*-macos-15-xlarge\` to+ \`macos-15-xlarge\`; never substitute one architecture for another. Explain your+ reasoning briefly in your final message.${HINT:+Focus hint from the operator: $HINT}EOF@@ -102,7 +103,7 @@ jobs:exit 0ficase "$runner" in- ubuntu-22.04|ubuntu-22.04-arm|ubuntu-24.04|macos-14-xlarge|macos-15-xlarge|windows-latest) ;;+ ubuntu-22.04|ubuntu-22.04-arm|ubuntu-24.04|macos-14-xlarge|macos-15-large|macos-15-xlarge|windows-latest) ;;*) echo "::error::Triage chose a non-allowlisted runner: '$runner'." >&2; exit 1 ;;esacecho "Triage chose runner: $runner"diff --git a/tests/library/defaultbrowsercontext-2.spec.ts b/tests/library/defaultbrowsercontext-2.spec.tsindex 823ad64927..f0eb64bd80 100644--- a/tests/library/defaultbrowsercontext-2.spec.ts+++ b/tests/library/defaultbrowsercontext-2.spec.ts@@ -20,6 +20,7 @@ import fs from 'fs';import path from 'path';it.skip(({ mode }) => mode !== 'default', 'Remote persistent contexts are not supported');+it.slow(({ browserName, isMac }) => browserName === 'firefox' && isMac && process.arch === 'x64', 'Persistent Firefox launches are slow on Intel macOS runners under load.');it('should support hasTouch option', async ({ server, launchPersistent }) => {const { page } = await launchPersistent({ hasTouch: true });