Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
chore: add test agents docs (#37461)
Pavel Feldmancde6c43868be
diff --git a/docs/src/test-agents-js.md b/docs/src/test-agents-js.mdnew file mode 100644index 0000000000..15b6537034--- /dev/null+++ b/docs/src/test-agents-js.md@@ -0,0 +1,217 @@+---+id: test-agents+title: "Agents"+---++# Playwright Agents++## Test Coverage in 1-2-3++Playwright’s agentic workflow makes it possible to generate test coverage in three straightforward steps.+These steps can be performed independently, manually, or as chained calls in an agentic loop.++1. **Plan**: A planning agent explores the app and produces a test plan in `specs/*.md`.++2. **Generate**: A generating agent transforms the plan into `tests/*.spec.ts` files. It executes actions live to verify selectors and flows, then emits testing code and assertions.++3. **Heal**: A healing agent executes the test suite and automatically repairs failing tests by applying diffs in place.++## 1. Plan++**Input**++* A clear request (e.g., “Generate a plan for guest checkout.”)+* A live entry point (URL) or a seed Playwright test that sets up the environment+* PRD (optional)++**Prompt**++```markdown+Ask `playwright-test-planner` agent to generate a test plan for "Guest Checkout" scenario.+Use `seed.spec.ts` as a seed test for the plan.+```++**Output**++* A Markdown test plan saved to `specs/guest-checkout.md`. The plan is human-readable but precise enough for test generation.++<details>+<summary>Example: specs/guest-checkout.md</summary>++```markdown+# Feature: Guest Checkout++## Purpose+Allow a user to purchase without creating an account.++## Preconditions+- Test seed `tests/seed.spec.ts`.+- Payment sandbox credentials available via env vars.++## Scenarios++### SC-1: Add single item to cart and purchase+**Steps**+1. Open home page.+2. Search for "Wireless Mouse".+3. Open product page and add to cart.+4. Proceed to checkout as guest.+5. Fill shipping and payment details.+6. Confirm order.++**Expected**+- Cart count increments after item is added.+- Checkout page shows item, price, tax, and total.+- Order confirmation number appears; status is "Processing".++### SC-2: Tax and shipping recalculation on address change+**Steps**+1. Start checkout with a CA address.+2. Change state to NY.++**Expected**+- Tax and shipping values recalculate.++## Data+- Product SKU: `WM-123`+- Payment: sandbox card `4111 1111 1111 1111`, valid expiry, CVV `123`.++## Methodology+*Optional notes about testing methodology*+```+</details>++## 2. Generate++The generating agent uses the Markdown plan to produce executable Playwright tests.+It verifies selectors and assertions live against the application. Playwright supports+generation hints and provides a catalog of assertions for efficient structural and+behavioral validation.++**Input**++* Markdown plan from `specs/`++**Prompt**++```markdown+Ask `playwright-test-generator` to generate tests for the guest checkout plan under `specs/`.+```++**Output**++* A test suite under `tests/`+* Tests may include initial errors that can be healed automatically++<details>+<summary>Example: tests/guest-checkout.spec.ts</summary>++```ts+import { test, expect } from '@playwright/test';++test.describe('Guest Checkout', () => {+ test('SC-1: add item and purchase', async ({ page }) => {+ await page.goto('/');+ await page.getByRole('searchbox', { name: /search/i }).fill('Wireless Mouse');+ await page.getByRole('button', { name: /search/i }).click();++ await page.getByRole('link', { name: /wireless mouse/i }).click();+ await page.getByRole('button', { name: /add to cart/i }).click();++ // Assertion: cart badge increments+ await expect(page.getByTestId('cart-badge')).toHaveText('1');++ await page.getByRole('link', { name: /checkout/i }).click();+ await page.getByRole('button', { name: /continue as guest/i }).click();++ // Fill checkout form+ await page.getByLabel('Email').fill(process.env.CHECKOUT_EMAIL!);+ await page.getByLabel('Full name').fill('Alex Guest');+ await page.getByLabel('Address').fill('1 Market St');+ await page.getByLabel('City').fill('San Francisco');+ await page.getByLabel('State').selectOption('CA');+ await page.getByLabel('ZIP').fill('94105');++ // Payment (sandbox)+ const frame = page.frameLocator('[data-testid="card-iframe"]');+ await frame.getByLabel('Card number').fill('4111111111111111');+ await frame.getByLabel('MM / YY').fill('12/30');+ await frame.getByLabel('CVC').fill('123');++ await page.getByRole('button', { name: /pay/i }).click();++ // Assertions: confirmation invariants+ await expect(page).toHaveURL(/\/orders\/\w+\/confirmation/);+ await expect(page.getByRole('heading', { name: /thank you/i })).toBeVisible();+ await expect(page.getByTestId('order-status')).toHaveText(/processing/i);++ // Optional visual check+ await expect(page.locator('[data-testid="order-summary"]')).toHaveScreenshot();+ });+});+```+</details>++## 3. Heal++When a test fails, the healing agent:++* Replays the failing steps+* Inspects the current UI to locate equivalent elements or flows+* Suggests a patch (e.g., locator update, wait adjustment, data fix)+* Re-runs the test until it passes or until guardrails stop the loop++**Input**++* Failing test name++**Prompt**++```markdown+Ask `playwright-test-healer` to fix all failing tests for the guest checkout scenario.+```++**Output**++* A passing test, or a skipped test if the functionality is broken++## Artifacts and Conventions++Follow a simple, auditable structure:++```bash+repo/+ .{claude|copilot|vscode|...}/ # agent definitions, tools, guardrails+ specs/ # human-readable test plans+ checkout-guest.md+ account-settings.md+ tests/ # generated Playwright tests+ seed.spec.ts+ checkout-guest.spec.ts+ account-settings.spec.ts+ playwright.config.ts+```++### Agents definitions++Agent definitions are collections of instructions and MCP tools. They are provided by+Playwright and should be regenerated whenever Playwright is updated.++Example for Claude Code subagents:++```bash+npx playwright init-agents --claude+```++### Specs in `specs/`++Specs are structured plans describing scenarios in human-readable terms. They include+steps, expected outcomes, and data. Specs can start from scratch or extend a seed test.++### Tests in `tests/`++Generated Playwright tests, aligned one-to-one with specs wherever feasible.++### Seed tests `seed.spec.ts`++Seed tests provide a ready-to-use `page` context to bootstrap execution.diff --git a/docs/src/test-configuration-js.md b/docs/src/test-configuration-js.mdindex e3831c1911..52e68c658c 100644--- a/docs/src/test-configuration-js.md+++ b/docs/src/test-configuration-js.md@@ -1,6 +1,6 @@---id: test-configuration-title: "Test configuration"+title: "Configuration"---## Introductiondiff --git a/docs/src/test-use-options-js.md b/docs/src/test-use-options-js.mdindex 0b46393941..50c9246e86 100644--- a/docs/src/test-use-options-js.md+++ b/docs/src/test-use-options-js.md@@ -1,6 +1,6 @@---id: test-use-options-title: "Test use options"+title: "Configuration (use)"---## Introductiondiff --git a/utils/doclint/cli.js b/utils/doclint/cli.jsindex a694cd0749..958a13e181 100755--- a/utils/doclint/cli.js+++ b/utils/doclint/cli.js@@ -197,6 +197,7 @@ async function run() {'java','css','js',+ 'markdown','ts','python','py',