Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

docs(tests-e2e): one set of e2e guidelines, enforced by eslint-plugin-playwright (#17104)

Othman Abu Ajamiehc47112b57e00
diff --git a/.agents/skills/playwright-e2e-testing/SKILL.md b/.agents/skills/playwright-e2e-testing/SKILL.md
deleted file mode 100644
index 632c2af382f..00000000000
--- a/.agents/skills/playwright-e2e-testing/SKILL.md
+++ /dev/null
@@ -1,1086 +0,0 @@
----
-name: playwright-e2e-testing
-description: "Playwright modern end-to-end testing framework with cross-browser automation, auto-wait, and built-in test runner"
-progressive_disclosure:
- entry_point:
- summary: "Playwright modern end-to-end testing framework with cross-browser automation, auto-wait, and built-in test runner"
- when_to_use: "When writing tests, implementing playwright-e2e-testing, or ensuring code quality."
- quick_start: "1. Review the core concepts below. 2. Apply patterns to your use case. 3. Follow best practices for implementation."
----
-# Playwright E2E Testing Skill
-
----
-progressive_disclosure:
- entry_point:
- summary: "Modern E2E testing framework with cross-browser automation and built-in test runner"
- when_to_use:
- - "When testing web applications end-to-end"
- - "When needing cross-browser testing"
- - "When testing user flows and interactions"
- - "When needing screenshot/video recording"
- quick_start:
- - "npm init playwright@latest"
- - "Choose TypeScript and test location"
- - "npx playwright test"
- - "npx playwright show-report"
- token_estimate:
- entry: 75-90
- full: 4200-5200
----
-
-<!-- ENTRY POINT - Load this section by default (75-90 tokens) -->
-
-## Overview
-
-Playwright is a modern end-to-end testing framework that provides cross-browser automation with a built-in test runner, auto-wait mechanisms, and excellent developer experience.
-
-### Key Features
-- **Auto-wait**: Automatically waits for elements to be ready
-- **Cross-browser**: Chromium, Firefox, WebKit support
-- **Built-in runner**: Parallel execution, retries, reporters
-- **Network control**: Mock and intercept network requests
-- **Debugging**: UI mode, trace viewer, inspector
-
----
-
-<!-- FULL CONTENT - Load on demand (4200-5200 tokens) -->
-
-## Installation
-
-```bash
-# Initialize new Playwright project
-npm init playwright@latest
-
-# Or add to existing project
-npm install -D @playwright/test
-
-# Install browsers
-npx playwright install
-```
-
-### Configuration
-
-```typescript
-// playwright.config.ts
-import { defineConfig, devices } from '@playwright/test';
-
-export default defineConfig({
- testDir: './tests',
- fullyParallel: true,
- forbidOnly: !!process.env.CI,
- retries: process.env.CI ? 2 : 0,
- workers: process.env.CI ? 1 : undefined,
- reporter: 'html',
-
- use: {
- baseURL: 'http://localhost:3000',
- trace: 'on-first-retry',
- screenshot: 'only-on-failure',
- },
-
- projects: [
- {
- name: 'chromium',
- use: { ...devices['Desktop Chrome'] },
- },
- {
- name: 'firefox',
- use: { ...devices['Desktop Firefox'] },
- },
- {
- name: 'webkit',
- use: { ...devices['Desktop Safari'] },
- },
- {
- name: 'Mobile Chrome',
- use: { ...devices['Pixel 5'] },
- },
- ],
-
- webServer: {
- command: 'npm run start',
- url: 'http://localhost:3000',
- reuseExistingServer: !process.env.CI,
- },
-});
-```
-
-## Fundamentals
-
-### Basic Test Structure
-
-```typescript
-import { test, expect } from '@playwright/test';
-
-test('basic test', async ({ page }) => {
- await page.goto('https://example.com');
-
- // Wait for element and check visibility
- const title = page.locator('h1');
- await expect(title).toBeVisible();
- await expect(title).toHaveText('Example Domain');
-
- // Get page title
- await expect(page).toHaveTitle(/Example/);
-});
-
-test.describe('User authentication', () => {
- test('should login successfully', async ({ page }) => {
- await page.goto('/login');
- await page.fill('[name="username"]', 'testuser');
- await page.fill('[name="password"]', 'password123');
- await page.click('button[type="submit"]');
-
- await expect(page).toHaveURL('/dashboard');
- await expect(page.locator('.welcome-message')).toContainText('Welcome');
- });
-
- test('should show error for invalid credentials', async ({ page }) => {
- await page.goto('/login');
- await page.fill('[name="username"]', 'invalid');
- await page.fill('[name="password"]', 'wrong');
- await page.click('button[type="submit"]');
-
- await expect(page.locator('.error-message')).toBeVisible();
- await expect(page.locator('.error-message')).toHaveText('Invalid credentials');
- });
-});
-```
-
-### Test Hooks
-
-```typescript
-import { test, expect } from '@playwright/test';
-
-test.describe('Dashboard tests', () => {
- test.beforeEach(async ({ page }) => {
- // Run before each test
- await page.goto('/dashboard');
- await page.waitForLoadState('networkidle');
- });
-
- test.afterEach(async ({ page }) => {
- // Cleanup after each test
- await page.close();
- });
-
- test.beforeAll(async ({ browser }) => {
- // Run once before all tests in describe block
- console.log('Starting test suite');
- });
-
- test.afterAll(async ({ browser }) => {
- // Run once after all tests
- console.log('Test suite complete');
- });
-
- test('displays user data', async ({ page }) => {
- await expect(page.locator('.user-name')).toBeVisible();
- });
-});
-```
-
-## Locator Strategies
-
-### Best Practice: Role-based Locators
-
-```typescript
-import { test, expect } from '@playwright/test';
-
-test('accessible locators', async ({ page }) => {
- await page.goto('/form');
-
- // By role (BEST - accessible and stable)
- await page.getByRole('button', { name: 'Submit' }).click();
- await page.getByRole('textbox', { name: 'Email' }).fill('[email protected]');
- await page.getByRole('checkbox', { name: 'Subscribe' }).check();
- await page.getByRole('link', { name: 'Learn more' }).click();
-
- // By label (good for forms)
- await page.getByLabel('Password').fill('secret123');
-
- // By placeholder
- await page.getByPlaceholder('Search...').fill('query');
-
- // By text
- await page.getByText('Welcome back').click();
- await page.getByText(/hello/i).isVisible();
-
- // By test ID (good for dynamic content)
- await page.getByTestId('user-profile').click();
-
- // By title
- await page.getByTitle('Close dialog').click();
-
- // By alt text (images)
- await page.getByAltText('User avatar').click();
-});
-```
-
-### CSS and XPath Locators
-
-```typescript
-test('CSS and XPath locators', async ({ page }) => {
- // CSS selectors
- await page.locator('button.primary').click();
- await page.locator('#user-menu').click();
- await page.locator('[data-testid="submit-btn"]').click();
- await page.locator('div.card:first-child').click();
-
- // XPath (use sparingly)
- await page.locator('xpath=//button[contains(text(), "Submit")]').click();
-
- // Chaining locators
- const form = page.locator('form#login-form');
- await form.locator('input[name="email"]').fill('[email protected]');
- await form.locator('button[type="submit"]').click();
-
- // Filter locators
- await page.getByRole('listitem')
- .filter({ hasText: 'Product 1' })
- .getByRole('button', { name: 'Add to cart' })
- .click();
-});
-```
-
-## Page Object Model
-
-### Page Class Pattern
-
-```typescript
-// pages/LoginPage.ts
-import { Page, Locator } from '@playwright/test';
-
-export class LoginPage {
- readonly page: Page;
- readonly usernameInput: Locator;
- readonly passwordInput: Locator;
- readonly submitButton: Locator;
- readonly errorMessage: Locator;
-
- constructor(page: Page) {
- this.page = page;
- this.usernameInput = page.getByLabel('Username');
- this.passwordInput = page.getByLabel('Password');
- this.submitButton = page.getByRole('button', { name: 'Log in' });
- this.errorMessage = page.locator('.error-message');
- }
-
- async goto() {
- await this.page.goto('/login');
- }
-
- async login(username: string, password: string) {
- await this.usernameInput.fill(username);
- await this.passwordInput.fill(password);
- await this.submitButton.click();
- }
-
- async expectErrorMessage(message: string) {
- await this.errorMessage.waitFor({ state: 'visible' });
- await expect(this.errorMessage).toHaveText(message);
- }
-}
-
-// pages/DashboardPage.ts
-export class DashboardPage {
- readonly page: Page;
- readonly welcomeMessage: Locator;
- readonly logoutButton: Locator;
-
- constructor(page: Page) {
- this.page = page;
- this.welcomeMessage = page.locator('.welcome-message');
- this.logoutButton = page.getByRole('button', { name: 'Logout' });
- }
-
- async waitForLoad() {
- await this.welcomeMessage.waitFor({ state: 'visible' });
- }
-
- async logout() {
- await this.logoutButton.click();
- }
-}
-
-// tests/auth.spec.ts
-import { test, expect } from '@playwright/test';
-import { LoginPage } from '../pages/LoginPage';
-import { DashboardPage } from '../pages/DashboardPage';
-
-test('successful login flow', async ({ page }) => {
- const loginPage = new LoginPage(page);
- const dashboard = new DashboardPage(page);
-
- await loginPage.goto();
- await loginPage.login('testuser', 'password123');
-
- await dashboard.waitForLoad();
- await expect(dashboard.welcomeMessage).toContainText('Welcome');
-});
-```
-
-### Component Pattern
-
-```typescript
-// components/NavigationComponent.ts
-import { Page, Locator } from '@playwright/test';
-
-export class NavigationComponent {
- readonly page: Page;
- readonly homeLink: Locator;
- readonly profileLink: Locator;
- readonly searchInput: Locator;
-
- constructor(page: Page) {
- this.page = page;
- const nav = page.locator('nav');
- this.homeLink = nav.getByRole('link', { name: 'Home' });
- this.profileLink = nav.getByRole('link', { name: 'Profile' });
- this.searchInput = nav.getByPlaceholder('Search...');
- }
-
- async navigateToProfile() {
- await this.profileLink.click();
- }
-
- async search(query: string) {
- await this.searchInput.fill(query);
- await this.searchInput.press('Enter');
- }
-}
-```
-
-## User Interactions
-
-### Form Interactions
-
-```typescript
-test('form interactions', async ({ page }) => {
- await page.goto('/form');
-
- // Text inputs
- await page.fill('input[name="email"]', '[email protected]');
- await page.type('textarea[name="message"]', 'Hello', { delay: 100 });
-
- // Checkboxes
- await page.check('input[type="checkbox"][name="subscribe"]');
- await page.uncheck('input[type="checkbox"][name="spam"]');
-
- // Radio buttons
- await page.check('input[type="radio"][value="option1"]');
-
- // Select dropdowns
- await page.selectOption('select[name="country"]', 'US');
- await page.selectOption('select[name="color"]', { label: 'Blue' });
- await page.selectOption('select[name="size"]', { value: 'large' });
-
- // Multi-select
- await page.selectOption('select[multiple]', ['value1', 'value2']);
-
- // File uploads
- await page.setInputFiles('input[type="file"]', 'path/to/file.pdf');
- await page.setInputFiles('input[type="file"]', [
- 'file1.jpg',
- 'file2.jpg'
- ]);
-
- // Clear file input
- await page.setInputFiles('input[type="file"]', []);
-});
-```
-
-### Mouse and Keyboard
-
-```typescript
-test('mouse and keyboard interactions', async ({ page }) => {
- // Click variations
- await page.click('button');
- await page.dblclick('button'); // Double click
- await page.click('button', { button: 'right' }); // Right click
- await page.click('button', { modifiers: ['Shift'] }); // Shift+click
-
- // Hover
- await page.hover('.tooltip-trigger');
- await expect(page.locator('.tooltip')).toBeVisible();
-
- // Drag and drop
- await page.dragAndDrop('#draggable', '#droppable');
-
- // Keyboard
- await page.keyboard.press('Enter');
- await page.keyboard.press('Control+A');
- await page.keyboard.type('Hello World');
- await page.keyboard.down('Shift');
- await page.keyboard.press('ArrowDown');
- await page.keyboard.up('Shift');
-
- // Focus
- await page.focus('input[name="email"]');
- await page.fill('input[name="email"]', '[email protected]');
-});
-```
-
-### Waiting Strategies
-
-```typescript
-test('waiting strategies', async ({ page }) => {
- // Wait for element
- await page.waitForSelector('.dynamic-content');
- await page.waitForSelector('.modal', { state: 'visible' });
- await page.waitForSelector('.loading', { state: 'hidden' });
-
- // Wait for load state
- await page.waitForLoadState('load');
- await page.waitForLoadState('domcontentloaded');
- await page.waitForLoadState('networkidle');
-
- // Wait for URL
- await page.waitForURL('**/dashboard');
- await page.waitForURL(/\/product\/\d+/);
-
- // Wait for function
- await page.waitForFunction(() => {
- return document.querySelectorAll('.item').length > 5;
- });
-
- // Wait for timeout (avoid if possible)
- await page.waitForTimeout(1000);
-
- // Wait for event
- await page.waitForEvent('load');
- await page.waitForEvent('popup');
-});
-```
-
-## Assertions
-
-### Common Assertions
-
-```typescript
-import { test, expect } from '@playwright/test';
-
-test('assertions', async ({ page }) => {
- await page.goto('/dashboard');
-
- // Visibility
- await expect(page.locator('.header')).toBeVisible();
- await expect(page.locator('.loading')).toBeHidden();
- await expect(page.locator('.optional')).not.toBeVisible();
-
- // Text content
- await expect(page.locator('h1')).toHaveText('Dashboard');
- await expect(page.locator('h1')).toContainText('Dash');
- await expect(page.locator('.message')).toHaveText(/welcome/i);
-
- // Attributes
- await expect(page.locator('button')).toBeEnabled();
- await expect(page.locator('button')).toBeDisabled();
- await expect(page.locator('input')).toHaveAttribute('type', 'email');
- await expect(page.locator('input')).toHaveValue('[email protected]');
-
- // CSS
- await expect(page.locator('.button')).toHaveClass('btn-primary');
- await expect(page.locator('.button')).toHaveClass(/btn-/);
- await expect(page.locator('.element')).toHaveCSS('color', 'rgb(255, 0, 0)');
-
- // Count
- await expect(page.locator('.item')).toHaveCount(5);
-
- // URL and title
- await expect(page).toHaveURL('http://localhost:3000/dashboard');
- await expect(page).toHaveURL(/dashboard$/);
- await expect(page).toHaveTitle('Dashboard - My App');
- await expect(page).toHaveTitle(/Dashboard/);
-
- // Screenshot comparison
- await expect(page).toHaveScreenshot('dashboard.png');
- await expect(page.locator('.widget')).toHaveScreenshot('widget.png');
-});
-```
-
-### Custom Assertions
-
-```typescript
-test('custom matchers', async ({ page }) => {
- // Soft assertions (continue test on failure)
- await expect.soft(page.locator('.title')).toHaveText('Welcome');
- await expect.soft(page.locator('.subtitle')).toBeVisible();
-
- // Multiple elements
- const items = page.locator('.item');
- await expect(items).toHaveCount(3);
- await expect(items.nth(0)).toContainText('First');
- await expect(items.nth(1)).toContainText('Second');
-
- // Poll assertions
- await expect(async () => {
- const response = await page.request.get('/api/status');
- expect(response.ok()).toBeTruthy();
- }).toPass({
- timeout: 10000,
- intervals: [1000, 2000, 5000],
- });
-});
-```
-
-## Authentication Patterns
-
-### Storage State Pattern
-
-```typescript
-// auth.setup.ts - Run once to save auth state
-import { test as setup } from '@playwright/test';
-
-const authFile = 'playwright/.auth/user.json';
-
-setup('authenticate', async ({ page }) => {
- await page.goto('/login');
- await page.fill('[name="username"]', 'testuser');
- await page.fill('[name="password"]', 'password123');
- await page.click('button[type="submit"]');
-
- await page.waitForURL('/dashboard');
-
- // Save authentication state
- await page.context().storageState({ path: authFile });
-});
-
-// playwright.config.ts
-export default defineConfig({
- projects: [
- {
- name: 'setup',
- testMatch: /.*\.setup\.ts/,
- },
- {
- name: 'chromium',
- use: {
- ...devices['Desktop Chrome'],
- storageState: authFile,
- },
- dependencies: ['setup'],
- },
- ],
-});
-
-// tests/dashboard.spec.ts - Already authenticated
-test('view dashboard', async ({ page }) => {
- await page.goto('/dashboard');
- // Already logged in!
- await expect(page.locator('.user-menu')).toBeVisible();
-});
-```
-
-### Multiple User Roles
-
-```typescript
-// fixtures/auth.ts
-import { test as base } from '@playwright/test';
-
-type Fixtures = {
- adminPage: Page;
- userPage: Page;
-};
-
-export const test = base.extend<Fixtures>({
- adminPage: async ({ browser }, use) => {
- const context = await browser.newContext({
- storageState: 'playwright/.auth/admin.json',
- });
- const page = await context.newPage();
- await use(page);
- await context.close();
- },
-
- userPage: async ({ browser }, use) => {
- const context = await browser.newContext({
- storageState: 'playwright/.auth/user.json',
- });
- const page = await context.newPage();
- await use(page);
- await context.close();
- },
-});
-
-// tests/permissions.spec.ts
-import { test } from '../fixtures/auth';
-
-test('admin can access admin panel', async ({ adminPage }) => {
- await adminPage.goto('/admin');
- await expect(adminPage.locator('.admin-panel')).toBeVisible();
-});
-
-test('regular user cannot access admin panel', async ({ userPage }) => {
- await userPage.goto('/admin');
- await expect(userPage.locator('.access-denied')).toBeVisible();
-});
-```
-
-## Network Control
-
-### Request Mocking
-
-```typescript
-test('mock API responses', async ({ page }) => {
- // Mock API response
- await page.route('**/api/users', route => {
- route.fulfill({
- status: 200,
- contentType: 'application/json',
- body: JSON.stringify({
- users: [
- { id: 1, name: 'John Doe' },
- { id: 2, name: 'Jane Smith' },
- ],
- }),
- });
- });
-
- await page.goto('/users');
- await expect(page.locator('.user-list')).toContainText('John Doe');
-});
-
-test('mock with conditions', async ({ page }) => {
- await page.route('**/api/**', route => {
- const url = route.request().url();
-
- if (url.includes('/users/1')) {
- route.fulfill({
- status: 200,
- body: JSON.stringify({ id: 1, name: 'Test User' }),
- });
- } else if (url.includes('/users')) {
- route.fulfill({
- status: 200,
- body: JSON.stringify({ users: [] }),
- });
- } else {
- route.continue();
- }
- });
-});
-
-test('simulate network errors', async ({ page }) => {
- await page.route('**/api/data', route => {
- route.abort('failed');
- });
-
- await page.goto('/data');
- await expect(page.locator('.error-message')).toBeVisible();
-});
-```
-
-### Request Interception
-
-```typescript
-test('intercept and modify requests', async ({ page }) => {
- // Modify request headers
- await page.route('**/api/**', route => {
- const headers = route.request().headers();
- route.continue({
- headers: {
- ...headers,
- 'X-Custom-Header': 'test-value',
- },
- });
- });
-
- // Modify POST data
- await page.route('**/api/submit', route => {
- const postData = route.request().postDataJSON();
- route.continue({
- postData: JSON.stringify({
- ...postData,
- timestamp: Date.now(),
- }),
- });
- });
-});
-
-test('wait for API response', async ({ page }) => {
- // Wait for specific request
- const responsePromise = page.waitForResponse('**/api/users');
- await page.click('button#load-users');
- const response = await responsePromise;
-
- expect(response.status()).toBe(200);
- const data = await response.json();
- expect(data.users).toHaveLength(10);
-});
-```
-
-## Test Organization
-
-### Custom Fixtures
-
-```typescript
-// fixtures/todos.ts
-import { test as base } from '@playwright/test';
-
-type TodoFixtures = {
- todoPage: TodoPage;
- createTodo: (title: string) => Promise<void>;
-};
-
-export const test = base.extend<TodoFixtures>({
- todoPage: async ({ page }, use) => {
- const todoPage = new TodoPage(page);
- await todoPage.goto();
- await use(todoPage);
- },
-
- createTodo: async ({ page }, use) => {
- const create = async (title: string) => {
- await page.fill('.new-todo', title);
- await page.press('.new-todo', 'Enter');
- };
- await use(create);
- },
-});
-
-// tests/todos.spec.ts
-import { test } from '../fixtures/todos';
-
-test('can create new todo', async ({ todoPage, createTodo }) => {
- await createTodo('Buy groceries');
- await expect(todoPage.todoItems).toHaveCount(1);
- await expect(todoPage.todoItems).toHaveText('Buy groceries');
-});
-```
-
-### Test Tags and Filtering
-
-```typescript
-test('smoke test', { tag: '@smoke' }, async ({ page }) => {
- await page.goto('/');
- await expect(page).toHaveTitle('Home');
-});
-
-test('regression test', { tag: ['@regression', '@critical'] }, async ({ page }) => {
- // Complex test
-});
-
-// Run: npx playwright test --grep @smoke
-// Run: npx playwright test --grep-invert @slow
-```
-
-## Visual Testing
-
-### Screenshot Comparison
-
-```typescript
-test('visual regression', async ({ page }) => {
- await page.goto('/dashboard');
-
- // Full page screenshot
- await expect(page).toHaveScreenshot('dashboard.png', {
- maxDiffPixels: 100,
- });
-
- // Element screenshot
- await expect(page.locator('.widget')).toHaveScreenshot('widget.png');
-
- // Full page with scroll
- await expect(page).toHaveScreenshot('full-page.png', {
- fullPage: true,
- });
-
- // Mask dynamic elements
- await expect(page).toHaveScreenshot('masked.png', {
- mask: [page.locator('.timestamp'), page.locator('.avatar')],
- });
-
- // Custom threshold
- await expect(page).toHaveScreenshot('comparison.png', {
- maxDiffPixelRatio: 0.05, // 5% difference allowed
- });
-});
-```
-
-### Video and Trace
-
-```typescript
-// playwright.config.ts
-export default defineConfig({
- use: {
- video: 'retain-on-failure',
- trace: 'on-first-retry',
- screenshot: 'only-on-failure',
- },
-});
-
-// Programmatic video
-test('record video', async ({ page }) => {
- await page.goto('/');
- // Test actions...
-
- // Video saved automatically to test-results/
-});
-
-// View trace: npx playwright show-trace trace.zip
-```
-
-## Parallel Execution
-
-### Test Sharding
-
-```typescript
-// playwright.config.ts
-export default defineConfig({
- fullyParallel: true,
- workers: process.env.CI ? 4 : undefined,
-});
-
-// Run shards in CI
-// npx playwright test --shard=1/4
-// npx playwright test --shard=2/4
-// npx playwright test --shard=3/4
-// npx playwright test --shard=4/4
-```
-
-### Serial Tests
-
-```typescript
-test.describe.configure({ mode: 'serial' });
-
-test.describe('order matters', () => {
- let orderId: string;
-
- test('create order', async ({ page }) => {
- // Create order
- orderId = await createOrder(page);
- });
-
- test('verify order', async ({ page }) => {
- // Use orderId from previous test
- await verifyOrder(page, orderId);
- });
-});
-```
-
-## CI/CD Integration
-
-### GitHub Actions
-
-```yaml
-# .github/workflows/playwright.yml
-name: Playwright Tests
-on:
- push:
- branches: [main, master]
- pull_request:
- branches: [main, master]
-
-jobs:
- test:
- timeout-minutes: 60
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
-
- - uses: actions/setup-node@v4
- with:
- node-version: 20
-
- - name: Install dependencies
- run: npm ci
-
- - name: Install Playwright Browsers
- run: npx playwright install --with-deps
-
- - name: Run Playwright tests
- run: npx playwright test
-
- - uses: actions/upload-artifact@v4
- if: always()
- with:
- name: playwright-report
- path: playwright-report/
- retention-days: 30
-```
-
-### Docker
-
-```dockerfile
-FROM mcr.microsoft.com/playwright:v1.40.0-jammy
-
-WORKDIR /app
-
-COPY package*.json ./
-RUN npm ci
-
-COPY . .
-
-CMD ["npx", "playwright", "test"]
-```
-
-## Debugging
-
-### UI Mode
-
-```bash
-# Interactive debugging
-npx playwright test --ui
-
-# Debug specific test
-npx playwright test --debug login.spec.ts
-
-# Step through test
-npx playwright test --headed --slow-mo=1000
-```
-
-### Trace Viewer
-
-```typescript
-// Generate trace
-test('with trace', async ({ page }) => {
- await page.context().tracing.start({ screenshots: true, snapshots: true });
-
- // Test actions
- await page.goto('/');
-
- await page.context().tracing.stop({ path: 'trace.zip' });
-});
-
-// View: npx playwright show-trace trace.zip
-```
-
-### Console Logs
-
-```typescript
-test('capture console', async ({ page }) => {
- page.on('console', msg => console.log(`Browser: ${msg.text()}`));
- page.on('pageerror', error => console.error(`Error: ${error.message}`));
-
- await page.goto('/');
-});
-```
-
-## Best Practices
-
-### 1. Use Stable Locators
-```typescript
-// ✅ Good - Role-based, stable
-await page.getByRole('button', { name: 'Submit' }).click();
-await page.getByLabel('Email').fill('[email protected]');
-
-// ❌ Bad - Fragile, implementation-dependent
-await page.click('button.btn-primary.submit-btn');
-await page.fill('div > form > input:nth-child(3)');
-```
-
-### 2. Leverage Auto-Waiting
-```typescript
-// ✅ Good - Auto-waits
-await page.click('button');
-await expect(page.locator('.result')).toBeVisible();
-
-// ❌ Bad - Manual waits
-await page.waitForTimeout(2000);
-await page.click('button');
-```
-
-### 3. Use Page Object Model
-```typescript
-// ✅ Good - Reusable, maintainable
-const loginPage = new LoginPage(page);
-await loginPage.login('user', 'pass');
-
-// ❌ Bad - Duplicated selectors
-await page.fill('[name="username"]', 'user');
-await page.fill('[name="password"]', 'pass');
-```
-
-### 4. Parallel-Safe Tests
-```typescript
-// ✅ Good - Isolated
-test('user signup', async ({ page }) => {
- const uniqueEmail = `user-${Date.now()}@test.com`;
- await signUp(page, uniqueEmail);
-});
-
-// ❌ Bad - Shared state
-test('user signup', async ({ page }) => {
- await signUp(page, '[email protected]'); // Conflicts in parallel
-});
-```
-
-### 5. Handle Flakiness
-```typescript
-// ✅ Good - Wait for network idle
-await page.goto('/', { waitUntil: 'networkidle' });
-await expect(page.locator('.data')).toBeVisible();
-
-// Configure retries
-test.describe(() => {
- test.use({ retries: 2 });
-
- test('flaky test', async ({ page }) => {
- // Test with auto-retry
- });
-});
-```
-
-## Common Patterns
-
-### Multi-Page Scenarios
-
-```typescript
-test('popup handling', async ({ page, context }) => {
- // Listen for new page
- const popupPromise = context.waitForEvent('page');
- await page.click('a[target="_blank"]');
- const popup = await popupPromise;
-
- await popup.waitForLoadState();
- await expect(popup).toHaveTitle('New Window');
- await popup.close();
-});
-```
-
-### Conditional Logic
-
-```typescript
-test('handle optional elements', async ({ page }) => {
- await page.goto('/');
-
- // Close modal if present
- const modal = page.locator('.modal');
- if (await modal.isVisible()) {
- await page.click('.modal .close-button');
- }
-
- // Or use count
- const cookieBanner = page.locator('.cookie-banner');
- if ((await cookieBanner.count()) > 0) {
- await page.click('.accept-cookies');
- }
-});
-```
-
-### Data-Driven Tests
-
-```typescript
-const testCases = [
- { input: 'hello', expected: 'HELLO' },
- { input: 'World', expected: 'WORLD' },
- { input: '123', expected: '123' },
-];
-
-for (const { input, expected } of testCases) {
- test(`transforms "${input}" to "${expected}"`, async ({ page }) => {
- await page.goto('/transform');
- await page.fill('input', input);
- await page.click('button');
- await expect(page.locator('.result')).toHaveText(expected);
- });
-}
-```
-
-## Resources
-
-- [Playwright Documentation](https://playwright.dev/)
-- [Best Practices Guide](https://playwright.dev/docs/best-practices)
-- [API Reference](https://playwright.dev/docs/api/class-playwright)
-- [GitHub Examples](https://github.com/microsoft/playwright/tree/main/examples)
diff --git a/.claude/rules/linear-tickets.md b/.claude/rules/linear-tickets.md
index ba5accc6083..3d3d10d4639 100644
--- a/.claude/rules/linear-tickets.md
+++ b/.claude/rules/linear-tickets.md
@@ -10,5 +10,5 @@ All teams (ENG, AI, SRE, GIT, PIE):
ENG tickets only:
- **Status is the approval.** In Triage or Backlog the scope is not approved: plan freely, but ask before the first code change, and continue if the engineer says "start anyway". Never move a ticket out of Backlog yourself. On Todo, move the ticket to In Progress at the first code change, and set the engineer as assignee if there is none.
- **Plan ends on the ticket.** Write the plan into `Acceptance criteria` and `Decisions`. Edit one section with `save_issue` `patch` from its heading to the next heading; for the last section (`Decisions`), use `append` or an anchor inside the section. Never rewrite the whole description, so a human edit made at the same time survives. A missing heading means the ticket was made without a template: add the heading, do not treat the section as empty. A changed decision also gets a one-line comment that says what changed.
-- **E2E marker.** Each acceptance criterion ends with ``E2E: `<test title>` `` or `no-e2e: <reason>`. Set the `no-e2e` label when any bullet has `no-e2e:`, and remove it when none does.
+- **E2E marker.** Each acceptance criterion ends with ``E2E: `<test title>` `` or `no-e2e: <reason>`. Set the `no-e2e` label when any bullet has `no-e2e:`, and remove it when none does. Write each E2E test by `packages/tests-e2e/README.md`.
- **Create** a ticket in ENG unless the engineer names a different team, with `save_issue` and `template: "Feature"` or `"Bug"`. That path strips the placeholder text, so fill each section as the brain page describes.
diff --git a/brain/knowledge/engineering/e2e-tests-and-monitors.md b/brain/knowledge/engineering/e2e-tests-and-monitors.md
index 649d3ea404f..c857d064554 100644
--- a/brain/knowledge/engineering/e2e-tests-and-monitors.md
+++ b/brain/knowledge/engineering/e2e-tests-and-monitors.md
@@ -6,13 +6,16 @@ icon: 🎭
One Playwright suite in `packages/tests-e2e` feeds three consumers that fail independently: CI on a fresh throwaway instance, Checkly monitors against **production Cloud**, and a single BetterStack monitor. A change that only breaks one of them looks green everywhere else, so it is worth knowing which reads what.
+How to write a spec lives in one place: "Writing a spec" in `packages/tests-e2e/README.md`. This page holds the gotchas only.
+
**Local / CI suite** — `playwright.config.ts` runs every `scenarios/**/*.spec.ts`; `AP_EDITION=ce` skips `scenarios/ee`. Boots the whole stack itself via the `webServer` block unless `E2E_INSTANCE_URL` is set. Global setup signs in once and saves the session to `.auth/user.json`, so specs start signed in.
-**Checkly monitors** — `checkly.config.ts` picks up the *same* `**/scenarios/**/*.spec.ts` files and runs them every 10 minutes with `baseURL: https://cloud.activepieces.com`, signing in with `E2E_EMAIL` / `E2E_PASSWORD`.
+**Checkly monitors** — `checkly.config.ts` picks up the *same* `**/scenarios/**/*.spec.ts` files and runs them every 10 minutes with `baseURL: https://cloud.activepieces.com`, signing in with `E2E_EMAIL` / `E2E_PASSWORD`. It runs the specs from its last deploy, not the specs on `main`.
**BetterStack monitor** — one standalone file, `scenarios/betterstack/*.flat.spec.js`.
## Gotchas
- **Checkly runs the shared specs against production Cloud**, so anything the page objects assume about the login screen has to hold on Cloud too — not just on the SMTP-less instance CI boots. This is the usual reason a UI change breaks the monitors but not CI. See the auth-card gotchas on [CE Authentication](../connections-auth/ce-authentication.md).
+- **Checkly deploys only by hand.** The `Deploy-Checkly` job in `emergency-cloud-deploy.yml` is the only path; nothing has called `e2e-tests-checkly.yml` since September 2025, and the `test-checkly` / `deploy-checkly` scripts are gone. A spec merged to `main` is not monitored until someone runs that workflow.
- **BetterStack does not read the repo — the repo pushes to it.** `.github/workflows/sync-betterstack-playwright.yml` fires on push to `main` and `PATCH`es the file's contents into the hardcoded monitor `4211060` as `playwright_script`. One-way and `main`-only: the monitor updates at *merge*, never on the PR, and any edit made in the BetterStack UI is silently overwritten by the next push.
- **The BetterStack file is deliberately flat and duplicated.** BetterStack executes one self-contained script, so it cannot `require` the `pages/` objects — its sign-in is a copy. Fix the page object and you have *not* fixed the monitor; both files need the change.
- **The `.flat.spec.js` runs nowhere else.** Playwright matches `*.spec.ts` and Checkly matches `**/scenarios/**/*.spec.ts`, so a broken flat file is invisible locally and in CI until it fails in BetterStack.
diff --git a/bun.lock b/bun.lock
index 1340c36f258..c8b9c369c87 100644
--- a/bun.lock
+++ b/bun.lock
@@ -11331,6 +11331,7 @@
"@playwright/test": "1.64.1",
"checkly": "6.9.10",
"dotenv": "17.2.3",
+ "eslint-plugin-playwright": "2.12.1",
},
},
"packages/web": {
@@ -16023,6 +16024,8 @@
"eslint-plugin-jsx-a11y": ["[email protected]", "", { "dependencies": { "@babel/runtime": "^7.20.7", "aria-query": "^5.1.3", "array-includes": "^3.1.6", "array.prototype.flatmap": "^1.3.1", "ast-types-flow": "^0.0.7", "axe-core": "^4.6.2", "axobject-query": "^3.1.1", "damerau-levenshtein": "^1.0.8", "emoji-regex": "^9.2.2", "has": "^1.0.3", "jsx-ast-utils": "^3.3.3", "language-tags": "=1.0.5", "minimatch": "^3.1.2", "object.entries": "^1.1.6", "object.fromentries": "^2.0.6", "semver": "^6.3.0" }, "peerDependencies": { "eslint": "^3 || ^4 || ^5 || ^6 || ^7 || ^8" } }, "sha512-63Bog4iIethyo8smBklORknVjB0T2dwB8Mr/hIC+fBS0uyHdYYpzM/Ed+YC8VxTjlXHEWFOdmgwcDn1U2L9VCA=="],
+ "eslint-plugin-playwright": ["[email protected]", "", { "dependencies": { "globals": "^17.3.0" }, "peerDependencies": { "eslint": ">=8.40.0" } }, "sha512-JhavNuYBIq7SE/Zg9Esf9RGHkXoDbzrEKS383cowzMyvZ4JMbmdNCN6CoW+RU1Cjzwcoea1tou0je58uXmT/9A=="],
+
"eslint-plugin-prettier": ["[email protected]", "", { "dependencies": { "prettier-linter-helpers": "^1.0.0" }, "peerDependencies": { "eslint": ">=7.28.0", "prettier": ">=2.0.0" } }, "sha512-f/0rXLXUt0oFYs8ra4w49wYZBG5GKZpAYsJSm6rnYL5uVDjd+zowwMwVZHnAjf4edNrKpCDYfXDgmRE/Ak7QyQ=="],
"eslint-plugin-react": ["[email protected]", "", { "dependencies": { "array-includes": "^3.1.6", "array.prototype.flatmap": "^1.3.1", "array.prototype.tosorted": "^1.1.1", "doctrine": "^2.1.0", "estraverse": "^5.3.0", "jsx-ast-utils": "^2.4.1 || ^3.0.0", "minimatch": "^3.1.2", "object.entries": "^1.1.6", "object.fromentries": "^2.0.6", "object.hasown": "^1.1.2", "object.values": "^1.1.6", "prop-types": "^15.8.1", "resolve": "^2.0.0-next.4", "semver": "^6.3.0", "string.prototype.matchall": "^4.0.8" }, "peerDependencies": { "eslint": "^3 || ^4 || ^5 || ^6 || ^7 || ^8" } }, "sha512-t2fBMa+XzonrrNkyVirzKlvn5RXzzPwRHtMvLAtVZrt8oxgnTQaYbU6SXTOO1mwQgp1y5+toMSKInnzGr0Knqg=="],
@@ -19719,6 +19722,8 @@
"eslint-plugin-jsx-a11y/semver": ["[email protected]", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="],
+ "eslint-plugin-playwright/globals": ["[email protected]", "", {}, "sha512-RwMTC61u7hrG3ZJMkZQOK4JLJrKS8QtoTii63EmWvFXHqycY9FObBJQCbsLxSO8kjKhWE5kV1GC+Vb1g3RI0Zg=="],
+
"eslint-plugin-react/doctrine": ["[email protected]", "", { "dependencies": { "esutils": "^2.0.2" } }, "sha512-35mSku4ZXK0vfCuHEDAwt55dg2jNajHZ1odvF+8SSr82EsZY4QmXfuWso8oEd8zRhVObSN18aM0CjSdoBX7zIw=="],
"eslint-plugin-react/semver": ["[email protected]", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="],
diff --git a/docs/handbook/engineering/playbooks/e2e-tests.mdx b/docs/handbook/engineering/playbooks/e2e-tests.mdx
index b78ad48db09..a2db1ddcf57 100644
--- a/docs/handbook/engineering/playbooks/e2e-tests.mdx
+++ b/docs/handbook/engineering/playbooks/e2e-tests.mdx
@@ -4,441 +4,14 @@ icon: 'clipboard-list'
---
<Info>
-This playbook is specifically about **full-stack browser E2E tests using Playwright** in `packages/tests-e2e/`. For the canonical 4-layer testing taxonomy across the whole monorepo (unit / integration / e2e / smoke) — and to decide whether your test actually belongs at the E2E layer at all — see the [Testing Strategy playbook](/handbook/engineering/playbooks/testing-strategy).
+This playbook is about **full-stack browser E2E tests using Playwright** in `packages/tests-e2e/`. To decide whether your test belongs at the E2E layer at all, see the [Testing Strategy playbook](/handbook/engineering/playbooks/testing-strategy).
</Info>
-## Overview
+## Writing a spec
-Our **full-stack browser E2E** suite uses Playwright to ensure critical user workflows function correctly across the application. The tests are organized using the Page Object Model pattern to maintain clean, reusable, and maintainable test code. This playbook outlines the structure, conventions, and best practices for writing e2e tests.
+The guidelines for writing a spec live in one place: the [`packages/tests-e2e` README](https://github.com/activepieces/activepieces/blob/main/packages/tests-e2e/README.md#writing-a-spec). It covers test data, selectors, waits, assertions, flaky specs and how to run the suite. This page does not repeat it.
-## Project Structure
+## Debugging a failed spec
-```
-packages/tests-e2e/
-├── scenarios/ # Test files (*.spec.ts)
-├── pages/ # Page Object Models
-│ ├── base.ts # Base page class
-│ ├── index.ts # Page exports
-│ ├── authentication.page.ts
-│ ├── builder.page.ts
-│ ├── flows.page.ts
-│ └── agent.page.ts
-├── helper/ # Utilities and configuration
-│ └── config.ts # Environment configuration
-├── playwright.config.ts # Playwright configuration
-└── project.json # Nx project configuration
-```
-
-This playbook provides a comprehensive guide for writing e2e tests following the established patterns in your codebase. It covers the Page Object Model structure, test organization, configuration management, and best practices for maintaining reliable e2e tests.
-
-## Page Object Model Pattern
-
-### Base Page Structure
-
-All page objects extend the `BasePage` class and follow a consistent structure:
-
-```typescript
-export class YourPage extends BasePage {
- url = `${configUtils.getConfig().instanceUrl}/your-path`;
-
- getters = {
- // Locator functions that return page elements
- elementName: (page: Page) => page.getByRole('button', { name: 'Button Text' }),
- };
-
- actions = {
- // Action functions that perform user interactions
- performAction: async (page: Page, params: { param1: string }) => {
- // Implementation
- },
- };
-}
-```
-
-### Page Object Guidelines
-
-#### ❌ Don't do
-```typescript
-// Direct element selection in test files
-test('should create flow', async ({ page }) => {
- await page.getByRole('button', { name: 'Create Flow' }).click();
- await page.getByText('From scratch').click();
- // Test logic mixed with element selection
-});
-```
-
-#### ✅ Do
-```typescript
-// flows.page.ts
-export class FlowsPage extends BasePage {
- getters = {
- createFlowButton: (page: Page) => page.getByRole('button', { name: 'Create Flow' }),
- fromScratchButton: (page: Page) => page.getByText('From scratch'),
- };
-
- actions = {
- newFlowFromScratch: async (page: Page) => {
- await this.getters.createFlowButton(page).click();
- await this.getters.fromScratchButton(page).click();
- },
- };
-}
-
-// integration.spec.ts
-test('should create flow', async ({ page }) => {
- await flowsPage.actions.newFlowFromScratch(page);
- // Clean test logic focused on behavior
-});
-```
-
-## Test Organization
-
-### Test File Structure
-
-Test files should be organized by feature or workflow:
-
-```typescript
-import { test, expect } from '@playwright/test';
-import {
- AuthenticationPage,
- FlowsPage,
- BuilderPage
-} from '../pages';
-import { configUtils } from '../helper/config';
-
-test.describe('Feature Name', () => {
- let authenticationPage: AuthenticationPage;
- let flowsPage: FlowsPage;
- let builderPage: BuilderPage;
-
- test.beforeEach(async () => {
- // Initialize page objects
- authenticationPage = new AuthenticationPage();
- flowsPage = new FlowsPage();
- builderPage = new BuilderPage();
- });
-
- test('should perform specific workflow', async ({ page }) => {
- // Test implementation
- });
-});
-```
-
-### Test Naming Conventions
-
-- Use descriptive test names that explain the expected behavior
-- Follow the pattern: `should [action] [expected result]`
-- Include context when relevant
-
-```typescript
-// Good test names
-test('should send Slack message via flow', async ({ page }) => {});
-test('should handle webhook with dynamic parameters', async ({ page }) => {});
-test('should authenticate user with valid credentials', async ({ page }) => {});
-
-// Avoid vague names
-test('should work', async ({ page }) => {});
-test('test flow', async ({ page }) => {});
-```
-
-## Configuration Management
-
-### Environment Configuration
-
-Use the centralized config utility to handle different environments:
-
-```typescript
-// helper/config.ts
-export const configUtils = {
- getConfig: (): Config => {
- return process.env.E2E_INSTANCE_URL ? prodConfig : localConfig;
- },
-};
-
-// Usage in pages
-export class AuthenticationPage extends BasePage {
- url = `${configUtils.getConfig().instanceUrl}/sign-in`;
-}
-```
-
-### Environment Variables
-
-Required environment variables for CI/CD:
-- `E2E_INSTANCE_URL`: Target application URL
-- `E2E_EMAIL`: Test user email
-- `E2E_PASSWORD`: Test user password
-
-## Writing Effective Tests
-
-### Test Structure
-
-Follow this pattern for comprehensive tests:
-
-```typescript
-test('should complete user workflow', async ({ page }) => {
- // 1. Set up test data and timeouts
- test.setTimeout(120000);
- const config = configUtils.getConfig();
-
- // 2. Authentication (if required)
- await authenticationPage.actions.signIn(page, {
- email: config.email,
- password: config.password
- });
-
- // 3. Navigate to relevant page
- await flowsPage.actions.navigate(page);
-
- // 4. Clean up existing data (if needed)
- await flowsPage.actions.cleanupExistingFlows(page);
-
- // 5. Perform the main workflow
- await flowsPage.actions.newFlowFromScratch(page);
- await builderPage.actions.waitFor(page);
- await builderPage.actions.selectInitialTrigger(page, {
- piece: 'Schedule',
- trigger: 'Every Hour'
- });
-
- // 6. Add assertions and validations
- await builderPage.actions.testFlowAndWaitForSuccess(page);
-
- // 7. Clean up (if needed)
- await builderPage.actions.exitRun(page);
-});
-```
-
-### Wait Strategies
-
-Use appropriate wait strategies instead of fixed timeouts:
-
-```typescript
-// Good - Wait for specific conditions
-await page.waitForURL('**/flows/**');
-await page.waitForSelector('.react-flow__nodes', { state: 'visible' });
-await page.waitForFunction(() => {
- const element = document.querySelector('.target-element');
- return element && element.textContent?.includes('Expected Text');
-}, { timeout: 10000 });
-
-// Avoid - Fixed timeouts
-await page.waitForTimeout(5000);
-```
-
-### Error Handling
-
-Implement proper error handling and cleanup:
-
-```typescript
-test('should handle errors gracefully', async ({ page }) => {
- try {
- await flowsPage.actions.navigate(page);
- // Test logic
- } catch (error) {
- // Log error details
- console.error('Test failed:', error);
- // Take screenshot for debugging
- await page.screenshot({ path: 'error-screenshot.png' });
- throw error;
- } finally {
- // Clean up resources
- await flowsPage.actions.cleanupExistingFlows(page);
- }
-});
-```
-
-## Best Practices
-
-### Element Selection
-
-Prefer semantic selectors over CSS selectors:
-
-```typescript
-// Good - Semantic selectors
-getters = {
- createButton: (page: Page) => page.getByRole('button', { name: 'Create Flow' }),
- emailField: (page: Page) => page.getByPlaceholder('[email protected]'),
- searchInput: (page: Page) => page.getByRole('textbox', { name: 'Search' }),
-};
-
-// Avoid - Fragile CSS selectors
-getters = {
- createButton: (page: Page) => page.locator('button.btn-primary'),
- emailField: (page: Page) => page.locator('input[type="email"]'),
-};
-```
-
-### Test Data Management
-
-Use dynamic test data to avoid conflicts:
-
-```typescript
-// Good - Dynamic test data
-const runVersion = Math.floor(Math.random() * 100000);
-const uniqueFlowName = `Test Flow ${Date.now()}`;
-
-// Avoid - Static test data
-const flowName = 'Test Flow';
-```
-
-### Assertions
-
-Use meaningful assertions that verify business logic:
-
-```typescript
-// Good - Business logic assertions
-await builderPage.actions.testFlowAndWaitForSuccess(page);
-const response = await apiRequest.get(urlWithParams);
-const body = await response.json();
-expect(body.targetRunVersion).toBe(runVersion.toString());
-
-// Avoid - Implementation details
-expect(await page.locator('.success-message').isVisible()).toBe(true);
-```
-
-## Running Tests
-
-### Local Development & Debugging with Checkly
-
-We use [Checkly](https://checklyhq.com/) to run and debug E2E tests. Checkly provides video recordings for each test run, making it easy to debug failures.
-
-```bash
-# Run tests with Checkly (includes video reporting)
-npx turbo run test-checkly --filter=tests-e2e
-```
-
-- Test results, including video recordings, are available in the Checkly dashboard.
-- You can debug failed tests by reviewing the video and logs provided by Checkly.
-
-### Deploying Tests
-
-Manual deployment is rarely needed, but you can trigger it with:
-
-```bash
-npx turbo run deploy-checkly --filter=tests-e2e
-```
-
-<Info>
-Tests are deployed to Checkly automatically after successful test runs in the CI pipeline.
-</Info>
-
-## Debugging Tests
-
-### 1. Checkly Videos and Reports
-
-When running tests with Checkly, each test execution is recorded and detailed reports are generated. This is the fastest way to debug failures:
-
-- **Video recordings**: Watch the exact browser session for any test run.
-- **Step-by-step logs**: Review detailed logs and screenshots for each test step.
-- **Access**: Open the Checkly dashboard and navigate to the relevant test run to view videos and reports.
-
-### 2. VSCode Extension
-
-For the best local debugging experience, install the **Playwright Test for VSCode** extension:
-
-1. Open VSCode Extensions (Ctrl+Shift+X)
-2. Search for "Playwright Test for VSCode"
-3. Install the extension by Microsoft
-
-**Benefits:**
-- Debug tests directly in VSCode with breakpoints
-- Step-through test execution
-- View test results and traces in the Test Explorer
-- Auto-completion for Playwright APIs
-- Integrated test runner
-
-### 3. Debugging Tips
-
-1. **Use Checkly dashboard**: Review videos and logs for failed tests.
-2. **Use VSCode Extension**: Set breakpoints directly in your test files.
-3. **Step Through**: Use F10 (step over) and F11 (step into) in debug mode.
-4. **Inspect Elements**: Use `await page.pause()` to pause execution and inspect the page.
-5. **Console Logs**: Add `console.log()` statements to track execution flow.
-6. **Manual Screenshots**: Take screenshots at critical points for visual debugging.
-
-```typescript
-test('should debug workflow', async ({ page }) => {
- await page.goto('/flows');
-
- // Pause execution for manual inspection
- await page.pause();
-
- // Take screenshot for debugging
- await page.screenshot({ path: 'debug-screenshot.png' });
-
- // Continue with test logic
- await flowsPage.actions.newFlowFromScratch(page);
-});
-```
-
-## Common Patterns
-
-### Authentication Flow
-
-```typescript
-test('should authenticate user', async ({ page }) => {
- const config = configUtils.getConfig();
-
- await authenticationPage.actions.signIn(page, {
- email: config.email,
- password: config.password
- });
-
- await agentPage.actions.waitFor(page);
-});
-```
-
-### Flow Creation and Testing
-
-```typescript
-test('should create and test flow', async ({ page }) => {
- await flowsPage.actions.navigate(page);
- await flowsPage.actions.cleanupExistingFlows(page);
- await flowsPage.actions.newFlowFromScratch(page);
-
- await builderPage.actions.waitFor(page);
- await builderPage.actions.selectInitialTrigger(page, {
- piece: 'Schedule',
- trigger: 'Every Hour'
- });
-
- await builderPage.actions.testFlowAndWaitForSuccess(page);
-});
-```
-
-### API Integration Testing
-
-```typescript
-test('should handle webhook integration', async ({ page }) => {
- const apiRequest = await page.context().request;
- const response = await apiRequest.get(urlWithParams);
- const body = await response.json();
-
- expect(body.targetRunVersion).toBe(expectedValue);
-});
-```
-
-## Maintenance Guidelines
-
-### Updating Selectors
-
-When UI changes occur:
-1. Update page object getters with new selectors
-2. Test the changes locally
-3. Update related tests if necessary
-4. Ensure all tests pass before merging
-
-### Adding New Tests
-
-1. Create or update relevant page objects
-2. Write test scenarios in appropriate spec files
-3. Follow the established patterns and conventions
-4. Add proper error handling and cleanup
-5. Test locally before submitting
-
-### Performance Considerations
-
-- Keep tests focused and avoid unnecessary steps
-- Use appropriate timeouts (not too short, not too long)
-- Clean up test data to avoid conflicts
-- Group related tests in the same describe block
+- **Trace viewer**: the E2E matrix uploads `test-results/` as a run artifact, and a spec that failed and retried holds a `trace.zip`. Open it with `npx playwright show-trace <trace.zip>` to see each action, the DOM and the network calls.
+- **VS Code**: the **Playwright Test for VSCode** extension (by Microsoft) runs a single spec with breakpoints and shows its trace.
diff --git a/docs/handbook/engineering/playbooks/testing-strategy.mdx b/docs/handbook/engineering/playbooks/testing-strategy.mdx
index 0edc46016ac..9036b4d72ce 100644
--- a/docs/handbook/engineering/playbooks/testing-strategy.mdx
+++ b/docs/handbook/engineering/playbooks/testing-strategy.mdx
@@ -83,5 +83,5 @@ This list is a backlog, not a to-do list for this playbook. Tackle items opportu
## Related documentation
-- [E2E Tests playbook](/handbook/engineering/playbooks/e2e-tests) — full-stack browser E2E deep dive
+- [E2E Tests playbook](/handbook/engineering/playbooks/e2e-tests) — where the E2E guidelines live, and how to debug a failed spec
- Per-package READMEs live at `packages/server/api/test/README.md`, `packages/server/engine/test/README.md`, `packages/server/worker/test/README.md`, `packages/tests-e2e/README.md`, and `smoke-test/README.md`.
diff --git a/packages/server/api/test/README.md b/packages/server/api/test/README.md
index d2d69f76821..cf73fd84827 100644
--- a/packages/server/api/test/README.md
+++ b/packages/server/api/test/README.md
@@ -46,4 +46,4 @@ This list is a backlog, not a to-do for this README. Items should be tackled opp
## Related documentation
- Canonical taxonomy: `../../../../docs/handbook/engineering/playbooks/testing-strategy.mdx` (repo-relative path — works from GitHub UI before the handbook PR is deployed).
-- Full-stack browser E2E (not covered here): `../../../../docs/handbook/engineering/playbooks/e2e-tests.mdx`
+- Full-stack browser E2E (not covered here): `../../../tests-e2e/README.md`
diff --git a/packages/tests-e2e/.eslintrc.json b/packages/tests-e2e/.eslintrc.json
index 9d9c0db55bb..d379c6a7c1e 100644
--- a/packages/tests-e2e/.eslintrc.json
+++ b/packages/tests-e2e/.eslintrc.json
@@ -8,7 +8,22 @@
},
{
"files": ["*.ts", "*.tsx"],
- "rules": {}
+ "extends": ["plugin:playwright/recommended"],
+ "rules": {
+ "playwright/no-wait-for-timeout": "error",
+ "playwright/no-wait-for-selector": "error",
+ "playwright/no-force-option": "error",
+ "playwright/no-element-handle": "error",
+ "playwright/no-page-pause": "error",
+ "playwright/no-eval": "error",
+ "playwright/no-conditional-in-test": "error",
+ "playwright/no-conditional-expect": "error",
+ "playwright/no-raw-locators": "error",
+ "playwright/no-nth-methods": "warn",
+ "playwright/no-skipped-test": ["error", { "allowConditional": true }],
+ "playwright/require-annotation-reason": "error",
+ "playwright/expect-expect": ["error", { "assertFunctionPatterns": ["^expect[A-Z]"] }]
+ }
},
{
"files": ["*.js", "*.jsx"],
diff --git a/packages/tests-e2e/CLAUDE.md b/packages/tests-e2e/CLAUDE.md
new file mode 100644
index 00000000000..99ee350f47d
--- /dev/null
+++ b/packages/tests-e2e/CLAUDE.md
@@ -0,0 +1 @@
diff --git a/packages/tests-e2e/README.md b/packages/tests-e2e/README.md
index 14ca518b409..45bf6d144e9 100644
--- a/packages/tests-e2e/README.md
+++ b/packages/tests-e2e/README.md
@@ -17,17 +17,44 @@ packages/tests-e2e/
## Writing a spec
-A spec must pass whatever the project already holds: nothing, many flows, or another run's flows.
+This section is the one set of E2E guidelines, for people and agents. Other pages link here instead of repeating it.
+
+Staging runs the CE specs in a shared, long-lived account after each deploy, and CI runs specs in parallel in one project, so a spec must pass whatever the project already holds: nothing, many flows, or another run's flows. It must never change data it did not create.
+
+Start from `scenarios/flows/flow-journey.spec.ts` and `pages/builder.page.ts`. Locators and actions live in a page object under `pages/`; the spec holds the steps and the expected outcomes.
+
+### Data
- Create what the spec needs, and name each flow and connection with `e2eFlows.uniqueName()` (prefix `E2E `).
- Pass each flow id to `flowCleanup.track()` and each connection id to `connectionCleanup.track()`. The fixtures delete them after the test, also when the test fails.
+- Delete only by the ids your test created. Never delete by a query, a name, or a prefix.
- Find your data by its id or unique name, never by its position in a list.
-- Wait for a visible result (a status text, a step output, a response), never for a fixed time.
-- Select by role and accessible name (`getByRole(…, { name })`, `getByLabel`). When an element has no name, give it one in the web app (a linked label or an `aria-label`); use a `data-testid` only when no name fits. Never locate a field by the text around it or by its position.
- The tests start signed in. Use `test.use({ storageState: { cookies: [], origins: [] } })` to start signed out.
- Use only pieces that need no outside service. A spec that needs a server calls a flow of its own, as `connection-step.spec.ts` does.
- A new piece must also go into the `PIECES` list of `e2e-matrix.yml`, because the matrix stack installs only those.
+### Selecting elements
+
+- Select by test id (`getByTestId`). When the element has none, add a `data-testid` in `packages/web` in the same PR. Name it in kebab-case, area first: `sign-in-email`, `test-step-button`.
+- Use role and accessible name (`getByRole(…, { name })`, `getByLabel`) only where a test id does not fit, such as a row you find by the unique name of your data.
+- Never select by a CSS class, XPath, the text around a field, or a position (`nth()`, `first()`, `last()`).
+- A test id does not replace a role and an accessible name. When you add one, also follow "Accessible markup" in `packages/web/AGENTS.md`.
+
+### Waiting and asserting
+
+- Wait for a visible result with a web-first assertion (`await expect(locator).toBeVisible()`), or for the API call with `page.waitForResponse()`. Never wait for a fixed time, `waitForSelector`, or `networkidle`.
+- Assert the outcome the ticket asks for, such as a run status, a step output, or a webhook response. A visible page or button is not an outcome.
+- Keep `if` out of specs. A spec runs the same steps on every run, so a green run always proves the same thing.
+- A page object may branch only when the app can correctly be in either of two states and the spec cannot choose which, such as an empty project or one with flows (see `AutomationsPage.createFlow`). Never branch to work around a bug or a flaky step: that hides the failure the spec exists to catch.
+- Name a page object method that asserts `expect…`, so lint counts it as an assertion.
+
+### Proving the spec
+
+- Run the spec. Show it fail without your change (or against the bug), then pass with it. Put both results in the PR.
+- When a spec fails, find the cause first: the app, the spec, or the environment. Never loosen a locator, remove an assertion, or change an expected value to match what the screen shows now.
+- Never skip a spec with `test.skip` or `test.fixme` to get a green run. A flaky spec follows [Flaky specs](#flaky-specs).
+- `npx turbo run lint --filter=tests-e2e` checks specs, page objects and fixtures with `eslint-plugin-playwright`.
+
## Flaky specs
A spec is flaky when it fails and then passes on a retry, or fails with no product bug. The run summary counts flaky specs.
@@ -42,12 +69,6 @@ A spec is flaky when it fails and then passes on a retry, or fails with no produ
```bash
# Local Playwright run
npm run test:e2e
-
-# Checkly (video recordings — preferred for debugging)
-npx turbo run test-checkly --filter=tests-e2e
-
-# Deploy to Checkly (CI does this automatically)
-npx turbo run deploy-checkly --filter=tests-e2e
```
## Environment
@@ -65,9 +86,10 @@ PRs do not run E2E unless they change the suite or carry the `ready-for-e2e` lab
- `.github/workflows/e2e-matrix.yml` — the only E2E suite that gates a release: builds the image and runs the suite in four CE/EE configurations. Runs on PRs that touch `packages/tests-e2e/**`, and from the Cloud gate.
- `.github/workflows/e2e.yml` — runs `e2e-matrix.yml` on a PR with the `ready-for-e2e` label, and again on each push while the label stays
- `.github/workflows/continuous-delivery-stg.yml` — runs the CE specs against staging after each deploy
-- `.github/workflows/e2e-tests-checkly.yml` — scheduled Checkly run (optional)
+- `.github/workflows/emergency-cloud-deploy.yml` — the only workflow that deploys the specs to Checkly, by hand. Nothing calls `e2e-tests-checkly.yml`, so Checkly runs the specs from its last deploy, not the specs on `main`.
## Related documentation
-- Deep dive: `../../docs/handbook/engineering/playbooks/e2e-tests.mdx` — Page Object Model patterns, selector guidelines, debugging with Checkly, VSCode extension.
+- Debugging a failed spec: `../../docs/handbook/engineering/playbooks/e2e-tests.mdx`
+- Gotchas of the suite, Checkly and BetterStack: `../../brain/knowledge/engineering/e2e-tests-and-monitors.md`
- Canonical taxonomy: `../../docs/handbook/engineering/playbooks/testing-strategy.mdx`
diff --git a/packages/tests-e2e/package.json b/packages/tests-e2e/package.json
index 1edecc4abb4..624967b1ad7 100644
--- a/packages/tests-e2e/package.json
+++ b/packages/tests-e2e/package.json
@@ -4,13 +4,14 @@
"private": true,
"scripts": {
"test-e2e": "npx playwright test --config=playwright.config.ts",
- "lint": "eslint 'scenarios/**/*.ts'"
+ "lint": "eslint 'scenarios/**/*.ts' 'pages/**/*.ts' 'fixtures/**/*.ts' global-setup.ts"
},
"dependencies": {},
"devDependencies": {
"@faker-js/faker": "10.5.0",
"@playwright/test": "1.64.1",
"checkly": "6.9.10",
- "dotenv": "17.2.3"
+ "dotenv": "17.2.3",
+ "eslint-plugin-playwright": "2.12.1"
}
}
diff --git a/packages/web/AGENTS.md b/packages/web/AGENTS.md
index ff16fc78a1e..09d7c1960aa 100644
--- a/packages/web/AGENTS.md
+++ b/packages/web/AGENTS.md
@@ -43,6 +43,11 @@ You are working in the Activepieces web application (`packages/web`).
- **Overflowing text must use `TextWithTooltip`** (from `@/components/custom/text-with-tooltip`). Wrap any text that may overflow its container (emails, IDs, long names) in `<TextWithTooltip tooltipMessage={text}><p className="...">{text}</p></TextWithTooltip>`. It auto-detects truncation and only shows the tooltip when the text actually overflows. Ensure parent flex containers have `min-w-0` so `truncate` works correctly.
- **Copy-to-clipboard UI must use `CopyToClipboardInput`** (from `@/components/custom/clipboard/copy-to-clipboard`). Never hand-roll a readonly `<Input>` glued to a copy `<Button>` with `navigator.clipboard.writeText`. `CopyToClipboardInput` handles the copied-state toggle, tooltip, styling, and optional download button. Pass `useInput={true}` for single-line values (links, keys) or `useInput={false}` for multi-line content (textarea). Use `fileName` only when the value should also be downloadable.
+## Accessible markup
+
+- **Every interactive element has a role and an accessible name.** Use a real `<button>` / `<a>` (the Shadcn `Button`), never a clickable `div`. Link each `FormLabel` to its input through `FormControl`. Give an icon-only button an `aria-label` with a translation key.
+- **A `data-testid` does not replace either.** E2E specs select by test id first (see "Writing a spec" in `packages/tests-e2e/README.md`), but screen readers and `getByRole` still need the role and the name. When you add a test id to an element, check that it has both.
+
## React Hook Form
- **Zod error messages must use `formErrors`** — For standard validation messages (e.g. required fields) use the `formErrors` constant from `@activepieces/shared`. For custom messages, add the key to `packages/web/public/locales/en/translation.json` first, then use the key string. `FormMessage` automatically calls `t()` on every error message, so the string must be a valid translation key.