Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
chore: introduce generator log (#37644)
Pavel Feldmanb68bf79e5da7
diff --git a/examples/todomvc/.claude/agents/playwright-test-generator.md b/examples/todomvc/.claude/agents/playwright-test-generator.mdindex 277ff679a4..ade2f9c0a3 100644--- a/examples/todomvc/.claude/agents/playwright-test-generator.md+++ b/examples/todomvc/.claude/agents/playwright-test-generator.md@@ -1,7 +1,7 @@---name: playwright-test-generatordescription: Use this agent when you need to create automated browser tests using Playwright. Examples: <example>Context: User wants to test a login flow on their web application. user: 'I need a test that logs into my app at localhost:3000 with username [email protected] and password 123456, then verifies the dashboard page loads' assistant: 'I'll use the generator agent to create and validate this login test for you' <commentary> The user needs a specific browser automation test created, which is exactly what the generator agent is designed for. </commentary></example><example>Context: User has built a new checkout flow and wants to ensure it works correctly. user: 'Can you create a test that adds items to cart, proceeds to checkout, fills in payment details, and confirms the order?' assistant: 'I'll use the generator agent to build a comprehensive checkout flow test' <commentary> This is a complex user journey that needs to be automated and tested, perfect for the generator agent. </commentary></example>-tools: Glob, Grep, Read, Write, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__test_setup_page+tools: Glob, Grep, Read, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__generator_log_step, mcp__playwright-test__generator_read_log, mcp__playwright-test__generator_setup_page, mcp__playwright-test__generator_write_testmodel: sonnetcolor: blue---@@ -10,31 +10,19 @@ You are a Playwright Test Generator, an expert in browser automation and end-to-Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validateapplication behavior.-Your process is methodical and thorough:--1. **Scenario Analysis**- - Carefully analyze the test scenario provided, identifying all user actions,- expected outcomes and validation points--2. **Interactive Execution**- - For each scenario, start with the `test_setup_page` tool to set up page for the scenario- - Use Playwright tools to manually execute each step of the scenario in real-time- - Verify that each action works as expected- - Identify the correct locators and interaction patterns- - Observe actual application behavior and responses- - Validate that assertions will work correctly--3. **Test Code Generation**-- After successfully completing the manual execution, generate clean, maintainable- @playwright/test source code that follows following convention:-- - One file per scenario, one test in a file- - Use seed test content (copyright, structure) to emit consistent tests.- - File name must be fs-friendly scenario name- - Test must be placed in a describe matching the top-level test plan item- - Test title must match the scenario name- - Includes a comment with the step text before each step execution+# For each test you generate+- Obtain the test plan with all the steps and verification specification+- Run the `generator_setup_page` tool to set up page for the scenario+- For each step and verification in the scenario, do the following:+ - Use Playwright tool to manually execute it in real-time.+ - Immediately after each tool execution, log the step via running `generator_log_step` tool.+- Retrieve generator log via `generator_read_log`+- Immediately after reading the test log, invoke `generator_write_test` with the generated source code+ - File should contain single test+ - File name must be fs-friendly scenario name+ - Test must be placed in a describe matching the top-level test plan item+ - Test title must match the scenario name+ - Includes a comment with the step text before each step execution<example-generation>For following plan:@@ -68,14 +56,14 @@ Your process is methodical and thorough:```</example-generation>-4. **Best practices**:- - Each test has clear, descriptive assertions that validate the expected behavior- - Includes proper error handling and meaningful failure messages- - Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)- - Do not improvise, do not add directives that were not asked for- - Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)- - Uses local variables for locators that are used multiple times- - Uses explicit waits rather than arbitrary timeouts- - Never waits for networkidle or use other discouraged or deprecated apis- - Is self-contained and can run independently- - Is deterministic and not prone to flaky behavior\ No newline at end of file+# Best practices+- Each test has clear, descriptive assertions that validate the expected behavior+- Includes proper error handling and meaningful failure messages+- Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)+- Do not improvise, do not add directives that were not asked for+- Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)+- Uses local variables for locators that are used multiple times+- Uses explicit waits rather than arbitrary timeouts+- Never waits for networkidle or use other discouraged or deprecated apis+- Is self-contained and can run independently+- Is deterministic and not prone to flaky behavior\ No newline at end of filediff --git a/examples/todomvc/.claude/agents/playwright-test-planner.md b/examples/todomvc/.claude/agents/playwright-test-planner.mdindex e2b17b94c3..9e468a85d8 100644--- a/examples/todomvc/.claude/agents/playwright-test-planner.md+++ b/examples/todomvc/.claude/agents/playwright-test-planner.md@@ -1,7 +1,7 @@---name: playwright-test-plannerdescription: Use this agent when you need to create comprehensive test plan for a web application or website. Examples: <example>Context: User wants to test a new e-commerce checkout flow. user: 'I need test scenarios for our new checkout process at https://mystore.com/checkout' assistant: 'I'll use the planner agent to navigate to your checkout page and create comprehensive test scenarios.' <commentary> The user needs test planning for a specific web page, so use the planner agent to explore and create test scenarios. </commentary></example><example>Context: User has deployed a new feature and wants thorough testing coverage. user: 'Can you help me test our new user dashboard at https://app.example.com/dashboard?' assistant: 'I'll launch the planner agent to explore your dashboard and develop detailed test scenarios.' <commentary> This requires web exploration and test scenario creation, perfect for the planner agent. </commentary></example>-tools: Glob, Grep, Read, Write, mcp__playwright-test__browser_click, mcp__playwright-test__browser_close, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_navigate_back, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_take_screenshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_wait_for, mcp__playwright-test__test_setup_page+tools: Glob, Grep, Read, Write, mcp__playwright-test__browser_click, mcp__playwright-test__browser_close, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_navigate_back, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_take_screenshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_wait_for, mcp__playwright-test__planner_setup_pagemodel: sonnetcolor: green---@@ -13,7 +13,7 @@ planning.You will:1. **Navigate and Explore**- - Invoke the `test_setup_page` tool once to set up page before using any other tools+ - Invoke the `planner_setup_page` tool once to set up page before using any other tools- Explore the browser snapshot- Do not take screenshots unless absolutely necessary- Use browser_* tools to navigate and discover interfacediff --git "a/examples/todomvc/.github/chatmodes/ \360\237\216\255 planner.chatmode.md" "b/examples/todomvc/.github/chatmodes/ \360\237\216\255 planner.chatmode.md"index 4f1446b86d..5eca16632b 100644--- "a/examples/todomvc/.github/chatmodes/ \360\237\216\255 planner.chatmode.md"+++ "b/examples/todomvc/.github/chatmodes/ \360\237\216\255 planner.chatmode.md"@@ -1,6 +1,6 @@---description: Use this agent when you need to create comprehensive test plan for a web application or website.-tools: ['edit/createFile', 'edit/createDirectory', 'search/fileSearch', 'search/textSearch', 'search/listDirectory', 'search/readFile', 'playwright-test/browser_click', 'playwright-test/browser_close', 'playwright-test/browser_console_messages', 'playwright-test/browser_drag', 'playwright-test/browser_evaluate', 'playwright-test/browser_file_upload', 'playwright-test/browser_handle_dialog', 'playwright-test/browser_hover', 'playwright-test/browser_navigate', 'playwright-test/browser_navigate_back', 'playwright-test/browser_network_requests', 'playwright-test/browser_press_key', 'playwright-test/browser_select_option', 'playwright-test/browser_snapshot', 'playwright-test/browser_take_screenshot', 'playwright-test/browser_type', 'playwright-test/browser_wait_for', 'playwright-test/test_setup_page']+tools: ['edit/createFile', 'edit/createDirectory', 'search/fileSearch', 'search/textSearch', 'search/listDirectory', 'search/readFile', 'playwright-test/browser_click', 'playwright-test/browser_close', 'playwright-test/browser_console_messages', 'playwright-test/browser_drag', 'playwright-test/browser_evaluate', 'playwright-test/browser_file_upload', 'playwright-test/browser_handle_dialog', 'playwright-test/browser_hover', 'playwright-test/browser_navigate', 'playwright-test/browser_navigate_back', 'playwright-test/browser_network_requests', 'playwright-test/browser_press_key', 'playwright-test/browser_select_option', 'playwright-test/browser_snapshot', 'playwright-test/browser_take_screenshot', 'playwright-test/browser_type', 'playwright-test/browser_wait_for', 'playwright-test/planner_setup_page']---You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test@@ -10,7 +10,7 @@ planning.You will:1. **Navigate and Explore**- - Invoke the `test_setup_page` tool once to set up page before using any other tools+ - Invoke the `planner_setup_page` tool once to set up page before using any other tools- Explore the browser snapshot- Do not take screenshots unless absolutely necessary- Use browser_* tools to navigate and discover interfacediff --git "a/examples/todomvc/.github/chatmodes/\360\237\216\255 generator.chatmode.md" "b/examples/todomvc/.github/chatmodes/\360\237\216\255 generator.chatmode.md"index 7bf8f247aa..7a594a877a 100644--- "a/examples/todomvc/.github/chatmodes/\360\237\216\255 generator.chatmode.md"+++ "b/examples/todomvc/.github/chatmodes/\360\237\216\255 generator.chatmode.md"@@ -1,37 +1,25 @@---description: Use this agent when you need to create automated browser tests using Playwright.-tools: ['edit/createFile', 'edit/createDirectory', 'search/fileSearch', 'search/textSearch', 'search/listDirectory', 'search/readFile', 'playwright-test/browser_click', 'playwright-test/browser_drag', 'playwright-test/browser_evaluate', 'playwright-test/browser_file_upload', 'playwright-test/browser_handle_dialog', 'playwright-test/browser_hover', 'playwright-test/browser_navigate', 'playwright-test/browser_press_key', 'playwright-test/browser_select_option', 'playwright-test/browser_snapshot', 'playwright-test/browser_type', 'playwright-test/browser_verify_element_visible', 'playwright-test/browser_verify_list_visible', 'playwright-test/browser_verify_text_visible', 'playwright-test/browser_verify_value', 'playwright-test/browser_wait_for', 'playwright-test/test_setup_page']+tools: ['search/fileSearch', 'search/textSearch', 'search/listDirectory', 'search/readFile', 'playwright-test/browser_click', 'playwright-test/browser_drag', 'playwright-test/browser_evaluate', 'playwright-test/browser_file_upload', 'playwright-test/browser_handle_dialog', 'playwright-test/browser_hover', 'playwright-test/browser_navigate', 'playwright-test/browser_press_key', 'playwright-test/browser_select_option', 'playwright-test/browser_snapshot', 'playwright-test/browser_type', 'playwright-test/browser_verify_element_visible', 'playwright-test/browser_verify_list_visible', 'playwright-test/browser_verify_text_visible', 'playwright-test/browser_verify_value', 'playwright-test/browser_wait_for', 'playwright-test/generator_log_step', 'playwright-test/generator_read_log', 'playwright-test/generator_setup_page', 'playwright-test/generator_write_test']---You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validateapplication behavior.-Your process is methodical and thorough:--1. **Scenario Analysis**- - Carefully analyze the test scenario provided, identifying all user actions,- expected outcomes and validation points--2. **Interactive Execution**- - For each scenario, start with the `test_setup_page` tool to set up page for the scenario- - Use Playwright tools to manually execute each step of the scenario in real-time- - Verify that each action works as expected- - Identify the correct locators and interaction patterns- - Observe actual application behavior and responses- - Validate that assertions will work correctly--3. **Test Code Generation**-- After successfully completing the manual execution, generate clean, maintainable- @playwright/test source code that follows following convention:-- - One file per scenario, one test in a file- - Use seed test content (copyright, structure) to emit consistent tests.- - File name must be fs-friendly scenario name- - Test must be placed in a describe matching the top-level test plan item- - Test title must match the scenario name- - Includes a comment with the step text before each step execution+# For each test you generate+- Obtain the test plan with all the steps and verification specification+- Run the `generator_setup_page` tool to set up page for the scenario+- For each step and verification in the scenario, do the following:+ - Use Playwright tool to manually execute it in real-time.+ - Immediately after each tool execution, log the step via running `generator_log_step` tool.+- Retrieve generator log via `generator_read_log`+- Immediately after reading the test log, invoke `generator_write_test` with the generated source code+ - File should contain single test+ - File name must be fs-friendly scenario name+ - Test must be placed in a describe matching the top-level test plan item+ - Test title must match the scenario name+ - Includes a comment with the step text before each step execution<example-generation>For following plan:@@ -65,16 +53,16 @@ Your process is methodical and thorough:```</example-generation>-4. **Best practices**:- - Each test has clear, descriptive assertions that validate the expected behavior- - Includes proper error handling and meaningful failure messages- - Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)- - Do not improvise, do not add directives that were not asked for- - Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)- - Uses local variables for locators that are used multiple times- - Uses explicit waits rather than arbitrary timeouts- - Never waits for networkidle or use other discouraged or deprecated apis- - Is self-contained and can run independently- - Is deterministic and not prone to flaky behavior+# Best practices+- Each test has clear, descriptive assertions that validate the expected behavior+- Includes proper error handling and meaningful failure messages+- Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)+- Do not improvise, do not add directives that were not asked for+- Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)+- Uses local variables for locators that are used multiple times+- Uses explicit waits rather than arbitrary timeouts+- Never waits for networkidle or use other discouraged or deprecated apis+- Is self-contained and can run independently+- Is deterministic and not prone to flaky behavior<example>Context: User wants to test a login flow on their web application. user: 'I need a test that logs into my app at localhost:3000 with username [email protected] and password 123456, then verifies the dashboard page loads' assistant: 'I'll use the generator agent to create and validate this login test for you' <commentary> The user needs a specific browser automation test created, which is exactly what the generator agent is designed for. </commentary></example><example>Context: User has built a new checkout flow and wants to ensure it works correctly. user: 'Can you create a test that adds items to cart, proceeds to checkout, fills in payment details, and confirms the order?' assistant: 'I'll use the generator agent to build a comprehensive checkout flow test' <commentary> This is a complex user journey that needs to be automated and tested, perfect for the generator agent. </commentary></example>\ No newline at end of filediff --git a/packages/playwright/src/agents/generator.md b/packages/playwright/src/agents/generator.mdindex b6eaa4006b..2ce327d15f 100644--- a/packages/playwright/src/agents/generator.md+++ b/packages/playwright/src/agents/generator.md@@ -7,7 +7,6 @@ tools:- ls- grep- read- - write- playwright-test/browser_click- playwright-test/browser_drag- playwright-test/browser_evaluate@@ -24,38 +23,29 @@ tools:- playwright-test/browser_verify_text_visible- playwright-test/browser_verify_value- playwright-test/browser_wait_for- - playwright-test/test_setup_page+ - playwright-test/generator_log_step+ - playwright-test/generator_read_log+ - playwright-test/generator_setup_page+ - playwright-test/generator_write_test---You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validateapplication behavior.-Your process is methodical and thorough:--1. **Scenario Analysis**- - Carefully analyze the test scenario provided, identifying all user actions,- expected outcomes and validation points--2. **Interactive Execution**- - For each scenario, start with the `test_setup_page` tool to set up page for the scenario- - Use Playwright tools to manually execute each step of the scenario in real-time- - Verify that each action works as expected- - Identify the correct locators and interaction patterns- - Observe actual application behavior and responses- - Validate that assertions will work correctly--3. **Test Code Generation**-- After successfully completing the manual execution, generate clean, maintainable- @playwright/test source code that follows following convention:-- - One file per scenario, one test in a file- - Use seed test content (copyright, structure) to emit consistent tests.- - File name must be fs-friendly scenario name- - Test must be placed in a describe matching the top-level test plan item- - Test title must match the scenario name- - Includes a comment with the step text before each step execution+# For each test you generate+- Obtain the test plan with all the steps and verification specification+- Run the `generator_setup_page` tool to set up page for the scenario+- For each step and verification in the scenario, do the following:+ - Use Playwright tool to manually execute it in real-time.+ - Immediately after each tool execution, log the step via running `generator_log_step` tool.+- Retrieve generator log via `generator_read_log`+- Immediately after reading the test log, invoke `generator_write_test` with the generated source code+ - File should contain single test+ - File name must be fs-friendly scenario name+ - Test must be placed in a describe matching the top-level test plan item+ - Test title must match the scenario name+ - Includes a comment with the step text before each step execution<example-generation>For following plan:@@ -89,17 +79,17 @@ Your process is methodical and thorough:```</example-generation>-4. **Best practices**:- - Each test has clear, descriptive assertions that validate the expected behavior- - Includes proper error handling and meaningful failure messages- - Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)- - Do not improvise, do not add directives that were not asked for- - Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)- - Uses local variables for locators that are used multiple times- - Uses explicit waits rather than arbitrary timeouts- - Never waits for networkidle or use other discouraged or deprecated apis- - Is self-contained and can run independently- - Is deterministic and not prone to flaky behavior+# Best practices+- Each test has clear, descriptive assertions that validate the expected behavior+- Includes proper error handling and meaningful failure messages+- Uses Playwright best practices (page.waitForLoadState, expect.toBeVisible, etc.)+- Do not improvise, do not add directives that were not asked for+- Uses reliable locators (preferring data-testid, role-based, or text-based selectors over fragile CSS selectors)+- Uses local variables for locators that are used multiple times+- Uses explicit waits rather than arbitrary timeouts+- Never waits for networkidle or use other discouraged or deprecated apis+- Is self-contained and can run independently+- Is deterministic and not prone to flaky behavior<example>Context: User wants to test a login flow on their web application.diff --git a/packages/playwright/src/agents/planner.md b/packages/playwright/src/agents/planner.mdindex 46a269bd6e..672f3f2387 100644--- a/packages/playwright/src/agents/planner.md+++ b/packages/playwright/src/agents/planner.md@@ -25,7 +25,7 @@ tools:- playwright-test/browser_take_screenshot- playwright-test/browser_type- playwright-test/browser_wait_for- - playwright-test/test_setup_page+ - playwright-test/planner_setup_page---You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test@@ -35,7 +35,7 @@ planning.You will:1. **Navigate and Explore**- - Invoke the `test_setup_page` tool once to set up page before using any other tools+ - Invoke the `planner_setup_page` tool once to set up page before using any other tools- Explore the browser snapshot- Do not take screenshots unless absolutely necessary- Use browser_* tools to navigate and discover interfacediff --git a/packages/playwright/src/mcp/sdk/mdb.ts b/packages/playwright/src/mcp/sdk/mdb.tsindex 2a1d23f12e..66c3ef8b34 100644--- a/packages/playwright/src/mcp/sdk/mdb.ts+++ b/packages/playwright/src/mcp/sdk/mdb.ts@@ -30,15 +30,23 @@ const mdbDebug = debug('pw:mcp:mdb');const errorsDebug = debug('pw:mcp:errors');const z = mcpBundle.z;+type StackEntry = {+ client: Client;+ toolNames: string[];+ resultPromise: ManualPromise<mcpServer.CallToolResult> | undefined;+};+export class MDBBackend implements mcpServer.ServerBackend {- private _stack: { client: Client, toolNames: string[], resultPromise: ManualPromise<mcpServer.CallToolResult> | undefined }[] = [];+ private _stack: StackEntry[] = [];private _interruptPromise: ManualPromise<mcpServer.CallToolResult> | undefined;private _topLevelBackend: mcpServer.ServerBackend;+ private _allowedOnPause: string[];private _clientInfo: mcpServer.ClientInfo | undefined;private _progress: mcpServer.CallToolResult['content'] = [];- constructor(topLevelBackend: mcpServer.ServerBackend) {+ constructor(topLevelBackend: mcpServer.ServerBackend, allowedOnPause: string[]) {this._topLevelBackend = topLevelBackend;+ this._allowedOnPause = allowedOnPause;}async initialize(server: mcpServer.Server, clientInfo: mcpServer.ClientInfo): Promise<void> {@@ -61,19 +69,30 @@ export class MDBBackend implements mcpServer.ServerBackend {const interruptPromise = new ManualPromise<mcpServer.CallToolResult>();this._interruptPromise = interruptPromise;- let [entry] = this._stack;- // Pop the client while the tool is not found.- while (entry && !entry.toolNames.includes(name)) {+ // Reset the stack if the command is in the reset commands.+ if (!this._allowedOnPause.includes(name)) {+ for (let i = 0; i < this._stack.length - 1; i++) {+ if (this._stack[i].toolNames.includes(name))+ break;+ await this._stack[i].client.close().catch(errorsDebug);+ this._stack.shift();+ }+ }++ // Otherwise find the client for the tool.+ let entry: StackEntry | undefined;+ for (let i = 0; i < this._stack.length; i++) {+ if (this._stack[i].toolNames.includes(name)) {+ entry = this._stack[i];+ break;+ }mdbDebug('popping client from stack for ', name);- this._stack.shift();- await entry.client.close().catch(errorsDebug);- entry = this._stack[0];}if (!entry)throw new Error(`Tool ${name} not found in the tool stack`);- const client = await this._client();+ const client = entry.client;const resultPromise = new ManualPromise<mcpServer.CallToolResult>();entry.resultPromise = resultPromise;@@ -116,7 +135,7 @@ export class MDBBackend implements mcpServer.ServerBackend {private async _pushClient(transport: Transport, introMessage?: string): Promise<mcpServer.CallToolResult> {mdbDebug('pushing client to the stack');- const client = new mcpBundle.Client({ name: 'Pushing client', version: '0.0.0' }, { capabilities: { roots: {} } });+ const client = new mcpBundle.Client({ name: 'Interrupting client', version: '0.0.0' }, { capabilities: { roots: {} } });client.setRequestHandler(mcpBundle.ListRootsRequestSchema, () => ({ roots: this._clientInfo?.roots || [] }));client.setRequestHandler(mcpBundle.PingRequestSchema, () => ({}));client.setNotificationHandler(mcpBundle.ProgressNotificationSchema, notification => {@@ -154,8 +173,8 @@ const pushToolsSchema = defineToolSchema({type: 'readOnly',});-export async function runMainBackend(backendFactory: mcpServer.ServerBackendFactory, options?: { port?: number }): Promise<string | undefined> {- const mdbBackend = new MDBBackend(backendFactory.create());+export async function runMainBackend(backendFactory: mcpServer.ServerBackendFactory, allowedOnPause: string[], options?: { port?: number }): Promise<string | undefined> {+ const mdbBackend = new MDBBackend(backendFactory.create(), allowedOnPause);// Start HTTP unconditionally.const factory: mcpServer.ServerBackendFactory = {...backendFactory,@@ -185,7 +204,7 @@ export async function runOnPauseBackendLoop(backend: mcpServer.ServerBackend, inawait mcpHttp.installHttpTransport(httpServer, factory);const url = mcpHttp.httpAddressToString(httpServer.address());- const client = new mcpBundle.Client({ name: 'On-pause client', version: '0.0.0' });+ const client = new mcpBundle.Client({ name: 'Pushing client', version: '0.0.0' });client.setRequestHandler(mcpBundle.PingRequestSchema, () => ({}));const transport = new mcpBundle.StreamableHTTPClientTransport(new URL(process.env.PLAYWRIGHT_DEBUGGER_MCP!));await client.connect(transport);diff --git a/packages/playwright/src/mcp/test/generatorTools.ts b/packages/playwright/src/mcp/test/generatorTools.tsnew file mode 100644index 0000000000..cf2d0a52ea--- /dev/null+++ b/packages/playwright/src/mcp/test/generatorTools.ts@@ -0,0 +1,123 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import fs from 'fs';+import path from 'path';++import { z } from '../sdk/bundle';+import { defineTestTool } from './testTool';+import { GeneratorJournal } from './testContext';++export const setupPage = defineTestTool({+ schema: {+ name: 'generator_setup_page',+ title: 'Setup generator page',+ description: 'Setup the page for test.',+ inputSchema: z.object({+ plan: z.string().describe('The plan for the test. This should be the actual test plan with all the steps.'),+ project: z.string().optional().describe('Project to use for setup. For example: "chromium", if no project is provided uses the first project in the config.'),+ seedFile: z.string().optional().describe('A seed file contains a single test that is used to setup the page for testing, for example: "tests/seed.spec.ts". If no seed file is provided, a default seed file is created.'),+ }),+ type: 'readOnly',+ },++ handle: async (context, params, progress) => {+ const seed = await context.getOrCreateSeedFile(params);+ context.generatorJournal = new GeneratorJournal(context.rootPath, params.plan, seed);+ await context.runSeedTest(seed.file, params.project, progress);+ return { content: [] };+ },+});++export const generatorLogStep = defineTestTool({+ schema: {+ name: 'generator_log_step',+ title: 'Log a test step',+ description: 'Log a successful test step.',+ inputSchema: z.object({+ title: z.string().describe('Title of the test step'),+ code: z.string().describe('The code for the last step only, as a markdown code block'),+ }),+ type: 'readOnly',+ },++ handle: async (context, params) => {+ if (!context.generatorJournal)+ throw new Error(`Please setup page using "${setupPage.schema.name}" first.`);+ context.generatorJournal.logStep(params.title, params.code);+ return { content: [] };+ },+});++export const generatorReadLog = defineTestTool({+ schema: {+ name: 'generator_read_log',+ title: 'Retrieve test log',+ description: 'Retrieve the performed test log',+ inputSchema: z.object({}),+ type: 'readOnly',+ },++ handle: async context => {+ if (!context.generatorJournal)+ throw new Error(`Please setup page using "${setupPage.schema.name}" first.`);+ const result = context.generatorJournal.journal();+ return { content: [{+ type: 'text',+ text: result,+ }] };+ },+});++export const generatorWriteTest = defineTestTool({+ schema: {+ name: 'generator_write_test',+ title: 'Write test',+ description: 'Write the generated test to the test file',+ inputSchema: z.object({+ fileName: z.string().describe('The file to write the test to'),+ code: z.string().describe('The generated test code'),+ }),+ type: 'destructive',+ },++ handle: async (context, params) => {+ if (!context.generatorJournal)+ throw new Error(`Please setup page using "${setupPage.schema.name}" first.`);++ const testRunner = context.existingTestRunner();+ if (!testRunner)+ throw new Error('No test runner found, please setup page and perform actions first.');+ const config = await testRunner.loadConfig();++ const dirs: string[] = [];+ for (const project of config.projects) {+ const testDir = path.relative(context.rootPath, project.project.testDir).replace(/\\/g, '/');+ const fileName = params.fileName.replace(/\\/g, '/');+ if (fileName.startsWith(testDir)) {+ await fs.promises.writeFile(path.resolve(context.rootPath, fileName), params.code);+ return {+ content: [{+ type: 'text',+ text: `### Result\nTest written to ${params.fileName}`,+ }]+ };+ }+ dirs.push(testDir);+ }+ throw new Error(`Test file did not match any of the test dirs: ${dirs.join(', ')}`);+ },+});diff --git a/packages/playwright/src/mcp/test/plannerTools.ts b/packages/playwright/src/mcp/test/plannerTools.tsnew file mode 100644index 0000000000..5fc7d41e4d--- /dev/null+++ b/packages/playwright/src/mcp/test/plannerTools.ts@@ -0,0 +1,37 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import { z } from '../sdk/bundle';+import { defineTestTool } from './testTool';++export const setupPage = defineTestTool({+ schema: {+ name: 'planner_setup_page',+ title: 'Setup planner page',+ description: 'Setup the page for test planning',+ inputSchema: z.object({+ project: z.string().optional().describe('Project to use for setup. For example: "chromium", if no project is provided uses the first project in the config.'),+ seedFile: z.string().optional().describe('A seed file contains a single test that is used to setup the page for testing, for example: "tests/seed.spec.ts". If no seed file is provided, a default seed file is created.'),+ }),+ type: 'readOnly',+ },++ handle: async (context, params, progress) => {+ const seed = await context.getOrCreateSeedFile(params);+ await context.runSeedTest(seed.file, params.project, progress);+ return { content: [] };+ },+});diff --git a/packages/playwright/src/mcp/test/testBackend.ts b/packages/playwright/src/mcp/test/testBackend.tsindex a1c64052e0..ade9e83d60 100644--- a/packages/playwright/src/mcp/test/testBackend.ts+++ b/packages/playwright/src/mcp/test/testBackend.ts@@ -16,7 +16,9 @@import * as mcp from '../sdk/exports';import { TestContext } from './testContext';-import { listTests, runTests, debugTest, setupPage } from './testTools.js';+import * as testTools from './testTools.js';+import * as generatorTools from './generatorTools.js';+import * as plannerTools from './plannerTools.js';import { browserTools } from '../browser/tools';import { resolveConfigLocation } from '../../common/configLoader';@@ -25,10 +27,23 @@ import type { TestTool } from './testTool';export class TestServerBackend implements mcp.ServerBackend {readonly name = 'Playwright';readonly version = '0.0.1';- private _tools: TestTool<any>[] = [listTests, runTests, debugTest, setupPage];+ private _tools: TestTool<any>[] = [+ plannerTools.setupPage,+ generatorTools.setupPage,+ generatorTools.generatorLogStep,+ generatorTools.generatorReadLog,+ generatorTools.generatorWriteTest,+ testTools.listTests,+ testTools.runTests,+ testTools.debugTest,+ ];private _context: TestContext;private _configOption: string | undefined;+ static allowedOnPause: string[] = [+ generatorTools.generatorLogStep.schema.name,+ ];+constructor(configOption: string | undefined, options?: { muteConsole?: boolean, headless?: boolean }) {this._context = new TestContext(options);this._configOption = configOption;diff --git a/packages/playwright/src/mcp/test/testContext.ts b/packages/playwright/src/mcp/test/testContext.tsindex 08904527ca..d8d4b27192 100644--- a/packages/playwright/src/mcp/test/testContext.ts+++ b/packages/playwright/src/mcp/test/testContext.ts@@ -14,15 +14,63 @@* limitations under the License.*/+import fs from 'fs';+import path from 'path';++import { noColors, escapeRegExp } from 'playwright-core/lib/utils';++import { terminalScreen } from '../../reporters/base';+import ListReporter from '../../reporters/list';+import { StringWriteStream } from './streams';+import { fileExistsAsync } from '../../util';import { TestRunner, TestRunnerEvent } from '../../runner/testRunner';+import { ensureSeedTest, seedProject } from './seed';+import type { ProgressCallback } from '../sdk/server';import type { ConfigLocation } from '../../common/config';+export type SeedFile = {+ file: string;+ content: string;+};++export class GeneratorJournal {+ private _rootPath: string;+ private _plan: string;+ private _seed: SeedFile;+ private _steps: { title: string, code: string }[];++ constructor(rootPath: string, plan: string, seed: SeedFile) {+ this._rootPath = rootPath;+ this._plan = plan;+ this._seed = seed;+ this._steps = [];+ }++ logStep(title: string, code: string) {+ this._steps.push({ title, code });+ }++ journal() {+ const result: string[] = [];+ result.push(`# Plan`);+ result.push(this._plan);+ result.push(`# Seed file: ${path.relative(this._rootPath, this._seed.file)}`);+ result.push('```ts');+ result.push(this._seed.content);+ result.push('```');+ result.push(`# Steps`);+ result.push(this._steps.map(step => `### ${step.title}\n${step.code}`).join('\n\n'));+ return result.join('\n\n');+ }+}+export class TestContext {private _testRunner: TestRunner | undefined;readonly options?: { muteConsole?: boolean, headless?: boolean };configLocation!: ConfigLocation;rootPath!: string;+ generatorJournal: GeneratorJournal | undefined;constructor(options?: { muteConsole?: boolean, headless?: boolean }) {this.options = options;@@ -33,6 +81,10 @@ export class TestContext {this.rootPath = rootPath || configLocation.configDir;}+ existingTestRunner(): TestRunner | undefined {+ return this._testRunner;+ }+async createTestRunner(): Promise<TestRunner> {if (this._testRunner)await this._testRunner.stopTests();@@ -46,6 +98,72 @@ export class TestContext {return testRunner;}+ async getOrCreateSeedFile(params: { project?: string, seedFile?: string }) {+ const configDir = this.configLocation.configDir;+ const testRunner = await this.createTestRunner();+ const config = await testRunner.loadConfig();++ let seedFile: string | undefined;+ if (!params.seedFile) {+ seedFile = await ensureSeedTest(config, params.project, false);+ } else {+ const candidateFiles: string[] = [];+ const testDir = seedProject(config, params.project).project.testDir;+ candidateFiles.push(path.resolve(testDir, params.seedFile));+ candidateFiles.push(path.resolve(configDir, params.seedFile));+ candidateFiles.push(path.resolve(this.rootPath, params.seedFile));+ for (const candidateFile of candidateFiles) {+ if (await fileExistsAsync(candidateFile)) {+ seedFile = candidateFile;+ break;+ }+ }+ if (!seedFile)+ throw new Error('seed test not found.');+ }++ const seedFileContent = await fs.promises.readFile(seedFile, 'utf8');+ return { file: seedFile, content: seedFileContent };+ }++ async runSeedTest(seedFile: string, projectName: string | undefined, progress: ProgressCallback) {+ const { screen } = this.createScreen(progress);+ const configDir = this.configLocation.configDir;+ const reporter = new ListReporter({ configDir, screen });+ const testRunner = await this.createTestRunner();++ const result = await testRunner.runTests(reporter, {+ headed: !this.options?.headless,+ locations: ['/' + escapeRegExp(seedFile) + '/'],+ projects: projectName ? [projectName] : undefined,+ timeout: 0,+ workers: 1,+ pauseAtEnd: true,+ disableConfigReporters: true,+ failOnLoadErrors: true,+ });++ // Ideally, we should check that page was indeed created and browser mcp has kicked in.+ // However, that is handled in the upper layer, so hard to check here.+ if (result.status === 'passed' && !reporter.suite?.allTests().length)+ throw new Error('seed test not found.');++ if (result.status !== 'passed')+ throw new Error('Errors while running the seed test.');+ }++ createScreen(progress: ProgressCallback) {+ const stream = new StringWriteStream(progress);+ const screen = {+ ...terminalScreen,+ isTTY: false,+ colors: noColors,+ stdout: stream as unknown as NodeJS.WriteStream,+ stderr: stream as unknown as NodeJS.WriteStream,+ };+ return { screen, stream };+ }+async close() {}}diff --git a/packages/playwright/src/mcp/test/testTools.ts b/packages/playwright/src/mcp/test/testTools.tsindex 01cd9a1cb4..a3c86ed080 100644--- a/packages/playwright/src/mcp/test/testTools.ts+++ b/packages/playwright/src/mcp/test/testTools.ts@@ -14,21 +14,10 @@* limitations under the License.*/-import fs from 'fs';-import path from 'path';--import { noColors, escapeRegExp } from 'playwright-core/lib/utils';-import { z } from '../sdk/bundle';-import { terminalScreen } from '../../reporters/base';import ListReporter from '../../reporters/list';import ListModeReporter from '../../reporters/listModeReporter';-import { ensureSeedTest, seedProject } from './seed';import { defineTestTool } from './testTool';-import { StringWriteStream } from './streams';-import { fileExistsAsync } from '../../util';--import type { ProgressCallback } from '../sdk/server';export const listTests = defineTestTool({schema: {@@ -40,7 +29,7 @@ export const listTests = defineTestTool({},handle: async (context, _, progress) => {- const { screen } = createScreen(progress);+ const { screen } = context.createScreen(progress);const reporter = new ListModeReporter({ screen, includeTestId: true });const testRunner = await context.createTestRunner();await testRunner.listTests(reporter, {});@@ -62,7 +51,7 @@ export const runTests = defineTestTool({},handle: async (context, params, progress) => {- const { screen } = createScreen(progress);+ const { screen } = context.createScreen(progress);const configDir = context.configLocation.configDir;const reporter = new ListReporter({ configDir, screen, includeTestId: true, prefixStdio: 'out' });const testRunner = await context.createTestRunner();@@ -91,7 +80,7 @@ export const debugTest = defineTestTool({},handle: async (context, params, progress) => {- const { screen } = createScreen(progress);+ const { screen } = context.createScreen(progress);const configDir = context.configLocation.configDir;const reporter = new ListReporter({ configDir, screen, includeTestId: true, prefixStdio: 'out' });const testRunner = await context.createTestRunner();@@ -108,85 +97,3 @@ export const debugTest = defineTestTool({return { content: [] };},});--export const setupPage = defineTestTool({- schema: {- name: 'test_setup_page',- title: 'Setup page',- description: 'Setup the page for test.',- inputSchema: z.object({- project: z.string().optional().describe('Project to use for setup. For example: "chromium", if no project is provided uses the first project in the config.'),- seedFile: z.string().optional().describe('A seed file contains a single test that is used to setup the page for testing, for example: "tests/seed.spec.ts". If no seed file is provided, a default seed file is created.'),- }),- type: 'readOnly',- },-- handle: async (context, params, progress) => {- const { screen } = createScreen(progress);- const configDir = context.configLocation.configDir;- const reporter = new ListReporter({ configDir, screen });- const testRunner = await context.createTestRunner();- const config = await testRunner.loadConfig();-- let seedFile: string | undefined;- if (!params.seedFile) {- seedFile = await ensureSeedTest(config, params.project, false);- } else {- const candidateFiles: string[] = [];- const testDir = seedProject(config, params.project).project.testDir;- candidateFiles.push(path.resolve(testDir, params.seedFile));- candidateFiles.push(path.resolve(configDir, params.seedFile));- candidateFiles.push(path.resolve(context.rootPath, params.seedFile));- for (const candidateFile of candidateFiles) {- if (await fileExistsAsync(candidateFile)) {- seedFile = candidateFile;- break;- }- }- if (!seedFile)- throw new Error('seed test not found.');- }--- const seedFileContent = await fs.promises.readFile(seedFile, 'utf8');- progress({ message: `### Seed test-File: ${path.relative(context.rootPath, seedFile)}-\`\`\`ts-${seedFileContent}-\`\`\`-` });-- const result = await testRunner.runTests(reporter, {- headed: !context.options?.headless,- locations: ['/' + escapeRegExp(seedFile) + '/'],- projects: params.project ? [params.project] : undefined,- timeout: 0,- workers: 1,- pauseAtEnd: true,- disableConfigReporters: true,- failOnLoadErrors: true,- });-- // Ideally, we should check that page was indeed created and browser mcp has kicked in.- // However, that is handled in the upper layer, so hard to check here.- if (result.status === 'passed' && !reporter.suite?.allTests().length)- throw new Error('seed test not found.');-- if (result.status !== 'passed')- throw new Error('Errors while running the seed test.');-- return { content: [] };- },-});--function createScreen(progress: ProgressCallback) {- const stream = new StringWriteStream(progress);- const screen = {- ...terminalScreen,- isTTY: false,- colors: noColors,- stdout: stream as unknown as NodeJS.WriteStream,- stderr: stream as unknown as NodeJS.WriteStream,- };- return { screen, stream };-}diff --git a/packages/playwright/src/program.ts b/packages/playwright/src/program.tsindex 79d93a8ce1..82857da2ef 100644--- a/packages/playwright/src/program.ts+++ b/packages/playwright/src/program.ts@@ -169,7 +169,13 @@ function addTestMCPServerCommand(program: Command) {version: packageJSON.version,create: () => new TestServerBackend(options.config, { muteConsole: options.port === undefined, headless: options.headless }),};- const mdbUrl = await runMainBackend(backendFactory, { port: options.port === undefined ? undefined : +options.port });+ const mdbUrl = await runMainBackend(+ backendFactory,+ TestServerBackend.allowedOnPause,+ {+ port: options.port === undefined ? undefined : +options.port+ },+ );if (mdbUrl)console.error('MCP Listening on: ', mdbUrl);});diff --git a/tests/mcp/generator.spec.ts b/tests/mcp/generator.spec.tsnew file mode 100644index 0000000000..722ce96258--- /dev/null+++ b/tests/mcp/generator.spec.ts@@ -0,0 +1,248 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import fs from 'fs';++import { test, expect, writeFiles } from './fixtures';++test.use({ mcpServerType: 'test-mcp' });++test('generator_setup_page', async ({ startClient }) => {+ await writeFiles({+ 'a.test.ts': `+ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+ `,+ });++ const { client } = await startClient();+ const response = await client.callTool({+ name: 'generator_setup_page',+ arguments: {+ plan: 'Test plan',+ seedFile: 'a.test.ts',+ },+ });++ expect(response).toHaveTextResponse(expect.stringContaining(`### Paused at end of test. ready for interaction++### Page state+- Page URL: about:blank+- Page Title:+- Page Snapshot:+\`\`\`yaml+- button "Submit" [ref=e2]+\`\`\`+`));++ await client.callTool({+ name: 'generator_log_step',+ arguments: {+ title: 'Click submit button',+ code: `\`\`\`ts+await page.getByRole('button', { name: 'Submit' }).click();+\`\`\``,+ },+ });++ expect(await client.callTool({+ name: 'generator_read_log',+ arguments: {},+ })).toHaveTextResponse(`# Plan++Test plan++# Seed file: a.test.ts++\`\`\`ts+++ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+++\`\`\`++# Steps++### Click submit button+\`\`\`ts+await page.getByRole('button', { name: 'Submit' }).click();+\`\`\``);+});++test('click after generator_log_action', async ({ startClient }) => {+ await writeFiles({+ 'a.test.ts': `+ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+ `,+ });++ const { client } = await startClient();+ await client.callTool({+ name: 'generator_setup_page',+ arguments: {+ plan: 'Test plan',+ seedFile: 'a.test.ts',+ },+ });++ await client.callTool({+ name: 'generator_log_step',+ arguments: {+ title: 'Click submit button',+ code: `\`\`\`ts+await page.getByRole('button', { name: 'Submit' }).click();+\`\`\``,+ },+ });++ expect(await client.callTool({+ name: 'browser_click',+ arguments: {+ element: 'Submit button',+ ref: 'e2',+ },+ })).toHaveResponse({+ code: `await page.getByRole('button', { name: 'Submit' }).click();`,+ pageState: expect.stringContaining(`- button "Submit"`),+ });++ expect(await client.callTool({+ name: 'generator_read_log',+ arguments: {},+ })).toHaveTextResponse(`# Plan++Test plan++# Seed file: a.test.ts++\`\`\`ts+++ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+++\`\`\`++# Steps++### Click submit button+\`\`\`ts+await page.getByRole('button', { name: 'Submit' }).click();+\`\`\``);+});++test('generator_setup_page is required', async ({ startClient }) => {+ await writeFiles({+ 'a.test.ts': `+ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+ `,+ });++ const { client } = await startClient();++ expect(await client.callTool({+ name: 'generator_log_step',+ arguments: {+ title: 'Click submit button',+ code: `\`\`\`ts+await page.getByRole('button', { name: 'Submit' }).click();+\`\`\``,+ },+ })).toHaveResponse({+ result: expect.stringContaining(`Please setup page using "generator_setup_page" first.`),+ isError: true,+ });++ expect(await client.callTool({+ name: 'generator_read_log',+ arguments: {},+ })).toHaveResponse({+ result: expect.stringContaining(`Please setup page using "generator_setup_page" first.`),+ isError: true,+ });++ expect(await client.callTool({+ name: 'generator_write_test',+ arguments: {+ fileName: 'a.test.ts',+ code: '// Test content',+ },+ })).toHaveResponse({+ result: expect.stringContaining(`Please setup page using "generator_setup_page" first.`),+ isError: true,+ });+});++test('generator_write_test', async ({ startClient }, testInfo) => {+ await writeFiles({+ 'a.test.ts': `+ import { test, expect } from '@playwright/test';+ test.beforeEach(async ({ page }) => {+ await page.setContent('<button>Submit</button>');+ });+ test('template', async ({ page }) => {+ });+ `,+ });++ const { client } = await startClient();++ await client.callTool({+ name: 'generator_setup_page',+ arguments: {+ plan: 'Test plan',+ seedFile: 'a.test.ts',+ },+ });++ expect(await client.callTool({+ name: 'generator_write_test',+ arguments: {+ fileName: 'a.test.ts',+ code: `// Test content`,+ },+ })).toHaveResponse({+ result: expect.stringContaining(`Test written to a.test.ts`),+ });++ const code = fs.readFileSync(testInfo.outputPath('a.test.ts'), 'utf8');+ expect(code).toBe(`// Test content`);+});diff --git a/tests/mcp/mdb.spec.ts b/tests/mcp/mdb.spec.tsindex 1c83725ab8..e51046f6de 100644--- a/tests/mcp/mdb.spec.ts+++ b/tests/mcp/mdb.spec.ts@@ -131,7 +131,7 @@ async function startMDBAndCLI(): Promise<{ mdbUrl: string }> {create: () => new CLIBackend(mdbUrlBox)};- const mdbUrl = (await runMainBackend(cliBackendFactory, { port: 0 }))!;+ const mdbUrl = (await runMainBackend(cliBackendFactory, [], { port: 0 }))!;mdbUrlBox.mdbUrl = mdbUrl;return { mdbUrl };}diff --git a/tests/mcp/test-setup-page.spec.ts b/tests/mcp/planner.spec.tssimilarity index 86%rename from tests/mcp/test-setup-page.spec.tsrename to tests/mcp/planner.spec.tsindex 6afcd9f5dd..8268894212 100644--- a/tests/mcp/test-setup-page.spec.ts+++ b/tests/mcp/planner.spec.ts@@ -22,7 +22,7 @@ import url from 'url';test.use({ mcpServerType: 'test-mcp' });-test('test_setup_page', async ({ startClient }) => {+test('planner_setup_page', async ({ startClient }) => {await writeFiles({'a.test.ts': `import { test, expect } from '@playwright/test';@@ -36,13 +36,12 @@ test('test_setup_page', async ({ startClient }) => {const { client } = await startClient();const response = await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'a.test.ts',},});- expect(response).toHaveTextResponse(expect.stringContaining(`### Seed test\nFile: a.test.ts`));expect(response).toHaveTextResponse(expect.stringContaining(`### Paused at end of test. ready for interaction### Page state@@ -66,7 +65,7 @@ test('test_setup_page', async ({ startClient }) => {});});-test('test_setup_page seed resolution', async ({ startClient }) => {+test('planner_setup_page seed resolution', async ({ startClient }) => {await writeFiles({'playwright.config.ts': `module.exports = {@@ -85,7 +84,7 @@ test('test_setup_page seed resolution', async ({ startClient }) => {// Relative to test dir.expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'seed.test.ts',},@@ -93,14 +92,14 @@ test('test_setup_page seed resolution', async ({ startClient }) => {// Relative to config dir.expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'tests/seed.test.ts',},})).toHaveTextResponse(expect.stringContaining(`### Paused at end of test.`));});-test('test_setup_page seed resolution - rootPath', async ({ startClient }) => {+test('planner_setup_page seed resolution - rootPath', async ({ startClient }) => {await writeFiles({'packages/my-app/configs/playwright.config.ts': `module.exports = {@@ -121,14 +120,14 @@ test('test_setup_page seed resolution - rootPath', async ({ startClient }) => {});expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'packages/my-app/tests/seed.test.ts',},})).toHaveTextResponse(expect.stringContaining(`### Paused at end of test.`));});-test('test_setup_page with dependencies', async ({ startClient }) => {+test('planner_setup_page with dependencies', async ({ startClient }) => {const baseDir = await writeFiles({'playwright.config.ts': `module.exports = {@@ -155,7 +154,7 @@ test('test_setup_page with dependencies', async ({ startClient }) => {const { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'seed.test.ts',project: 'chromium',@@ -168,7 +167,7 @@ test('test_setup_page with dependencies', async ({ startClient }) => {expect(fs.existsSync(path.join(baseDir, 'test-results', 'seed-template-ignored', 'template.txt'))).toBe(false);});-test('test_setup_page (loading error)', async ({ startClient }) => {+test('planner_setup_page (loading error)', async ({ startClient }) => {await writeFiles({'seed.test.ts': `throw new Error('loading error');@@ -176,17 +175,17 @@ test('test_setup_page (loading error)', async ({ startClient }) => {});const { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'seed.test.ts',},})).toHaveTextResponse(expect.stringContaining('Error: loading error'));});-test('test_setup_page (wrong test location)', async ({ startClient }) => {+test('planner_setup_page (wrong test location)', async ({ startClient }) => {const { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {seedFile: 'a.test.ts',},@@ -194,15 +193,15 @@ test('test_setup_page (wrong test location)', async ({ startClient }) => {Error: seed test not found.`);});-test('test_setup_page (no test location)', async ({ startClient }) => {+test('planner_setup_page (no test location)', async ({ startClient }) => {const { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {},})).toHaveTextResponse(expect.stringContaining(`### Paused at end of test. ready for interaction`));});-test('test_setup_page chooses top-level project', async ({ startClient }) => {+test('planner_setup_page chooses top-level project', async ({ startClient }) => {const baseDir = await writeFiles({'playwright.config.ts': `module.exports = {@@ -216,7 +215,7 @@ test('test_setup_page chooses top-level project', async ({ startClient }) => {const { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {},})).toHaveTextResponse(expect.stringContaining(`### Paused at end of test. ready for interaction`));@@ -224,7 +223,7 @@ test('test_setup_page chooses top-level project', async ({ startClient }) => {expect(fs.existsSync(path.join(baseDir, 'two', 'seed.spec.ts'))).toBe(true);});-test('test_setup_page without location respects testsDir', async ({ startClient }) => {+test('planner_setup_page without location respects testsDir', async ({ startClient }) => {await writeFiles({'playwright.config.ts': `module.exports = {@@ -242,7 +241,7 @@ test('test_setup_page without location respects testsDir', async ({ startClientconst { client } = await startClient();expect(await client.callTool({- name: 'test_setup_page',+ name: 'planner_setup_page',arguments: {},})).toHaveTextResponse(expect.stringContaining(`### Paused at end of test. ready for interaction`));diff --git a/tests/mcp/test-debug.spec.ts b/tests/mcp/test-debug.spec.tsindex 0c52d06e31..0c3b031ef2 100644--- a/tests/mcp/test-debug.spec.ts+++ b/tests/mcp/test-debug.spec.ts@@ -229,6 +229,44 @@ test('test_debug / evaluate', async ({ startClient }) => {});});+test('test_debug / evaluate x 2', async ({ startClient }) => {+ const { client, id } = await prepareDebugTest(startClient);+ await client.callTool({+ name: 'test_debug',+ arguments: {+ test: { id, title: 'fail' },+ },+ });+ expect(await client.callTool({+ name: 'browser_evaluate',+ arguments: {+ function: '() => 21+21',+ },+ })).toHaveResponse({+ result: `42`,+ });++ expect(await client.callTool({+ name: 'test_debug',+ arguments: {+ test: { id, title: 'fail' },+ },+ })).toEqual({+ content: [+ { type: 'text', text: expect.stringContaining(`Paused on error`) },+ ]+ });++ expect(await client.callTool({+ name: 'browser_evaluate',+ arguments: {+ function: '() => 21+23',+ },+ })).toHaveResponse({+ result: `44`,+ });+});+test('test_debug / evaluate (with element)', async ({ startClient }) => {const { client, id } = await prepareDebugTest(startClient);await client.callTool({