Skip to content

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

chore: run headlessly in copilot cli (#38188)

Pavel Feldman129ac60f466d
diff --git a/examples/todomvc/.github/agents/playwright-test-generator.agent.md b/examples/todomvc/.github/agents/playwright-test-generator.agent.md
index e52ddbf616..b3722976b5 100644
--- a/examples/todomvc/.github/agents/playwright-test-generator.agent.md
+++ b/examples/todomvc/.github/agents/playwright-test-generator.agent.md
@@ -31,6 +31,15 @@ tools:
- playwright-test/generator_setup_page
- playwright-test/generator_write_test
model: Claude Sonnet 4
+mcp-servers:
+ playwright-test:
+ type: stdio
+ command: npx
+ args:
+ - playwright
+ - run-test-mcp-server
+ tools:
+ - "*"
---
You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.
diff --git a/examples/todomvc/.github/agents/playwright-test-healer.agent.md b/examples/todomvc/.github/agents/playwright-test-healer.agent.md
index a77d07b45a..2e044868b0 100644
--- a/examples/todomvc/.github/agents/playwright-test-healer.agent.md
+++ b/examples/todomvc/.github/agents/playwright-test-healer.agent.md
@@ -13,6 +13,15 @@ tools:
- playwright-test/test_list
- playwright-test/test_run
model: Claude Sonnet 4
+mcp-servers:
+ playwright-test:
+ type: stdio
+ command: npx
+ args:
+ - playwright
+ - run-test-mcp-server
+ tools:
+ - "*"
---
You are the Playwright Test Healer, an expert test automation engineer specializing in debugging and
diff --git a/examples/todomvc/.github/agents/playwright-test-planner.agent.md b/examples/todomvc/.github/agents/playwright-test-planner.agent.md
index 8c51e58c8e..6ac9a5f1f3 100644
--- a/examples/todomvc/.github/agents/playwright-test-planner.agent.md
+++ b/examples/todomvc/.github/agents/playwright-test-planner.agent.md
@@ -4,7 +4,6 @@ description: Use this agent when you need to create comprehensive test plan for
a web application or website
tools:
- search
- - edit
- playwright-test/browser_click
- playwright-test/browser_close
- playwright-test/browser_console_messages
@@ -23,7 +22,17 @@ tools:
- playwright-test/browser_type
- playwright-test/browser_wait_for
- playwright-test/planner_setup_page
+ - playwright-test/planner_save_plan
model: Claude Sonnet 4
+mcp-servers:
+ playwright-test:
+ type: stdio
+ command: npx
+ args:
+ - playwright
+ - run-test-mcp-server
+ tools:
+ - "*"
---
You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test
@@ -61,52 +70,7 @@ You will:
5. **Create Documentation**
- Save your test plan as requested:
- - Executive summary of the tested page/application
- - Individual scenarios as separate sections
- - Each scenario formatted with numbered steps
- - Each test case with proposed file name for implementation
- - Clear expected results for verification
-
-<example-spec>
-# TodoMVC Application - Comprehensive Test Plan
-
-## Application Overview
-
-The TodoMVC application is a React-based todo list manager that provides core task management functionality. The
-application features:
-
-- **Task Management**: Add, edit, complete, and delete individual todos
-- **Bulk Operations**: Mark all todos as complete/incomplete and clear all completed todos
-- **Filtering**: View todos by All, Active, or Completed status
-- **URL Routing**: Support for direct navigation to filtered views via URLs
-- **Counter Display**: Real-time count of active (incomplete) todos
-- **Persistence**: State maintained during session (browser refresh behavior not tested)
-
-## Test Scenarios
-
-### 1. Adding New Todos
-
-**Seed:** `tests/seed.spec.ts`
-
-#### 1.1 Add Valid Todo
-
-**File** `tests/adding-new-todos/add-valid-todo.spec.ts`
-
-**Steps:**
-1. Click in the "What needs to be done?" input field
-2. Type "Buy groceries"
-3. Press Enter key
-
-**Expected Results:**
-- Todo appears in the list with unchecked checkbox
-- Counter shows "1 item left"
-- Input field is cleared and ready for next entry
-- Todo list controls become visible (Mark all as complete checkbox)
-
-#### 1.2
-...
-</example-spec>
+ Submit your test plan using `planner_save_plan` tool.
**Quality Standards**:
- Write steps that are specific enough for any tester to follow
diff --git a/packages/playwright/src/agents/generateAgents.ts b/packages/playwright/src/agents/generateAgents.ts
index 5184c6a9cd..5a5dff14a9 100644
--- a/packages/playwright/src/agents/generateAgents.ts
+++ b/packages/playwright/src/agents/generateAgents.ts
@@ -243,26 +243,13 @@ export class CopilotGenerator {
await deleteFile(`.github/chatmodes/ 🎭 planner.chatmode.md`, 'legacy planner chatmode');
await deleteFile(`.github/chatmodes/🎭 generator.chatmode.md`, 'legacy generator chatmode');
await deleteFile(`.github/chatmodes/🎭 healer.chatmode.md`, 'legacy healer chatmode');
+ await deleteFile(`.github/agents/ 🎭 planner.agent.md`, 'legacy planner agent');
+ await deleteFile(`.github/agents/🎭 generator.agent.md`, 'legacy generator agent');
+ await deleteFile(`.github/agents/🎭 healer.agent.md`, 'legacy healer agent');
await VSCodeGenerator.appendToMCPJson();
- const cwdFolder = path.basename(process.cwd());
- const mcpConfig = {
- 'mcpServers': {
- 'playwright-test': {
- 'type': 'stdio',
- 'command': 'npx',
- 'args': [
- `--prefix=/home/runner/work/${cwdFolder}/${cwdFolder}`,
- 'playwright',
- 'run-test-mcp-server',
- '--headless',
- `--config=/home/runner/work/${cwdFolder}/${cwdFolder}`
- ],
- 'tools': ['*']
- }
- }
- };
+ const mcpConfig = { mcpServers: CopilotGenerator.mcpServers };
if (!fs.existsSync('.github/copilot-setup-steps.yml')) {
const yaml = fs.readFileSync(path.join(__dirname, 'copilot-setup-steps.yml'), 'utf-8');
@@ -283,10 +270,11 @@ export class CopilotGenerator {
const examples = agent.examples.length ? ` Examples: ${agent.examples.map(example => `<example>${example}</example>`).join('')}` : '';
const lines: string[] = [];
const header = {
- name: agent.header.name,
- description: agent.header.description + examples,
- tools: agent.header.tools,
- model: 'Claude Sonnet 4',
+ 'name': agent.header.name,
+ 'description': agent.header.description + examples,
+ 'tools': agent.header.tools,
+ 'model': 'Claude Sonnet 4',
+ 'mcp-servers': CopilotGenerator.mcpServers,
};
lines.push(`---`);
lines.push(yaml.stringify(header) + `---`);
@@ -295,6 +283,18 @@ export class CopilotGenerator {
lines.push('');
return lines.join('\n');
}
+
+ static mcpServers = {
+ 'playwright-test': {
+ 'type': 'stdio',
+ 'command': 'npx',
+ 'args': [
+ 'playwright',
+ 'run-test-mcp-server'
+ ],
+ 'tools': ['*']
+ },
+ };
}
export class VSCodeGenerator {
diff --git a/packages/playwright/src/mcp/test/testContext.ts b/packages/playwright/src/mcp/test/testContext.ts
index e27b1fa505..d556d806a5 100644
--- a/packages/playwright/src/mcp/test/testContext.ts
+++ b/packages/playwright/src/mcp/test/testContext.ts
@@ -15,6 +15,7 @@
*/
import fs from 'fs';
+import os from 'os';
import path from 'path';
import { noColors, escapeRegExp } from 'playwright-core/lib/utils';
@@ -75,6 +76,7 @@ export class TestContext {
private _pushClient: MDBPushClientCallback;
private _testRunner: TestRunner | undefined;
readonly options?: { muteConsole?: boolean, headless?: boolean };
+ readonly computedHeaded: boolean;
configLocation!: ConfigLocation;
rootPath!: string;
generatorJournal: GeneratorJournal | undefined;
@@ -82,6 +84,10 @@ export class TestContext {
constructor(pushClient: MDBPushClientCallback, options?: { muteConsole?: boolean, headless?: boolean }) {
this._pushClient = pushClient;
this.options = options;
+ if (options?.headless !== undefined)
+ this.computedHeaded = !options.headless;
+ else
+ this.computedHeaded = !process.env.CI && !(os.platform() === 'linux' && !process.env.DISPLAY);
}
initialize(rootPath: string | undefined, configLocation: ConfigLocation) {
@@ -159,7 +165,7 @@ export class TestContext {
async runSeedTest(seedFile: string, projectName: string, progress: ProgressCallback) {
await this.runWithGlobalSetup(async (testRunner, reporter) => {
const result = await testRunner.runTests(reporter, {
- headed: !this.options?.headless,
+ headed: this.computedHeaded,
locations: ['/' + escapeRegExp(seedFile) + '/'],
projects: [projectName],
timeout: 0,
diff --git a/packages/playwright/src/mcp/test/testTools.ts b/packages/playwright/src/mcp/test/testTools.ts
index 096d6bca90..5087a3ba19 100644
--- a/packages/playwright/src/mcp/test/testTools.ts
+++ b/packages/playwright/src/mcp/test/testTools.ts
@@ -80,7 +80,7 @@ export const debugTest = defineTestTool({
handle: async (context, params, progress) => {
await context.runWithGlobalSetup(async (testRunner, reporter) => {
await testRunner.runTests(reporter, {
- headed: !context.options?.headless,
+ headed: context.computedHeaded,
testIds: [params.test.id],
// For automatic recovery
timeout: 0,