Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
feat(test): support per-project webServer configuration (#40869)
Yury Semikhatskyed9185086f4c
diff --git a/docs/src/api/params.md b/docs/src/api/params.mdindex eae785f347..491395efd7 100644--- a/docs/src/api/params.md+++ b/docs/src/api/params.md@@ -1965,3 +1965,24 @@ In this config:1. Since `snapshotPathTemplate` resolves to relative path, it will be resolved relative to `configDir`.1. Forward slashes `"/"` can be used as path separators on any platform.+## test-config-web-server-options+* langs: js+- type: ?<[Object]|[Array]<[Object]>>+ - `command` <[string]> Shell command to start. For example `npm run start`..+ - `cwd` ?<[string]> Current working directory of the spawned process, defaults to the directory of the configuration file.+ - `env` ?<[Object]<[string], [string]>> Environment variables to set for the command, `process.env` by default.+ - `gracefulShutdown` ?<[Object]> How to shut down the process. If unspecified, the process group is forcefully `SIGKILL`ed. If set to `{ signal: 'SIGTERM', timeout: 500 }`, the process group is sent a `SIGTERM` signal, followed by `SIGKILL` if it doesn't exit within 500ms. You can also use `SIGINT` as the signal instead. A `0` timeout means no `SIGKILL` will be sent. Windows doesn't support `SIGTERM` and `SIGINT` signals, so this option is ignored on Windows. Note that shutting down a Docker container requires `SIGTERM`.+ - `signal` <["SIGINT"|"SIGTERM"]>+ - `timeout` <[int]>+ - `ignoreHTTPSErrors` ?<[boolean]> Whether to ignore HTTPS errors when fetching the `url`. Defaults to `false`.+ - `name` ?<[string]> Specifies a custom name for the web server. This name will be prefixed to log messages. Defaults to `[WebServer]`.+ - `port` ?<[int]> The port that your http server is expected to appear on. It does wait until it accepts connections. Either `port` or `url` should be specified.+ - `reuseExistingServer` ?<[boolean]> If true, it will re-use an existing server on the `port` or `url` when available. If no server is running on that `port` or `url`, it will run the command to start a new server. If `false`, it will throw if an existing process is listening on the `port` or `url`. This should be commonly set to `!process.env.CI` to allow the local dev server when running tests locally.+ - `stderr` ?<["pipe"|"ignore"]> Whether to pipe the stderr of the command to the process stderr or ignore it. Defaults to `"pipe"`.+ - `stdout` ?<["pipe"|"ignore"]> If `"pipe"`, it will pipe the stdout of the command to the process stdout. If `"ignore"`, it will ignore the stdout of the command. Default to `"ignore"`.+ - `wait` ?<[Object]> Consider command started only when given output has been produced.+ - `stdout` ?<[RegExp]> Regular expression to wait for in the `stdout` of the command output. Named capture groups are stored in the environment, for example `/Listening on port (?<my_server_port>\d+)/` will store the port number in `process.env['MY_SERVER_PORT']`.+ - `stderr` ?<[RegExp]> Regular expression to wait for in the `stderr` of the command output. Named capture groups are stored in the environment, for example `/Listening on port (?<my_server_port>\d+)/` will store the port number in `process.env['MY_SERVER_PORT']`.+ - `timeout` ?<[int]> How long to wait for the process to start up and be available in milliseconds. Defaults to 60000.+ - `url` ?<[string]> The url on your http server that is expected to return a 2xx, 3xx, 400, 401, 402, or 403 status code when the server is ready to accept connections. Redirects (3xx status codes) are being followed and the new location is checked. Either `port` or `url` should be specified.+diff --git a/docs/src/test-api/class-testconfig.md b/docs/src/test-api/class-testconfig.mdindex 88b059a35a..83cae0624a 100644--- a/docs/src/test-api/class-testconfig.md+++ b/docs/src/test-api/class-testconfig.md@@ -690,26 +690,8 @@ export default defineConfig({});```-## property: TestConfig.webServer+## property: TestConfig.webServer = %%-test-config-web-server-options-%%* since: v1.10-- type: ?<[Object]|[Array]<[Object]>>- - `command` <[string]> Shell command to start. For example `npm run start`..- - `cwd` ?<[string]> Current working directory of the spawned process, defaults to the directory of the configuration file.- - `env` ?<[Object]<[string], [string]>> Environment variables to set for the command, `process.env` by default.- - `gracefulShutdown` ?<[Object]> How to shut down the process. If unspecified, the process group is forcefully `SIGKILL`ed. If set to `{ signal: 'SIGTERM', timeout: 500 }`, the process group is sent a `SIGTERM` signal, followed by `SIGKILL` if it doesn't exit within 500ms. You can also use `SIGINT` as the signal instead. A `0` timeout means no `SIGKILL` will be sent. Windows doesn't support `SIGTERM` and `SIGINT` signals, so this option is ignored on Windows. Note that shutting down a Docker container requires `SIGTERM`.- - `signal` <["SIGINT"|"SIGTERM"]>- - `timeout` <[int]>- - `ignoreHTTPSErrors` ?<[boolean]> Whether to ignore HTTPS errors when fetching the `url`. Defaults to `false`.- - `name` ?<[string]> Specifies a custom name for the web server. This name will be prefixed to log messages. Defaults to `[WebServer]`.- - `port` ?<[int]> The port that your http server is expected to appear on. It does wait until it accepts connections. Either `port` or `url` should be specified.- - `reuseExistingServer` ?<[boolean]> If true, it will re-use an existing server on the `port` or `url` when available. If no server is running on that `port` or `url`, it will run the command to start a new server. If `false`, it will throw if an existing process is listening on the `port` or `url`. This should be commonly set to `!process.env.CI` to allow the local dev server when running tests locally.- - `stderr` ?<["pipe"|"ignore"]> Whether to pipe the stderr of the command to the process stderr or ignore it. Defaults to `"pipe"`.- - `stdout` ?<["pipe"|"ignore"]> If `"pipe"`, it will pipe the stdout of the command to the process stdout. If `"ignore"`, it will ignore the stdout of the command. Default to `"ignore"`.- - `wait` ?<[Object]> Consider command started only when given output has been produced.- - `stdout` ?<[RegExp]> Regular expression to wait for in the `stdout` of the command output. Named capture groups are stored in the environment, for example `/Listening on port (?<my_server_port>\d+)/` will store the port number in `process.env['MY_SERVER_PORT']`.- - `stderr` ?<[RegExp]> Regular expression to wait for in the `stderr` of the command output. Named capture groups are stored in the environment, for example `/Listening on port (?<my_server_port>\d+)/` will store the port number in `process.env['MY_SERVER_PORT']`.- - `timeout` ?<[int]> How long to wait for the process to start up and be available in milliseconds. Defaults to 60000.- - `url` ?<[string]> The url on your http server that is expected to return a 2xx, 3xx, 400, 401, 402, or 403 status code when the server is ready to accept connections. Redirects (3xx status codes) are being followed and the new location is checked. Either `port` or `url` should be specified.Launch a development web server (or multiple) during the tests.diff --git a/docs/src/test-api/class-testproject.md b/docs/src/test-api/class-testproject.mdindex ce55beb069..6cb2a98e14 100644--- a/docs/src/test-api/class-testproject.md+++ b/docs/src/test-api/class-testproject.md@@ -395,6 +395,48 @@ export default defineConfig({Use [`property: TestConfig.use`] to change this option for all projects.+## property: TestProject.webServer = %%-test-config-web-server-options-%%+* since: v1.61++Launch a development web server (or multiple) before running tests in this project. See [`property: TestConfig.webServer`] for the shape of each entry.++A per-project `webServer` is only launched when the project is selected (either directly via `--project` or indirectly through dependencies). This is useful when only a subset of your projects need a local backend, while others run against a deployed environment.++Per-project web servers are launched in addition to any top-level [`property: TestConfig.webServer`].++**Usage**++```js title="playwright.config.ts"+import { defineConfig } from '@playwright/test';++export default defineConfig({+ projects: [+ {+ name: 'functional',+ grepInvert: /@smoke/,+ use: { baseURL: 'http://localhost:3000' },+ webServer: [+ {+ command: 'npm run start',+ url: 'http://localhost:3000',+ reuseExistingServer: !process.env.CI,+ },+ {+ command: 'npm run mock-server',+ port: 3001,+ reuseExistingServer: !process.env.CI,+ },+ ],+ },+ {+ name: 'smoke',+ grep: /@smoke/,+ use: { baseURL: 'https://production.app.com' },+ },+ ],+});+```+## property: TestProject.workers* since: v1.52- type: ?<[int]|[string]>diff --git a/packages/playwright/src/common/config.ts b/packages/playwright/src/common/config.tsindex a1098ef099..475386f853 100644--- a/packages/playwright/src/common/config.ts+++ b/packages/playwright/src/common/config.ts@@ -129,7 +129,8 @@ export class FullConfigInternal {}// When no projects are defined, do not use config.workers as a hard limit for project.workers.- const projectConfigs = configCLIOverrides.projects || userConfig.projects || [{ ...userConfig, workers: undefined }];+ // Strip webServer from the implicit default project — it is already accounted for at the top level.+ const projectConfigs = configCLIOverrides.projects || userConfig.projects || [{ ...userConfig, workers: undefined, webServer: undefined }];this.projects = projectConfigs.map(p => new FullProjectInternal(configDir, userConfig, this, p, this.configCLIOverrides, packageJsonDir));resolveProjectDependencies(this.projects);this._assignUniqueProjectIds(this.projects);@@ -161,6 +162,7 @@ export class FullProjectInternal {readonly respectGitIgnore: boolean;readonly snapshotPathTemplate: string | undefined;readonly workers: number | undefined;+ readonly webServers: NonNullable<FullConfig['webServer']>[];id = '';deps: FullProjectInternal[] = [];teardown: FullProjectInternal | undefined;@@ -169,6 +171,8 @@ export class FullProjectInternal {this.fullConfig = fullConfig;const testDir = takeFirst(pathResolve(configDir, projectConfig.testDir), pathResolve(configDir, config.testDir), fullConfig.configDir);this.snapshotPathTemplate = takeFirst(projectConfig.snapshotPathTemplate, config.snapshotPathTemplate);+ const webServer = projectConfig.webServer;+ this.webServers = Array.isArray(webServer) ? webServer : webServer ? [webServer] : [];this.project = {grep: takeFirst(projectConfig.grep, config.grep, defaultGrep),diff --git a/packages/playwright/src/common/configLoader.ts b/packages/playwright/src/common/configLoader.tsindex 166313e063..b7bd1c9b1b 100644--- a/packages/playwright/src/common/configLoader.ts+++ b/packages/playwright/src/common/configLoader.ts@@ -328,6 +328,19 @@ function validateProject(file: string, project: Project, title: string) {else if (typeof project.workers === 'string' && !project.workers.endsWith('%'))throw errorWithFile(file, `${title}.workers must be a number or percentage`);}++ if ('webServer' in project && project.webServer !== undefined) {+ const webServer = project.webServer;+ const isArray = Array.isArray(webServer);+ const items = isArray ? webServer : [webServer];+ items.forEach((item, index) => {+ const itemTitle = isArray ? `${title}.webServer[${index}]` : `${title}.webServer`;+ if (!item || typeof item !== 'object')+ throw errorWithFile(file, `${itemTitle} must be an object`);+ if (item.command !== undefined && (typeof item.command !== 'string' || !item.command))+ throw errorWithFile(file, `${itemTitle}.command must be a non-empty string`);+ });+ }}export function resolveConfigLocation(configFile: string | undefined): ConfigLocation {diff --git a/packages/playwright/src/plugins/index.ts b/packages/playwright/src/plugins/index.tsindex 4f2e878a4b..509874f2e4 100644--- a/packages/playwright/src/plugins/index.ts+++ b/packages/playwright/src/plugins/index.ts@@ -30,6 +30,9 @@ export interface TestRunnerPlugin {export type TestRunnerPluginRegistration = {factory: TestRunnerPlugin | (() => TestRunnerPlugin | Promise<TestRunnerPlugin>);instance?: TestRunnerPlugin;+ // When set, the plugin is only set up when the project (or their+ // transitive closure of dependencies/teardowns) is selected to run.+ projectId?: string;};export { webServer } from './webServerPlugin';diff --git a/packages/playwright/src/plugins/webServerPlugin.ts b/packages/playwright/src/plugins/webServerPlugin.tsindex 804bf96968..777abd25dc 100644--- a/packages/playwright/src/plugins/webServerPlugin.ts+++ b/packages/playwright/src/plugins/webServerPlugin.ts@@ -24,7 +24,7 @@ import { raceAgainstDeadline } from '@isomorphic/timeoutRunner';import { isURLAvailable } from '@utils/network';import { launchProcess } from '@utils/processLauncher';-import type { TestRunnerPlugin } from '.';+import type { TestRunnerPlugin, TestRunnerPluginRegistration } from '.';import type { FullConfig } from '../../types/testReporter';import type { FullConfigInternal } from '../common';import type { ReporterV2 } from '../reporters/reporterV2';@@ -258,27 +258,35 @@ export const webServer = (options: WebServerPluginOptions): TestRunnerPlugin =>return new WebServerPlugin(options, false);};-export const webServerPluginsForConfig = (config: FullConfigInternal): TestRunnerPlugin[] => {+export const webServerPluginsForConfig = (config: FullConfigInternal): TestRunnerPluginRegistration[] => {const shouldSetBaseUrl = !!config.config.webServer;- const webServerPlugins = [];- for (const webServerConfig of config.webServers) {- if (webServerConfig.port && webServerConfig.url)- throw new Error(`Either 'port' or 'url' should be specified in config.webServer.`);-- let url: string | undefined;- if (webServerConfig.port || webServerConfig.url) {- url = webServerConfig.url || `http://localhost:${webServerConfig.port}`;-- // We only set base url when only the port is given. That's a legacy mode we have regrets about.- if (shouldSetBaseUrl && !webServerConfig.url)- process.env.PLAYWRIGHT_TEST_BASE_URL = url;- }- webServerPlugins.push(new WebServerPlugin({ ...webServerConfig, url }, webServerConfig.port !== undefined));+ const plugins: TestRunnerPluginRegistration[] = [];+ for (const webServerConfig of config.webServers)+ plugins.push({ factory: createWebServerPlugin(webServerConfig, shouldSetBaseUrl) });++ for (const project of config.projects) {+ for (const webServerConfig of project.webServers)+ plugins.push({ factory: createWebServerPlugin(webServerConfig, false), projectId: project.id });}- return webServerPlugins;+ return plugins;};+function createWebServerPlugin(webServerConfig: WebServerPluginOptions & { port?: number }, shouldSetBaseUrl: boolean): TestRunnerPlugin {+ if (webServerConfig.port && webServerConfig.url)+ throw new Error(`Either 'port' or 'url' should be specified in config.webServer.`);++ let url: string | undefined;+ if (webServerConfig.port || webServerConfig.url) {+ url = webServerConfig.url || `http://localhost:${webServerConfig.port}`;++ // We only set base url when only the port is given. That's a legacy mode we have regrets about.+ if (shouldSetBaseUrl && !webServerConfig.url)+ process.env.PLAYWRIGHT_TEST_BASE_URL = url;+ }+ return new WebServerPlugin({ ...webServerConfig, url }, webServerConfig.port !== undefined);+}+function prefixOutputLines(output: string, prefixName: string = 'WebServer'): string {const lastIsNewLine = output[output.length - 1] === '\n';let lines = output.split('\n');diff --git a/packages/playwright/src/runner/tasks.ts b/packages/playwright/src/runner/tasks.tsindex c2a3a9b5dd..a6196a8115 100644--- a/packages/playwright/src/runner/tasks.ts+++ b/packages/playwright/src/runner/tasks.ts@@ -25,7 +25,7 @@ import { removeFolders } from '@utils/fileUtils';import { Dispatcher } from './dispatcher';import { collectProjectsAndTestFiles, createRootSuite, loadFileSuites, loadGlobalHook, loadTestList } from './loadUtils';-import { buildDependentProjects, buildTeardownToSetupsMap, filterProjects } from './projectUtils';+import { buildDependentProjects, buildProjectsClosure, buildTeardownToSetupsMap, filterProjects } from './projectUtils';import { applySuggestedRebaselines, clearSuggestedRebaselines } from './rebase';import { TaskRunner } from './taskRunner';import { detectChangedTestFiles } from './vcs';@@ -96,12 +96,16 @@ export class TestRun {readonly loadFileFilters: Matcher[] = [];readonly preOnlyTestFilters: TestCaseFilter[] = [];readonly postShardTestFilters: TestCaseFilter[] = [];+ readonly projectClosureIds: Set<string>;constructor(config: FullConfigInternal, reporter: InternalReporter, options?: TestRunOptions) {this.config = config;this.options = options ?? {};this.reporter = reporter;this.filteredProjects = filterProjects(config.projects, this.options.projectFilter);+ this.projectClosureIds = new Set();+ for (const project of buildProjectsClosure(this.filteredProjects).keys())+ this.projectClosureIds.add(project.id);}onTestPaused(params: TestPausedParams) {@@ -191,12 +195,14 @@ export function createReportBeginTask(): Task<TestRun> {export function createPluginSetupTasks(config: FullConfigInternal): Task<TestRun>[] {return config.plugins.map(plugin => ({title: 'plugin setup',- setup: async ({ reporter }) => {+ setup: async testRun => {+ if (plugin.projectId && !testRun.projectClosureIds.has(plugin.projectId))+ return;if (typeof plugin.factory === 'function')plugin.instance = await plugin.factory();elseplugin.instance = plugin.factory;- await plugin.instance?.setup?.(config.config, config.configDir, reporter);+ await plugin.instance?.setup?.(config.config, config.configDir, testRun.reporter);},teardown: async () => {await plugin.instance?.teardown?.();@@ -208,6 +214,8 @@ function createPluginBeginTask(plugin: TestRunnerPluginRegistration): Task<TestRreturn {title: 'plugin begin',setup: async testRun => {+ if (plugin.projectId && !testRun.projectClosureIds.has(plugin.projectId))+ return;await plugin.instance?.begin?.(testRun.rootSuite!);},teardown: async () => {diff --git a/packages/playwright/src/runner/testRunner.ts b/packages/playwright/src/runner/testRunner.tsindex 0d40a9ac13..3f1f250dee 100644--- a/packages/playwright/src/runner/testRunner.ts+++ b/packages/playwright/src/runner/testRunner.ts@@ -392,7 +392,7 @@ export class TestRunner extends EventEmitter<TestRunnerEventMap> {const config = await configLoader.loadConfig(this.configLocation, overrides);// Preserve plugin instances between setup and build.if (!this._plugins) {- webServerPluginsForConfig(config).forEach(p => config.plugins.push({ factory: p }));+ config.plugins.push(...webServerPluginsForConfig(config));addGitCommitInfoPlugin(config);this._plugins = config.plugins || [];} else {@@ -442,7 +442,7 @@ export async function runAllTestsWithConfig(config: FullConfigInternal, options:addGitCommitInfoPlugin(config);// Legacy webServer support.- webServerPluginsForConfig(config).forEach(p => config.plugins.push({ factory: p }));+ config.plugins.push(...webServerPluginsForConfig(config));const filteredProjects = filterProjects(config.projects, options.projectFilter);const reporters = await createReporters(config, options.listMode ? 'list' : 'test', undefined, options);diff --git a/packages/playwright/types/test.d.ts b/packages/playwright/types/test.d.tsindex 34651bff3b..52e56752ee 100644--- a/packages/playwright/types/test.d.ts+++ b/packages/playwright/types/test.d.ts@@ -129,6 +129,54 @@ interface TestProject<TestArgs = {}, WorkerArgs = {}> {* all projects.*/use?: UseOptions<TestArgs, WorkerArgs>;+ /**+ * Launch a development web server (or multiple) before running tests in this project. See+ * [testConfig.webServer](https://playwright.dev/docs/api/class-testconfig#test-config-web-server) for the shape of+ * each entry.+ *+ * A per-project `webServer` is only launched when the project is selected (either directly via `--project` or+ * indirectly through dependencies). This is useful when only a subset of your projects need a local backend, while+ * others run against a deployed environment.+ *+ * Per-project web servers are launched in addition to any top-level+ * [testConfig.webServer](https://playwright.dev/docs/api/class-testconfig#test-config-web-server).+ *+ * **Usage**+ *+ * ```js+ * // playwright.config.ts+ * import { defineConfig } from '@playwright/test';+ *+ * export default defineConfig({+ * projects: [+ * {+ * name: 'functional',+ * grepInvert: /@smoke/,+ * use: { baseURL: 'http://localhost:3000' },+ * webServer: [+ * {+ * command: 'npm run start',+ * url: 'http://localhost:3000',+ * reuseExistingServer: !process.env.CI,+ * },+ * {+ * command: 'npm run mock-server',+ * port: 3001,+ * reuseExistingServer: !process.env.CI,+ * },+ * ],+ * },+ * {+ * name: 'smoke',+ * grep: /@smoke/,+ * use: { baseURL: 'https://production.app.com' },+ * },+ * ],+ * });+ * ```+ *+ */+ webServer?: TestConfigWebServer | TestConfigWebServer[];/*** List of projects that need to run before any test in this project runs. Dependencies can be useful for configuring* the global setup actions in a way that every action is in a form of a test. Passing `--no-deps` argument ignoresdiff --git a/tests/playwright-test/web-server.spec.ts b/tests/playwright-test/web-server.spec.tsindex c171637bad..e8f491d415 100644--- a/tests/playwright-test/web-server.spec.ts+++ b/tests/playwright-test/web-server.spec.ts@@ -931,7 +931,7 @@ test('should throw helpful error when command is empty', async ({ runInlineTest`,}, undefined);expect(result.exitCode).toBe(1);- expect(result.output).toContain('config.webServer.command cannot be empty');+ expect(result.output).toContain('webServer[0].command must be a non-empty string');});for (const stdio of ['stdout', 'stderr']) {@@ -990,3 +990,190 @@ for (const stdio of ['stdout', 'stderr']) {expect(result.output).toContain('My server port is 123');});}++test.describe('per-project webServer', () => {+ test('should launch only servers for the selected project', async ({ runInlineTest }, { workerIndex }) => {+ const portA = workerIndex * 4 + 10600;+ const portB = workerIndex * 4 + 10601;+ const result = await runInlineTest({+ 'test.spec.ts': `+ import { test, expect } from '@playwright/test';+ test('connect', async ({ baseURL, page }) => {+ await page.goto('/hello');+ expect(await page.textContent('body')).toBe('hello');+ });+ `,+ 'playwright.config.ts': `+ module.exports = {+ projects: [+ {+ name: 'with-server',+ use: { baseURL: 'http://localhost:${portA}' },+ webServer: {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${portA}',+ url: 'http://localhost:${portA}/hello',+ name: 'ServerA',+ },+ },+ {+ name: 'no-server',+ use: { baseURL: 'http://localhost:${portB}' },+ webServer: {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${portB}',+ url: 'http://localhost:${portB}/hello',+ name: 'ServerB',+ },+ },+ ],+ };+ `,+ }, { project: 'with-server' });+ expect(result.exitCode).toBe(0);+ expect(result.passed).toBe(1);+ expect(result.output).toContain('[ServerA]');+ expect(result.output).not.toContain('[ServerB]');+ });++ test('should launch a per-project server for a project running as dependency', async ({ runInlineTest }, { workerIndex }) => {+ const port = workerIndex * 2 + 10610;+ const result = await runInlineTest({+ 'setup.spec.ts': `+ import { test, expect } from '@playwright/test';+ test('warm up', async ({ baseURL, request }) => {+ const r = await request.get('/hello');+ expect(await r.text()).toBe('hello');+ });+ `,+ 'test.spec.ts': `+ import { test, expect } from '@playwright/test';+ test('use', async ({ baseURL, page }) => {+ await page.goto('/hello');+ expect(await page.textContent('body')).toBe('hello');+ });+ `,+ 'playwright.config.ts': `+ module.exports = {+ use: { baseURL: 'http://localhost:${port}' },+ projects: [+ {+ name: 'setup',+ testMatch: /setup\\.spec\\.ts/,+ webServer: {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${port}',+ url: 'http://localhost:${port}/hello',+ name: 'SetupServer',+ },+ },+ {+ name: 'main',+ testMatch: /test\\.spec\\.ts/,+ dependencies: ['setup'],+ },+ ],+ };+ `,+ }, { project: 'main' });+ expect(result.exitCode).toBe(0);+ expect(result.passed).toBe(2);+ expect(result.output).toContain('[SetupServer]');+ });++ test('should launch top-level webServer regardless of selected project', async ({ runInlineTest }, { workerIndex }) => {+ const topPort = workerIndex * 2 + 10620;+ const projPort = workerIndex * 2 + 10621;+ const result = await runInlineTest({+ 'test.spec.ts': `+ import { test, expect } from '@playwright/test';+ test('check both', async ({ request }) => {+ expect((await (await request.get('http://localhost:${topPort}/hello')).text())).toBe('hello');+ });+ `,+ 'playwright.config.ts': `+ module.exports = {+ webServer: {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${topPort}',+ url: 'http://localhost:${topPort}/hello',+ name: 'TopServer',+ },+ projects: [+ {+ name: 'A',+ webServer: {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${projPort}',+ url: 'http://localhost:${projPort}/hello',+ name: 'ProjAServer',+ },+ },+ {+ name: 'B',+ },+ ],+ };+ `,+ }, { project: 'B' });+ expect(result.exitCode).toBe(0);+ expect(result.passed).toBe(1);+ expect(result.output).toContain('[TopServer]');+ expect(result.output).not.toContain('[ProjAServer]');+ });++ test('should accept an array of webServer per project', async ({ runInlineTest }, { workerIndex }) => {+ const port1 = workerIndex * 2 + 10630;+ const port2 = workerIndex * 2 + 10631;+ const result = await runInlineTest({+ 'test.spec.ts': `+ import { test, expect } from '@playwright/test';+ test('connect', async ({ request }) => {+ expect(await (await request.get('http://localhost:${port1}/hello')).text()).toBe('hello');+ expect(await (await request.get('http://localhost:${port2}/hello')).text()).toBe('hello');+ });+ `,+ 'playwright.config.ts': `+ module.exports = {+ projects: [+ {+ name: 'A',+ webServer: [+ {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${port1}',+ url: 'http://localhost:${port1}/hello',+ name: 'ServerOne',+ },+ {+ command: 'node ${JSON.stringify(SIMPLE_SERVER_PATH)} ${port2}',+ url: 'http://localhost:${port2}/hello',+ name: 'ServerTwo',+ },+ ],+ },+ ],+ };+ `,+ });+ expect(result.exitCode).toBe(0);+ expect(result.passed).toBe(1);+ expect(result.output).toContain('[ServerOne]');+ expect(result.output).toContain('[ServerTwo]');+ });++ test('should validate project.webServer.command', async ({ runInlineTest }) => {+ const result = await runInlineTest({+ 'test.spec.ts': `+ import { test } from '@playwright/test';+ test('pass', async ({}) => {});+ `,+ 'playwright.config.ts': `+ module.exports = {+ projects: [+ {+ name: 'A',+ webServer: { command: '', url: 'http://localhost:1' },+ },+ ],+ };+ `,+ });+ expect(result.exitCode).toBe(1);+ expect(result.output).toContain('webServer.command must be a non-empty string');+ });+});diff --git a/utils/generate_types/overrides-test.d.ts b/utils/generate_types/overrides-test.d.tsindex ac5c365801..d6ec5c49bb 100644--- a/utils/generate_types/overrides-test.d.ts+++ b/utils/generate_types/overrides-test.d.ts@@ -50,6 +50,7 @@ type UseOptions<TestArgs, WorkerArgs> = Partial<WorkerArgs> & Partial<TestArgs>;interface TestProject<TestArgs = {}, WorkerArgs = {}> {use?: UseOptions<TestArgs, WorkerArgs>;+ webServer?: TestConfigWebServer | TestConfigWebServer[];}export interface Project<TestArgs = {}, WorkerArgs = {}> extends TestProject<TestArgs, WorkerArgs> {