Skip to content

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

feat(test-runner): add httpCache config option (#41687)

Pavel Feldmanc04da5281162
diff --git a/docs/src/test-api/class-testconfig.md b/docs/src/test-api/class-testconfig.md
index 848882fe2e..ffe92beda8 100644
--- a/docs/src/test-api/class-testconfig.md
+++ b/docs/src/test-api/class-testconfig.md
@@ -68,6 +68,115 @@ The structure of the git commit metadata is subject to change.
:::
+## property: TestConfig.httpCache
+* since: v1.62
+- type: ?<[Object]>
+ - `dir` <[string]> Directory where the cache is stored, resolved relative to the configuration file.
+ - `match` ?<[string]|[RegExp]|[HttpCachePolicy]> Limits or customizes what is cached. A glob pattern or regular expression restricts caching to requests whose URL matches; a callback returns a per-request decision (see [HttpCachePolicy]). When omitted, every request is considered with the default behavior.
+ - `proxy` ?<[Object]> Upstream proxy for cache misses.
+ - `server` <[string]> Proxy to be used for all requests. HTTP and SOCKS proxies are supported, for example `http://myproxy.com:3128` or `socks5://myproxy.com:3128`. Short form `myproxy.com:3128` is considered an HTTP proxy.
+ - `bypass` ?<[string]> Optional comma-separated domains to bypass proxy.
+ - `username` ?<[string]> Optional username to use if HTTP proxy requires authentication.
+ - `password` ?<[string]> Optional password to use if HTTP proxy requires authentication.
+
+Records network responses to disk and replays them on later runs, so large static
+dependencies are downloaded from a remote server once instead of on every run. This is
+most useful against a slow or remote environment such as staging.
+
+When `httpCache` is set, Playwright starts a caching proxy for the run and routes all
+browser traffic through it. On the first run, eligible responses are recorded under `dir`;
+on subsequent runs they are served from disk without reaching the network. A single proxy
+is shared by all workers, so a resource fetched by one worker is a cache hit for the rest,
+and the cache persists across runs until you delete `dir`.
+
+Loopback traffic (`localhost`, `127.0.0.1`) is never cached — a local dev server already
+serves from disk, so there is nothing to optimize. The cache targets remote origins.
+
+**What is cached by default**
+
+With no `match`, the cache stores only **shared static assets**: successful `GET` requests
+the browser makes for a static subresource — a script, stylesheet, image, font, or media
+element — as reported by the request's `Sec-Fetch-Dest` metadata. These bytes do not depend
+on who is signed in, so replaying them into a fresh browser context is always safe, which
+keeps tests that each create their own context isolated by construction.
+
+The following are therefore **not** cached by default:
+
+* `fetch`/`XMLHttpRequest` (API) requests and top-level documents — their request
+ destination is not a static subresource. This is the dynamic, per-user surface.
+* Any response marked `Cache-Control: no-store`.
+* Any response carrying a personalization signal: `Cache-Control: private`, a `Set-Cookie`
+ header, or `Vary: Cookie`/`Vary: Authorization`.
+
+The `Authorization` and `Cookie` request headers are deliberately ignored when deciding
+what to cache. On a gated staging environment these are a shared environment credential
+attached to every request, not a per-user identity, so caching on their presence would be
+wrong.
+
+Freshness directives (`max-age`, `no-cache`, `Expires`) are ignored: once a response is
+recorded it is replayed until `dir` is deleted, keeping runs deterministic. `Vary` is
+honored — responses are keyed by the request-header values they vary on, and `Vary: *` is
+never stored.
+
+**Customizing with `match`**
+
+A string or [RegExp] restricts caching to requests whose URL matches; other requests pass
+straight through to the network. For full control, pass a callback that returns a decision
+object per request:
+
+* `disposition` — `'cache'` force-stores the response and serves it back, `'no-cache'`
+ bypasses the cache entirely, and `'default'` (or an empty object) applies the rules above.
+* `identity` — a stable principal id (such as a session token) that partitions the cache.
+ Entries recorded under one identity are never served to a request with a different one,
+ so per-user content can be cached without leaking across contexts. The value is hashed
+ into the cache key and never written to disk.
+
+Set `proxy` to fetch cache misses through an upstream proxy — for example, to reach a
+staging environment that is only accessible behind one. Browsers connect to the caching
+proxy, which chains to `proxy` for anything not served from disk.
+
+**Usage**
+
+Cache shared static assets from a staging server with zero configuration:
+
+```js title="playwright.config.ts"
+import { defineConfig } from '@playwright/test';
+
+export default defineConfig({
+ httpCache: { dir: './.network-cache' },
+});
+```
+
+Fetch cache misses through an upstream proxy:
+
+```js title="playwright.config.ts"
+import { defineConfig } from '@playwright/test';
+
+export default defineConfig({
+ httpCache: { dir: './.network-cache', proxy: { server: 'http://myproxy.com:3128' } },
+});
+```
+
+Take control per request — force-cache a per-user API response with session isolation, and
+bypass the cache for others:
+
+```js title="playwright.config.ts"
+import { defineConfig } from '@playwright/test';
+
+export default defineConfig({
+ httpCache: {
+ dir: './.network-cache',
+ match: request => {
+ if (request.url.includes('/api/config'))
+ return { disposition: 'cache', identity: request.headers.get('authorization') };
+ if (request.url.includes('/telemetry'))
+ return { disposition: 'no-cache' };
+ return {};
+ },
+ },
+});
+```
+
## property: TestConfig.expect
* since: v1.10
- type: ?<[Object]>
diff --git a/packages/playwright/src/common/config.ts b/packages/playwright/src/common/config.ts
index 0287e87d3e..c971e8b8b1 100644
--- a/packages/playwright/src/common/config.ts
+++ b/packages/playwright/src/common/config.ts
@@ -48,6 +48,7 @@ export class FullConfigInternal {
readonly projects: FullProjectInternal[] = [];
readonly singleTSConfigPath?: string;
readonly captureGitInfo: Config['captureGitInfo'];
+ readonly httpCache: Config['httpCache'];
readonly retryStrategy: 'immediate' | 'isolated';
defineConfigWasUsed = false;
@@ -68,6 +69,7 @@ export class FullConfigInternal {
this.plugins = (privateConfiguration?.plugins || []).map((p: any) => ({ factory: p }));
this.singleTSConfigPath = pathResolve(configDir, userConfig.tsconfig);
this.captureGitInfo = userConfig.captureGitInfo;
+ this.httpCache = resolveHttpCache(userConfig.httpCache, configDir);
this.retryStrategy = takeFirst(userConfig.retryStrategy, 'immediate');
this.globalSetups = (Array.isArray(userConfig.globalSetup) ? userConfig.globalSetup : [userConfig.globalSetup]).map(s => resolveScript(s, configDir)).filter(script => script !== undefined);
@@ -218,6 +220,12 @@ function resolveReporters(reporters: Config['reporter'], rootDir: string): Repor
});
}
+function resolveHttpCache(httpCache: Config['httpCache'], configDir: string): Config['httpCache'] {
+ if (!httpCache)
+ return undefined;
+ return { ...httpCache, dir: path.resolve(configDir, httpCache.dir) };
+}
+
function resolveWorkers(workers: string | number): number {
if (typeof workers === 'string') {
if (workers.endsWith('%')) {
diff --git a/packages/playwright/src/index.ts b/packages/playwright/src/index.ts
index 34f9d6deda..a8b7cd2d5f 100644
--- a/packages/playwright/src/index.ts
+++ b/packages/playwright/src/index.ts
@@ -226,7 +226,7 @@ const playwrightFixtures: Fixtures<TestFixtures, WorkerFixtures, UtilityTestFixt
}, { scope: 'worker', option: true, box: true }],
video: ['off', { scope: 'worker', option: true, box: true }],
- _browserOptions: [async ({ playwright, headless, channel, launchOptions }, use) => {
+ _browserOptions: [async ({ playwright, headless, channel, launchOptions }, use, workerInfo) => {
const options: LaunchOptions = {
handleSIGINT: false,
...launchOptions,
@@ -237,6 +237,9 @@ const playwrightFixtures: Fixtures<TestFixtures, WorkerFixtures, UtilityTestFixt
options.headless = headless;
if (channel !== undefined)
options.channel = channel;
+ const cacheProxy = cacheProxySettings();
+ if (cacheProxy)
+ options.proxy = cacheProxy;
playwright._defaultLaunchOptions = options;
await use(options);
@@ -367,6 +370,12 @@ const playwrightFixtures: Fixtures<TestFixtures, WorkerFixtures, UtilityTestFixt
if (serviceWorkers !== undefined)
options.serviceWorkers = serviceWorkers;
+ const cacheProxy = cacheProxySettings();
+ if (cacheProxy) {
+ options.proxy = cacheProxy;
+ options.ignoreHTTPSErrors = true;
+ }
+
await use({
...contextOptions,
...options,
@@ -575,6 +584,14 @@ function resolveClientCerticates(clientCertificates: ClientCertificates): Client
const kTracingStarted = Symbol('kTracingStarted');
+function cacheProxySettings(): { server: string, bypass: string } | undefined {
+ const server = process.env.PLAYWRIGHT_TEST_CACHE_PROXY;
+ // The cache targets remote environments (e.g. staging). Loopback is bypassed
+ // so a local dev server - which already serves from disk - is never cached,
+ // consistently across browsers.
+ return server ? { server, bypass: 'localhost, 127.0.0.1, ::1' } : undefined;
+}
+
function connectOptionsFromEnv() {
const wsEndpoint = process.env.PW_TEST_CONNECT_WS_ENDPOINT;
if (!wsEndpoint)
diff --git a/packages/playwright/src/plugins/DEPS.list b/packages/playwright/src/plugins/DEPS.list
index af3c8c1c20..1d04d3acb1 100644
--- a/packages/playwright/src/plugins/DEPS.list
+++ b/packages/playwright/src/plugins/DEPS.list
@@ -1,5 +1,7 @@
[*]
@isomorphic/**
@utils/**
+cacheProxy/**
+../common/
node_modules/colors/safe
node_modules/debug
diff --git a/packages/playwright/src/plugins/cacheProxy/cache.ts b/packages/playwright/src/plugins/cacheProxy/cache.ts
new file mode 100644
index 0000000000..49840f0a66
--- /dev/null
+++ b/packages/playwright/src/plugins/cacheProxy/cache.ts
@@ -0,0 +1,192 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import fs from 'fs';
+import path from 'path';
+
+import { calculateSha1, createGuid } from '@utils/crypto';
+import { existsAsync } from '@utils/fileUtils';
+
+import { computeVaryFields, varyMatches } from './cacheSemantics';
+
+import type { ProxyHeaders } from './cacheSemantics';
+
+const CACHE_VERSION = 1;
+const INLINE_THRESHOLD = 8 * 1024;
+
+export type CachedResponse = {
+ status: number;
+ statusText: string;
+ headers: [string, string][];
+ body: Buffer;
+};
+
+type CacheRecord = {
+ k: string; // key: sha1(method + '\n' + url)
+ u: string; // url, verbatim so the index stays greppable
+ s: number; // status
+ st: string; // statusText
+ hh: [string, string][]; // headers, verbatim (incl. content-encoding)
+ ts: number; // record time, epoch seconds
+ v?: [string, string][]; // Vary fields: request header values this variant is keyed on
+ c?: string; // inline body (base64), or...
+ f?: string; // ...content-addressed blob (sha1)
+};
+
+export class ResponseCache {
+ private _dir: string;
+ private _blobsDir: string;
+ private _indexFile: string;
+ private _metaFile: string;
+ private _index = new Map<string, CacheRecord[]>();
+ private _writeChain: Promise<void> = Promise.resolve();
+ private _initialized = false;
+
+ constructor(dir: string) {
+ this._dir = path.resolve(dir);
+ this._blobsDir = path.join(this._dir, 'blobs');
+ this._indexFile = path.join(this._dir, 'index.jsonl');
+ this._metaFile = path.join(this._dir, 'meta.json');
+ }
+
+ async load() {
+ let content: string;
+ try {
+ content = await fs.promises.readFile(this._indexFile, 'utf8');
+ } catch {
+ return; // No cache yet.
+ }
+ // Discard an incompatible cache instead of mis-parsing it.
+ if (await this._onDiskVersion() !== CACHE_VERSION) {
+ await fs.promises.rm(this._indexFile, { force: true }).catch(() => {});
+ await fs.promises.rm(this._blobsDir, { recursive: true, force: true }).catch(() => {});
+ return;
+ }
+ for (const line of content.split('\n')) {
+ if (!line.trim())
+ continue;
+ try {
+ const record = JSON.parse(line) as CacheRecord;
+ this._variants(record.k).push(record);
+ } catch {
+ // Ignore a malformed line rather than failing the whole cache.
+ }
+ }
+ }
+
+ static key(method: string, url: string, identity: string = ''): string {
+ return calculateSha1(`${method}\n${url}\n${identity}`);
+ }
+
+ async get(key: string, requestHeaders: ProxyHeaders): Promise<CachedResponse | undefined> {
+ const record = this._index.get(key)?.find(candidate => varyMatches(candidate.v, requestHeaders));
+ if (!record)
+ return undefined;
+ try {
+ const body = record.c !== undefined ? Buffer.from(record.c, 'base64') : await fs.promises.readFile(this._blobPath(record.f!));
+ return { status: record.s, statusText: record.st, headers: record.hh, body };
+ } catch {
+ return undefined; // Corrupt/missing blob - treat as a miss.
+ }
+ }
+
+ async set(key: string, url: string, requestHeaders: ProxyHeaders, response: CachedResponse) {
+ const vary = computeVaryFields(response.headers, requestHeaders);
+ const variants = this._variants(key);
+ if (variants.some(existing => sameVary(existing.v, vary)))
+ return;
+ const record: CacheRecord = {
+ k: key,
+ u: url,
+ s: response.status,
+ st: response.statusText,
+ hh: response.headers,
+ ts: Math.floor(Date.now() / 1000),
+ };
+ if (vary)
+ record.v = vary;
+ if (response.body.length >= INLINE_THRESHOLD) {
+ const hash = calculateSha1(response.body);
+ await this._writeBlob(hash, response.body);
+ record.f = hash;
+ } else {
+ record.c = response.body.toString('base64');
+ }
+ variants.push(record);
+ await this._append(record);
+ }
+
+ async flush() {
+ await this._writeChain;
+ }
+
+ private _variants(key: string): CacheRecord[] {
+ let variants = this._index.get(key);
+ if (!variants) {
+ variants = [];
+ this._index.set(key, variants);
+ }
+ return variants;
+ }
+
+ private _blobPath(hash: string): string {
+ return path.join(this._blobsDir, hash.slice(0, 2), hash);
+ }
+
+ private async _writeBlob(hash: string, body: Buffer) {
+ const dest = this._blobPath(hash);
+ if (await existsAsync(dest))
+ return;
+ await fs.promises.mkdir(path.dirname(dest), { recursive: true });
+ const tmp = `${dest}.tmp-${createGuid()}`;
+ await fs.promises.writeFile(tmp, body);
+ try {
+ await fs.promises.rename(tmp, dest);
+ } catch {
+ await fs.promises.rm(tmp, { force: true }).catch(() => {});
+ }
+ }
+
+ private _append(record: CacheRecord): Promise<void> {
+ const line = JSON.stringify(record) + '\n';
+ const run = async () => {
+ if (!this._initialized) {
+ await fs.promises.mkdir(this._dir, { recursive: true });
+ await fs.promises.writeFile(this._metaFile, JSON.stringify({ version: CACHE_VERSION }, null, 2), { flag: 'w' });
+ this._initialized = true;
+ }
+ await fs.promises.appendFile(this._indexFile, line);
+ };
+ const result = this._writeChain.then(run, run);
+ this._writeChain = result.catch(() => {});
+ return result;
+ }
+
+ private async _onDiskVersion(): Promise<number | undefined> {
+ try {
+ const meta = JSON.parse(await fs.promises.readFile(this._metaFile, 'utf8'));
+ return typeof meta.version === 'number' ? meta.version : undefined;
+ } catch {
+ return undefined;
+ }
+ }
+}
+
+function sameVary(a: [string, string][] | undefined, b: [string, string][] | undefined): boolean {
+ if (!a || !b)
+ return !a && !b;
+ return a.length === b.length && a.every(([name, value], i) => b[i][0] === name && b[i][1] === value);
+}
diff --git a/packages/playwright/src/plugins/cacheProxy/cacheSemantics.ts b/packages/playwright/src/plugins/cacheProxy/cacheSemantics.ts
new file mode 100644
index 0000000000..c3beb8f7da
--- /dev/null
+++ b/packages/playwright/src/plugins/cacheProxy/cacheSemantics.ts
@@ -0,0 +1,108 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import type http from 'http';
+
+// RFC 9111 6.1 heuristically cacheable status codes, minus 206 (partial) and
+// temporary redirects (302/307), which only cache with explicit freshness that
+// this cache intentionally ignores.
+const CACHEABLE_STATUS = new Set([200, 203, 204, 300, 301, 308, 404, 405, 410, 414, 501]);
+const SEC_FETCH_DEST_STATIC = new Set(['audio', 'audioworklet', 'embed', 'font', 'image', 'manifest', 'object', 'paintworklet', 'script', 'serviceworker', 'sharedworker', 'style', 'track', 'video', 'worker', 'xslt']);
+
+export type ProxyHeaders = [string, string][] | http.IncomingHttpHeaders;
+
+export function isCacheableStatus(status: number): boolean {
+ return CACHEABLE_STATUS.has(status);
+}
+
+export function isHardBlocked(requestHeaders: ProxyHeaders, status: number, responseHeaders: ProxyHeaders): boolean {
+ if (!isCacheableStatus(status))
+ return true;
+ if (parseCacheControl(requestHeaders).has('no-store'))
+ return true;
+ if (parseCacheControl(responseHeaders).has('no-store'))
+ return true;
+ if (varyNames(responseHeaders).includes('*'))
+ return true;
+ return false;
+}
+
+export function isDefaultStorable(requestHeaders: ProxyHeaders, status: number, responseHeaders: ProxyHeaders): boolean {
+ if (isHardBlocked(requestHeaders, status, responseHeaders))
+ return false;
+ if (isPersonalized(responseHeaders))
+ return false;
+ if (status === 301 || status === 308)
+ return true;
+ return SEC_FETCH_DEST_STATIC.has(headerValue(requestHeaders, 'sec-fetch-dest'));
+}
+
+function isPersonalized(responseHeaders: ProxyHeaders): boolean {
+ if (parseCacheControl(responseHeaders).has('private'))
+ return true;
+ if (headerValues(responseHeaders, 'set-cookie').length)
+ return true;
+ const vary = varyNames(responseHeaders);
+ return vary.includes('cookie') || vary.includes('authorization');
+}
+
+export function parseCacheControl(headers: ProxyHeaders): Set<string> {
+ const directives = new Set<string>();
+ for (const value of headerValues(headers, 'cache-control')) {
+ for (const part of value.split(',')) {
+ const name = part.split('=')[0].trim().toLowerCase();
+ if (name)
+ directives.add(name);
+ }
+ }
+ return directives;
+}
+
+export function computeVaryFields(responseHeaders: ProxyHeaders, requestHeaders: ProxyHeaders): [string, string][] | undefined {
+ const names = varyNames(responseHeaders).filter(name => name !== '*');
+ if (!names.length)
+ return undefined;
+ return names.map(name => [name, headerValue(requestHeaders, name)]);
+}
+
+export function varyMatches(fields: [string, string][] | undefined, requestHeaders: ProxyHeaders): boolean {
+ if (!fields)
+ return true;
+ return fields.every(([name, value]) => headerValue(requestHeaders, name) === value);
+}
+
+function varyNames(headers: ProxyHeaders): string[] {
+ return headerValues(headers, 'vary').flatMap(value => value.split(',').map(name => name.trim().toLowerCase()).filter(Boolean));
+}
+
+function headerValue(headers: ProxyHeaders, name: string): string {
+ return headerValues(headers, name).join(', ');
+}
+
+function headerValues(headers: ProxyHeaders, name: string): string[] {
+ if (Array.isArray(headers)) {
+ const result: string[] = [];
+ for (const [key, value] of headers) {
+ if (key.toLowerCase() === name)
+ result.push(value);
+ }
+ return result;
+ }
+ const value = headers[name];
+ if (value === undefined)
+ return [];
+ return Array.isArray(value) ? value : [value];
+}
diff --git a/packages/playwright/src/plugins/cacheProxy/server.ts b/packages/playwright/src/plugins/cacheProxy/server.ts
new file mode 100644
index 0000000000..9a1d25d92b
--- /dev/null
+++ b/packages/playwright/src/plugins/cacheProxy/server.ts
@@ -0,0 +1,356 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import http from 'http';
+import https from 'https';
+import net from 'net';
+import tls from 'tls';
+
+import { generateSelfSignedCertificate } from '@utils/crypto';
+import { httpHappyEyeballsAgent, httpsHappyEyeballsAgent } from '@utils/happyEyeballs';
+import { createHttpServer, createHttpsServer, createProxyAgent, shouldBypassProxy, startHttpServer } from '@utils/network';
+import { urlMatches } from '@isomorphic/urlMatch';
+
+import { ResponseCache } from './cache';
+import { isDefaultStorable, isHardBlocked } from './cacheSemantics';
+
+import type { CachedResponse } from './cache';
+import type { ProxySettings } from '@utils/network';
+
+const HOP_BY_HOP = new Set(['connection', 'keep-alive', 'proxy-authenticate', 'proxy-authorization', 'te', 'trailer', 'transfer-encoding', 'upgrade', 'proxy-connection']);
+const LOOPBACK_BYPASS = 'localhost, 127.0.0.1, ::1, [::1]';
+
+export type HttpCacheDecision = {
+ disposition?: 'cache' | 'no-cache' | 'default';
+ identity?: string | null;
+};
+export type HttpCachePolicy = (request: Request) => HttpCacheDecision;
+export type HttpCacheMatch = string | RegExp | HttpCachePolicy | undefined;
+
+export type CacheEntry = { cache: ResponseCache, match: HttpCacheMatch };
+
+type Selection = { cache: ResponseCache, read: boolean, write: 'force' | 'never' | 'default', identity: string };
+
+export class CacheProxy {
+ private _entry: CacheEntry;
+ private _proxy: ProxySettings | undefined;
+ private _proxyAgent: http.Agent | undefined;
+ private _httpServer: http.Server;
+ private _httpsServer: https.Server;
+ private _inflight = new Map<string, Promise<CachedResponse | undefined>>();
+
+ constructor(entry: CacheEntry, proxy?: ProxySettings) {
+ this._entry = entry;
+ this._proxy = proxy;
+ this._proxyAgent = createProxyAgent(proxy, undefined, { keepAlive: true });
+ const { cert, key } = generateSelfSignedCertificate();
+
+ this._httpServer = createHttpServer((req, res) => this._handleRequest(req, res, false));
+ this._httpServer.on('connect', (req, socket, head) => this._onConnect(req, socket as net.Socket, head));
+ this._httpServer.on('upgrade', (req, socket, head) => this._onUpgrade(req, socket as net.Socket, head, false));
+ this._httpsServer = createHttpsServer({ cert, key }, (req, res) => this._handleRequest(req, res, true));
+ this._httpsServer.on('upgrade', (req, socket, head) => this._onUpgrade(req, socket as net.Socket, head, true));
+ }
+
+ async start(): Promise<string> {
+ await startHttpServer(this._httpServer, { host: '127.0.0.1', port: 0 });
+ const address = this._httpServer.address() as net.AddressInfo;
+ return `http://127.0.0.1:${address.port}`;
+ }
+
+ async stop() {
+ await Promise.all([
+ new Promise<void>(resolve => this._httpServer.close(() => resolve())),
+ new Promise<void>(resolve => this._httpsServer.close(() => resolve())),
+ ]);
+ this._proxyAgent?.destroy();
+ }
+
+ private _onConnect(req: http.IncomingMessage, socket: net.Socket, head: Buffer) {
+ socket.on('error', () => {});
+ socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
+ // A CONNECT tunnel carries either TLS (https / wss) or a plaintext HTTP
+ // request (http / ws). Peek the first byte - 0x16 is the TLS handshake -
+ // and hand the socket to the matching server instead of assuming TLS.
+ // Pause first so no data is lost between the sniff and the server attaching.
+ const route = (first: Buffer) => {
+ socket.pause();
+ socket.unshift(first);
+ if (first[0] === 0x16) {
+ this._httpsServer.emit('connection', socket);
+ } else {
+ // The http server does not resume a socket handed over while paused
+ // (the TLS server does), so the tunnelled request would never parse.
+ this._httpServer.emit('connection', socket);
+ socket.resume();
+ }
+ };
+ if (head && head.length)
+ route(head);
+ else
+ socket.once('data', route);
+ }
+
+ // A throwing match callback or any handler error must never take down the
+ // runner process the proxy lives in.
+ private _handleRequest(req: http.IncomingMessage, res: http.ServerResponse, isTls: boolean) {
+ req.on('error', () => {});
+ Promise.resolve().then(() => this._onRequest(req, res, isTls)).catch(() => {
+ if (!res.headersSent)
+ res.writeHead(502);
+ res.end();
+ });
+ }
+
+ private async _onRequest(req: http.IncomingMessage, res: http.ServerResponse, isTls: boolean) {
+ req.url = requestUrl(req, isTls);
+ const url = req.url;
+ const method = req.method || 'GET';
+
+ const selected = method === 'GET' ? this._select(req, url) : undefined;
+ if (!selected || (!selected.read && selected.write === 'never')) {
+ this._passThrough(req, res, url, method);
+ return;
+ }
+
+ const key = ResponseCache.key('GET', url, selected.identity);
+ if (selected.read) {
+ const cached = await selected.cache.get(key, req.headers);
+ if (cached) {
+ writeResponse(res, cached);
+ return;
+ }
+ }
+ this._serveMiss(selected, key, url, req, res);
+ }
+
+ private _serveMiss(selected: Selection, key: string, url: string, req: http.IncomingMessage, res: http.ServerResponse) {
+ // Coalesce concurrent misses onto a single upstream fetch, but only while
+ // that fetch is being buffered for storage - a streamed (non-storable)
+ // response can't be shared, so waiters on it fetch their own.
+ const inflight = this._inflight.get(key);
+ if (inflight) {
+ void inflight.then(response => {
+ if (response)
+ writeResponse(res, response);
+ else
+ this._fetchAndServe(selected, key, url, req, res, false);
+ });
+ return;
+ }
+ this._fetchAndServe(selected, key, url, req, res, true);
+ }
+
+ private _fetchAndServe(selected: Selection, key: string, url: string, req: http.IncomingMessage, res: http.ServerResponse, coalesce: boolean) {
+ // Waiters receive the buffered response, or undefined when this fetch
+ // ended up streaming (or failing) and cannot be shared.
+ let finish: (response?: CachedResponse) => void = () => {};
+ if (coalesce) {
+ const buffered = new Promise<CachedResponse | undefined>(resolve => finish = response => {
+ this._inflight.delete(key);
+ resolve(response);
+ });
+ this._inflight.set(key, buffered);
+ }
+ const fail = () => {
+ finish();
+ failResponse(res);
+ };
+
+ const upstream = this._requestUpstream(url, 'GET', forwardHeaders(req.headers));
+ upstream.on('error', fail);
+ res.on('close', () => upstream.destroy());
+ upstream.on('response', proxyRes => {
+ proxyRes.on('error', fail);
+ const status = proxyRes.statusCode || 502;
+ const store = selected.write === 'force'
+ ? !isHardBlocked(req.headers, status, proxyRes.headers)
+ : selected.write === 'default' && isDefaultStorable(req.headers, status, proxyRes.headers);
+ // Stream anything we won't store straight through, so SSE, long-poll, and
+ // large dynamic responses are never buffered in the runner process.
+ if (!store) {
+ finish();
+ res.writeHead(status, filterHeadersFlat(proxyRes.rawHeaders));
+ proxyRes.pipe(res);
+ return;
+ }
+ const chunks: Buffer[] = [];
+ proxyRes.on('data', chunk => chunks.push(chunk));
+ proxyRes.on('end', () => {
+ const response: CachedResponse = {
+ status,
+ statusText: proxyRes.statusMessage || '',
+ headers: pairsFromRaw(proxyRes.rawHeaders).filter(([name]) => !HOP_BY_HOP.has(name.toLowerCase())),
+ body: Buffer.concat(chunks),
+ };
+ finish(response);
+ selected.cache.set(key, url, req.headers, response).catch(() => {});
+ writeResponse(res, response);
+ });
+ });
+ upstream.end();
+ }
+
+ private _select(req: http.IncomingMessage, url: string): Selection | undefined {
+ // The cache targets remote environments; loopback always passes through,
+ // regardless of how the browser or context was pointed at the proxy.
+ let parsed: URL;
+ try {
+ parsed = new URL(url);
+ } catch {
+ return undefined;
+ }
+ if (shouldBypassProxy(parsed, LOOPBACK_BYPASS))
+ return undefined;
+ const entry = this._entry;
+ let decision: HttpCacheDecision = {};
+ if (typeof entry.match === 'function')
+ decision = entry.match(toWebRequest(req, url)) || {};
+ else if (entry.match !== undefined && !urlMatches(undefined, url, entry.match))
+ return undefined;
+ return { cache: entry.cache, identity: decision.identity ?? '', ...resolveDisposition(decision) };
+ }
+
+ private _passThrough(req: http.IncomingMessage, res: http.ServerResponse, url: string, method: string) {
+ const fail = () => failResponse(res);
+ const upstream = this._requestUpstream(url, method, forwardHeaders(req.headers));
+ upstream.on('response', proxyRes => {
+ proxyRes.on('error', fail);
+ res.writeHead(proxyRes.statusCode || 502, filterHeadersFlat(proxyRes.rawHeaders));
+ proxyRes.pipe(res);
+ });
+ upstream.on('error', fail);
+ req.on('error', () => upstream.destroy());
+ req.pipe(upstream);
+ }
+
+ private _requestUpstream(url: string, method: string, headers: http.OutgoingHttpHeaders): http.ClientRequest {
+ const parsed = new URL(url);
+ const isHttps = parsed.protocol === 'https:';
+ const mod = isHttps ? https : http;
+ const direct = this._proxyAgent === undefined || shouldBypassProxy(parsed, this._proxy?.bypass);
+ const agent = direct ? (isHttps ? httpsHappyEyeballsAgent : httpHappyEyeballsAgent) : this._proxyAgent;
+ return mod.request(parsed, {
+ method,
+ headers,
+ agent,
+ rejectUnauthorized: false,
+ });
+ }
+
+ private _onUpgrade(req: http.IncomingMessage, socket: net.Socket, head: Buffer, isTls: boolean) {
+ socket.on('error', () => {});
+ const target = new URL(requestUrl(req, isTls));
+ const host = target.hostname;
+ const port = Number(target.port) || (isTls ? 443 : 80);
+ const onConnect = () => {
+ const lines = [`${req.method} ${target.pathname}${target.search} HTTP/1.1`];
+ for (const [name, value] of Object.entries(req.headers))
+ lines.push(`${name}: ${Array.isArray(value) ? value.join(', ') : value}`);
+ upstream.write(lines.join('\r\n') + '\r\n\r\n');
+ if (head && head.length)
+ upstream.write(head);
+ socket.pipe(upstream);
+ upstream.pipe(socket);
+ };
+ const upstream = isTls
+ ? tls.connect({ host, port, rejectUnauthorized: false, servername: net.isIP(host) ? undefined : host }, onConnect)
+ : net.connect({ host, port }, onConnect);
+ upstream.on('error', () => socket.destroy());
+ }
+}
+
+// Match callbacks receive a standard WHATWG Request. Unlike a browser's,
+// Node's Request keeps "forbidden" headers (host, cookie, sec-fetch-*), so
+// no information is lost; the copy also keeps callbacks from mutating the
+// live incoming request.
+function toWebRequest(req: http.IncomingMessage, url: string): Request {
+ const headers: [string, string][] = [];
+ for (const [name, value] of Object.entries(req.headers)) {
+ if (value === undefined)
+ continue;
+ if (Array.isArray(value))
+ headers.push(...value.map(item => [name, item] as [string, string]));
+ else
+ headers.push([name, value]);
+ }
+ return new Request(url, { method: req.method, headers });
+}
+
+// Tunnelled requests arrive in origin form (`GET /path`), direct proxy
+// requests in absolute form; reconstruct via the Host header when needed.
+function requestUrl(req: http.IncomingMessage, isTls: boolean): string {
+ const raw = req.url || '/';
+ if (/^\w+:\/\//.test(raw))
+ return raw;
+ return `${isTls ? 'https' : 'http'}://${req.headers.host}${raw}`;
+}
+
+function resolveDisposition(decision: HttpCacheDecision): { read: boolean, write: 'force' | 'never' | 'default' } {
+ if (decision.disposition === 'no-cache')
+ return { read: false, write: 'never' };
+ if (decision.disposition === 'cache')
+ return { read: true, write: 'force' };
+ return { read: true, write: 'default' };
+}
+
+function failResponse(res: http.ServerResponse) {
+ if (!res.headersSent) {
+ res.writeHead(502);
+ res.end();
+ } else {
+ res.destroy();
+ }
+}
+
+
+function forwardHeaders(headers: http.IncomingHttpHeaders): http.OutgoingHttpHeaders {
+ const result: http.OutgoingHttpHeaders = {};
+ for (const [name, value] of Object.entries(headers)) {
+ if (value !== undefined && !HOP_BY_HOP.has(name.toLowerCase()))
+ result[name] = value;
+ }
+ return result;
+}
+
+function writeResponse(res: http.ServerResponse, cached: CachedResponse) {
+ const raw: string[] = [];
+ for (const [name, value] of cached.headers) {
+ const lower = name.toLowerCase();
+ if (HOP_BY_HOP.has(lower) || lower === 'content-length')
+ continue;
+ raw.push(name, value);
+ }
+ raw.push('Content-Length', String(cached.body.length));
+ res.writeHead(cached.status, cached.statusText || undefined, raw);
+ res.end(cached.body);
+}
+
+function filterHeadersFlat(rawHeaders: string[]): string[] {
+ const result: string[] = [];
+ for (let i = 0; i + 1 < rawHeaders.length; i += 2) {
+ if (!HOP_BY_HOP.has(rawHeaders[i].toLowerCase()))
+ result.push(rawHeaders[i], rawHeaders[i + 1]);
+ }
+ return result;
+}
+
+function pairsFromRaw(rawHeaders: string[]): [string, string][] {
+ const pairs: [string, string][] = [];
+ for (let i = 0; i + 1 < rawHeaders.length; i += 2)
+ pairs.push([rawHeaders[i], rawHeaders[i + 1]]);
+ return pairs;
+}
diff --git a/packages/playwright/src/plugins/cacheProxyPlugin.ts b/packages/playwright/src/plugins/cacheProxyPlugin.ts
new file mode 100644
index 0000000000..23ce518f81
--- /dev/null
+++ b/packages/playwright/src/plugins/cacheProxyPlugin.ts
@@ -0,0 +1,46 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { CacheProxy } from './cacheProxy/server';
+import { ResponseCache } from './cacheProxy/cache';
+
+import type { TestRunnerPlugin } from '.';
+import type { FullConfigInternal } from '../common';
+
+export const cacheProxyPluginForConfig = (config: FullConfigInternal): TestRunnerPlugin[] => {
+ const httpCache = config.httpCache;
+ if (!httpCache)
+ return [];
+
+ let cache: ResponseCache | undefined;
+ let proxy: CacheProxy | undefined;
+ return [{
+ name: 'playwright:cache-proxy',
+ setup: async () => {
+ cache = new ResponseCache(httpCache.dir);
+ await cache.load();
+ proxy = new CacheProxy({ cache, match: httpCache.match }, httpCache.proxy);
+ process.env.PLAYWRIGHT_TEST_CACHE_PROXY = await proxy.start();
+ },
+ teardown: async () => {
+ delete process.env.PLAYWRIGHT_TEST_CACHE_PROXY;
+ await proxy?.stop();
+ await cache?.flush();
+ proxy = undefined;
+ cache = undefined;
+ },
+ }];
+};
diff --git a/packages/playwright/src/runner/testRunner.ts b/packages/playwright/src/runner/testRunner.ts
index 9f8b9c37d5..65fc0cbde0 100644
--- a/packages/playwright/src/runner/testRunner.ts
+++ b/packages/playwright/src/runner/testRunner.ts
@@ -29,6 +29,7 @@ import { FSWatcher } from './fsWatcher';
import { baseFullConfig } from '../isomorphic/teleReceiver';
import { addGitCommitInfoPlugin } from '../plugins/gitCommitInfoPlugin';
import { webServerPluginsForConfig } from '../plugins/webServerPlugin';
+import { cacheProxyPluginForConfig } from '../plugins/cacheProxyPlugin';
import { internalScreen } from '../reporters/base';
import { InternalReporter } from '../reporters/internalReporter';
import { serializeError } from '../util';
@@ -394,6 +395,7 @@ export class TestRunner extends EventEmitter<TestRunnerEventMap> {
// Preserve plugin instances between setup and build.
if (!this._plugins) {
webServerPluginsForConfig(config).forEach(p => config.plugins.push({ factory: p }));
+ cacheProxyPluginForConfig(config).forEach(p => config.plugins.push({ factory: p }));
addGitCommitInfoPlugin(config);
this._plugins = config.plugins || [];
} else {
@@ -449,6 +451,7 @@ export async function runAllTestsWithConfig(config: FullConfigInternal, options:
// Legacy webServer support.
webServerPluginsForConfig(config).forEach(p => config.plugins.push({ factory: p }));
+ cacheProxyPluginForConfig(config).forEach(p => config.plugins.push({ factory: p }));
const filteredProjects = filterProjects(config.projects, options.projectFilter);
const reporters = await createReporters(config, options.listMode ? 'list' : 'test', undefined, options);
diff --git a/packages/playwright/types/test.d.ts b/packages/playwright/types/test.d.ts
index 44d6742a8c..46005f1248 100644
--- a/packages/playwright/types/test.d.ts
+++ b/packages/playwright/types/test.d.ts
@@ -1415,6 +1415,141 @@ interface TestConfig<TestArgs = {}, WorkerArgs = {}> {
*/
grepInvert?: RegExp|Array<RegExp>;
+ /**
+ * Records network responses to disk and replays them on later runs, so large static dependencies are downloaded from
+ * a remote server once instead of on every run. This is most useful against a slow or remote environment such as
+ * staging.
+ *
+ * When `httpCache` is set, Playwright starts a caching proxy for the run and routes all browser traffic through it.
+ * On the first run, eligible responses are recorded under `dir`; on subsequent runs they are served from disk without
+ * reaching the network. A single proxy is shared by all workers, so a resource fetched by one worker is a cache hit
+ * for the rest, and the cache persists across runs until you delete `dir`.
+ *
+ * Loopback traffic (`localhost`, `127.0.0.1`) is never cached — a local dev server already serves from disk, so there
+ * is nothing to optimize. The cache targets remote origins.
+ *
+ * **What is cached by default**
+ *
+ * With no `match`, the cache stores only **shared static assets**: successful `GET` requests the browser makes for a
+ * static subresource — a script, stylesheet, image, font, or media element — as reported by the request's
+ * `Sec-Fetch-Dest` metadata. These bytes do not depend on who is signed in, so replaying them into a fresh browser
+ * context is always safe, which keeps tests that each create their own context isolated by construction.
+ *
+ * The following are therefore **not** cached by default:
+ * - `fetch`/`XMLHttpRequest` (API) requests and top-level documents — their request destination is not a static
+ * subresource. This is the dynamic, per-user surface.
+ * - Any response marked `Cache-Control: no-store`.
+ * - Any response carrying a personalization signal: `Cache-Control: private`, a `Set-Cookie` header, or `Vary:
+ * Cookie`/`Vary: Authorization`.
+ *
+ * The `Authorization` and `Cookie` request headers are deliberately ignored when deciding what to cache. On a gated
+ * staging environment these are a shared environment credential attached to every request, not a per-user identity,
+ * so caching on their presence would be wrong.
+ *
+ * Freshness directives (`max-age`, `no-cache`, `Expires`) are ignored: once a response is recorded it is replayed
+ * until `dir` is deleted, keeping runs deterministic. `Vary` is honored — responses are keyed by the request-header
+ * values they vary on, and `Vary: *` is never stored.
+ *
+ * **Customizing with `match`**
+ *
+ * A string or [RegExp] restricts caching to requests whose URL matches; other requests pass straight through to the
+ * network. For full control, pass a callback that returns a decision object per request:
+ * - `disposition` — `'cache'` force-stores the response and serves it back, `'no-cache'` bypasses the cache
+ * entirely, and `'default'` (or an empty object) applies the rules above.
+ * - `identity` — a stable principal id (such as a session token) that partitions the cache. Entries recorded under
+ * one identity are never served to a request with a different one, so per-user content can be cached without
+ * leaking across contexts. The value is hashed into the cache key and never written to disk.
+ *
+ * Set `proxy` to fetch cache misses through an upstream proxy — for example, to reach a staging environment that is
+ * only accessible behind one. Browsers connect to the caching proxy, which chains to `proxy` for anything not served
+ * from disk.
+ *
+ * **Usage**
+ *
+ * Cache shared static assets from a staging server with zero configuration:
+ *
+ * ```js
+ * // playwright.config.ts
+ * import { defineConfig } from '@playwright/test';
+ *
+ * export default defineConfig({
+ * httpCache: { dir: './.network-cache' },
+ * });
+ * ```
+ *
+ * Fetch cache misses through an upstream proxy:
+ *
+ * ```js
+ * // playwright.config.ts
+ * import { defineConfig } from '@playwright/test';
+ *
+ * export default defineConfig({
+ * httpCache: { dir: './.network-cache', proxy: { server: 'http://myproxy.com:3128' } },
+ * });
+ * ```
+ *
+ * Take control per request — force-cache a per-user API response with session isolation, and bypass the cache for
+ * others:
+ *
+ * ```js
+ * // playwright.config.ts
+ * import { defineConfig } from '@playwright/test';
+ *
+ * export default defineConfig({
+ * httpCache: {
+ * dir: './.network-cache',
+ * match: request => {
+ * if (request.url.includes('/api/config'))
+ * return { disposition: 'cache', identity: request.headers.get('authorization') };
+ * if (request.url.includes('/telemetry'))
+ * return { disposition: 'no-cache' };
+ * return {};
+ * },
+ * },
+ * });
+ * ```
+ *
+ */
+ httpCache?: {
+ /**
+ * Directory where the cache is stored, resolved relative to the configuration file.
+ */
+ dir: string;
+
+ /**
+ * Limits or customizes what is cached. A glob pattern or regular expression restricts caching to requests whose URL
+ * matches; a callback returns a per-request decision (see [HttpCachePolicy]). When omitted, every request is
+ * considered with the default behavior.
+ */
+ match?: string|RegExp|HttpCachePolicy;
+
+ /**
+ * Upstream proxy for cache misses.
+ */
+ proxy?: {
+ /**
+ * Proxy to be used for all requests. HTTP and SOCKS proxies are supported, for example `http://myproxy.com:3128` or
+ * `socks5://myproxy.com:3128`. Short form `myproxy.com:3128` is considered an HTTP proxy.
+ */
+ server: string;
+
+ /**
+ * Optional comma-separated domains to bypass proxy.
+ */
+ bypass?: string;
+
+ /**
+ * Optional username to use if HTTP proxy requires authentication.
+ */
+ username?: string;
+
+ /**
+ * Optional password to use if HTTP proxy requires authentication.
+ */
+ password?: string;
+ };
+ };
+
/**
* Whether to skip snapshot expectations, such as `expect(value).toMatchSnapshot()` and `await
* expect(page).toHaveScreenshot()`.
@@ -2025,6 +2160,32 @@ export interface Config<TestArgs = {}, WorkerArgs = {}> extends TestConfig<TestA
export type Metadata = { [key: string]: any };
+/**
+ * A per-request caching decision. Every field is optional; an empty object applies the
+ * default behavior.
+ */
+export type HttpCacheDecision = {
+ /**
+ * `'cache'` serves a stored response and force-stores on a miss, `'no-cache'` bypasses the cache (fetch fresh, do
+ * not store), and `'default'` applies the default rules (cache shared static assets only). Defaults to `'default'`.
+ */
+ disposition?: 'cache' | 'no-cache' | 'default';
+
+ /**
+ * A stable principal id, such as a session token, that partitions the cache. Entries recorded under one identity
+ * are never served to a request with a different one, so per-user content can be cached without leaking across
+ * contexts. The value is hashed into the cache key and never written to disk. `null` is treated as anonymous, so
+ * `request.headers.get()` results can be passed directly.
+ */
+ identity?: string | null;
+};
+
+/**
+ * Called for each request to decide how it is cached. Receives a standard
+ * [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request). See [`property: TestConfig.httpCache`].
+ */
+export type HttpCachePolicy = (request: Request) => HttpCacheDecision;
+
/**
* Resolved configuration which is accessible via
* [testInfo.config](https://playwright.dev/docs/api/class-testinfo#test-info-config) and is passed to the test
diff --git a/packages/utils/network.ts b/packages/utils/network.ts
index 3b3790754c..1929ab56f9 100644
--- a/packages/utils/network.ts
+++ b/packages/utils/network.ts
@@ -103,7 +103,7 @@ export function httpRequest(params: HTTPRequestParams, onResponse: (r: http.Inco
return { cancel: e => cancelRequest(e) };
}
-function shouldBypassProxy(url: URL, bypass?: string): boolean {
+export function shouldBypassProxy(url: URL, bypass?: string): boolean {
if (!bypass)
return false;
const domains = bypass.split(',').map(s => {
@@ -124,7 +124,7 @@ function normalizeProxyURL(proxy: string): URL {
return new URL(proxy);
}
-export function createProxyAgent(proxy?: ProxySettings, forUrl?: URL) {
+export function createProxyAgent(proxy?: ProxySettings, forUrl?: URL, agentOptions?: http.AgentOptions) {
if (!proxy)
return;
if (forUrl && proxy.bypass && shouldBypassProxy(forUrl, proxy.bypass))
@@ -140,7 +140,7 @@ export function createProxyAgent(proxy?: ProxySettings, forUrl?: URL) {
else if (proxyURL.protocol === 'socks4:')
proxyURL.protocol = 'socks4a:';
- return new SocksProxyAgent(proxyURL);
+ return new SocksProxyAgent(proxyURL, agentOptions);
}
if (proxy.username) {
proxyURL.username = proxy.username;
@@ -149,11 +149,11 @@ export function createProxyAgent(proxy?: ProxySettings, forUrl?: URL) {
if (forUrl && ['ws:', 'wss:'].includes(forUrl.protocol)) {
// Force CONNECT method for WebSockets.
- return new HttpsProxyAgent(proxyURL);
+ return new HttpsProxyAgent(proxyURL, agentOptions);
}
// TODO: This branch should be different from above. We should use HttpProxyAgent conditional on proxyURL.protocol instead of always using CONNECT method.
- return new HttpsProxyAgent(proxyURL);
+ return new HttpsProxyAgent(proxyURL, agentOptions);
}
export function createHttpServer(requestListener?: (req: http.IncomingMessage, res: http.ServerResponse) => void): http.Server;
diff --git a/tests/library/unit/cache-proxy.spec.ts b/tests/library/unit/cache-proxy.spec.ts
new file mode 100644
index 0000000000..89060af57a
--- /dev/null
+++ b/tests/library/unit/cache-proxy.spec.ts
@@ -0,0 +1,598 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { test as base, expect } from '@playwright/test';
+import http from 'http';
+import https from 'https';
+import net from 'net';
+import fs from 'fs';
+import path from 'path';
+import { WebSocket, WebSocketServer } from 'ws';
+import { HttpsProxyAgent } from 'https-proxy-agent';
+import { generateSelfSignedCertificate } from '@utils/crypto';
+import { CacheProxy } from '../../../packages/playwright/src/plugins/cacheProxy/server';
+import { ResponseCache } from '../../../packages/playwright/src/plugins/cacheProxy/cache';
+import type { CacheEntry } from '../../../packages/playwright/src/plugins/cacheProxy/server';
+import type { ProxySettings } from '@utils/network';
+
+// A non-loopback name that resolves to 127.0.0.1, so the proxy's loopback
+// bypass does not skip our local origin.
+const HOST = 'fake-localhost-127-0-0-1.nip.io';
+
+type RouteHandler = (req: http.IncomingMessage, res: http.ServerResponse) => void;
+type Origin = {
+ port: number;
+ url: (p: string) => string;
+ loopbackUrl: (p: string) => string;
+ wsUrl: (p: string) => string;
+ setRoute: (p: string, h: RouteHandler) => void;
+ hits: (p: string) => number;
+ onWebSocket: (h: (ws: WebSocket) => void) => void;
+ close: () => Promise<void>;
+};
+
+async function startOrigin(tls?: https.ServerOptions): Promise<Origin> {
+ const hits = new Map<string, number>();
+ const routes = new Map<string, RouteHandler>();
+ const handler: RouteHandler = (req, res) => {
+ const p = new URL(req.url || '/', 'http://x').pathname;
+ hits.set(p, (hits.get(p) || 0) + 1);
+ const route = routes.get(p);
+ if (route) {
+ route(req, res);
+ } else {
+ res.writeHead(404);
+ res.end('not found');
+ }
+ };
+ const server = tls ? https.createServer(tls, handler) : http.createServer(handler);
+ const wss = new WebSocketServer({ server });
+ let wsHandler: ((ws: WebSocket) => void) | undefined;
+ wss.on('connection', ws => wsHandler?.(ws));
+ await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
+ const port = (server.address() as net.AddressInfo).port;
+ const scheme = tls ? 'https' : 'http';
+ return {
+ port,
+ url: p => `${scheme}://${HOST}:${port}${p}`,
+ loopbackUrl: p => `${scheme}://127.0.0.1:${port}${p}`,
+ wsUrl: p => `${tls ? 'wss' : 'ws'}://${HOST}:${port}${p}`,
+ setRoute: (p, h) => routes.set(p, h),
+ hits: p => hits.get(p) || 0,
+ onWebSocket: h => { wsHandler = h; },
+ close: () => new Promise<void>(resolve => {
+ for (const client of wss.clients)
+ client.terminate();
+ wss.close();
+ server.closeAllConnections();
+ server.close(() => resolve());
+ }),
+ };
+}
+
+type Result = { status: number, headers: http.IncomingHttpHeaders, body: string };
+
+// Drive a forward-proxy GET (absolute request target) through the proxy and
+// buffer the whole response.
+function drive(proxyAddress: string, targetUrl: string, headers: Record<string, string> = {}, method = 'GET'): Promise<Result> {
+ const proxy = new URL(proxyAddress);
+ const target = new URL(targetUrl);
+ return new Promise((resolve, reject) => {
+ const req = http.request({ host: proxy.hostname, port: Number(proxy.port), method, path: targetUrl, headers: { host: target.host, ...headers } }, res => {
+ const chunks: Buffer[] = [];
+ res.on('data', chunk => chunks.push(chunk));
+ res.on('end', () => resolve({ status: res.statusCode || 0, headers: res.headers, body: Buffer.concat(chunks).toString() }));
+ res.on('error', reject);
+ });
+ req.on('error', reject);
+ req.end();
+ });
+}
+
+// Resolve as soon as the first body chunk arrives, without waiting for the
+// response to end - used to prove streaming responses are not buffered.
+function driveFirstChunk(proxyAddress: string, targetUrl: string, headers: Record<string, string> = {}): Promise<{ status: number, chunk: string }> {
+ const proxy = new URL(proxyAddress);
+ const target = new URL(targetUrl);
+ return new Promise((resolve, reject) => {
+ const req = http.request({ host: proxy.hostname, port: Number(proxy.port), path: targetUrl, headers: { host: target.host, ...headers } }, res => {
+ res.once('data', chunk => {
+ resolve({ status: res.statusCode || 0, chunk: chunk.toString() });
+ req.destroy();
+ });
+ res.on('error', () => {});
+ });
+ req.on('error', reject);
+ req.end();
+ });
+}
+
+// Drive an https (MITM) GET through the proxy via CONNECT.
+function driveTls(proxyAddress: string, targetUrl: string, headers: Record<string, string> = {}): Promise<Result> {
+ const agent = new HttpsProxyAgent(proxyAddress);
+ return new Promise((resolve, reject) => {
+ const req = https.request(targetUrl, { agent, rejectUnauthorized: false, headers } as https.RequestOptions, res => {
+ const chunks: Buffer[] = [];
+ res.on('data', chunk => chunks.push(chunk));
+ res.on('end', () => resolve({ status: res.statusCode || 0, headers: res.headers, body: Buffer.concat(chunks).toString() }));
+ res.on('error', reject);
+ });
+ req.on('error', reject);
+ req.end();
+ });
+}
+
+function wsEcho(proxyAddress: string, wsUrl: string): Promise<string> {
+ return new Promise((resolve, reject) => {
+ const ws = new WebSocket(wsUrl, { agent: new HttpsProxyAgent(proxyAddress), rejectUnauthorized: false } as any);
+ ws.on('open', () => ws.send('ping'));
+ ws.on('message', data => { resolve(data.toString()); ws.close(); });
+ ws.on('error', reject);
+ });
+}
+
+const response = (body: string, headers: [string, string][] = []): { status: number, statusText: string, headers: [string, string][], body: Buffer } =>
+ ({ status: 200, statusText: 'OK', headers, body: Buffer.from(body) });
+
+type Fixtures = {
+ origin: Origin;
+ httpsOrigin: Origin;
+ cacheDir: () => string;
+ startProxy: (entry: CacheEntry, upstream?: ProxySettings) => Promise<string>;
+};
+
+const it = base.extend<Fixtures>({
+ origin: async ({}, use) => {
+ const origin = await startOrigin();
+ await use(origin);
+ await origin.close();
+ },
+ httpsOrigin: async ({}, use) => {
+ const origin = await startOrigin(generateSelfSignedCertificate());
+ await use(origin);
+ await origin.close();
+ },
+ cacheDir: async ({}, use, testInfo) => {
+ let index = 0;
+ await use(() => testInfo.outputPath('cache-' + (index++)));
+ },
+ startProxy: async ({}, use) => {
+ const started: CacheProxy[] = [];
+ await use(async (entry, upstream) => {
+ const proxy = new CacheProxy(entry, upstream);
+ const address = await proxy.start();
+ started.push(proxy);
+ return address;
+ });
+ await Promise.all(started.map(proxy => proxy.stop()));
+ },
+});
+
+// Convenience: a single-cache proxy over a fresh directory.
+async function cachingProxy(startProxy: Fixtures['startProxy'], cacheDir: Fixtures['cacheDir'], match: CacheEntry['match'] = undefined, upstream?: ProxySettings) {
+ const dir = cacheDir();
+ const cache = new ResponseCache(dir);
+ const address = await startProxy({ cache, match }, upstream);
+ return { address, dir, cache };
+}
+
+it.describe('ResponseCache', () => {
+ it('key is deterministic and identity-sensitive', () => {
+ const a = ResponseCache.key('GET', 'http://x/a');
+ expect(ResponseCache.key('GET', 'http://x/a')).toBe(a);
+ expect(ResponseCache.key('GET', 'http://x/a', '')).toBe(a);
+ expect(ResponseCache.key('GET', 'http://x/b')).not.toBe(a);
+ expect(ResponseCache.key('GET', 'http://x/a', 'alice')).not.toBe(ResponseCache.key('GET', 'http://x/a', 'bob'));
+ });
+
+ it('roundtrips status, headers and body', async ({ cacheDir }) => {
+ const cache = new ResponseCache(cacheDir());
+ await cache.load();
+ const key = ResponseCache.key('GET', 'http://x/a');
+ expect(await cache.get(key, {})).toBeUndefined();
+ await cache.set(key, 'http://x/a', {}, response('hello', [['content-type', 'text/plain']]));
+ const got = await cache.get(key, {});
+ expect(got!.status).toBe(200);
+ expect(got!.headers).toEqual([['content-type', 'text/plain']]);
+ expect(got!.body.toString()).toBe('hello');
+ });
+
+ it('stores small bodies inline and large bodies as blobs', async ({ cacheDir }) => {
+ const dir = cacheDir();
+ const cache = new ResponseCache(dir);
+ await cache.load();
+ await cache.set(ResponseCache.key('GET', 'http://x/small'), 'http://x/small', {}, response('tiny'));
+ const big = Buffer.alloc(9000, 0x61);
+ await cache.set(ResponseCache.key('GET', 'http://x/big'), 'http://x/big', {}, { status: 200, statusText: 'OK', headers: [], body: big });
+ await cache.flush();
+ const index = fs.readFileSync(path.join(dir, 'index.jsonl'), 'utf8');
+ expect(index).toContain('"c":'); // inline
+ expect(index).toContain('"f":'); // blob reference
+ expect(fs.existsSync(path.join(dir, 'blobs'))).toBe(true);
+ expect((await cache.get(ResponseCache.key('GET', 'http://x/big'), {}))!.body.length).toBe(9000);
+ });
+
+ it('persists across cache instances', async ({ cacheDir }) => {
+ const dir = cacheDir();
+ const key = ResponseCache.key('GET', 'http://x/a');
+ const first = new ResponseCache(dir);
+ await first.load();
+ await first.set(key, 'http://x/a', {}, response('persisted'));
+ await first.flush();
+ const second = new ResponseCache(dir);
+ await second.load();
+ expect((await second.get(key, {}))!.body.toString()).toBe('persisted');
+ });
+
+ it('discards the cache on version mismatch', async ({ cacheDir }) => {
+ const dir = cacheDir();
+ const key = ResponseCache.key('GET', 'http://x/a');
+ const first = new ResponseCache(dir);
+ await first.load();
+ await first.set(key, 'http://x/a', {}, response('v'));
+ await first.flush();
+ fs.writeFileSync(path.join(dir, 'meta.json'), JSON.stringify({ version: 999 }));
+ const second = new ResponseCache(dir);
+ await second.load();
+ expect(await second.get(key, {})).toBeUndefined();
+ });
+
+ it('load is a no-op without an index', async ({ cacheDir }) => {
+ const cache = new ResponseCache(cacheDir());
+ await cache.load();
+ expect(await cache.get(ResponseCache.key('GET', 'http://x/a'), {})).toBeUndefined();
+ });
+
+ it('keys variants by their vary fields', async ({ cacheDir }) => {
+ const cache = new ResponseCache(cacheDir());
+ await cache.load();
+ const key = ResponseCache.key('GET', 'http://x/v');
+ await cache.set(key, 'http://x/v', { 'x-foo': 'a' }, response('A', [['vary', 'x-foo']]));
+ await cache.set(key, 'http://x/v', { 'x-foo': 'b' }, response('B', [['vary', 'x-foo']]));
+ expect((await cache.get(key, { 'x-foo': 'a' }))!.body.toString()).toBe('A');
+ expect((await cache.get(key, { 'x-foo': 'b' }))!.body.toString()).toBe('B');
+ expect(await cache.get(key, { 'x-foo': 'c' })).toBeUndefined();
+ });
+
+ it('dedups identical variants', async ({ cacheDir }) => {
+ const dir = cacheDir();
+ const cache = new ResponseCache(dir);
+ await cache.load();
+ const key = ResponseCache.key('GET', 'http://x/a');
+ await cache.set(key, 'http://x/a', {}, response('one'));
+ await cache.set(key, 'http://x/a', {}, response('two'));
+ await cache.flush();
+ const lines = fs.readFileSync(path.join(dir, 'index.jsonl'), 'utf8').trim().split('\n');
+ expect(lines).toHaveLength(1);
+ expect((await cache.get(key, {}))!.body.toString()).toBe('one');
+ });
+});
+
+it.describe('default caching', () => {
+ it('caches a static subresource and replays it', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/img', (req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('pixels'); });
+ const { address, dir, cache } = await cachingProxy(startProxy, cacheDir);
+ const first = await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ expect(first.status).toBe(200);
+ expect(first.body).toBe('pixels');
+ expect(origin.hits('/img')).toBe(1);
+ const second = await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ expect(second.body).toBe('pixels');
+ expect(second.headers['content-type']).toBe('image/png');
+ expect(origin.hits('/img')).toBe(1);
+ await cache.flush();
+ expect(fs.existsSync(path.join(dir, 'index.jsonl'))).toBe(true);
+ });
+
+ it('does not cache requests without fetch metadata', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/x', (req, res) => res.end('body'));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/x'));
+ await drive(address, origin.url('/x'));
+ expect(origin.hits('/x')).toBe(2);
+ });
+
+ it('does not cache xhr/fetch (empty destination)', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/api', (req, res) => res.end('data'));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/api'), { 'sec-fetch-dest': 'empty' });
+ await drive(address, origin.url('/api'), { 'sec-fetch-dest': 'empty' });
+ expect(origin.hits('/api')).toBe(2);
+ });
+
+ for (const directive of ['no-store', 'private']) {
+ it(`does not cache Cache-Control: ${directive}`, async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/r', (req, res) => { res.writeHead(200, { 'cache-control': directive }); res.end('x'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/r')).toBe(2);
+ });
+ }
+
+ it('does not cache responses with Set-Cookie', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/r', (req, res) => { res.writeHead(200, { 'set-cookie': 'sid=1' }); res.end('x'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/r')).toBe(2);
+ });
+
+ for (const vary of ['cookie', 'authorization', '*']) {
+ it(`does not cache Vary: ${vary}`, async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/r', (req, res) => { res.writeHead(200, { 'vary': vary }); res.end('x'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/r')).toBe(2);
+ });
+ }
+
+ it('keys cached variants by Vary', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/v', (req, res) => { res.writeHead(200, { 'vary': 'x-variant', 'sec-fetch-dest': 'image' }); res.end('variant:' + (req.headers['x-variant'] || '')); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect((await drive(address, origin.url('/v'), { 'sec-fetch-dest': 'image', 'x-variant': 'a' })).body).toBe('variant:a');
+ expect((await drive(address, origin.url('/v'), { 'sec-fetch-dest': 'image', 'x-variant': 'b' })).body).toBe('variant:b');
+ expect(origin.hits('/v')).toBe(2);
+ expect((await drive(address, origin.url('/v'), { 'sec-fetch-dest': 'image', 'x-variant': 'a' })).body).toBe('variant:a');
+ expect(origin.hits('/v')).toBe(2);
+ });
+
+ for (const status of [301, 308]) {
+ it(`caches ${status} redirects`, async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/old', (req, res) => { res.writeHead(status, { location: '/new' }); res.end(); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect((await drive(address, origin.url('/old'))).status).toBe(status);
+ expect((await drive(address, origin.url('/old'))).status).toBe(status);
+ expect(origin.hits('/old')).toBe(1);
+ });
+ }
+
+ it('does not cache 302 temporary redirects', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/old', (req, res) => { res.writeHead(302, { location: '/new' }); res.end(); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/old'));
+ await drive(address, origin.url('/old'));
+ expect(origin.hits('/old')).toBe(2);
+ });
+
+ it('does not cache non-GET requests', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/p', (req, res) => res.end('x'));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/p'), { 'sec-fetch-dest': 'image' }, 'POST');
+ await drive(address, origin.url('/p'), { 'sec-fetch-dest': 'image' }, 'POST');
+ expect(origin.hits('/p')).toBe(2);
+ });
+
+ it('honors a request-side Cache-Control: no-store', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/r', (req, res) => { res.writeHead(200); res.end('x'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image', 'cache-control': 'no-store' });
+ await drive(address, origin.url('/r'), { 'sec-fetch-dest': 'image', 'cache-control': 'no-store' });
+ expect(origin.hits('/r')).toBe(2);
+ });
+});
+
+it.describe('match callback', () => {
+ it('restricts caching to a URL glob', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/assets/a', (req, res) => res.end('a'));
+ origin.setRoute('/api/b', (req, res) => res.end('b'));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: '**/assets/**' });
+ await drive(address, origin.url('/assets/a'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/assets/a'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/api/b'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/api/b'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/assets/a')).toBe(1);
+ expect(origin.hits('/api/b')).toBe(2);
+ });
+
+ it("force-caches with disposition 'cache' regardless of metadata", async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/api', (req, res) => res.end('data'));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: () => ({ disposition: 'cache' }) });
+ await drive(address, origin.url('/api'));
+ await drive(address, origin.url('/api'));
+ expect(origin.hits('/api')).toBe(1);
+ });
+
+ it("bypasses the cache with disposition 'no-cache'", async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/img', (req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('x'); });
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: () => ({ disposition: 'no-cache' }) });
+ await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/img')).toBe(2);
+ });
+
+ it('partitions the cache by identity', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/me', (req, res) => res.end('user:' + (req.headers['x-user'] || 'guest')));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: request => ({ disposition: 'cache', identity: request.headers.get('x-user') }) });
+ expect((await drive(address, origin.url('/me'), { 'x-user': 'alice' })).body).toBe('user:alice');
+ expect((await drive(address, origin.url('/me'), { 'x-user': 'bob' })).body).toBe('user:bob');
+ expect(origin.hits('/me')).toBe(2);
+ // Alice's own entry, never bob's.
+ expect((await drive(address, origin.url('/me'), { 'x-user': 'alice' })).body).toBe('user:alice');
+ expect((await drive(address, origin.url('/me'))).body).toBe('user:guest');
+ expect(origin.hits('/me')).toBe(3);
+ });
+
+ it('invokes the callback once per request with a WHATWG Request', async ({ origin, startProxy, cacheDir }) => {
+ const seen: { url: string, method: string, dest: string | null, host: string | null, cookie: string | null }[] = [];
+ origin.setRoute('/probe', (req, res) => res.end('x'));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: request => {
+ expect(request).toBeInstanceOf(Request);
+ seen.push({
+ url: request.url,
+ method: request.method,
+ dest: request.headers.get('sec-fetch-dest'),
+ host: request.headers.get('host'),
+ cookie: request.headers.get('cookie'),
+ });
+ return {};
+ } });
+ await drive(address, origin.url('/probe'), { 'sec-fetch-dest': 'image', 'cookie': 'sid=1' });
+ expect(seen).toHaveLength(1);
+ expect(seen[0]).toEqual({
+ url: origin.url('/probe'),
+ method: 'GET',
+ dest: 'image',
+ host: `${HOST}:${origin.port}`,
+ cookie: 'sid=1', // Node's Request keeps "forbidden" headers.
+ });
+ });
+
+ it('passes through a URL that does not match the glob', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/api/b', (req, res) => res.end('b'));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: '**/assets/**' });
+ await drive(address, origin.url('/api/b'), { 'sec-fetch-dest': 'image' });
+ await drive(address, origin.url('/api/b'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/api/b')).toBe(2);
+ });
+});
+
+it.describe('streaming and coalescing', () => {
+ it('streams a non-storable response without buffering', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/stream', (req, res) => { res.writeHead(200, { 'content-type': 'text/plain' }); res.write('first-chunk'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ const { chunk } = await driveFirstChunk(address, origin.url('/stream'));
+ expect(chunk).toBe('first-chunk');
+ });
+
+ it('coalesces concurrent identical misses into one upstream fetch', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/slow', (req, res) => setTimeout(() => res.end('slow'), 200));
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: () => ({ disposition: 'cache' }) });
+ const results = await Promise.all(Array.from({ length: 8 }, () => drive(address, origin.url('/slow'))));
+ expect(results.map(r => r.body)).toEqual(Array(8).fill('slow'));
+ expect(origin.hits('/slow')).toBe(1);
+ });
+
+ it('does not coalesce non-storable responses', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/slow', (req, res) => setTimeout(() => res.end('slow'), 200));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ const results = await Promise.all(Array.from({ length: 5 }, () => drive(address, origin.url('/slow'))));
+ expect(results.map(r => r.body)).toEqual(Array(5).fill('slow'));
+ expect(origin.hits('/slow')).toBe(5);
+ });
+});
+
+it.describe('websockets', () => {
+ it('tunnels a plaintext ws:// connection', async ({ origin, startProxy, cacheDir }) => {
+ origin.onWebSocket(ws => ws.on('message', () => ws.send('pong')));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect(await wsEcho(address, origin.wsUrl('/ws'))).toBe('pong');
+ });
+
+ it('tunnels a secure wss:// connection through MITM', async ({ httpsOrigin, startProxy, cacheDir }) => {
+ httpsOrigin.onWebSocket(ws => ws.on('message', () => ws.send('pong')));
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect(await wsEcho(address, httpsOrigin.wsUrl('/ws'))).toBe('pong');
+ });
+});
+
+it.describe('https MITM', () => {
+ it('caches https responses via TLS termination', async ({ httpsOrigin, startProxy, cacheDir }) => {
+ httpsOrigin.setRoute('/img', (req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('secure'); });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect((await driveTls(address, httpsOrigin.url('/img'), { 'sec-fetch-dest': 'image' })).body).toBe('secure');
+ expect((await driveTls(address, httpsOrigin.url('/img'), { 'sec-fetch-dest': 'image' })).body).toBe('secure');
+ expect(httpsOrigin.hits('/img')).toBe(1);
+ });
+});
+
+it.describe('errors and edge cases', () => {
+ it('returns 502 when the upstream is unreachable', async ({ origin, startProxy, cacheDir }) => {
+ const url = origin.url('/gone');
+ await origin.close();
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ expect((await drive(address, url, { 'sec-fetch-dest': 'image' })).status).toBe(502);
+ });
+
+ it('survives a throwing match callback', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/ok', (req, res) => res.end('ok'));
+ let boom = true;
+ const dir = cacheDir();
+ const address = await startProxy({ cache: new ResponseCache(dir), match: () => {
+ if (boom)
+ throw new Error('boom');
+ return {};
+ } });
+ expect((await drive(address, origin.url('/ok'))).status).toBe(502);
+ boom = false;
+ expect((await drive(address, origin.url('/ok'))).body).toBe('ok');
+ });
+
+ it('passes loopback traffic through without caching', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/img', (req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('x'); });
+ const { address, dir } = await cachingProxy(startProxy, cacheDir);
+ // Addressed over loopback (127.0.0.1) instead of the resolvable name.
+ expect((await drive(address, origin.loopbackUrl('/img'), { 'sec-fetch-dest': 'image' })).body).toBe('x');
+ await drive(address, origin.loopbackUrl('/img'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/img')).toBe(2);
+ expect(fs.existsSync(path.join(dir, 'index.jsonl'))).toBe(false);
+ });
+
+ it('preserves response headers and body verbatim on replay', async ({ origin, startProxy, cacheDir }) => {
+ origin.setRoute('/img', (req, res) => {
+ res.writeHead(200, { 'content-type': 'image/svg+xml', 'x-custom': 'kept', 'content-encoding': 'identity' });
+ res.end('<svg/>');
+ });
+ const { address } = await cachingProxy(startProxy, cacheDir);
+ await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ const replayed = await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/img')).toBe(1);
+ expect(replayed.headers['content-type']).toBe('image/svg+xml');
+ expect(replayed.headers['x-custom']).toBe('kept');
+ expect(replayed.body).toBe('<svg/>');
+ });
+});
+
+it.describe('upstream proxy chaining', () => {
+ it('fetches misses through the configured upstream proxy', async ({ origin, startProxy, cacheDir }) => {
+ const tunnelled: string[] = [];
+ const upstream = http.createServer((req, res) => { res.writeHead(502); res.end(); });
+ upstream.on('connect', (req, socket, head) => {
+ tunnelled.push(req.url!);
+ const [host, port] = req.url!.split(':');
+ const target = net.connect({ host, port: Number(port), family: 4 }, () => {
+ socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
+ if (head.length)
+ target.write(head);
+ socket.pipe(target);
+ target.pipe(socket);
+ });
+ target.on('error', () => socket.destroy());
+ socket.on('error', () => target.destroy());
+ });
+ await new Promise<void>(resolve => upstream.listen(0, '127.0.0.1', resolve));
+ const upstreamUrl = `http://127.0.0.1:${(upstream.address() as net.AddressInfo).port}`;
+
+ origin.setRoute('/img', (req, res) => { res.writeHead(200, { 'content-type': 'image/png' }); res.end('x'); });
+ try {
+ const { address } = await cachingProxy(startProxy, cacheDir, undefined, { server: upstreamUrl });
+ expect((await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' })).body).toBe('x');
+ expect(tunnelled.length).toBeGreaterThan(0);
+ await drive(address, origin.url('/img'), { 'sec-fetch-dest': 'image' });
+ expect(origin.hits('/img')).toBe(1); // second is a cache hit, no new tunnel
+ } finally {
+ await new Promise<void>(resolve => upstream.close(() => resolve()));
+ }
+ });
+});
diff --git a/tests/playwright-test/cache-proxy.spec.ts b/tests/playwright-test/cache-proxy.spec.ts
new file mode 100644
index 0000000000..b6b6f801c7
--- /dev/null
+++ b/tests/playwright-test/cache-proxy.spec.ts
@@ -0,0 +1,536 @@
+/**
+ * Copyright (c) Microsoft Corporation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { test, expect } from './playwright-test-fixtures';
+import fs from 'fs';
+import http from 'http';
+import net from 'net';
+import path from 'path';
+
+async function startUpstreamProxy(): Promise<{ url: string, log: string[], stop: () => Promise<void> }> {
+ const log: string[] = [];
+ const server = http.createServer((req, res) => {
+ res.writeHead(502);
+ res.end();
+ });
+ server.on('connect', (req, socket, head) => {
+ log.push(req.url!);
+ const [host, port] = req.url!.split(':');
+ // 'localhost' resolves to ::1 first, but test servers listen on IPv4.
+ const target = net.connect({ port: Number(port), host, family: 4 }, () => {
+ socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
+ if (head && head.length)
+ target.write(head);
+ socket.pipe(target);
+ target.pipe(socket);
+ });
+ target.on('error', () => socket.destroy());
+ socket.on('error', () => target.destroy());
+ });
+ await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
+ const url = `http://127.0.0.1:${(server.address() as net.AddressInfo).port}`;
+ return { url, log, stop: () => new Promise(resolve => server.close(() => resolve())) };
+}
+
+const loadImage = `src => new Promise(f => { const img = document.createElement('img'); img.onload = img.onerror = f; img.src = src; })`;
+const loadScript = `src => new Promise(f => { const s = document.createElement('script'); s.onload = s.onerror = f; s.src = src; document.head.appendChild(s); })`;
+
+// httpCache bypasses loopback, so tests reach the local server through a
+// non-loopback name that still resolves to 127.0.0.1 - forcing traffic through
+// the caching proxy the way a remote (staging) host would.
+const HOST = 'fake-localhost-127-0-0-1.nip.io';
+const remote = (server: { PORT: number }, tls = false) => `${tls ? 'https' : 'http'}://${HOST}:${server.PORT}`;
+const remoteHost = (server: { PORT: number }) => `${HOST}:${server.PORT}`;
+
+test('should record and replay responses across runs', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ httpsServer.setRoute('/asset.png', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'content-type': 'image/png' });
+ res.end('payload');
+ });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/asset.png');
+ });
+ `,
+ };
+
+ const result1 = await runInlineTest(files, { workers: 1 });
+ expect(result1.exitCode).toBe(0);
+ expect(hits).toBe(1);
+ expect(fs.existsSync(path.join(cacheDir, 'index.jsonl'))).toBe(true);
+ expect(JSON.parse(fs.readFileSync(path.join(cacheDir, 'meta.json'), 'utf8')).version).toBe(1);
+
+ const hitsAfterRecord = hits;
+ const result2 = await runInlineTest(files, { workers: 1 });
+ expect(result2.exitCode).toBe(0);
+ expect(hits).toBe(hitsAfterRecord); // Served entirely from cache, no new httpsServer hit.
+
+ fs.writeFileSync(path.join(cacheDir, 'meta.json'), JSON.stringify({ version: 999 }));
+ const result3 = await runInlineTest(files, { workers: 1 });
+ expect(result3.exitCode).toBe(0);
+ expect(hits).toBe(hitsAfterRecord + 1); // Went back to the httpsServer.
+});
+
+test('should cache https responses via MITM', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ httpsServer.setRoute('/secure.png', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'content-type': 'image/png' });
+ res.end('secure payload');
+ });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/secure.png');
+ });
+ `,
+ };
+
+ const result1 = await runInlineTest(files, { workers: 1 });
+ expect(result1.exitCode).toBe(0);
+ expect(hits).toBe(1); // Proxy terminated TLS, fetched once, cached.
+
+ const hitsAfterRecord = hits;
+ const result2 = await runInlineTest(files, { workers: 1 });
+ expect(result2.exitCode).toBe(0);
+ expect(hits).toBe(hitsAfterRecord); // Replayed from cache over HTTPS, no new hit.
+});
+
+test('should tunnel secure WebSockets through the MITM proxy', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ httpsServer.onceWebSocketConnection(ws => ws.on('message', () => ws.send('pong')));
+
+ const result = await runInlineTest({
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('wss', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ const echoed = await page.evaluate(host => new Promise(resolve => {
+ const ws = new WebSocket('wss://' + host + '/ws');
+ ws.onopen = () => ws.send('ping');
+ ws.onmessage = e => resolve(e.data);
+ ws.onerror = () => resolve('error');
+ }), '${remoteHost(httpsServer)}');
+ expect(echoed).toBe('pong');
+ });
+ `,
+ }, { workers: 1 });
+
+ expect(result.exitCode).toBe(0);
+ expect(result.passed).toBe(1);
+});
+
+test('should tunnel plaintext WebSockets', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ server.onceWebSocketConnection(ws => ws.on('message', () => ws.send('pong')));
+
+ const result = await runInlineTest({
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('ws', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ const echoed = await page.evaluate(host => new Promise(resolve => {
+ const ws = new WebSocket('ws://' + host + '/ws');
+ ws.onopen = () => ws.send('ping');
+ ws.onmessage = e => resolve(e.data);
+ ws.onerror = () => resolve('error');
+ }), '${remoteHost(server)}');
+ expect(echoed).toBe('pong');
+ });
+ `,
+ }, { workers: 1 });
+
+ expect(result.exitCode).toBe(0);
+ expect(result.passed).toBe(1);
+});
+
+test('should stream non-storable responses without buffering', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ // A response that emits a chunk and then never ends; buffering would hang.
+ server.setRoute('/stream', (req, res) => {
+ res.writeHead(200, { 'content-type': 'text/plain' });
+ res.write('first-chunk');
+ });
+
+ const result = await runInlineTest({
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('stream', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ const chunk = await page.evaluate(async () => {
+ const res = await fetch('/stream');
+ const reader = res.body.getReader();
+ const { value } = await reader.read();
+ await reader.cancel();
+ return new TextDecoder().decode(value);
+ });
+ expect(chunk).toBe('first-chunk');
+ });
+ `,
+ }, { workers: 1, timeout: 15000 });
+
+ expect(result.exitCode).toBe(0);
+ expect(result.passed).toBe(1);
+});
+
+test('should cache only shared static assets by default', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ const counts: Record<string, number> = { asset: 0, xhr: 0, private: 0, cookie: 0, noStore: 0 };
+ httpsServer.setRoute('/asset', (req, res) => { ++counts.asset; res.end('a'); });
+ httpsServer.setRoute('/api/data', (req, res) => { ++counts.xhr; res.end('data'); });
+ httpsServer.setRoute('/private', (req, res) => { ++counts.private; res.writeHead(200, { 'cache-control': 'private' }); res.end('a'); });
+ httpsServer.setRoute('/cookie', (req, res) => { ++counts.cookie; res.writeHead(200, { 'set-cookie': 'sid=1' }); res.end('a'); });
+ httpsServer.setRoute('/no-store', (req, res) => { ++counts.noStore; res.writeHead(200, { 'cache-control': 'no-store' }); res.end('a'); });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ // Loaded as image subresources (Sec-Fetch-Dest: image).
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/asset');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/private');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/cookie');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/no-store');
+ // Loaded as an XHR/fetch (Sec-Fetch-Dest: empty).
+ await page.evaluate(() => fetch('/api/data').then(r => r.text()));
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ await runInlineTest(files, { workers: 1 });
+ expect(counts.asset).toBe(1); // Static subresource -> cached, replayed.
+ expect(counts.xhr).toBe(2); // fetch/XHR -> not cached by default.
+ expect(counts.private).toBe(2); // Cache-Control: private -> personalized, not cached.
+ expect(counts.cookie).toBe(2); // Set-Cookie -> personalized, not cached.
+ expect(counts.noStore).toBe(2); // Cache-Control: no-store -> never cached.
+});
+
+test('should isolate cached entries by identity', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ server.setRoute('/me', (req, res) => { ++hits; res.end('user:' + (req.headers['x-user'] || 'guest')); });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: {
+ dir: ${JSON.stringify(cacheDir)},
+ match: request => ({ disposition: 'cache', identity: request.headers.get('x-user') }),
+ } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ const fetchAs = user => page.evaluate(u =>
+ fetch('/me', { headers: { 'x-user': u } }).then(r => r.text()), user);
+ expect(await fetchAs('alice')).toBe('user:alice');
+ expect(await fetchAs('bob')).toBe('user:bob');
+ expect(await fetchAs('alice')).toBe('user:alice'); // Alice's own entry, not bob's.
+ expect(await fetchAs('')).toBe('user:guest'); // Guest identity -> its own namespace.
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(3); // alice, bob, guest each fetched once; the repeat 'alice' was a hit.
+
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(3); // All three identities replayed from cache.
+});
+
+test('should key cached variants by Vary', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ server.setRoute('/variant', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'vary': 'x-variant' });
+ res.end('variant:' + (req.headers['x-variant'] || 'none'));
+ });
+
+ const files = {
+ // Force-cache so the fetch is stored; this isolates the Vary keying.
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)}, match: () => ({ disposition: 'cache' }) } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ const fetchVariant = v => page.evaluate(variant =>
+ fetch('/variant', { headers: { 'x-variant': variant } }).then(r => r.text()), v);
+ expect(await fetchVariant('a')).toBe('variant:a');
+ expect(await fetchVariant('b')).toBe('variant:b');
+ expect(await fetchVariant('a')).toBe('variant:a'); // From cache, still the 'a' body.
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(2); // Two distinct variants fetched once each; the repeat 'a' was a hit.
+
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(2); // Both variants replayed from cache.
+});
+
+test('should not cache Vary: * responses', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ server.setRoute('/star', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'vary': '*' });
+ res.end('a');
+ });
+
+ const files = {
+ // Even when caching is forced, Vary: * is hard-blocked and never stored.
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)}, match: () => ({ disposition: 'cache' }) } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ await page.evaluate(() => fetch('/star').then(r => r.text()));
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(2); // Vary: * is not storable -> network every run.
+});
+
+test('should cache redirects', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let redirects = 0;
+ server.setRoute('/old', (req, res) => {
+ ++redirects;
+ res.writeHead(301, { 'location': '/new' });
+ res.end();
+ });
+ server.setRoute('/new', (req, res) => { res.end('arrived'); });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ expect(await page.evaluate(() => fetch('/old').then(r => r.text()))).toBe('arrived');
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ expect(redirects).toBe(1);
+
+ await runInlineTest(files, { workers: 1 });
+ expect(redirects).toBe(1); // 301 replayed from cache.
+});
+
+test('should not cache loopback traffic', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ server.setRoute('/asset.png', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'content-type': 'image/png' });
+ res.end('payload');
+ });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)} } };`,
+ // Reached over loopback (localhost), which the cache proxy bypasses.
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${server.EMPTY_PAGE}');
+ await page.evaluate(${loadImage}, '${server.PREFIX}/asset.png');
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ await runInlineTest(files, { workers: 1 });
+ expect(hits).toBe(2); // Loopback is not proxied -> never cached.
+ expect(fs.existsSync(path.join(cacheDir, 'index.jsonl'))).toBe(false);
+});
+
+test('should respect the match callback disposition', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let pinned = 0;
+ let volatileScript = 0;
+ let normalScript = 0;
+ httpsServer.setRoute('/api/pinned', (req, res) => { ++pinned; res.end('pinned'); });
+ httpsServer.setRoute('/volatile.js', (req, res) => {
+ ++volatileScript;
+ res.writeHead(200, { 'content-type': 'application/javascript' });
+ res.end(';');
+ });
+ httpsServer.setRoute('/normal.js', (req, res) => {
+ ++normalScript;
+ res.writeHead(200, { 'content-type': 'application/javascript' });
+ res.end(';');
+ });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: {
+ dir: ${JSON.stringify(cacheDir)},
+ match: request => {
+ if (request.url.includes('/api/pinned'))
+ return { disposition: 'cache' };
+ if (request.url.includes('/volatile'))
+ return { disposition: 'no-cache' };
+ return {};
+ },
+ } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ await page.evaluate(() => fetch('/api/pinned').then(r => r.text()));
+ await page.evaluate(${loadScript}, '${remote(httpsServer, true)}/volatile.js');
+ await page.evaluate(${loadScript}, '${remote(httpsServer, true)}/normal.js');
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ await runInlineTest(files, { workers: 1 });
+ expect(pinned).toBe(1); // XHR force-cached by the policy.
+ expect(volatileScript).toBe(2); // Static resource forced to the network.
+ expect(normalScript).toBe(1); // Default behavior - static, cached.
+});
+
+test('should coalesce concurrent identical requests into one upstream fetch', async ({ runInlineTest, server }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let hits = 0;
+ server.setRoute('/slow', (req, res) => {
+ ++hits;
+ setTimeout(() => {
+ res.writeHead(200, { 'content-type': 'text/plain' });
+ res.end('slow payload');
+ }, 300);
+ });
+
+ const result = await runInlineTest({
+ // Force-cache everything so that concurrent fetch() misses coalesce.
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)}, match: () => ({ disposition: 'cache' }) } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('parallel', async ({ page }) => {
+ await page.goto('${remote(server)}/empty.html');
+ const bodies = await page.evaluate(() => Promise.all(
+ Array.from({ length: 8 }, () => fetch('${remote(server)}/slow').then(r => r.text()))));
+ expect(bodies).toEqual(Array(8).fill('slow payload'));
+ });
+ `,
+ }, { workers: 1 });
+
+ expect(result.exitCode).toBe(0);
+ expect(hits).toBe(1); // 8 concurrent misses coalesced into a single upstream fetch.
+});
+
+test('should only cache requests matching the filter', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ let assets = 0;
+ let api = 0;
+ httpsServer.setRoute('/assets/logo.png', (req, res) => { ++assets; res.end('logo'); });
+ httpsServer.setRoute('/api/data.png', (req, res) => { ++api; res.end('data'); });
+
+ const files = {
+ 'playwright.config.ts': `export default { httpCache: { dir: ${JSON.stringify(cacheDir)}, match: '**/assets/**' } };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/assets/logo.png');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/api/data.png');
+ });
+ `,
+ };
+
+ await runInlineTest(files, { workers: 1 });
+ expect(assets).toBe(1);
+ expect(api).toBe(1);
+
+ await runInlineTest(files, { workers: 1 });
+ expect(assets).toBe(1); // Matched the filter -> served from cache.
+ expect(api).toBe(2); // Not matched -> always hits the network.
+});
+
+test('should chain to the configured proxy', async ({ runInlineTest, httpsServer }, testInfo) => {
+ const cacheDir = testInfo.outputPath('.network-cache');
+ const upstreamProxy = await startUpstreamProxy();
+ let hits = 0;
+ httpsServer.setRoute('/asset.png', (req, res) => {
+ ++hits;
+ res.writeHead(200, { 'content-type': 'image/png' });
+ res.end('payload');
+ });
+
+ const files = {
+ 'playwright.config.ts': `export default {
+ httpCache: { dir: ${JSON.stringify(cacheDir)}, proxy: { server: ${JSON.stringify(upstreamProxy.url)} } },
+ };`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('load', async ({ page }) => {
+ await page.goto('${remote(httpsServer, true)}/empty.html');
+ await page.evaluate(${loadImage}, '${remote(httpsServer, true)}/asset.png');
+ });
+ `,
+ };
+
+ try {
+ const result1 = await runInlineTest(files, { workers: 1 });
+ expect(result1.exitCode).toBe(0);
+ expect(hits).toBe(1);
+ expect(upstreamProxy.log.length).toBeGreaterThan(0); // Cache misses went through the configured proxy.
+
+ const result2 = await runInlineTest(files, { workers: 1 });
+ expect(result2.exitCode).toBe(0);
+ expect(hits).toBe(1); // Still served from cache when chaining through a proxy.
+ } finally {
+ await upstreamProxy.stop();
+ }
+});
+
+test('should not start the proxy without httpCache', async ({ runInlineTest }) => {
+ const result = await runInlineTest({
+ 'playwright.config.ts': `export default {};`,
+ 'a.test.ts': `
+ import { test, expect } from '@playwright/test';
+ test('no proxy', async ({ page }) => {
+ expect(process.env.PLAYWRIGHT_TEST_CACHE_PROXY).toBeUndefined();
+ });
+ `,
+ }, { workers: 1 });
+ expect(result.exitCode).toBe(0);
+ expect(result.passed).toBe(1);
+});
diff --git a/utils/generate_types/overrides-test.d.ts b/utils/generate_types/overrides-test.d.ts
index 713979b3bf..ccb62d8e7b 100644
--- a/utils/generate_types/overrides-test.d.ts
+++ b/utils/generate_types/overrides-test.d.ts
@@ -74,6 +74,32 @@ export interface Config<TestArgs = {}, WorkerArgs = {}> extends TestConfig<TestA
export type Metadata = { [key: string]: any };
+/**
+ * A per-request caching decision. Every field is optional; an empty object applies the
+ * default behavior.
+ */
+export type HttpCacheDecision = {
+ /**
+ * `'cache'` serves a stored response and force-stores on a miss, `'no-cache'` bypasses the cache (fetch fresh, do
+ * not store), and `'default'` applies the default rules (cache shared static assets only). Defaults to `'default'`.
+ */
+ disposition?: 'cache' | 'no-cache' | 'default';
+
+ /**
+ * A stable principal id, such as a session token, that partitions the cache. Entries recorded under one identity
+ * are never served to a request with a different one, so per-user content can be cached without leaking across
+ * contexts. The value is hashed into the cache key and never written to disk. `null` is treated as anonymous, so
+ * `request.headers.get()` results can be passed directly.
+ */
+ identity?: string | null;
+};
+
+/**
+ * Called for each request to decide how it is cached. Receives a standard
+ * [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request). See [`property: TestConfig.httpCache`].
+ */
+export type HttpCachePolicy = (request: Request) => HttpCacheDecision;
+
export interface FullConfig<TestArgs = {}, WorkerArgs = {}> {
projects: FullProject<TestArgs, WorkerArgs>[];
reporter: ReporterDescription[];