Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

perf(web): serve the remote dev server over HTTP/2 with compression and linked source maps (#8287)

Wolf Mermelsteina90748a2f917
diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.md
index ad583d61df..90a169ab79 100644
--- a/apps/web/AGENTS.md
+++ b/apps/web/AGENTS.md
@@ -23,7 +23,7 @@ cd apps/web && PORT=3003 bun run dev # any free port in 3000-3009
`import.meta.env.MODE === 'development'` resolves the service clients to `https://dev.macro.com` through Vite’s same-origin development proxy. Sign in on the development hostname; existing `.macro.com` cookies are separate. Vite prints `https://<hostname>:<port>/app/`; `https://localhost:<port>/app/` also works. Trust `infra/local/certs/ca.pem` once in the visiting browser (see `infra/local/certs/README.md`). `MACRO_DEV_HTTPS=false` restores HTTP for workflows that require it. Notes:
-- On a remote devbox, open the app through Tailscale Serve rather than `https://<hostname>:<port>`. Vite drops to HTTP/1.1 while its development proxy is on, so the browser fetches the ~7k dev modules six at a time and pays a round trip for each (about 45 s at 30 ms). Run `sudo tailscale serve --bg --https=443 https+insecure://localhost:<port>` on the devbox and open `https://<devbox>.<tailnet>.ts.net/app/`, which serves HTTP/2 with a trusted certificate. Sign in again there. Serve offers HTTPS ports 443, 8443, and 10000.
+- On a remote devbox, open `https://<hostname>:<port>/app/` directly. The dev server serves HTTP/2, gzips modules and links source maps instead of inlining them, so a cold load of the ~7k dev modules moves ~13 MB instead of ~100 MB. Trust the CA on the visiting machine: Chrome caches nothing from a site whose certificate warning was clicked through, so every reload re-downloads every module. Tailscale Serve is an alternative that needs no CA: `sudo tailscale serve --bg --https=443 https+insecure://localhost:<port>`, then open `https://<devbox>.<tailnet>.ts.net/app/` and sign in again there (Serve offers HTTPS ports 443, 8443, and 10000). `MACRO_DEV_COMPRESSION=false` and `MACRO_DEV_INLINE_SOURCEMAPS=true` restore Vite's defaults.
- Never assume port 3000 or 3002 is yours. Check with `lsof -nP -iTCP:<port> -sTCP:LISTEN -t` and confirm the owner's worktree via `lsof -p <pid> | awk '$4=="cwd"'`. Take a free port instead of killing another session's server, and reuse one only if its cwd is this worktree.
- The `.cursor/*.sh` scripts and the `run-app` skill are **Cursor Cloud** entry points. On a local machine they prompt for sudo and are the wrong tool. Only backend (Rust) changes need the local stack.
diff --git a/apps/web/scripts/dev-compression.test.ts b/apps/web/scripts/dev-compression.test.ts
new file mode 100644
index 0000000000..525e3cb174
--- /dev/null
+++ b/apps/web/scripts/dev-compression.test.ts
@@ -0,0 +1,73 @@
+import { get } from 'node:http';
+import type { AddressInfo } from 'node:net';
+import { gunzipSync } from 'node:zlib';
+import { createServer } from 'vite';
+import { expect, it } from 'vitest';
+import { devCompression } from './dev-compression';
+
+it('gzips single-shot module responses and passes everything else through', async () => {
+ const module = `export const value = ${JSON.stringify('x'.repeat(4000))};\n`;
+ const vite = await createServer({
+ configFile: false,
+ appType: 'custom',
+ logLevel: 'silent',
+ plugins: [devCompression({})],
+ optimizeDeps: { noDiscovery: true, include: [] },
+ server: { host: '127.0.0.1', port: 0 },
+ });
+ vite.middlewares.use((req, res) => {
+ res.setHeader('Content-Type', 'text/javascript');
+ if (req.url === '/module.js') return res.end(module);
+ if (req.url === '/small.js') return res.end('export {};');
+ res.write(module.slice(0, 2000));
+ res.end(module.slice(2000));
+ });
+ try {
+ await vite.listen();
+ const { port } = vite.httpServer?.address() as AddressInfo;
+ const fetchRaw = (path: string, acceptEncoding: string) =>
+ new Promise<{ encoding: string | undefined; body: Buffer }>(
+ (resolve, reject) => {
+ get(
+ {
+ host: '127.0.0.1',
+ port,
+ path,
+ headers: { 'accept-encoding': acceptEncoding },
+ },
+ (res) => {
+ const chunks: Buffer[] = [];
+ res.on('data', (chunk: Buffer) => chunks.push(chunk));
+ res.on('end', () =>
+ resolve({
+ encoding: res.headers['content-encoding'],
+ body: Buffer.concat(chunks),
+ })
+ );
+ res.on('error', reject);
+ }
+ ).on('error', reject);
+ }
+ );
+
+ const compressed = await fetchRaw('/module.js', 'gzip, br');
+ expect(compressed.encoding).toBe('gzip');
+ expect(compressed.body.length).toBeLessThan(module.length / 10);
+ expect(gunzipSync(compressed.body).toString()).toBe(module);
+
+ expect(await fetchRaw('/module.js', 'identity')).toEqual({
+ encoding: undefined,
+ body: Buffer.from(module),
+ });
+ expect(await fetchRaw('/small.js', 'gzip')).toEqual({
+ encoding: undefined,
+ body: Buffer.from('export {};'),
+ });
+ expect(await fetchRaw('/streamed.js', 'gzip')).toEqual({
+ encoding: undefined,
+ body: Buffer.from(module),
+ });
+ } finally {
+ await vite.close();
+ }
+});
diff --git a/apps/web/scripts/dev-compression.ts b/apps/web/scripts/dev-compression.ts
new file mode 100644
index 0000000000..9b608af8f2
--- /dev/null
+++ b/apps/web/scripts/dev-compression.ts
@@ -0,0 +1,87 @@
+import type { IncomingMessage, ServerResponse } from 'node:http';
+import { constants, gzip } from 'node:zlib';
+import type { Connect, Plugin } from 'vite';
+
+const COMPRESSIBLE =
+ /^(text\/|application\/(javascript|json|wasm|xml|manifest\+json)|image\/svg)/;
+const MIN_BYTES = 1024;
+
+/**
+ * Gzips the dev server's transformed modules. Vite sends them uncompressed,
+ * which is fine on localhost but ~80 MB per cold load to a remote browser.
+ * Only responses sent in a single `end(body)` call are compressed (Vite's
+ * module and dependency responses); streamed static files pass through.
+ */
+export function devCompression(
+ env: Record<string, string | undefined> = process.env
+): Plugin {
+ const middleware: Connect.NextHandleFunction = (req, res, next) => {
+ if (!/\bgzip\b/.test(String(req.headers['accept-encoding'] ?? '')))
+ return next();
+ compressSingleShot(req, res);
+ next();
+ };
+ return {
+ name: 'macro-dev-compression',
+ apply: (_config, { command, isPreview }) =>
+ command === 'serve' && !isPreview && env.MACRO_DEV_COMPRESSION !== 'false',
+ configureServer(server) {
+ // Before Vite's own middlewares, so module responses pass through it.
+ server.middlewares.use(middleware);
+ },
+ };
+}
+
+function compressSingleShot(_req: IncomingMessage, res: ServerResponse) {
+ let streamed = false;
+ const write = res.write;
+ res.write = function (this: ServerResponse, ...args: Parameters<typeof write>) {
+ streamed = true;
+ return write.apply(this, args);
+ } as typeof write;
+
+ const end = res.end;
+ // One signature, so TypeScript can resolve `call` with just a body.
+ const endWith = end as (this: ServerResponse, body: Buffer) => ServerResponse;
+ res.end = function (this: ServerResponse, ...args: unknown[]) {
+ const [body] = args;
+ const restore = () => {
+ res.write = write;
+ res.end = end;
+ };
+ const buffer =
+ typeof body === 'string'
+ ? Buffer.from(body)
+ : Buffer.isBuffer(body)
+ ? body
+ : body instanceof Uint8Array
+ ? Buffer.from(body)
+ : undefined;
+ const type = String(res.getHeader('content-type') ?? '');
+ if (
+ streamed ||
+ res.headersSent ||
+ !buffer ||
+ buffer.length < MIN_BYTES ||
+ res.statusCode !== 200 ||
+ res.getHeader('content-encoding') ||
+ !COMPRESSIBLE.test(type)
+ ) {
+ restore();
+ return end.apply(this, args as Parameters<typeof end>);
+ }
+ gzip(buffer, { level: constants.Z_BEST_SPEED }, (error, compressed) => {
+ restore();
+ if (error) {
+ endWith.call(this, buffer);
+ return;
+ }
+ res.setHeader('content-encoding', 'gzip');
+ res.setHeader('content-length', compressed.length);
+ const vary = res.getHeader('vary');
+ res.setHeader('vary', vary ? `${vary}, Accept-Encoding` : 'Accept-Encoding');
+ endWith.call(this, compressed);
+ });
+ return this;
+ } as typeof end;
+}
diff --git a/apps/web/scripts/dev-https.test.ts b/apps/web/scripts/dev-https.test.ts
index 368b5235b9..b87a95ecdc 100644
--- a/apps/web/scripts/dev-https.test.ts
+++ b/apps/web/scripts/dev-https.test.ts
@@ -1,5 +1,6 @@
import { execFileSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
+import { connect as connectHttp2, constants as http2Constants } from 'node:http2';
import { get, request } from 'node:https';
import { createServer, resolveConfig } from 'vite';
import { expect, it } from 'vitest';
@@ -126,3 +127,50 @@ it.each(['build', 'preview'] as const)(
expect(config.server.https).toBeUndefined();
}
);
+
+it('keeps the HTTP/2 session when a reload cancels thousands of module requests', async () => {
+ const vite = await createServer({
+ configFile: false,
+ appType: 'custom',
+ logLevel: 'silent',
+ plugins: [devHttps({})],
+ optimizeDeps: { noDiscovery: true, include: [] },
+ server: { host: '127.0.0.1', port: 0 },
+ });
+ vite.middlewares.use((_req, res) => {
+ setTimeout(() => res.end('module'), 50);
+ });
+ try {
+ await vite.listen();
+ const address = vite.httpServer?.address();
+ if (!address || typeof address === 'string')
+ throw new Error('Vite has no TCP listener');
+ const session = connectHttp2(`https://localhost:${address.port}`, {
+ host: '127.0.0.1',
+ servername: 'localhost',
+ ca,
+ });
+ try {
+ await new Promise((resolve) => session.once('connect', resolve));
+ const abandoned = Array.from({ length: 3000 }, (_, i) =>
+ session.request({ ':path': `/app/module-${i}` })
+ );
+ for (const stream of abandoned) stream.on('error', () => {});
+ await new Promise((resolve) => setTimeout(resolve, 20));
+ for (const stream of abandoned) stream.close(http2Constants.NGHTTP2_CANCEL);
+ await new Promise((resolve) => setTimeout(resolve, 300));
+
+ const status = await new Promise((resolve, reject) => {
+ const stream = session.request({ ':path': '/app/' });
+ stream.on('response', (headers) => resolve(headers[':status']));
+ stream.on('error', reject);
+ stream.resume();
+ });
+ expect(status).toBe(200);
+ } finally {
+ session.destroy();
+ }
+ } finally {
+ await vite.close();
+ }
+});
diff --git a/apps/web/scripts/dev-https.ts b/apps/web/scripts/dev-https.ts
index 1572eef409..75d69632c1 100644
--- a/apps/web/scripts/dev-https.ts
+++ b/apps/web/scripts/dev-https.ts
@@ -3,6 +3,7 @@ import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
+import type { ServerOptions } from 'node:https';
import type { Plugin } from 'vite';
/** Standalone browser development shares the local stack's stable CA. */
@@ -39,7 +40,12 @@ export function devHttps(
https: {
cert: readFileSync(join(directory, 'server.pem')),
key: readFileSync(join(directory, 'server-key.pem')),
- },
+ // Reloading mid-load cancels thousands of module streams at
+ // once. Node's HTTP/2 rapid-reset guard (1000 resets per burst)
+ // then tears down the session and the new page loads blank.
+ streamResetBurst: 1_000_000,
+ streamResetRate: 1_000_000,
+ } as ServerOptions,
// Tailscale Serve proxies a remote devbox over HTTP/2 under its
// MagicDNS name; Vite's HMR socket rejects hosts not listed here.
allowedHosts: [hostname, '.ts.net'],
diff --git a/apps/web/scripts/dev-sourcemaps.test.ts b/apps/web/scripts/dev-sourcemaps.test.ts
new file mode 100644
index 0000000000..1ee9030ee4
--- /dev/null
+++ b/apps/web/scripts/dev-sourcemaps.test.ts
@@ -0,0 +1,67 @@
+import { get } from 'node:http';
+import type { AddressInfo } from 'node:net';
+import { createServer } from 'vite';
+import { expect, it } from 'vitest';
+import { devExternalSourcemaps } from './dev-sourcemaps';
+
+it('links inline source maps instead of sending them with every module', async () => {
+ const map = JSON.stringify({
+ version: 3,
+ sources: ['NewChatPage.tsx'],
+ sourcesContent: ['export const heading = "What should we work on?";'],
+ mappings: 'AAAA',
+ });
+ const base64 = Buffer.from(map).toString('base64');
+ const vite = await createServer({
+ configFile: false,
+ appType: 'custom',
+ logLevel: 'silent',
+ plugins: [devExternalSourcemaps({})],
+ optimizeDeps: { noDiscovery: true, include: [] },
+ server: { host: '127.0.0.1', port: 0 },
+ });
+ vite.middlewares.use((req, res) => {
+ if (req.url === '/module.js')
+ return res.end(
+ `export const heading = "x";\n//# sourceMappingURL=data:application/json;base64,${base64}`
+ );
+ if (req.url === '/style.css')
+ return res.end(
+ `.a{color:red}\n/*# sourceMappingURL=data:application/json;base64,${base64} */`
+ );
+ res.end('export const plain = 1;');
+ });
+ try {
+ await vite.listen();
+ const { port } = vite.httpServer?.address() as AddressInfo;
+ const fetchText = (path: string) =>
+ new Promise<{ status: number | undefined; body: string }>(
+ (resolve, reject) => {
+ get({ host: '127.0.0.1', port, path }, (res) => {
+ let body = '';
+ res.setEncoding('utf8');
+ res.on('data', (chunk) => {
+ body += chunk;
+ });
+ res.on('end', () => resolve({ status: res.statusCode, body }));
+ }).on('error', reject);
+ }
+ );
+
+ const module = await fetchText('/module.js');
+ const linked = /\/\/# sourceMappingURL=(\/@macro-sourcemap\/[0-9a-f]{40}\.map)$/.exec(
+ module.body
+ );
+ expect(module.body.startsWith('export const heading = "x";\n')).toBe(true);
+ expect(linked).not.toBeNull();
+ expect(await fetchText(linked?.[1] ?? '')).toEqual({ status: 200, body: map });
+
+ expect((await fetchText('/style.css')).body).toBe(
+ `.a{color:red}\n/*# sourceMappingURL=${linked?.[1]} */`
+ );
+ expect((await fetchText('/plain.js')).body).toBe('export const plain = 1;');
+ expect((await fetchText('/@macro-sourcemap/missing.map')).status).toBe(404);
+ } finally {
+ await vite.close();
+ }
+});
diff --git a/apps/web/scripts/dev-sourcemaps.ts b/apps/web/scripts/dev-sourcemaps.ts
new file mode 100644
index 0000000000..f2434092b0
--- /dev/null
+++ b/apps/web/scripts/dev-sourcemaps.ts
@@ -0,0 +1,81 @@
+import { createHash } from 'node:crypto';
+import type { ServerResponse } from 'node:http';
+import type { Connect, Plugin } from 'vite';
+
+const PREFIX = '/@macro-sourcemap/';
+const INLINE_MAP =
+ /\n(\/\/# sourceMappingURL=|\/\*# sourceMappingURL=)data:application\/json;(?:charset=utf-8;)?base64,([A-Za-z0-9+/=]+)( \*\/)?\s*$/;
+/** Maps are content-addressed, so this bounds memory, not correctness. */
+const MAX_MAPS = 20_000;
+
+/**
+ * Serves the dev server's source maps from their own URLs instead of inline.
+ * Vite appends every module's map as a base64 data URL, which is ~70% of the
+ * bytes a remote browser downloads; browsers fetch a linked map only while
+ * DevTools is open. Register after `devCompression` so the slimmer body is
+ * what gets compressed.
+ */
+export function devExternalSourcemaps(
+ env: Record<string, string | undefined> = process.env
+): Plugin {
+ const maps = new Map<string, Buffer>();
+
+ const middleware: Connect.NextHandleFunction = (req, res, next) => {
+ if (req.url?.startsWith(PREFIX)) {
+ const map = maps.get(req.url.slice(PREFIX.length));
+ if (!map) {
+ res.statusCode = 404;
+ res.end('Source map expired; reload the module');
+ return;
+ }
+ res.setHeader('Content-Type', 'application/json');
+ res.setHeader('Cache-Control', 'max-age=31536000,immutable');
+ res.end(map);
+ return;
+ }
+ linkInlineMaps(res, (map) => {
+ const name = `${createHash('sha1').update(map).digest('hex')}.map`;
+ maps.delete(name);
+ maps.set(name, map);
+ if (maps.size > MAX_MAPS) {
+ const oldest = maps.keys().next().value;
+ if (oldest) maps.delete(oldest);
+ }
+ return `${PREFIX}${name}`;
+ });
+ next();
+ };
+
+ return {
+ name: 'macro-dev-external-sourcemaps',
+ apply: (_config, { command, isPreview }) =>
+ command === 'serve' &&
+ !isPreview &&
+ env.MACRO_DEV_INLINE_SOURCEMAPS !== 'true',
+ configureServer(server) {
+ server.middlewares.use(middleware);
+ },
+ };
+}
+
+function linkInlineMaps(res: ServerResponse, store: (map: Buffer) => string) {
+ const end = res.end;
+ // One signature, so TypeScript can resolve `call` with just a body.
+ const endWith = end as (this: ServerResponse, body: string) => ServerResponse;
+ res.end = function (this: ServerResponse, ...args: unknown[]) {
+ res.end = end;
+ const [body] = args;
+ const text =
+ typeof body === 'string'
+ ? body
+ : Buffer.isBuffer(body)
+ ? body.toString('utf8')
+ : undefined;
+ const match = text === undefined ? null : INLINE_MAP.exec(text);
+ if (!text || !match) return end.apply(this, args as Parameters<typeof end>);
+ const [inline, comment, base64, close = ''] = match;
+ const url = store(Buffer.from(base64 ?? '', 'base64'));
+ const linked = `${text.slice(0, text.length - inline.length)}\n${comment}${url}${close}`;
+ return endWith.call(this, linked);
+ } as typeof end;
+}
diff --git a/apps/web/scripts/hosted-dev-proxy.test.ts b/apps/web/scripts/hosted-dev-proxy.test.ts
index 7334e9b678..f85ccbe9d3 100644
--- a/apps/web/scripts/hosted-dev-proxy.test.ts
+++ b/apps/web/scripts/hosted-dev-proxy.test.ts
@@ -1,6 +1,7 @@
import { createHash } from 'node:crypto';
import { readFileSync } from 'node:fs';
import { createServer as createHttpServer } from 'node:http';
+import { connect as connectHttp2 } from 'node:http2';
import { get, request } from 'node:https';
import type { AddressInfo, Socket } from 'node:net';
import { createServer, resolveConfig } from 'vite';
@@ -207,6 +208,78 @@ it('forwards authenticated HTTP and WS over HTTPS, rewrites cookies, and rejects
}
});
+it('keeps HTTP/2 for remote browsers while proxying', async () => {
+ const backend = createHttpServer((req, res) => {
+ res.setHeader('Content-Type', 'application/json');
+ res.setHeader('Connection', 'keep-alive');
+ res.setHeader(
+ 'Set-Cookie',
+ 'dev-macro-refresh-token=fixture; Domain=macro.com; Path=/; Secure; HttpOnly'
+ );
+ res.end(JSON.stringify({ path: req.url, origin: req.headers.origin ?? null }));
+ });
+ await new Promise<void>((resolve) => backend.listen(0, '127.0.0.1', resolve));
+ const upstream = `http://127.0.0.1:${(backend.address() as AddressInfo).port}`;
+ const vite = await createServer({
+ configFile: false,
+ appType: 'custom',
+ logLevel: 'silent',
+ plugins: [devHttps({}), hostedDevProxy({}, { gateway: upstream })],
+ optimizeDeps: { noDiscovery: true, include: [] },
+ server: { host: '127.0.0.1', port: 0 },
+ });
+ try {
+ await vite.listen();
+ const address = vite.httpServer?.address();
+ if (!address || typeof address === 'string')
+ throw new Error('No Vite listener');
+ const session = connectHttp2(`https://localhost:${address.port}`, {
+ host: '127.0.0.1',
+ servername: 'localhost',
+ ca: readFileSync(
+ new URL('../../../infra/local/certs/ca.pem', import.meta.url)
+ ),
+ });
+ try {
+ const response = await new Promise<{
+ status: number | undefined;
+ cookies: string[] | undefined;
+ body: string;
+ }>((resolve, reject) => {
+ const stream = session.request({
+ ':path': '/__macro_dev/gateway/auth/user/me',
+ origin: `https://localhost:${address.port}`,
+ });
+ let status: number | undefined;
+ let cookies: string[] | undefined;
+ let body = '';
+ stream.on('response', (headers) => {
+ status = headers[':status'];
+ cookies = headers['set-cookie'];
+ });
+ stream.setEncoding('utf8');
+ stream.on('data', (chunk) => {
+ body += chunk;
+ });
+ stream.on('end', () => resolve({ status, cookies, body }));
+ stream.on('error', reject);
+ stream.end();
+ });
+ expect(session.alpnProtocol).toBe('h2');
+ expect(response).toEqual({
+ status: 200,
+ cookies: ['dev-macro-refresh-token=fixture; Path=/; Secure; HttpOnly'],
+ body: JSON.stringify({ path: '/auth/user/me', origin: null }),
+ });
+ } finally {
+ session.close();
+ }
+ } finally {
+ await vite.close();
+ await new Promise<void>((resolve) => backend.close(() => resolve()));
+ }
+});
+
it.each([
{ TAURI_ENV_PLATFORM: 'linux' },
{ MACRO_LOCAL_BACKEND_PROXY: 'https://localhost:8090' },
diff --git a/apps/web/scripts/hosted-dev-proxy.ts b/apps/web/scripts/hosted-dev-proxy.ts
index 85f9bba613..96bde5fbc1 100644
--- a/apps/web/scripts/hosted-dev-proxy.ts
+++ b/apps/web/scripts/hosted-dev-proxy.ts
@@ -1,20 +1,186 @@
-import type { IncomingMessage } from 'node:http';
-import type { Plugin, ProxyOptions } from 'vite';
+import { lookup as dnsLookup, type LookupAddress } from 'node:dns';
+import {
+ Agent as HttpAgent,
+ type IncomingHttpHeaders,
+ type IncomingMessage,
+ request as httpRequest,
+ type ServerResponse,
+} from 'node:http';
+import type { Http2ServerRequest } from 'node:http2';
+import { Agent as HttpsAgent, request as httpsRequest } from 'node:https';
+import type { LookupFunction, Socket } from 'node:net';
+import type { Plugin } from 'vite';
import {
DEVELOPMENT_PROXY_PREFIX,
DEVELOPMENT_PROXY_TARGETS,
} from '../src/lib/core/constant/developmentProxy';
+type ProxyRequest = IncomingMessage | Http2ServerRequest;
+
+/** Connection-level and HTTP/2 pseudo headers never cross the proxy. */
+const HOP_BY_HOP = new Set([
+ 'connection',
+ 'keep-alive',
+ 'proxy-connection',
+ 'transfer-encoding',
+ 'upgrade',
+ 'te',
+ 'trailer',
+ 'http2-settings',
+ 'host',
+]);
+
+const DNS_TTL_MS = 60_000;
+
+/**
+ * Resolves each upstream host once a minute. Every proxied request otherwise
+ * does its own lookup, and bursts of them fail with EAI_AGAIN on dev boxes.
+ */
+function cachedLookup(): LookupFunction {
+ const cache = new Map<string, { at: number; addresses: LookupAddress[] }>();
+ return (hostname, options, callback) => {
+ const respond = (addresses: LookupAddress[]) => {
+ if (options.all) return callback(null, addresses);
+ const [first] = addresses;
+ if (!first) return callback(new Error(`No address for ${hostname}`), '');
+ callback(null, first.address, first.family);
+ };
+ const cached = cache.get(hostname);
+ if (cached && Date.now() - cached.at < DNS_TTL_MS) {
+ respond(cached.addresses);
+ return;
+ }
+ dnsLookup(hostname, { all: true }, (error, addresses) => {
+ if (error) {
+ // A stale answer beats failing the request on a transient DNS error.
+ if (cached) respond(cached.addresses);
+ else callback(error, '');
+ return;
+ }
+ cache.set(hostname, { at: Date.now(), addresses });
+ respond(addresses);
+ });
+ };
+}
+
+function hostOf(req: ProxyRequest) {
+ return req.headers.host ?? req.headers[':authority'];
+}
+
+function forwardedHeaders(headers: IncomingHttpHeaders, host: string) {
+ const forwarded: IncomingHttpHeaders = {};
+ for (const [name, value] of Object.entries(headers)) {
+ if (name.startsWith(':') || HOP_BY_HOP.has(name) || name === 'origin')
+ continue;
+ forwarded[name] = value;
+ }
+ // isCrossOrigin already vetted the browser's origin. Hosted services
+ // allowlist origins (sync-service 403s any https:// dev host), so forward
+ // like a server-side client.
+ forwarded.host = host;
+ return forwarded;
+}
+
+/** Hosted auth cookies have Domain=macro.com; scope them to the dev host. */
+function withoutCookieDomain(cookies: string[] | undefined) {
+ return cookies?.map((cookie) => cookie.replace(/;\s*Domain=[^;]*/i, ''));
+}
+
/** Keep dev credentials on this hostname while forwarding to fixed hosted APIs. */
export function hostedDevProxy(
env: Record<string, string | undefined> = process.env,
targets: Record<string, string> = DEVELOPMENT_PROXY_TARGETS
): Plugin {
let protocol = 'https';
- const isCrossOrigin = (req: IncomingMessage) =>
+ const isCrossOrigin = (req: ProxyRequest) =>
req.headers['sec-fetch-site'] === 'cross-site' ||
(req.headers.origin !== undefined &&
- req.headers.origin !== `${protocol}://${req.headers.host}`);
+ req.headers.origin !== `${protocol}://${hostOf(req)}`);
+
+ const lookup = cachedLookup();
+ const agents = {
+ 'http:': new HttpAgent({ keepAlive: true, lookup }),
+ 'https:': new HttpsAgent({ keepAlive: true, lookup }),
+ };
+
+ /** The upstream URL for a proxied path, or undefined when it is not one. */
+ const resolve = (url: string | undefined) => {
+ if (!url?.startsWith(`${DEVELOPMENT_PROXY_PREFIX}/`)) return;
+ for (const [name, target] of Object.entries(targets)) {
+ const prefix = `${DEVELOPMENT_PROXY_PREFIX}/${name}`;
+ if (!url.startsWith(prefix)) continue;
+ const rest = url.slice(prefix.length);
+ if (rest !== '' && !/^[/?]/.test(rest)) continue;
+ return new URL(rest.startsWith('/') ? rest : `/${rest}`, target);
+ }
+ };
+
+ const send = (upstream: URL, headers: IncomingHttpHeaders, method = 'GET') =>
+ (upstream.protocol === 'https:' ? httpsRequest : httpRequest)(upstream, {
+ method,
+ headers,
+ agent: agents[upstream.protocol as 'http:' | 'https:'],
+ });
+
+ const proxyHttp = (
+ upstream: URL,
+ req: ProxyRequest,
+ // Connect hands middleware a ServerResponse (Node's HTTP/2 compat API matches it).
+ res: ServerResponse,
+ logError: (message: string) => void
+ ) => {
+ const outgoing = send(
+ upstream,
+ forwardedHeaders(req.headers, upstream.host),
+ req.method
+ );
+ outgoing.on('response', (incoming) => {
+ const headers: Record<string, string | string[]> = {};
+ for (const [name, value] of Object.entries(incoming.headers)) {
+ if (value === undefined || HOP_BY_HOP.has(name)) continue;
+ headers[name] =
+ name === 'set-cookie'
+ ? (withoutCookieDomain(value as string[]) ?? [])
+ : value;
+ }
+ res.writeHead(incoming.statusCode ?? 502, headers);
+ incoming.pipe(res as NodeJS.WritableStream);
+ });
+ outgoing.on('error', (error) => {
+ logError(`hosted dev proxy: ${req.url}: ${error.message}`);
+ if (res.headersSent) {
+ res.destroy();
+ return;
+ }
+ res.writeHead(502, { 'content-type': 'text/plain' });
+ res.end(`Proxy error: ${error.message}`);
+ });
+ req.pipe(outgoing);
+ };
+
+ const proxyUpgrade = (upstream: URL, req: IncomingMessage, socket: Socket) => {
+ const outgoing = send(upstream, {
+ ...forwardedHeaders(req.headers, upstream.host),
+ connection: 'Upgrade',
+ upgrade: req.headers.upgrade,
+ });
+ outgoing.on('upgrade', (incoming, upstreamSocket, head) => {
+ const lines = [`HTTP/1.1 101 ${incoming.statusMessage ?? 'Switching Protocols'}`];
+ for (let i = 0; i < incoming.rawHeaders.length; i += 2) {
+ lines.push(`${incoming.rawHeaders[i]}: ${incoming.rawHeaders[i + 1]}`);
+ }
+ socket.write(`${lines.join('\r\n')}\r\n\r\n`);
+ if (head.length) socket.write(head);
+ upstreamSocket.pipe(socket).pipe(upstreamSocket);
+ upstreamSocket.on('error', () => socket.destroy());
+ socket.on('error', () => upstreamSocket.destroy());
+ });
+ outgoing.on('response', (incoming) => {
+ socket.end(`HTTP/1.1 ${incoming.statusCode} ${incoming.statusMessage}\r\n\r\n`);
+ });
+ outgoing.on('error', () => socket.destroy());
+ outgoing.end();
+ };
return {
name: 'macro-hosted-dev-proxy',
@@ -27,58 +193,36 @@ export function hostedDevProxy(
!env.TAURI_DEV_HOST &&
env.MACRO_DEV_PROXY !== 'false',
config() {
- return {
- define: { 'import.meta.env.VITE_DEV_PROXY': true },
- server: {
- proxy: Object.fromEntries(
- Object.entries(targets).map(([name, target]) => {
- const prefix = `${DEVELOPMENT_PROXY_PREFIX}/${name}`;
- const options: ProxyOptions = {
- target,
- changeOrigin: true,
- ws: true,
- // Hosted auth cookies have Domain=macro.com. Keep Secure and
- // HttpOnly, but scope them to the visiting development host.
- cookieDomainRewrite: '',
- rewrite: (path) => {
- const rest = path.slice(prefix.length);
- return rest.startsWith('/') ? rest : `/${rest}`;
- },
- // isCrossOrigin already vetted the browser's origin. Hosted
- // services allowlist origins (sync-service 403s any https://
- // dev host), so forward like a server-side client.
- configure(proxy) {
- proxy.on('proxyReq', (request) => {
- request.removeHeader('origin');
- });
- proxy.on('proxyReqWs', (request, req, socket) => {
- if (isCrossOrigin(req)) {
- request.destroy();
- socket.destroy();
- return;
- }
- request.removeHeader('origin');
- });
- },
- };
- return [`^${prefix}(?:/|\\?|$)`, options];
- })
- ),
- },
- };
+ // A middleware rather than `server.proxy`: Vite falls back to HTTP/1.1
+ // whenever `server.proxy` is set, and remote browsers then fetch
+ // thousands of dev modules over six connections.
+ return { define: { 'import.meta.env.VITE_DEV_PROXY': true } };
},
configureServer(server) {
protocol = server.config.server.https ? 'https' : 'http';
+ const logError = (message: string) => server.config.logger.error(message);
server.middlewares.use((req, res, next) => {
- if (
- req.url?.startsWith(`${DEVELOPMENT_PROXY_PREFIX}/`) &&
- isCrossOrigin(req)
- ) {
+ const upstream = resolve(req.url);
+ if (!upstream) return next();
+ if (isCrossOrigin(req)) {
res.statusCode = 403;
res.end('Cross-origin development proxy requests are not allowed');
return;
}
- next();
+ proxyHttp(upstream, req, res, logError);
+ });
+ server.httpServer?.on('upgrade', (req: IncomingMessage, socket: Socket) => {
+ const upstream = resolve(req.url);
+ if (!upstream) return;
+ if (isCrossOrigin(req)) {
+ socket.destroy();
+ return;
+ }
+ proxyUpgrade(upstream, req, socket);
+ });
+ server.httpServer?.once('close', () => {
+ agents['http:'].destroy();
+ agents['https:'].destroy();
});
},
};
diff --git a/apps/web/src/index.tsx b/apps/web/src/index.tsx
index d9cf729977..0896a009bf 100644
--- a/apps/web/src/index.tsx
+++ b/apps/web/src/index.tsx
@@ -15,8 +15,11 @@ import { Telemetry } from '@macro-inc/observability';
import { initMonochromeIcons } from '@ui/utils/monochromeIcons';
import { ErrorBoundary, render } from 'solid-js/web';
import { FatalError } from './components/app/FatalError';
+import { perfMark } from './observability/perf-marks';
import { Root } from './routes/Root';
+perfMark('boot.entry-evaluated');
+
// Override global fetch with platformFetch for Tauri compatibility
// Skip localhost requests (dev server) to avoid breaking HMR
if (isTauri()) {
@@ -99,7 +102,9 @@ const renderApp = () => {
};
async function main() {
+ perfMark('boot.main');
await initializeBrowserObservability();
+ perfMark('boot.observability-ready');
console.log('App Version ', import.meta.env.__APP_VERSION__);
@@ -113,7 +118,9 @@ async function main() {
});
}
+ perfMark('boot.render-start');
renderApp();
+ perfMark('boot.render-returned');
}
main().catch((error) => Telemetry.error(error, { surface: 'app-startup' }));
diff --git a/apps/web/src/lib/core/util/fetchWithToken.ts b/apps/web/src/lib/core/util/fetchWithToken.ts
index 846b489099..6c015f7147 100644
--- a/apps/web/src/lib/core/util/fetchWithToken.ts
+++ b/apps/web/src/lib/core/util/fetchWithToken.ts
@@ -1,3 +1,4 @@
+import { perfMark } from '@app/observability/perf-marks';
import { ENABLE_BEARER_TOKEN_AUTH } from '@core/constant/featureFlags';
import { SERVER_HOSTS } from '@core/constant/servers';
import { Telemetry } from '@macro-inc/observability';
@@ -120,6 +121,7 @@ export async function fetchToken(): Promise<TokenResult> {
if (tokenPromise == null) {
const generation = refreshGeneration;
tokenPromise = (async () => {
+ perfMark('auth.refresh-start');
try {
const result = await fetchWithCredentials(
`${SERVER_HOSTS['auth-service']}/jwt/refresh`,
@@ -134,6 +136,7 @@ export async function fetchToken(): Promise<TokenResult> {
}
);
+ perfMark('auth.refresh-end', { ok: result.isOk() });
if (result.isErr()) {
const failure = err(result.error);
if (
diff --git a/apps/web/src/lib/split-router/solid/outlet.tsx b/apps/web/src/lib/split-router/solid/outlet.tsx
index f742d6672a..5529f23d99 100644
--- a/apps/web/src/lib/split-router/solid/outlet.tsx
+++ b/apps/web/src/lib/split-router/solid/outlet.tsx
@@ -1,7 +1,11 @@
+import { perfMark } from '@app/observability/perf-marks';
import {
type Component,
+ createEffect,
createMemo,
type JSX,
+ on,
+ onMount,
Show,
Suspense,
useContext,
@@ -43,6 +47,16 @@ function RouteOutlet(props: { fallback?: OutletFallback }) {
{ equals: (left, right) => left?.key === right?.key }
);
+ createEffect(
+ on(match, (resolved) => {
+ if (resolved)
+ perfMark('route.matched', {
+ route: resolved.key,
+ depth: resolved.depth,
+ });
+ })
+ );
+
const context: PaneContextValue = {
pane: parent.pane,
entry: parent.entry,
@@ -65,12 +79,19 @@ function RouteOutlet(props: { fallback?: OutletFallback }) {
</Suspense>
)}
</Show>
+ <RouteMounted route={resolved.key} />
</PaneContext.Provider>
)}
</Show>
);
}
+/** Marks when a matched route's component tree has been created and attached. */
+function RouteMounted(props: { route: string }) {
+ onMount(() => perfMark('route.mounted', { route: props.route }));
+ return null;
+}
+
/**
* Renders the next route component in the enclosing pane's branch. Its view
* stays mounted while its route id and `remountKey` are unchanged; params
diff --git a/apps/web/src/observability/perf-marks.ts b/apps/web/src/observability/perf-marks.ts
new file mode 100644
index 0000000000..11dd32b81f
--- /dev/null
+++ b/apps/web/src/observability/perf-marks.ts
@@ -0,0 +1,15 @@
+/**
+ * Named User Timing marks for load and navigation phases (`macro.<name>`).
+ * They show up in DevTools Performance traces and PerformanceObserver; with
+ * `VITE_PERF_LOG=true` each one is also logged as `[perf] <name> <ms>`.
+ */
+export function perfMark(name: string, detail?: Record<string, unknown>) {
+ if (typeof performance === 'undefined') return;
+ const mark = performance.mark(`macro.${name}`, { detail });
+ if (import.meta.env.VITE_PERF_LOG === 'true') {
+ console.info(
+ `[perf] ${name} ${Math.round(mark.startTime)}ms`,
+ ...(detail ? [JSON.stringify(detail)] : [])
+ );
+ }
+}
diff --git a/apps/web/src/routes/Root.tsx b/apps/web/src/routes/Root.tsx
index b5b615d77c..993296c60e 100644
--- a/apps/web/src/routes/Root.tsx
+++ b/apps/web/src/routes/Root.tsx
@@ -53,6 +53,7 @@ import {
} from '@theme/utils/themeUtils';
import { detect } from 'detect-browser';
import { createEffect, type JSX, on, onCleanup, onMount } from 'solid-js';
+import { perfMark } from '../observability/perf-marks';
import { AppRouterView } from './app-router-view';
import { useSyncLoginCookie } from './use-sync-login-cookie';
@@ -198,6 +199,7 @@ export function Root() {
});
onMount(() => {
+ perfMark('boot.root-mounted');
applyTheme(resolveActiveThemeId());
ensureMinimalThemeContrast();
});
diff --git a/apps/web/src/vite-env.d.ts b/apps/web/src/vite-env.d.ts
index 7d089cb8da..acd963d693 100644
--- a/apps/web/src/vite-env.d.ts
+++ b/apps/web/src/vite-env.d.ts
@@ -11,6 +11,8 @@ interface ImportMetaEnv {
readonly VITE_OTEL_EXPORTER_URL?: string;
readonly VITE_OTEL_ENV?: string;
readonly VITE_ENABLE_BROWSER_OTEL?: string;
+ /** `true` logs each `perfMark` as a `[perf]` console line. */
+ readonly VITE_PERF_LOG?: string;
readonly VITE_ENABLE_REMINDERS?: string;
readonly VITE_DISABLE_BROWSER_TURSO_CACHE?: string;
}
diff --git a/apps/web/vite.base.ts b/apps/web/vite.base.ts
index ee4d3cbe7a..24da7ec4e9 100644
--- a/apps/web/vite.base.ts
+++ b/apps/web/vite.base.ts
@@ -10,7 +10,9 @@ import wasm from 'vite-plugin-wasm';
import tsconfigpaths from 'vite-tsconfig-paths';
// @ts-ignore
import { version } from './package.json';
+import { devCompression } from './scripts/dev-compression';
import { devHttps } from './scripts/dev-https';
+import { devExternalSourcemaps } from './scripts/dev-sourcemaps';
import { hostedDevProxy } from './scripts/hosted-dev-proxy';
import { keepImportMetaDev } from './scripts/keep-import-meta-dev';
import { localDevServer } from './scripts/local-dev-server';
@@ -318,6 +320,8 @@ export const createAppViteConfig = (): UserConfigFn => {
},
plugins: [
devHttps(),
+ devCompression(),
+ devExternalSourcemaps(),
hostedDevProxy(),
// solidDevtools({ autoname: true }),
pureGeneratedZodSchemas(),
diff --git a/docs/development/running-locally.md b/docs/development/running-locally.md
index 96e9beacc5..0f407eb1eb 100644
--- a/docs/development/running-locally.md
+++ b/docs/development/running-locally.md
@@ -85,6 +85,14 @@ local stack's internal Vite server keep using HTTP automatically; the stack
provides HTTPS through its proxy. Builds and preview servers are unchanged.
`MACRO_DEV_PROXY=false` opts out of the hosted API proxy.
+The dev server speaks HTTP/2, gzips module responses, and links each module's
+source map (`/@macro-sourcemap/<hash>.map`) instead of inlining it; browsers only
+fetch maps while DevTools is open. On a slow link this is the difference between
+~100 MB and ~13 MB per load. Trust the CA rather than clicking through the
+certificate warning: Chrome caches nothing from a site with a certificate error.
+`MACRO_DEV_COMPRESSION=false` and `MACRO_DEV_INLINE_SOURCEMAPS=true` restore
+Vite's defaults.
+
## Run the local stack
The local stack runs without Doppler. It runs Postgres, Redis, LocalStack, OpenSearch, Kafka, and FusionAuth in Docker, with dummy AWS credentials and fixed test secrets.