Enhanced ChatGPT Clone: Features Agents, MCP, Skills, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active
TypeScript aianthropicartifactsawsazurechatgptchatgpt-cloneclaudeclonedeepseekgeminigooglegpt-5librechatmcpo1openairesponses-apivisionwebui
📱 test: Add Mobile Chat Performance Benchmark (#15510)
Danny Avila0cb1118e082b
diff --git a/e2e/benchmarks-mobile-chat/README.md b/e2e/benchmarks-mobile-chat/README.mdnew file mode 100644index 0000000000..f1ad245840--- /dev/null+++ b/e2e/benchmarks-mobile-chat/README.md@@ -0,0 +1,43 @@+# Mobile Chat Performance Benchmark++Measures long-conversation behavior with Playwright's iPhone 13 descriptor: a+`390x664` browser viewport on a `390x844` screen. It uses a local production+build and the mock-model pipeline; no provider credentials or external model+calls are used.++The benchmark seeds 150 user/assistant turns (300 rendered message rows), then+captures six phases:++- sitting idle on an empty chat as a control;+- loading and progressively mounting the long transcript;+- sitting idle for three seconds after the transcript settles;+- repeated full-history scrolling;+- three streamed continuation turns with long Markdown responses;+- typing after the transcript grows to 306 rows.++Each scenario records Chromium `Performance` domain counters (main-thread task,+script, layout and style time), long tasks, heap size, and DOM node count. It+runs twice: first without react-scan for lower-overhead browser measurements,+then with react-scan to attribute render counts and duration to components.++This is a desktop-hosted Chromium measurement at an iPhone viewport, not a+physical-device battery or thermal measurement. Use it to reproduce and compare+frontend changes locally; confirm energy impact separately on a real iPhone.++## Run++Extract the pinned instrumentation bundle outside the repository so npm cannot+re-resolve LibreChat's dependencies, prepare the production client, then run+the benchmark:++```bash+mkdir -p /tmp/librechat-react-scan+npm pack [email protected] --pack-destination /tmp/librechat-react-scan+tar -xzf /tmp/librechat-react-scan/react-scan-0.5.7.tgz -C /tmp/librechat-react-scan+npm run e2e:prepare+REACT_SCAN_PATH=/tmp/librechat-react-scan/package/dist/auto.global.js \+ npx playwright test --config=e2e/playwright.config.mobile-chat-perf.ts+```++JSON snapshots are attached to the Playwright results under+`e2e/benchmarks/.test-results/mobile-chat`.diff --git a/e2e/benchmarks-mobile-chat/RESULTS.md b/e2e/benchmarks-mobile-chat/RESULTS.mdnew file mode 100644index 0000000000..47f88b0f7e--- /dev/null+++ b/e2e/benchmarks-mobile-chat/RESULTS.md@@ -0,0 +1,56 @@+# Mobile Chat Performance Baseline++Captured September 2, 2026 from LibreChat `519e64769` on an Apple M4 Max with+macOS 26.6, Node.js 24.16.0, Playwright 1.62.1, and Chromium at the iPhone 13+descriptor's `390x664` viewport on a `390x844` screen. The production client+was used.++The stress conversation contained 150 user/assistant turns (300 rows) with+prose, lists, code blocks, and tables. The continuation phase streamed three+additional rich Markdown replies. `busy` is Chromium main-thread task time+divided by elapsed wall time; it is a useful comparison proxy, not device CPU+utilization.++## Raw browser measurements++| Phase | Wall | Main-thread busy | Task | Script | Style | Long tasks (total/worst) | Heap at end | DOM nodes |+| ----------------------- | --------: | ---------------: | --------: | -------: | -----: | -----------------------: | ----------: | --------: |+| Empty chat idle | 3,007 ms | 15.9% | 479 ms | 273 ms | 64 ms | 102/102 ms | 52.6 MB | 2,063 |+| Load 300 rows | 2,909 ms | 84.4% | 2,455 ms | 1,717 ms | 119 ms | 347/250 ms | 271.2 MB | 121,371 |+| Long chat idle | 3,030 ms | 14.5% | 439 ms | 5 ms | 132 ms | 0/0 ms | 349.5 MB | 123,427 |+| Four full scroll cycles | 3,787 ms | 90.9% | 3,441 ms | 1,107 ms | 343 ms | 0/0 ms | 348.4 MB | 123,587 |+| Three continuations | 11,182 ms | 95.2% | 10,646 ms | 4,208 ms | 872 ms | 2,791/325 ms | 511.6 MB | 126,234 |+| Typing after stress | 1,357 ms | 71.1% | 965 ms | 206 ms | 145 ms | 0/0 ms | 541.7 MB | 127,013 |++Heap figures are point-in-time values without forced garbage collection, so+they do not establish a memory leak.++## React Scan diagnostic++React Scan observed 44,521 render records while loading, 2,852 while scrolling,+201,013 during the three streamed continuations, and 1,334 while typing. The+settled long conversation produced only 26 render records during its+three-second idle phase. This points away from a continuous React rerender loop+as the source of settled-idle work, but shows a very large amount of React work+during streaming.++React Scan materially perturbed the workload. During continuation, Chromium+script time increased from 4,208 ms to 12,332 ms and task time increased from+10,646 ms to 14,991 ms. Its instrumented production-build timings were reported+as zero and many component names were minified. Therefore the raw run is the+performance baseline; React Scan is useful here only for comparative render+counts and for narrowing a follow-up development-build profile.++## Interpretation++This run supports the report that long LibreChat conversations can sustain high+frontend work while loading, scrolling, and streaming. It does not prove that+ordinary LibreChat use consumes more energy than other apps, and it cannot+measure iPhone thermals or Mobile Safari behavior.++The largest scaling signal is the document size: after progressive mounting+settles, all 300 rows remain mounted at roughly 120,000 DOM nodes. The current+progressive-row strategy optimizes the first commit but deliberately converges+to the complete DOM. A real-device Safari energy trace and a controlled A/B+against a bounded/virtualized message DOM are the next tests needed to connect+this browser result to phone heating.diff --git a/e2e/benchmarks-mobile-chat/mobile-chat.perf.spec.ts b/e2e/benchmarks-mobile-chat/mobile-chat.perf.spec.tsnew file mode 100644index 0000000000..8ec07f2bad--- /dev/null+++ b/e2e/benchmarks-mobile-chat/mobile-chat.perf.spec.ts@@ -0,0 +1,222 @@+import { randomUUID } from 'node:crypto';+import { expect, test } from '@playwright/test';+import type { Page, TestInfo } from '@playwright/test';+import {+ clearUserConversations,+ deleteConversations,+ deleteMessagesByConversation,+ seedConversations,+ seedMessages,+} from '../specs/mock/db';+import {+ MOCK_ENDPOINTS,+ NEW_CHAT_PATH,+ messagesView,+ selectMockEndpoint,+ sendMessage,+} from '../specs/mock/helpers';+import { getE2EUser } from '../setup/user';+import {+ attachBrowserPhases,+ createBrowserProbe,+ formatBrowserPhase,+ installBrowserPerf,+} from '../perf/browser';+import type { BrowserPhase } from '../perf/browser';+import {+ attachSnapshot,+ installReactScan,+ resetPerf,+ snapshotPerf,+ topComponents,+ totals,+} from '../perf/scan';+import type { PerfSnapshot } from '../perf/scan';+import { buildStressMessages, ROWS, STREAM_END_MARKER, TURNS } from './payload';++const userEmail = getE2EUser().email;+const FIXTURES = [+ { id: randomUUID(), label: 'raw', title: 'Mobile performance raw metrics' },+ { id: randomUUID(), label: 'scan', title: 'Mobile performance react scan' },+] as const;+const CONTINUATIONS = 3;++interface ScenarioResults {+ browser: Record<string, BrowserPhase>;+ scan: Record<string, PerfSnapshot>;+}++function messageRows(page: Page) {+ return messagesView(page).locator('.message-render');+}++async function stressScroll(page: Page): Promise<void> {+ const scroller = messagesView(page).locator('.scrollbar-gutter-stable');+ await scroller.evaluate(async (element) => {+ const scrollTo = (target: number) =>+ new Promise<void>((resolve) => {+ const startedAt = performance.now();+ const from = element.scrollTop;+ const duration = 450;+ const step = (now: number) => {+ const progress = Math.min(1, (now - startedAt) / duration);+ element.scrollTop = from + (target - from) * progress;+ if (progress < 1) {+ requestAnimationFrame(step);+ return;+ }+ resolve();+ };+ requestAnimationFrame(step);+ });+ for (let pass = 0; pass < 4; pass += 1) {+ await scrollTo(0);+ await scrollTo(element.scrollHeight);+ }+ });+}++async function runPhase(+ name: string,+ page: Page,+ results: ScenarioResults,+ reactScan: boolean,+ action: () => Promise<void>,+): Promise<void> {+ const probe = await createBrowserProbe(page);+ if (reactScan) {+ await resetPerf(page);+ }+ await probe.start();+ await action();+ results.browser[name] = await probe.finish();+ if (reactScan) {+ results.scan[name] = await snapshotPerf(page);+ }+}++async function runScenario(+ page: Page,+ testInfo: TestInfo,+ fixture: (typeof FIXTURES)[number],+ reactScan: boolean,+): Promise<void> {+ await installBrowserPerf(page);+ if (reactScan) {+ await installReactScan(page);+ }++ await page.goto(NEW_CHAT_PATH, { timeout: 120_000 });+ const results: ScenarioResults = { browser: {}, scan: {} };++ await runPhase('idle-empty', page, results, reactScan, async () => {+ await page.waitForTimeout(3_000);+ });++ await runPhase('load', page, results, reactScan, async () => {+ await page.goto(`/c/${fixture.id}`, { timeout: 120_000 });+ await expect(messageRows(page)).toHaveCount(ROWS, { timeout: 120_000 });+ });++ await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);++ await runPhase('idle-long', page, results, reactScan, async () => {+ await page.waitForTimeout(3_000);+ });++ await runPhase('scroll', page, results, reactScan, async () => {+ await stressScroll(page);+ });++ await runPhase('continue', page, results, reactScan, async () => {+ for (let turn = 1; turn <= CONTINUATIONS; turn += 1) {+ await sendMessage(page, `Continue mobile stress turn ${turn}`);+ await expect(messagesView(page).getByText(STREAM_END_MARKER, { exact: true })).toHaveCount(+ turn,+ { timeout: 90_000 },+ );+ await expect(page.getByRole('button', { name: 'Stop generating' })).toBeHidden({+ timeout: 30_000,+ });+ }+ });+ await expect(messageRows(page)).toHaveCount(ROWS + CONTINUATIONS * 2, { timeout: 30_000 });++ await runPhase('typing', page, results, reactScan, async () => {+ const input = page.getByRole('textbox', { name: 'Message input' });+ await input.click();+ await input.pressSequentially(+ 'Typing after a three-hundred-message transcript should keep the transcript quiet.',+ { delay: 10 },+ );+ });++ console.log(+ `\n=== Mobile chat stress: ${reactScan ? 'react-scan diagnostic' : 'raw browser metrics'} ` ++ `(390x664 viewport, 390x844 screen, ${TURNS} seeded turns / ${ROWS} rows, ` ++ `${CONTINUATIONS} continuations) ===`,+ );+ for (const [name, phase] of Object.entries(results.browser)) {+ console.log(formatBrowserPhase(name, phase));+ }+ if (reactScan) {+ let capturedRenders = 0;+ for (const [name, snapshot] of Object.entries(results.scan)) {+ const phaseTotals = totals(snapshot);+ capturedRenders += phaseTotals.renders;+ console.log(+ `${name} react renders=${phaseTotals.renders} render-time=${phaseTotals.time.toFixed(0)}ms`,+ );+ for (const line of topComponents(snapshot, 12)) {+ console.log(` ${line}`);+ }+ await attachSnapshot(testInfo, `${name}-react-scan.json`, snapshot, {+ seededRows: ROWS,+ continuations: CONTINUATIONS,+ });+ }+ expect(capturedRenders).toBeGreaterThan(0);+ }+ await attachBrowserPhases(testInfo, 'browser-metrics.json', results.browser);++ for (const phase of Object.values(results.browser)) {+ expect(phase.elapsedMs).toBeGreaterThan(0);+ expect(phase.taskMs).toBeGreaterThanOrEqual(0);+ expect(Number.isFinite(phase.busyPercent)).toBe(true);+ }+}++test.describe.configure({ mode: 'serial' });++test.describe('iPhone-sized long-chat performance', () => {+ test.beforeAll(async () => {+ await clearUserConversations(userEmail);+ await seedConversations(+ userEmail,+ FIXTURES.map((fixture, index) => ({+ conversationId: fixture.id,+ title: fixture.title,+ updatedAt: new Date(Date.now() - index * 60_000),+ })),+ );+ for (const fixture of FIXTURES) {+ await seedMessages(userEmail, fixture.id, buildStressMessages(fixture.label));+ }+ });++ test.afterAll(async () => {+ const ids = FIXTURES.map((fixture) => fixture.id);+ await deleteMessagesByConversation(ids);+ await deleteConversations(ids);+ });++ test('captures browser CPU proxies without react-scan overhead', async ({ page }, testInfo) => {+ test.setTimeout(10 * 60 * 1000);+ await runScenario(page, testInfo, FIXTURES[0], false);+ });++ test('attributes renders with react-scan enabled', async ({ page }, testInfo) => {+ test.setTimeout(10 * 60 * 1000);+ await runScenario(page, testInfo, FIXTURES[1], true);+ });+});diff --git a/e2e/benchmarks-mobile-chat/payload.ts b/e2e/benchmarks-mobile-chat/payload.tsnew file mode 100644index 0000000000..883f7258f6--- /dev/null+++ b/e2e/benchmarks-mobile-chat/payload.ts@@ -0,0 +1,62 @@+import type { SeedMessage } from '../specs/mock/db';++export const TURNS = 150;+export const ROWS = TURNS * 2;+export const STREAM_END_MARKER = 'MOBILE-PERF-STREAM-END';++const PROSE =+ 'This mobile performance transcript deliberately combines prose, lists, tables, and code so each mounted message exercises realistic chat rendering. ';++function assistantBody(turn: number): string {+ let body = `## Mobile stress turn ${turn}\n\n${PROSE}${PROSE}\n\n`;+ body += `- First observation for turn ${turn}\n`;+ body += `- Second observation for turn ${turn}\n\n`;+ if (turn % 5 === 0) {+ body +=+ '```ts\nexport function mobileSample(value: number): number {\n return value * 2;\n}\n```\n\n';+ }+ if (turn % 7 === 0) {+ body +=+ '| Metric | Value |\n| --- | ---: |\n' + `| messages | ${turn * 2} |\n| turn | ${turn} |\n\n`;+ }+ return body;+}++export function buildStressMessages(label: string): SeedMessage[] {+ const messages: SeedMessage[] = [];+ let parentMessageId = '00000000-0000-0000-0000-000000000000';+ for (let turn = 1; turn <= TURNS; turn += 1) {+ const userMessageId = `${label}-user-${turn}`;+ messages.push({+ messageId: userMessageId,+ parentMessageId,+ text: `Mobile stress prompt ${turn}: summarize the measurements for this turn.`,+ isCreatedByUser: true,+ sender: 'User',+ });+ const assistantMessageId = `${label}-assistant-${turn}`;+ messages.push({+ messageId: assistantMessageId,+ parentMessageId: userMessageId,+ text: assistantBody(turn),+ isCreatedByUser: false,+ sender: 'Assistant',+ });+ parentMessageId = assistantMessageId;+ }+ return messages;+}++export function buildContinuationReply(): string {+ let body = '# Continued mobile stress response\n\n';+ for (let section = 1; section <= 16; section += 1) {+ body += `## Stream section ${section}\n\n${PROSE}${PROSE}\n\n`;+ body += `1. Render the streamed section ${section}.\n`;+ body += `2. Keep the long transcript mounted behind it.\n\n`;+ if (section % 4 === 0) {+ body +=+ '```ts\nexport function streamedValue(input: number): number {\n return input + 1;\n}\n```\n\n';+ }+ }+ return `${body}${STREAM_END_MARKER}\n`;+}diff --git a/e2e/perf/browser.ts b/e2e/perf/browser.tsnew file mode 100644index 0000000000..a04cb434f9--- /dev/null+++ b/e2e/perf/browser.ts@@ -0,0 +1,153 @@+import type { CDPSession, Page, TestInfo } from '@playwright/test';++const METRIC_NAMES = [+ 'Timestamp',+ 'TaskDuration',+ 'ScriptDuration',+ 'LayoutDuration',+ 'RecalcStyleDuration',+ 'LayoutCount',+ 'RecalcStyleCount',+ 'JSHeapUsedSize',+ 'Nodes',+] as const;++type MetricName = (typeof METRIC_NAMES)[number];+type MetricMap = Record<MetricName, number>;++export interface BrowserPhase {+ elapsedMs: number;+ taskMs: number;+ scriptMs: number;+ layoutMs: number;+ styleMs: number;+ busyPercent: number;+ layoutCount: number;+ styleCount: number;+ heapDeltaBytes: number;+ heapEndBytes: number;+ nodesEnd: number;+ longTasks: number[];+}++interface BrowserPerfGlobal {+ longTasks: number[];+ reset(): void;+ drain(): void;+}++declare global {+ interface Window {+ __BROWSER_PERF__: BrowserPerfGlobal;+ }+}++const LONG_TASK_OBSERVER = `(() => {+ const perf = {+ longTasks: [],+ observer: null,+ drain() {+ if (!this.observer) {+ return;+ }+ for (const entry of this.observer.takeRecords()) {+ this.longTasks.push(entry.duration);+ }+ },+ reset() {+ this.drain();+ this.longTasks = [];+ },+ };+ window.__BROWSER_PERF__ = perf;+ try {+ perf.observer = new PerformanceObserver((list) => {+ for (const entry of list.getEntries()) {+ perf.longTasks.push(entry.duration);+ }+ });+ perf.observer.observe({ type: 'longtask', buffered: true });+ } catch (_error) {+ /* longtask unsupported: totals stay empty */+ }+})();`;++async function readMetrics(session: CDPSession): Promise<MetricMap> {+ const response = await session.send('Performance.getMetrics');+ const values = new Map(response.metrics.map(({ name, value }) => [name, value]));+ return Object.fromEntries(METRIC_NAMES.map((name) => [name, values.get(name) ?? 0])) as MetricMap;+}++export async function installBrowserPerf(page: Page): Promise<void> {+ await page.addInitScript({ content: LONG_TASK_OBSERVER });+}++export async function createBrowserProbe(page: Page): Promise<{+ start(): Promise<void>;+ finish(): Promise<BrowserPhase>;+}> {+ const session = await page.context().newCDPSession(page);+ await session.send('Performance.enable');+ let startMetrics: MetricMap | null = null;++ return {+ async start() {+ await page.evaluate(() => window.__BROWSER_PERF__.reset());+ startMetrics = await readMetrics(session);+ },+ async finish() {+ if (!startMetrics) {+ throw new Error('Browser performance probe must be started before it is finished');+ }+ const endMetrics = await readMetrics(session);+ const longTasks = await page.evaluate(() => {+ window.__BROWSER_PERF__.drain();+ return window.__BROWSER_PERF__.longTasks.slice();+ });+ const secondsToMs = (name: MetricName) =>+ (endMetrics[name] - (startMetrics?.[name] ?? 0)) * 1000;+ const elapsedMs = secondsToMs('Timestamp');+ const taskMs = secondsToMs('TaskDuration');+ return {+ elapsedMs,+ taskMs,+ scriptMs: secondsToMs('ScriptDuration'),+ layoutMs: secondsToMs('LayoutDuration'),+ styleMs: secondsToMs('RecalcStyleDuration'),+ busyPercent: elapsedMs > 0 ? (taskMs / elapsedMs) * 100 : 0,+ layoutCount: endMetrics.LayoutCount - startMetrics.LayoutCount,+ styleCount: endMetrics.RecalcStyleCount - startMetrics.RecalcStyleCount,+ heapDeltaBytes: endMetrics.JSHeapUsedSize - startMetrics.JSHeapUsedSize,+ heapEndBytes: endMetrics.JSHeapUsedSize,+ nodesEnd: endMetrics.Nodes,+ longTasks,+ };+ },+ };+}++export function formatBrowserPhase(name: string, phase: BrowserPhase): string {+ const longTaskTotal = phase.longTasks.reduce((sum, duration) => sum + duration, 0);+ const worstLongTask = phase.longTasks.reduce((max, duration) => Math.max(max, duration), 0);+ return (+ `${name.padEnd(18)} wall=${phase.elapsedMs.toFixed(0).padStart(6)}ms ` ++ `busy=${phase.busyPercent.toFixed(1).padStart(5)}% ` ++ `task=${phase.taskMs.toFixed(0).padStart(6)}ms ` ++ `script=${phase.scriptMs.toFixed(0).padStart(6)}ms ` ++ `layout=${phase.layoutMs.toFixed(0).padStart(5)}ms ` ++ `style=${phase.styleMs.toFixed(0).padStart(5)}ms ` ++ `longtasks=${longTaskTotal.toFixed(0)}/${worstLongTask.toFixed(0)}ms ` ++ `heap=${(phase.heapEndBytes / 1024 / 1024).toFixed(1)}MB nodes=${phase.nodesEnd}`+ );+}++export async function attachBrowserPhases(+ testInfo: TestInfo,+ name: string,+ phases: Record<string, BrowserPhase>,+): Promise<void> {+ await testInfo.attach(name, {+ body: JSON.stringify(phases, null, 2),+ contentType: 'application/json',+ });+}diff --git a/e2e/playwright.config.mobile-chat-perf.ts b/e2e/playwright.config.mobile-chat-perf.tsnew file mode 100644index 0000000000..a8e4b8f2e2--- /dev/null+++ b/e2e/playwright.config.mobile-chat-perf.ts@@ -0,0 +1,36 @@+import { defineConfig, devices } from '@playwright/test';+import mockConfig from './playwright.config.mock';+import { buildContinuationReply } from './benchmarks-mobile-chat/payload';++const mockServers = Array.isArray(mockConfig.webServer) ? mockConfig.webServer : [];+if (mockConfig.webServer && !Array.isArray(mockConfig.webServer)) {+ mockServers.push(mockConfig.webServer);+}+const benchmarkModelEnv = {+ MOCK_LLM_REPLY: buildContinuationReply(),+ MOCK_LLM_CHUNK_DELAY_MS: '1',+};++export default defineConfig({+ ...mockConfig,+ testDir: 'benchmarks-mobile-chat',+ outputDir: 'benchmarks/.test-results/mobile-chat',+ fullyParallel: false,+ timeout: 10 * 60 * 1000,+ retries: 0,+ workers: 1,+ reporter: [['line']],+ webServer: mockServers.map((server) => ({+ ...server,+ env: { ...server.env, ...benchmarkModelEnv },+ })),+ projects: [+ {+ name: 'iPhone 13 (Chromium)',+ use: {+ ...devices['iPhone 13'],+ browserName: 'chromium',+ },+ },+ ],+});diff --git a/package.json b/package.jsonindex f4f5962c79..a60c89de17 100644--- a/package.json+++ b/package.json@@ -72,6 +72,7 @@"e2e:benchmark:agents": "npm run e2e:prepare && playwright test --config=e2e/playwright.config.benchmark.ts agent-startup.latency.spec.ts","e2e:benchmark:navigation": "npm run e2e:prepare && playwright test --config=e2e/playwright.config.navigation-perf.ts","e2e:benchmark:reasoning": "npm run e2e:prepare && playwright test --config=e2e/playwright.config.reasoning-perf.ts",+ "e2e:benchmark:mobile-chat": "npm run e2e:prepare && playwright test --config=e2e/playwright.config.mobile-chat-perf.ts","e2e:bombadil": "npm run e2e:prepare && playwright test --config=e2e/playwright.config.bombadil.ts","e2e:bombadil:run": "playwright test --config=e2e/playwright.config.bombadil.ts","e2e:bombadil:branch-reload": "npm run e2e:prepare && cross-env BOMBADIL_SPECIFICATION=branch-reload.specification.ts BOMBADIL_TIME_LIMIT=30s playwright test --config=e2e/playwright.config.bombadil.ts",