Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

Improve UI tests + CI: artifact uploads, WebKit skip narrowing, settle-wait migrations (#4061)

LearningCircuit8597e429cc83
diff --git a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
index 983a299f05..ef1eaa83e3 100644
--- a/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
+++ b/.github/PULL_REQUEST_TEMPLATE/pull_request_template.md
@@ -1,3 +1,12 @@
## Description
Fixes #
+
+## CI test coverage
+
+By default, this PR runs the unit/lint checks. Heavy E2E suites are label-gated — add the label that matches what you touched so the relevant workflow runs:
+
+- `test:puppeteer` or `test:e2e` — Puppeteer E2E suite (~30–60 min; uses paid LLM/search API quotas).
+- `ldr_research` or `ldr_research_static` — LDR research integration workflow that posts findings as a PR comment.
+
+WebKit/Mobile Safari tests run on the daily 02:00 UTC schedule and at release; the responsive UI suite runs at release and on manual dispatch.
diff --git a/.github/labels.yml b/.github/labels.yml
index 5f1adb7215..a6636ecd9b 100644
--- a/.github/labels.yml
+++ b/.github/labels.yml
@@ -25,10 +25,21 @@
description: Author has responded; needs codeowner re-review
# LDR research trigger labels (used by .github/workflows/e2e-research-test.yml
# and .github/workflows/issue-research.yml — applied manually by maintainers,
-# auto-removed by the workflow after the run completes):
+# auto-removed by the workflow after the run completes).
+# Descriptions are phrased as direct instructions so the AI code reviewer
+# (.github/workflows/ai-code-reviewer.yml) can decide when to apply them.
- name: ldr_research
color: '7057ff'
- description: Trigger LDR research on this PR or issue
+ description: Apply for substantive logic/architecture changes — when in doubt, add it. Runs full LDR research.
- name: ldr_research_static
color: '7057ff'
- description: Trigger LDR static-query regression run on this PR
+ description: Cheaper variant of ldr_research for routine code changes — favor running this over no analysis.
+# E2E test trigger labels (used by .github/workflows/puppeteer-e2e-tests.yml).
+# Both labels trigger the same Puppeteer suite; descriptions guide the AI
+# reviewer toward running the suite for any PR that touches the web stack.
+- name: 'test:puppeteer'
+ color: '0e8a16'
+ description: Apply if PR touches UI, web routes, auth, or templates — when in doubt, add it. Runs Puppeteer E2E.
+- name: 'test:e2e'
+ color: '1d76db'
+ description: Alias for test:puppeteer (same Puppeteer workflow). Apply one; both trigger the same job.
diff --git a/.github/workflows/responsive-ui-tests-enhanced.yml b/.github/workflows/responsive-ui-tests-enhanced.yml
index 2a56bb80dd..e9bbe972c7 100644
--- a/.github/workflows/responsive-ui-tests-enhanced.yml
+++ b/.github/workflows/responsive-ui-tests-enhanced.yml
@@ -203,10 +203,23 @@ jobs:
# Only fail if critical failures
exit "$TEST_EXIT_CODE"
- # NOTE: Screenshot generation and upload steps have been removed to fix actionlint warnings.
- # The actionlint pre-commit hook fails on "if: false" conditions, which were used to disable
- # these steps. Screenshots are not needed in CI, but can be re-enabled for visual regression
- # testing by uncommenting the workflow steps below (remove the 'if: false' conditions).
+ - name: Upload viewport test results
+ if: always()
+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+ with:
+ name: ui-test-results-${{ matrix.viewport }}
+ path: |
+ tests/ui_tests/test-output.log
+ tests/ui_tests/test-summary.md
+ tests/ui_tests/screenshots/
+ tests/ui_tests/responsive/
+ src/server.log
+ retention-days: 7
+ if-no-files-found: ignore
+
+ # NOTE: Earlier screenshot generation/upload steps were removed to fix actionlint warnings
+ # caused by "if: false" disablement. Screenshots are now captured into the artifact above
+ # when they exist. Visual regression workflows can re-enable dedicated screenshot steps later.
- name: Upload to GitHub Pages (if available)
if: always() && github.event_name == 'pull_request'
@@ -255,7 +268,9 @@ jobs:
post-results:
needs: ui-tests
runs-on: ubuntu-latest
- if: always() && github.event_name == 'pull_request'
+ # Workflow has no pull_request trigger; gating on event_name == 'pull_request' kept this job
+ # from ever running. Always run so the combined report is built for workflow_dispatch / release.
+ if: always()
steps:
- name: Harden the runner (Audit all outbound calls)
diff --git a/tests/ui_tests/auth_helper.js b/tests/ui_tests/auth_helper.js
index fbbc1b8d08..2d60c7ea61 100644
--- a/tests/ui_tests/auth_helper.js
+++ b/tests/ui_tests/auth_helper.js
@@ -887,8 +887,57 @@ async function safeClick(page, selectorOrElement, options = {}) {
}
}
+/**
+ * Wait for a selector to be present and visually stable.
+ *
+ * Use this instead of `await delay(N)` after an action that triggers UI to
+ * settle (e.g., a button click that reveals a panel, a typed input that
+ * triggers debounced validation). It resolves as soon as the element's
+ * bounding box hasn't moved for `idleMs` of `requestAnimationFrame` ticks,
+ * up to a 3s in-page budget.
+ *
+ * DO NOT replace `delay()` calls that intentionally exercise wall-clock
+ * behavior (e.g., a 10s timer the app is supposed to respect) — those tests
+ * need real elapsed time, not a settle wait.
+ *
+ * @param {Page} page - Puppeteer page object
+ * @param {string} selector - CSS selector to wait for
+ * @param {Object} [options]
+ * @param {number} [options.timeout=5000] Max time to wait for the selector to appear
+ * @param {number} [options.idleMs=200] Final settle pause after layout stabilizes
+ */
+async function waitForStable(page, selector, options = {}) {
+ const timeout = options.timeout || 5000;
+ const idleMs = options.idleMs ?? 200;
+
+ await page.waitForSelector(selector, { visible: true, timeout });
+ await page.evaluate(async (sel, idle) => {
+ const el = document.querySelector(sel);
+ if (!el) return;
+ let last = el.getBoundingClientRect();
+ const start = Date.now();
+ // Bounded in-page poll: returns as soon as the layout settles, or after
+ // 3s if the element keeps moving (caller can extend via outer logic).
+ while (Date.now() - start < 3000) {
+ await new Promise((r) => requestAnimationFrame(r));
+ const now = el.getBoundingClientRect();
+ if (
+ now.x === last.x &&
+ now.y === last.y &&
+ now.width === last.width &&
+ now.height === last.height
+ ) {
+ await new Promise((r) => setTimeout(r, idle));
+ return;
+ }
+ last = now;
+ }
+ }, selector, idleMs);
+}
+
module.exports = AuthHelper;
module.exports.safeClick = safeClick;
+module.exports.waitForStable = waitForStable;
module.exports.AUTH_CONFIG = AUTH_CONFIG;
module.exports.Timer = Timer;
module.exports.CI_TEST_USER = CI_TEST_USER;
diff --git a/tests/ui_tests/playwright/tests/all-pages-mobile.spec.js b/tests/ui_tests/playwright/tests/all-pages-mobile.spec.js
index 03a16c04e4..8e9940132f 100644
--- a/tests/ui_tests/playwright/tests/all-pages-mobile.spec.js
+++ b/tests/ui_tests/playwright/tests/all-pages-mobile.spec.js
@@ -251,7 +251,7 @@ test.describe('All Pages - Touch Targets', () => {
test.describe('All Pages - Content Above Mobile Nav', () => {
for (const pageInfo of PAGES) {
- test(`${pageInfo.name} content not hidden behind mobile nav`, async ({ page, isMobile }) => {
+ test(`${pageInfo.name} content not hidden behind mobile nav`, async ({ page, isMobile, browserName }) => {
if (!isMobile) {
test.skip();
return;
@@ -366,10 +366,19 @@ test.describe('All Pages - Content Above Mobile Nav', () => {
}, MOBILE_NAV_SELECTOR);
} catch (evalError) {
const msg = String(evalError && evalError.message || evalError);
- if (msg.includes('Target page, context or browser has been closed')) {
- // `test.skip(condition, reason)` throws internally to halt the
- // test, so no explicit return is needed here.
- test.skip(true, `WebKit closed the context mid-evaluate on ${pageInfo.name} — flake`);
+ // Same upstream WebKit race surfaces under two wordings depending on
+ // Playwright version; treat both as the documented flake (issue #4060).
+ if (
+ msg.includes('Target page, context or browser has been closed') ||
+ msg.includes('Execution context was destroyed')
+ ) {
+ // Only WebKit/Mobile Safari is known to hit this race. For every other
+ // browser, the error is a real regression — let it surface. `test.skip`
+ // throws internally to halt the test, so no explicit return is needed.
+ test.skip(
+ browserName === 'webkit',
+ `WebKit closed the context mid-evaluate on ${pageInfo.name} — flake (issue #4060)`
+ );
}
throw evalError;
}
diff --git a/tests/ui_tests/test_form_validation_aria_ci.js b/tests/ui_tests/test_form_validation_aria_ci.js
index 42c0d16fb2..cad1c41ba5 100644
--- a/tests/ui_tests/test_form_validation_aria_ci.js
+++ b/tests/ui_tests/test_form_validation_aria_ci.js
@@ -13,7 +13,39 @@
* Run: node test_form_validation_aria_ci.js
*/
-const { setupTest, teardownTest, TestResults, log, delay, navigateTo, withTimeout } = require('./test_lib');
+const { setupTest, teardownTest, TestResults, log, navigateTo, withTimeout } = require('./test_lib');
+
+// Helpers used by the validation tests below. We wait on real DOM state instead
+// of fixed delays so the tests stay correct on slow CI runners without padding
+// the wall-clock time on fast ones.
+async function waitForQueryReady(page, timeout = 5000) {
+ await page.waitForSelector('#query', { visible: true, timeout });
+ // The FormValidator hook is what actually gates "loading UI shown / not shown"
+ // on submit. addValidation() inserts a .ldr-field-error sibling next to #query,
+ // so we wait for that DOM marker — it proves the validator is registered, not
+ // just that the class is on window.
+ await page.waitForFunction(
+ () => {
+ const q = document.querySelector('#query');
+ if (!q || typeof window.FormValidator !== 'function') return false;
+ const sibling = q.nextElementSibling;
+ return !!sibling && sibling.classList.contains('ldr-field-error');
+ },
+ { timeout }
+ ).catch(() => {});
+}
+async function waitForValidationState(page, expectInvalid, timeout = 2000) {
+ await page.waitForFunction(
+ (wantInvalid) => {
+ const q = document.querySelector('#query');
+ if (!q) return false;
+ const isInvalid = q.classList.contains('ldr-field-invalid');
+ return wantInvalid ? isInvalid : !isInvalid;
+ },
+ { timeout },
+ expectInvalid
+ ).catch(() => {});
+}
// ============================================================================
// ARIA Attribute Tests
@@ -43,7 +75,13 @@ const AriaAttributeTests = {
async formValidatorLoaded(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ // Wait for the FormValidator module to actually attach instead of a fixed delay.
+ await page.waitForFunction(
+ () => typeof window.FormValidator === 'function' &&
+ typeof window.formValidators === 'object' &&
+ window.formValidators !== null,
+ { timeout: 5000 }
+ ).catch(() => {});
const result = await page.evaluate(() => {
return {
@@ -65,7 +103,7 @@ const AriaAttributeTests = {
const InlineValidationTests = {
async emptySubmitShowsInlineError(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// Clear query and submit
await page.evaluate(() => {
@@ -78,7 +116,7 @@ const InlineValidationTests = {
// Click submit
await page.click('button[type="submit"], .start-research, #start-research').catch(() => {});
- await delay(300);
+ await waitForValidationState(page, true);
const result = await page.evaluate(() => {
const query = document.querySelector('#query');
@@ -136,7 +174,7 @@ const InlineValidationTests = {
async blurOnEmptyShowsError(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// Clear query field
await page.evaluate(() => {
@@ -149,10 +187,10 @@ const InlineValidationTests = {
// Focus then blur the query field
await page.focus('#query');
- await delay(100);
+ await page.waitForFunction(() => document.activeElement?.id === 'query', { timeout: 1000 }).catch(() => {});
// Click somewhere else to trigger blur
await page.$eval('#query', el => el.blur());
- await delay(300);
+ await waitForValidationState(page, true);
const result = await page.evaluate(() => {
const query = document.querySelector('#query');
@@ -177,7 +215,7 @@ const InlineValidationTests = {
async errorClearsOnValidSubmit(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// First trigger the error by submitting empty
await page.evaluate(() => {
@@ -188,7 +226,7 @@ const InlineValidationTests = {
}
});
await page.click('button[type="submit"], .start-research, #start-research').catch(() => {});
- await delay(300);
+ await waitForValidationState(page, true);
// Verify error is shown
const errorShown = await page.evaluate(() => {
@@ -203,7 +241,11 @@ const InlineValidationTests = {
// Now type a valid query - this should clear the error on next submit attempt
await page.focus('#query');
await page.keyboard.type('Test research query');
- await delay(200);
+ // Wait for the typed value to land in the field instead of a fixed sleep.
+ await page.waitForFunction(
+ () => (document.querySelector('#query')?.value || '').includes('Test research'),
+ { timeout: 2000 }
+ ).catch(() => {});
// Verify the text was actually entered
const queryValue = await page.evaluate(() => {
@@ -225,7 +267,8 @@ const InlineValidationTests = {
// Click submit - the validation should pass and clear errors
await page.click('button[type="submit"], .start-research, #start-research').catch(() => {});
- await delay(1000);
+ // Wait for the ldr-field-invalid class to clear instead of a fixed sleep.
+ await waitForValidationState(page, false);
const result = await page.evaluate(() => {
const query = document.querySelector('#query');
@@ -249,8 +292,13 @@ const InlineValidationTests = {
},
async noLoadingUiOnEmptySubmit(page, baseUrl) {
- await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ // This test asserts no loading UI appears on an empty submit. A previous test
+ // leaves the page on `/` (navigateTo skips when paths match) with a stale
+ // .ldr-loading-overlay from a half-finished real submission. We need a
+ // genuinely fresh page here, so force a reload via page.goto instead of
+ // relying on the same-path optimization.
+ await page.goto(`${baseUrl}/`, { waitUntil: 'domcontentloaded' });
+ await waitForQueryReady(page);
// Clear query
await page.evaluate(() => {
@@ -291,7 +339,7 @@ const InlineValidationTests = {
const ErrorElementTests = {
async errorElementHasAriaLive(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// Trigger validation to create error element
await page.evaluate(() => {
@@ -302,9 +350,9 @@ const ErrorElementTests = {
}
});
await page.focus('#query');
- await delay(100);
+ await page.waitForFunction(() => document.activeElement?.id === 'query', { timeout: 1000 }).catch(() => {});
await page.$eval('#query', el => el.blur());
- await delay(300);
+ await waitForValidationState(page, true);
const result = await page.evaluate(() => {
const query = document.querySelector('#query');
@@ -346,7 +394,7 @@ const ErrorElementTests = {
async errorPositionedAfterTextarea(page, baseUrl) {
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// Trigger validation
await page.evaluate(() => {
@@ -354,9 +402,9 @@ const ErrorElementTests = {
if (query) query.value = '';
});
await page.focus('#query');
- await delay(100);
+ await page.waitForFunction(() => document.activeElement?.id === 'query', { timeout: 1000 }).catch(() => {});
await page.$eval('#query', el => el.blur());
- await delay(300);
+ await waitForValidationState(page, true);
const result = await page.evaluate(() => {
const query = document.querySelector('#query');
@@ -391,7 +439,7 @@ const ErrorElementTests = {
}
await navigateTo(page, `${baseUrl}/`);
- await delay(500);
+ await waitForQueryReady(page);
// Clear and submit
await page.evaluate(() => {
@@ -405,7 +453,7 @@ const ErrorElementTests = {
});
await page.click('button[type="submit"], .start-research, #start-research').catch(() => {});
- await delay(300);
+ await waitForValidationState(page, true);
const result = await page.evaluate(() => {
const activeElement = document.activeElement;
diff --git a/tests/ui_tests/test_research_cancellation.js b/tests/ui_tests/test_research_cancellation.js
index 1a1a029200..a31b777aff 100644
--- a/tests/ui_tests/test_research_cancellation.js
+++ b/tests/ui_tests/test_research_cancellation.js
@@ -75,8 +75,17 @@ async function startResearch(page, query) {
log(`Navigation wait: ${e.message}`, 'info');
}
- // Wait for page to stabilize after navigation
- await delay(2000);
+ // The cancellation flow needs the research/progress page; wait for either the
+ // URL to land or any progress UI to appear instead of a fixed sleep.
+ await page.waitForFunction(
+ () => {
+ const p = window.location.pathname;
+ return p.includes('/research/') ||
+ p.includes('/progress') ||
+ !!document.querySelector('.cancel-btn, .stop-btn, .progress-info, .progress-text');
+ },
+ { timeout: 5000 }
+ ).catch(() => {});
// Check if we're on a research page (URL-based check is safest during navigation)
const url = page.url();
@@ -113,8 +122,8 @@ async function testCancellation() {
await startResearch(page, 'Complex analysis of quantum computing applications');
- // Wait a bit then cancel
- await delay(3000);
+ // The cancel button only renders once the research starts; the waitForSelector
+ // below is the real signal we need, so no settle delay is required here.
// Find and click cancel button - use waitForSelector to ensure page is stable
let cancelButton = null;
@@ -134,7 +143,15 @@ async function testCancellation() {
});
await cancelButton.click();
- await delay(2000);
+ // Poll for the server-acknowledged cancel state instead of a fixed sleep;
+ // .catch(() => {}) keeps behavior identical when the page navigates away.
+ await page.waitForFunction(
+ () => {
+ const status = document.querySelector('.status, .research-status')?.textContent || '';
+ return /cancel|stop|suspend/i.test(status);
+ },
+ { timeout: 5000 }
+ ).catch(() => {});
// Check if research was cancelled - wrap in try-catch for navigation safety
try {
@@ -209,7 +226,15 @@ async function testCancellation() {
});
await cancelButton2.click();
- await delay(2000);
+ // Same pattern as the early-cancel branch: wait for the cancel state to
+ // surface in the UI rather than sleeping for a fixed interval.
+ await page.waitForFunction(
+ () => {
+ const status = document.querySelector('.status-badge, .research-status')?.textContent || '';
+ return /cancel|stop|suspend/i.test(status);
+ },
+ { timeout: 5000 }
+ ).catch(() => {});
try {
const afterCancel = await page.evaluate(() => {
@@ -283,7 +308,15 @@ async function testCancellation() {
// Click first resume button
await resumeButtons[0].click();
- await delay(2000);
+ // Wait for the resume to actually route to a research/progress page
+ // instead of using a fixed sleep.
+ await page.waitForFunction(
+ () => {
+ const p = window.location.pathname;
+ return p.includes('/research') || p.includes('/progress');
+ },
+ { timeout: 5000 }
+ ).catch(() => {});
// Check if research resumed
const url = page.url();
@@ -302,8 +335,8 @@ async function testCancellation() {
await startResearch(page, 'Quick test for immediate cancellation');
- // Cancel immediately
- await delay(500); // Very short delay
+ // The waitForSelector below already waits for the cancel button to render;
+ // no extra settle delay is needed.
try {
const quickCancel = await page.waitForSelector('.cancel-btn, .stop-btn', { timeout: 3000 });