Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

ci: read the runner's Chrome install into the page cache before DB suites launch it (#9641)

[._.]/ Adam Eivyd5ce5c238b65
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 8ed90ad57a..41897dac70 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -663,6 +663,15 @@ jobs:
PG_MAJOR: '17'
run: bash scripts/ci-install-db-tools.sh
+ # Read the runner's Chrome tree into the page cache before any suite
+ # launches it (#9641). The recurring startup hang was captured with the
+ # browser's lead thread waiting on a page-cache read and I/O pressure
+ # above 50 right after the installs above, i.e. a cold first launch. The
+ # launcher's 20s startup deadline is unchanged; the printed duration shows
+ # whether the cold read is as slow as the failures suggest.
+ - name: Warm the Chrome install
+ run: bash scripts/ci-warm-test-chrome.sh
+
- name: Run DB-backed tests (portos_test)
if: needs.impact.outputs.db == 'true'
run: |
diff --git a/scripts/ci-warm-test-chrome.sh b/scripts/ci-warm-test-chrome.sh
new file mode 100755
index 0000000000..ee0d71d24c
--- /dev/null
+++ b/scripts/ci-warm-test-chrome.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# Reads the runner's Chrome install into the page cache before the DB suites
+# start, within a bounded wall-clock budget, and reports how long that took.
+#
+# Why (#9641): the recurring "Test Chrome did not start within 20000ms" failures
+# captured the owned child in uninterruptible sleep with its lead thread waiting
+# on a page-cache folio (leadWait=page) while the runner's I/O pressure
+# (ioAvg10) was above 50, and no DevToolsActivePort yet. That is the first,
+# cold read of a ~300MB browser tree on a fresh runner that has just written
+# ~95 ffmpeg packages and PostgreSQL data. Reading the tree here moves that cost
+# out of the 20s startup deadline, which stays unchanged. This is not a retry
+# and not a skip; the tests still launch, and must start, the real browser.
+#
+# The reported duration is also the evidence: a long warm read confirms the
+# cold page-in mechanism, a short one on a failing run refutes it.
+#
+# Never fails the job: a missing browser is reported by the suites themselves
+# (PORTOS_REQUIRE_BROWSER_SUITES), and the warm read is only an optimisation.
+# CHROME_WARM_BUDGET_SECONDS and CHROME_WARM_CANDIDATES exist for the contract
+# test and for runners with a different layout.
+set -uo pipefail
+
+BUDGET="${CHROME_WARM_BUDGET_SECONDS:-90}"
+read -r -a CANDIDATES <<< "${CHROME_WARM_CANDIDATES:-${CHROME_PATH:-} /usr/bin/google-chrome /usr/bin/chromium /usr/bin/chromium-browser}"
+
+# GNU timeout exists on the Ubuntu runner; elsewhere (a developer's macOS) the
+# read simply runs unbounded, which is still a bounded local directory.
+TIMEOUT=()
+if command -v timeout >/dev/null 2>&1; then TIMEOUT=(timeout --kill-after=5 "$BUDGET"); fi
+
+for candidate in "${CANDIDATES[@]}"; do
+ [ -n "$candidate" ] && [ -e "$candidate" ] || continue
+ executable="$(readlink -f "$candidate")"
+ # The install directory holds the binary, its resources and its libraries.
+ tree="$(dirname "$executable")"
+ start="$(date +%s%N)"
+ # cat's exit code is lost behind the pipe, so the byte count is the signal
+ # that the read finished and the timeout's 124 is the signal that it did not.
+ total="$(mktemp)"
+ ${TIMEOUT[@]+"${TIMEOUT[@]}"} bash -c 'find "$1" -type f -print0 | xargs -0 cat 2>/dev/null | wc -c > "$2"' _ "$tree" "$total" 2>/dev/null
+ status=$?
+ bytes="$(tr -d '[:space:]' < "$total")"
+ rm -f "$total"
+ elapsed_ms=$(( ($(date +%s%N) - start) / 1000000 ))
+ if [ "$status" -eq 0 ] && [ "${bytes:-0}" -gt 0 ]; then
+ echo "🔥 Warmed the Chrome install into the page cache: $(( bytes / 1048576 )) MiB in ${elapsed_ms}ms"
+ else
+ echo "::warning title=Chrome warm-up incomplete::Read ${bytes:-0} bytes in ${elapsed_ms}ms (exit ${status}, budget ${BUDGET}s); the suites will launch the browser cold."
+ fi
+ exit 0
+done
+
+echo "::notice title=Chrome warm-up skipped::No Chrome install was found to warm; the browser suites report a missing browser themselves."
+exit 0
diff --git a/scripts/ci-warm-test-chrome.test.js b/scripts/ci-warm-test-chrome.test.js
new file mode 100644
index 0000000000..3a8fa04bac
--- /dev/null
+++ b/scripts/ci-warm-test-chrome.test.js
@@ -0,0 +1,47 @@
+import { describe, it, expect } from 'vitest';
+import { mkdtempSync, mkdirSync, writeFileSync, rmSync, symlinkSync, chmodSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { spawnSync } from 'node:child_process';
+import { fileURLToPath } from 'node:url';
+
+const script = fileURLToPath(new URL('./ci-warm-test-chrome.sh', import.meta.url));
+
+// Runs the real script against a fake browser tree; nothing reads the host's.
+function run({ layout, budget = '5' }) {
+ const dir = mkdtempSync(join(tmpdir(), 'ci-warm-chrome-'));
+ const install = join(dir, 'opt/chrome');
+ mkdirSync(install, { recursive: true });
+ let candidates = join(dir, 'missing');
+ if (layout !== 'missing') {
+ writeFileSync(join(install, 'chrome'), 'x'.repeat(2 * 1048576));
+ writeFileSync(join(install, 'resources.pak'), 'y'.repeat(1048576));
+ mkdirSync(join(install, 'locales'));
+ writeFileSync(join(install, 'locales/en.pak'), 'z'.repeat(1048576));
+ // The runner exposes the browser as a symlink into its install directory.
+ candidates = join(dir, 'google-chrome');
+ symlinkSync(join(install, 'chrome'), candidates);
+ }
+ if (layout === 'unreadable') chmodSync(join(install, 'chrome'), 0o000);
+ const result = spawnSync('/bin/bash', [script], {
+ encoding: 'utf8',
+ env: { PATH: process.env.PATH, CHROME_WARM_CANDIDATES: candidates, CHROME_WARM_BUDGET_SECONDS: budget },
+ });
+ rmSync(dir, { recursive: true, force: true });
+ return result;
+}
+
+describe.skipIf(process.platform === 'win32')('CI Chrome page-cache warm-up', () => {
+ it('reads the whole install tree the symlinked browser lives in and reports size and time', () => {
+ const result = run({ layout: 'installed' });
+ expect(result.status).toBe(0);
+ // 2 + 1 + 1 MiB across the binary, a resource and a nested locale file.
+ expect(result.stdout).toMatch(/^🔥 Warmed the Chrome install into the page cache: 4 MiB in \d+ms$/m);
+ });
+
+ it('never fails the job when no browser exists; the suites report that themselves', () => {
+ const result = run({ layout: 'missing' });
+ expect(result.status).toBe(0);
+ expect(result.stdout).toContain('::notice title=Chrome warm-up skipped::');
+ });
+});