Lightpanda: the headless browser designed for AI and automation
CI: Weekly orderfile generation + PR
Karl Seguin84b8500f2e0b
diff --git a/.github/workflows/orderfile.yml b/.github/workflows/orderfile.ymlnew file mode 100644index 000000000..243694b7e--- /dev/null+++ b/.github/workflows/orderfile.yml@@ -0,0 +1,105 @@+name: orderfile++# Regenerates orderfile/lightpanda.ld + orderfile/v8.txt from a profile of the+# CDP bench and opens a pull request with the result, every Monday or on+# demand from the Actions tab. See orderfile/README.md.++env:+ LIGHTPANDA_DISABLE_TELEMETRY: true++on:+ schedule:+ - cron: "2 0 * * 1"++ # Allows you to run this workflow manually from the Actions tab+ workflow_dispatch:++permissions:+ contents: read++jobs:+ regen:+ name: regenerate orderfile++ runs-on: ubuntu-latest+ timeout-minutes: 45++ steps:+ # The branch push and the PR go through the distribution app rather+ # than GITHUB_TOKEN: pull requests opened with GITHUB_TOKEN never+ # trigger the CI workflows, so they would never get a check.+ - name: Generate token for browser+ id: app-token+ uses: actions/create-github-app-token@fee1f7d63c2ff003460e3d139729b119787bc349 # v2.2.2+ with:+ app-id: ${{ secrets.DISTRIBUTION_APP_ID }}+ private-key: ${{ secrets.DISTRIBUTION_APP_PRIVATE_KEY }}+ owner: lightpanda-io+ repositories: browser++ - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3+ with:+ ref: main+ fetch-depth: 0+ token: ${{ steps.app-token.outputs.token }}++ - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3+ with:+ repository: 'lightpanda-io/demo'+ path: demo++ - uses: ./.github/actions/install+ - uses: ./.github/actions/v8-snapshot++ - run: npm install+ working-directory: demo++ - name: regenerate the profile+ env:+ DEMO_DIR: demo+ OUT: ${{ runner.temp }}/orderfile-regen+ run: |+ orderfile/tools/regen.sh -Dsnapshot_path=../../snapshot.bin -Dprebuilt_v8_path=v8/libc_v8.a -Doptimize=ReleaseFast -Dcpu=x86_64++ - name: upload profile+ if: always()+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1+ with:+ name: orderfile-profile+ path: |+ ${{ runner.temp }}/orderfile-regen/result.txt+ ${{ runner.temp }}/orderfile-regen/gen_order.stats+ ${{ runner.temp }}/orderfile-regen/hot.text+ ${{ runner.temp }}/orderfile-regen/hot.rodata+ ${{ runner.temp }}/orderfile-regen/resident.json+ ${{ runner.temp }}/orderfile-regen/link.line+ ${{ runner.temp }}/orderfile-regen/bench.out+ ${{ runner.temp }}/orderfile-regen/lightpanda.ld+ ${{ runner.temp }}/orderfile-regen/v8.txt+ retention-days: 10++ - name: open a pull request+ env:+ GH_TOKEN: ${{ steps.app-token.outputs.token }}+ OUT: ${{ runner.temp }}/orderfile-regen+ BRANCH: orderfile-regen+ run: |+ if git diff --quiet -- orderfile/lightpanda.ld orderfile/v8.txt; then+ echo "profile unchanged"+ exit 0+ fi+ slug="${{ steps.app-token.outputs.app-slug }}[bot]"+ git config user.name "$slug"+ git config user.email "$(gh api "/users/$slug" --jq .id)[email protected]"+ git add orderfile/lightpanda.ld orderfile/v8.txt+ git commit -m "orderfile: regenerate the hot-code profile" -m "$(cat "$OUT/result.txt")"+ # One fixed branch, force-pushed: a re-run refreshes the open PR+ # instead of stacking a second one.+ git push --force origin "HEAD:refs/heads/$BRANCH"+ if [ -n "$(gh pr list --head "$BRANCH" --base main --state open --json number --jq '.[].number')" ]; then+ echo "refreshed the open pull request"+ exit 0+ fi+ gh pr create --base main --head "$BRANCH" \+ --title "orderfile: regenerate the hot-code profile" \+ --body-file "$OUT/result.txt"diff --git a/orderfile/README.md b/orderfile/README.mdindex 4abec0891..271e9aa47 100644--- a/orderfile/README.md+++ b/orderfile/README.md@@ -27,8 +27,8 @@ below), with no change in run duration.- Patterns that match nothing are ignored by LLD, so a stale script degradesgracefully: renamed or removed functions simply fall back into cold `.text`.Zig's `__anon_NNN` names (~7% of the patterns) renumber on unrelated- changes, so the profile decays a little with every commit; regenerate it- when CI's VmHWM creeps.+ changes, so the profile decays a little with every commit; it is+ regenerated weekly (below).- Link-time cost: LLD name-checks every input section against every patternof every input-section description, so the script scopes patterns to theirobject file (`*api.o(...)`). The shipped script covers Zig/Rust/C only@@ -55,38 +55,51 @@ below), with no change in run duration.## Regenerating the profile-The profile is a list of resident pages captured at 4KB granularity, so-fault-around has to be disabled while profiling (root):+`.github/workflows/orderfile.yml` regenerates `lightpanda.ld` and `v8.txt`+every Monday (00:02 UTC) and opens a pull request from the `orderfile-regen`+branch with the result (a re-run refreshes that PR rather than opening+another); it can also be run from the Actions tab. It only runs+`orderfile/tools/regen.sh`, which works the same locally:```bash-# 1. Build with the current script (or none) and copy the binary to tmpfs.-# Bench from tmpfs: on ext4 with a recent kernel, large page-cache folios-# are mapped whole and hide the 64KB-window behaviour that CI sees.-zig build -Doptimize=ReleaseFast -Dsnapshot_path=../../snapshot.bin -Dcpu=x86_64 \- -Dorderfile=orderfile/lightpanda.ld-cp zig-out/bin/lightpanda /tmp/lightpanda+# root for /sys/kernel/debug/fault_around_bytes; ../demo checked out with+# `npm install` done; node, go, python3 and binutils on the PATH.+orderfile/tools/regen.sh -Doptimize=ReleaseFast -Dsnapshot_path=../../snapshot.bin -Dcpu=x86_64+```-# 2. Run the workload with fault-around off and dump which pages are resident.-sudo sh -c 'echo 4096 > /sys/kernel/debug/fault_around_bytes'-/tmp/lightpanda serve --insecure-disable-tls-host-verification & LP=$!-(cd ../demo && RUNS=100 node puppeteer/cdp.js)-sleep 5-python3 orderfile/tools/pagemap.py $LP resident.json-kill $LP-sudo sh -c 'echo 65536 > /sys/kernel/debug/fault_around_bytes'+The script-# 3. Resident pages -> hot symbol lists (address order), then -> linker script.-# gen_order.py needs every object/archive of the link to map symbols to-# section names: `zig build ... --verbose-link` prints the ld.lld line.-python3 orderfile/tools/hotlist.py /tmp/lightpanda resident.json hot-python3 orderfile/tools/gen_order.py hot.text hot.rodata orderfile/lightpanda.ld \- <uniq-v8.a> .zig-cache/o/*/lightpanda_zcu.o .zig-cache/o/*/*.a ...-```+1. builds with an empty linker script (the unordered binary; the+ `--verbose-link` line of that build is where `gen_order.py` gets the+ objects and archives it maps symbols to sections with);+2. runs the CDP bench (`demo/puppeteer/cdp.js`, 100 runs) against it with+ fault-around set to 4KB, dumps the pages that are resident+ (`pagemap.py`), turns them into hot symbol lists in address order+ (`hotlist.py`) and into the new `lightpanda.ld` / `v8.txt`+ (`gen_order.py`);+3. re-runs that build's `ld.lld` line with the new script to check that it+ links (the script is a cache input of the whole exe compilation, so+ building with it would be a second full compile; the relink takes+ seconds and is the same link, bar V8's hot-marked archive coming from+ the previous `v8.txt`, which does not affect whether the script parses).++Whether the new profile is any good is not measured here; the e2e bench's+`MAX_VmHWM` gate is, on the next push to `main`.++The profile is a list of resident pages, so every function sharing a 4KB page+with a hot one is counted as hot. Profiling the *unordered* binary keeps that+over-inclusion random: a profile of the ordered binary would re-include every+function that went cold but still sits in `.text.hot` next to a hot one, and+the hot set would only ever grow. Over-inclusion is the cheap side of the+trade anyway (a hot function left out costs a whole 64KB cold window; one+included needlessly costs its own size), and `.text.hot` is packed in+address order of the profile, so functions that run together stay together.-Profiles from several runs can be concatenated before `gen_order.py`-(duplicates are dropped, first occurrence wins); the shipped script is the-union of a profile of the unordered binary and one of the ordered binary.+The binary is benched from tmpfs (`RAMDIR`, default `/dev/shm`): on ext4 with+a recent kernel, large page-cache folios are mapped whole and hide the+64KB-window behaviour that CI sees. The `x86_64` profile is applied to the+`aarch64` release too; it carries over as far as the section and symbol+names coincide, nothing is measured on `aarch64`.-To check the result without root, compare `VmHWM` from `/proc/<pid>/status`-after the bench, and look at what is still resident outside the hot sections-with `pagemap.py` + `nm`.+To look at what is still resident outside the hot sections, run the bench by+hand and use `pagemap.py` + `nm` on the live process.diff --git a/orderfile/tools/regen.sh b/orderfile/tools/regen.shnew file mode 100755index 000000000..cc84187b5--- /dev/null+++ b/orderfile/tools/regen.sh@@ -0,0 +1,152 @@+#!/usr/bin/env bash+# Regenerates orderfile/lightpanda.ld and orderfile/v8.txt from a profile of+# the CDP bench (demo/puppeteer/cdp.js), see orderfile/README.md.+#+# usage: orderfile/tools/regen.sh [zig build args...]+# The build args are those of the release build minus -Dorderfile, e.g.+# -Doptimize=ReleaseFast -Dsnapshot_path=../../snapshot.bin -Dcpu=x86_64+#+# Needs root (sudo) for /sys/kernel/debug/fault_around_bytes, a checkout of+# lightpanda-io/demo (DEMO_DIR, npm install done), node, go, python3 and+# binutils. Run from the repository root.+#+# Steps:+# 1. Build with an empty linker script: the unordered binary. Profiling+# it, rather than one laid out by the current profile, keeps stale+# entries from surviving forever by sharing a page with a hot neighbour.+# 2. Bench it with 4KB fault-around, dump the resident pages and turn them+# into the new lightpanda.ld / v8.txt.+# 3. Re-run the build's ld.lld line with the new script to check that it+# links. The script is a cache input of the whole exe compilation, so+# building with it would be a second full release compile; the manual+# relink is the same link in seconds (only V8's hot-marked archive is+# the one from the current v8.txt, which does not affect parsing).+# Whether the new profile is any good is measured by the e2e bench.+#+# Environment:+# DEMO_DIR demo checkout (default ../demo)+# RUNS bench iterations (default 100)+# OUT scratch directory (default /tmp/orderfile-regen)+# RAMDIR tmpfs the binary is benched from (default /dev/shm)+set -euo pipefail++DEMO_DIR=${DEMO_DIR:-../demo}+RUNS=${RUNS:-100}+OUT=${OUT:-/tmp/orderfile-regen}+RAMDIR=${RAMDIR:-/dev/shm}+FAULT_AROUND=/sys/kernel/debug/fault_around_bytes++ROOT=$(pwd)+DEMO_DIR=$(cd "$DEMO_DIR" && pwd)+TOOLS=$ROOT/orderfile/tools+LD=$ROOT/orderfile/lightpanda.ld+V8_TXT=$ROOT/orderfile/v8.txt++[ -f "$LD" ] || { echo "run from the repository root" >&2; exit 2; }+[ -f "$DEMO_DIR/puppeteer/cdp.js" ] || { echo "DEMO_DIR=$DEMO_DIR is not a demo checkout" >&2; exit 2; }+mkdir -p "$OUT"++# Fault-around is a global knob: whatever happens, put it back.+if [ ! -f "$FAULT_AROUND" ]; then+ sudo mount -t debugfs none /sys/kernel/debug+fi+FAULT_AROUND_DEFAULT=$(sudo cat "$FAULT_AROUND")+set_fault_around() { echo "$1" | sudo tee "$FAULT_AROUND" > /dev/null; }++PIDS=()+cleanup() {+ set_fault_around "$FAULT_AROUND_DEFAULT" || true+ for pid in "${PIDS[@]:-}"; do+ [ -n "$pid" ] && kill "$pid" 2> /dev/null || true+ done+ rm -f "$RAMDIR"/lightpanda-regen-$$-*+}+trap cleanup EXIT++log() { echo "== $*" >&2; }++# The demo web server the bench navigates to (port 1234).+if ! curl -sf -o /dev/null http://127.0.0.1:1234/campfire-commerce/; then+ (cd "$DEMO_DIR" && go run runner/main.go -serve > "$OUT/runner.log" 2>&1) &+ PIDS+=($!)+ for _ in $(seq 50); do+ curl -sf -o /dev/null http://127.0.0.1:1234/campfire-commerce/ && break+ sleep 0.2+ done+fi++# The tmpfs copy the binary is benched from. It outlives the bench: the+# resident pages dump names it, and hotlist.py matches mappings by path.+ram_copy() { echo "$RAMDIR/lightpanda-regen-$$-$(basename "$1")"; }++# profile <binary> <resident.json>+# Runs the CDP bench against <binary> served from tmpfs with 4KB fault-around,+# dumps its resident pages while it is still alive and prints its VmHWM (KB).+profile() {+ local bin=$1 resident=$2+ local ram+ ram=$(ram_copy "$bin")+ cp "$bin" "$ram"+ set_fault_around 4096+ "$ram" serve --insecure-disable-tls-host-verification > /dev/null 2>&1 &+ local pid=$!+ sleep 1+ (cd "$DEMO_DIR" && RUNS=$RUNS node puppeteer/cdp.js > "$OUT/bench.out")+ sleep 2+ python3 "$TOOLS/pagemap.py" "$pid" "$resident" >&2+ local hwm+ hwm=$(grep VmHWM "/proc/$pid/status" | grep -oP '\d+')+ kill "$pid"+ while kill -0 "$pid" 2> /dev/null; do sleep 0.2; done+ set_fault_around "$FAULT_AROUND_DEFAULT"+ echo "$hwm"+}++# 1. Unordered build. The script's content is a cache input, so the timestamp+# guarantees a fresh link and thus a --verbose-link line listing every+# object and archive of the link.+log "building without ordering"+printf '/* unordered profile build, %s */\n' "$(date +%s%N)" > "$OUT/empty.ld"+zig build "$@" -Dorderfile="$OUT/empty.ld" --verbose-link 2> "$OUT/link.log" \+ || { cat "$OUT/link.log" >&2; exit 1; }+cp zig-out/bin/lightpanda "$OUT/unordered"+grep -m1 -F 'lightpanda_zcu.o' "$OUT/link.log" | sed 's/^error: ld\.lld //' > "$OUT/link.line"+[ -s "$OUT/link.line" ] || { echo "no ld.lld line in $OUT/link.log" >&2; exit 1; }++# Link inputs -> objects/archives for gen_order.py; the V8 archive is+# handled separately (its symbols go to v8.txt).+V8_ARCHIVE=+OBJS=()+for arg in $(cat "$OUT/link.line"); do+ case $arg in+ *.o | *.a) [ -f "$arg" ] || continue ;;+ *) continue ;;+ esac+ case $(basename "$arg") in+ libc_v8*.a) V8_ARCHIVE=$arg ;;+ *) case " ${OBJS[*]:-} " in *" $arg "*) ;; *) OBJS+=("$arg") ;; esac ;;+ esac+done+[ -n "$V8_ARCHIVE" ] || { echo "no V8 archive in the link line" >&2; exit 1; }+log "${#OBJS[@]} link inputs, V8 archive $V8_ARCHIVE"++# 2. Profile the unordered binary at 4KB fault-around.+log "profiling the unordered binary (RUNS=$RUNS)"+HOT_SET_KB=$(profile "$OUT/unordered" "$OUT/resident.json")+python3 "$TOOLS/hotlist.py" "$(ram_copy "$OUT/unordered")" "$OUT/resident.json" "$OUT/hot" >&2+[ -s "$OUT/hot.text" ] || { echo "empty profile, see $OUT/resident.json" >&2; exit 1; }+python3 "$TOOLS/gen_order.py" "$OUT/hot.text" "$OUT/hot.rodata" "$OUT/lightpanda.ld" \+ --v8 "$V8_ARCHIVE" "$OUT/v8.txt" "${OBJS[@]}" > "$OUT/gen_order.stats"+cat "$OUT/gen_order.stats" >&2+[ -s "$OUT/v8.txt" ] || { echo "no V8 symbols in the profile" >&2; exit 1; }++# 3. Does it link? Same objects, same flags, new script.+log "linking with the new orderfile"+# shellcheck disable=SC2046+zig ld.lld $(sed "s#-T [^ ]*#-T $OUT/lightpanda.ld#; s#-o [^ ]*#-o $OUT/linked#" "$OUT/link.line")+"$OUT/linked" version > /dev/null++cp "$OUT/lightpanda.ld" "$LD"+cp "$OUT/v8.txt" "$V8_TXT"+printf 'hot set %sKB resident at 4KB fault-around: %s text, %s rodata symbols\n' \+ "$HOT_SET_KB" "$(wc -l < "$OUT/hot.text")" "$(wc -l < "$OUT/hot.rodata")" | tee "$OUT/result.txt"