The open-source Arc replacement for macOS. Spaces, pinned tabs, split view, Little Windows and Arc's shortcuts on Chromium, with your Arc sidebar imported in one go.
Intel CEF: cross-compile x64 on the arm64 Mac, rebuild-cef.sh for patch rounds
Victor6d8b2f63fdd1
diff --git a/docs/release.md b/docs/release.mdindex b9706ce..737bf18 100644--- a/docs/release.md+++ b/docs/release.md@@ -237,6 +237,16 @@ from a different source. To change the build without a CEF version bump, bump th`codecsN` suffix of `CEF_RELEASE` and publish again. Needs Xcode's Metal toolchain(`xcodebuild -downloadComponent MetalToolchain`).+**Patch rounds and Intel.** Once `build-cef.sh` has built a version, a new or changed+patch goes through `rebuild-cef.sh`, which builds from that checkout for both CPUs+(minutes each) and never reruns CEF's patcher (rerunning it compiles Web Push out). The+Intel (`macosx64`) build is cross-compiled on the same Mac from the same sources, in its+own `out/Release_GN_x64` (~12 GB; the first one compiles everything):++ scripts/rebuild-cef.sh [arm64] [x64] # both when no CPU is given+ scripts/publish-cef.sh # arm64 asset+ CEF_ARCH=macosx64 scripts/publish-cef.sh # Intel asset, same release+**Staying current.** Chromium only ships security fixes for its newest version. The"CEF updates" workflow (`.github/workflows/cef-updates.yml`, daily, free Linux runner)runs `scripts/check-cef-update.sh`: when Spotify's index lists a newer stable CEF for@@ -244,7 +254,7 @@ macOS arm64, it opens one "Chromium engine update available" issue, assigned toowner, with the exact command. The build itself runs on the Mac, because GitHub's freemacOS runners are too small (3 cores, 14 GB disk, 6 hour jobs) and larger ones are billed:- scripts/update-cef.sh <cef_version> # build, publish, pin, rebuild Cra+ scripts/update-cef.sh <cef_version> # build + publish arm64 and x64, pin, rebuild CraA same-major update reuses the checkout: `build-cef.sh` moves it to the new Chromium tagitself, and ninja recompiles nearly everything (about 2½ hours on the M2 Max). A new Chromium@@ -255,5 +265,6 @@ issue and tag a release.- **Homebrew.** A cask (`brew install --cask cra`) pointing at the DMG and`SHA256SUMS.txt`.-- **Intel.** The CEF fetch pins `macosarm64`; a universal build needs both CEF- distributions and `lipo`.+- **Intel.** CEF is published for `macosx64` too (above). Still missing: an x86_64+ `swift build` in `scripts/build.sh`, a separate Intel DMG from the release workflow+ and its own Sparkle feed.diff --git a/scripts/build-cef.sh b/scripts/build-cef.shindex 306c82c..505d2ca 100755--- a/scripts/build-cef.sh+++ b/scripts/build-cef.sh@@ -20,9 +20,16 @@ eval "$(grep -E '^CEF_VERSION=' scripts/fetch-cef.sh)"# 154.0.34+g14c5a08+chromium-154.0.8037.98: CEF commit 14c5a08 on branch 8037.CEF_COMMIT="$(sed -E 's/.*\+g([0-9a-f]+)\+.*/\1/' <<<"$CEF_VERSION")"CEF_BRANCH="$(sed -E 's/.*chromium-[0-9]+\.[0-9]+\.([0-9]+)\..*/\1/' <<<"$CEF_VERSION")"-# Build for the host CPU by default (automate-git/make_distrib take --arm64-build /-# --x86-build); CEF_TARGET_ARCH (arm64 / x86) overrides.-CEF_TARGET_ARCH="${CEF_TARGET_ARCH:-$([ "$(uname -m)" = arm64 ] && echo arm64 || echo x86)}"+# Builds for the host CPU; CEF_TARGET_ARCH=x64 (or arm64) cross-compiles. automate-git+# and make_distrib take --arm64-build / --x64-build, and CEF's gn_args.py only writes+# out/Release_GN_<cpu> for the other CPU when CEF_ENABLE_AMD64 / CEF_ENABLE_ARM64 is set.+# For a second CPU on a checkout that is already built, scripts/rebuild-cef.sh is faster.+CEF_TARGET_ARCH="${CEF_TARGET_ARCH:-$([ "$(uname -m)" = arm64 ] && echo arm64 || echo x64)}"+case "$CEF_TARGET_ARCH" in+ arm64) export CEF_ENABLE_ARM64=1 ;;+ x64) export CEF_ENABLE_AMD64=1 ;;+ *) echo "CEF_TARGET_ARCH must be arm64 or x64, not $CEF_TARGET_ARCH" >&2; exit 1 ;;+esacARCH_FLAG="--${CEF_TARGET_ARCH}-build"echo "Building CEF $CEF_VERSION (branch $CEF_BRANCH, commit $CEF_COMMIT, $CEF_TARGET_ARCH)"# .noindex keeps Spotlight out of ~40 GB of sources.@@ -105,12 +112,13 @@ for p in "$PATCHES"/*.patch; dofidone# 4. Build. make_distrib takes the framework from cefclient.app.-(cd chromium/src && autoninja -C out/Release_GN_arm64 cefclient)+(cd chromium/src && autoninja -C "out/Release_GN_$CEF_TARGET_ARCH" cefclient)LC_ALL=C grep -c -a GCM.RegistrationCacheStatus \- "chromium/src/out/Release_GN_arm64/cefclient.app/Contents/Frameworks/Chromium Embedded Framework.framework/Chromium Embedded Framework" >/dev/null \+ "chromium/src/out/Release_GN_$CEF_TARGET_ARCH/cefclient.app/Contents/Frameworks/Chromium Embedded Framework.framework/Chromium Embedded Framework" >/dev/null \|| { echo "GCM is stubbed out (components_gcm_disable.patch won over gcm-enabled.patch)" >&2; exit 1; }# 5. Package. automate-git does not pass --no-symbols on for branches after 7151,# and without dSYMs (enable_dsyms=false) make_distrib then fails.-rm -rf chromium/src/cef/binary_distrib+# Only this CPU's package: the other CPU's may not be published yet.+rm -rf chromium/src/cef/binary_distrib/cef_binary_"${CEF_VERSION}"_macos"$([ "$CEF_TARGET_ARCH" = x64 ] && echo x64 || echo arm64)"_minimal*(cd chromium/src/cef/tools && python3 make_distrib.py --output-dir=../binary_distrib --allow-partial \--ninja-build "$ARCH_FLAG" --minimal --no-docs --no-symbols)diff --git a/scripts/publish-cef.sh b/scripts/publish-cef.shindex 46beeee..f6f56fe 100755--- a/scripts/publish-cef.sh+++ b/scripts/publish-cef.sh@@ -2,7 +2,8 @@# Publishes the CEF distribution built by scripts/build-cef.sh as a GitHub release# asset, where scripts/fetch-cef.sh (and CI) download it from. The release is a# pre-release so releases/latest keeps pointing at the newest Cra.-# scripts/publish-cef.sh uses CEF_RELEASE from fetch-cef.sh+# scripts/publish-cef.sh this Mac's CPU, CEF_RELEASE from fetch-cef.sh+# CEF_ARCH=macosx64 scripts/publish-cef.sh the Intel build, into the same releaseset -euo pipefailcd "$(dirname "$0")/.."@@ -13,7 +14,7 @@ ARCHIVE="$CEF_BUILD_DIR/chromium/src/cef/binary_distrib/cef_binary_${CEF_VERSIONASSET="$(mktemp -d)/cef_binary_${ARCH}_minimal.tar.bz2"ln -s "$ARCHIVE" "$ASSET"-NOTES="CEF ${CEF_VERSION} for ${ARCH}, built by scripts/build-cef.sh with proprietary codecs (H.264/AAC) and scripts/cef-patches. Downloaded by scripts/fetch-cef.sh; not a Cra release."+NOTES="CEF ${CEF_VERSION} for macOS, one asset per CPU (macosarm64, macosx64), built by scripts/build-cef.sh with proprietary codecs (H.264/AAC) and scripts/cef-patches. Downloaded by scripts/fetch-cef.sh; not a Cra release."if gh release view "$CEF_RELEASE" >/dev/null 2>&1; thengh release upload "$CEF_RELEASE" "$ASSET" --clobberelsediff --git a/scripts/rebuild-cef.sh b/scripts/rebuild-cef.shnew file mode 100755index 0000000..fc707a4--- /dev/null+++ b/scripts/rebuild-cef.sh@@ -0,0 +1,117 @@+#!/bin/bash+# Rebuilds Cra's CEF from the checkout scripts/build-cef.sh left, for one or both CPUs.+# scripts/rebuild-cef.sh arm64 and x64+# scripts/rebuild-cef.sh x64 one CPU+# Then publish each: scripts/publish-cef.sh (arm64), CEF_ARCH=macosx64 scripts/publish-cef.sh.+#+# For a patch round (a new or changed file in scripts/cef-patches) and for adding the+# second CPU (update-cef.sh runs it for x64). It skips automate-git and CEF's gclient+# hook: every patcher run puts components_gcm_disable.patch back over gcm-enabled.patch+# and compiles Web Push out. A new CEF version goes through scripts/update-cef.sh.+#+# Both CPUs share the ~42 GB of sources; each has its own out/Release_GN_<cpu> (~12 GB).+# The first x64 build compiles everything (hours); later rounds take minutes per CPU.+set -euo pipefail+cd "$(dirname "$0")/.."++CEF_BUILD_DIR="${CEF_BUILD_DIR:-$HOME/cef-build.noindex}"+SRC="$CEF_BUILD_DIR/chromium/src"+PATCHES="$PWD/scripts/cef-patches"+CPUS=("$@")+[ ${#CPUS[@]} -gt 0 ] || CPUS=(arm64 x64)+for cpu in "${CPUS[@]}"; do+ [[ "$cpu" =~ ^(arm64|x64)$ ]] || { echo "usage: scripts/rebuild-cef.sh [arm64] [x64]" >&2; exit 1; }+done++eval "$(grep -E '^CEF_VERSION=' scripts/fetch-cef.sh)"+eval "$(grep -E '^export GN_DEFINES=' scripts/build-cef.sh)"+CEF_COMMIT="$(sed -E 's/.*\+g([0-9a-f]+)\+.*/\1/' <<<"$CEF_VERSION")"+CHROMIUM="${CEF_VERSION##*chromium-}"++# The checkout must be the pinned version, built once by build-cef.sh.+[ -d "$SRC/cef" ] || { echo "No CEF checkout in $CEF_BUILD_DIR; run scripts/build-cef.sh" >&2; exit 1; }+HAVE_CHROMIUM="$(. "$SRC/chrome/VERSION" && echo "$MAJOR.$MINOR.$BUILD.$PATCH")"+HAVE_CEF="$(git -C "$SRC/cef" rev-parse HEAD)"+if [ "$HAVE_CHROMIUM" != "$CHROMIUM" ] || [[ "$HAVE_CEF" != "$CEF_COMMIT"* ]]; then+ echo "The checkout is Chromium $HAVE_CHROMIUM, CEF ${HAVE_CEF:0:7}; fetch-cef.sh pins $CEF_VERSION." >&2+ echo "Move it with scripts/update-cef.sh." >&2+ exit 1+fi+# One build at a time: both CPUs share the sources and every core.+if pgrep -fq 'siso ninja|autoninja'; then+ echo "Another Chromium build is running; wait for it." >&2+ exit 1+fi++export PATH="$CEF_BUILD_DIR/depot_tools:$PATH" # make_distrib needs its clang-format+export DEPOT_TOOLS_UPDATE=0+export CEF_ARCHIVE_FORMAT=tar.bz2++# Cra's patches: new ones go on, applied ones stay. A changed patch applies neither way:+# reverse its old version by hand (git -C chromium/src apply -R) and run again.+for p in "$PATCHES"/*.patch; do+ if git -C "$SRC" apply --check --reverse "$p" 2>/dev/null; then+ continue+ elif git -C "$SRC" apply --check "$p" 2>/dev/null; then+ git -C "$SRC" apply "$p" && echo "Applied $(basename "$p")"+ else+ echo "$(basename "$p") does not apply; reverse its old version first" >&2+ exit 1+ fi+done++for cpu in "${CPUS[@]}"; do+ OUT="out/Release_GN_$cpu"+ if [ ! -f "$SRC/$OUT/build.ninja" ]; then+ FREE_GB=$(( $(df -k "$CEF_BUILD_DIR" | awk 'NR==2 {print $4}') / 1024 / 1024 ))+ [ "$FREE_GB" -ge 25 ] || { echo "A first $cpu build needs ~15 GB plus packaging; $FREE_GB GB free" >&2; exit 1; }+ # The args.gn CEF's gclient_hook.py would write for this CPU (it differs from the+ # other CPU's only in target_cpu), without running the hook's patcher.+ echo "Configuring $OUT"+ mkdir -p "$SRC/$OUT"+ (cd "$SRC/cef/tools" && CEF_ENABLE_ARM64=1 CEF_ENABLE_AMD64=1 python3 -c "+import sys, gn_args+print(gn_args.GetConfigFileContents(gn_args.GetAllPlatformConfigs({}, quiet=True)[sys.argv[1]]), end='')+" "Release_GN_$cpu") > "$SRC/$OUT/args.gn"+ (cd "$SRC" && gn gen "$OUT")+ fi++ echo "Building CEF $CEF_VERSION for $cpu"+ START=$SECONDS+ # make_distrib copies the framework out of cefclient.app.+ (cd "$SRC" && autoninja -C "$OUT" cefsimple cefclient)++ PLATFORM="$([ "$cpu" = x64 ] && echo macosx64 || echo macosarm64)"+ NAME="cef_binary_${CEF_VERSION}_${PLATFORM}_minimal"+ rm -rf "$SRC/cef/binary_distrib/$NAME" "$SRC/cef/binary_distrib/$NAME".*+ # automate-git does not pass --no-symbols on for branches after 7151, and without+ # dSYMs (enable_dsyms=false) make_distrib then fails.+ (cd "$SRC/cef/tools" && python3 make_distrib.py --output-dir=../binary_distrib --allow-partial \+ --ninja-build "--$cpu-build" --minimal --no-docs --no-symbols)++ ARCHIVE="$SRC/cef/binary_distrib/$NAME.tar.bz2"+ FRAMEWORK="$SRC/cef/binary_distrib/$NAME/Release/Chromium Embedded Framework.framework/Chromium Embedded Framework"+ [ -f "$ARCHIVE" ] || { echo "make_distrib wrote no $ARCHIVE" >&2; exit 1; }+ WANT_ARCH="$([ "$cpu" = x64 ] && echo x86_64 || echo arm64)"+ [ "$(lipo -archs "$FRAMEWORK")" = "$WANT_ARCH" ] || { echo "Framework is $(lipo -archs "$FRAMEWORK"), not $WANT_ARCH" >&2; exit 1; }+ LC_ALL=C grep -qa "Cra Safe Storage" "$FRAMEWORK" || { echo "Framework names no Cra Safe Storage" >&2; exit 1; }+ LC_ALL=C grep -qa "GCM.RegistrationCacheStatus" "$FRAMEWORK" || { echo "GCM is compiled out (Web Push)" >&2; exit 1; }+ # Both CPUs come from the same patched sources, so the Cra.* page methods the patches+ # add must match the other CPU's package when there is one. Only NUL-terminated+ # strings count: a literal of 8 bytes or less, like Cra.load, compiles to an immediate,+ # which arm64 splits up but x86_64's movabs keeps whole ("Cra.loadH" in a plain grep).+ OTHER="$([ "$cpu" = x64 ] && echo macosarm64 || echo macosx64)"+ OTHER_FW="$SRC/cef/binary_distrib/cef_binary_${CEF_VERSION}_${OTHER}_minimal/Release/Chromium Embedded Framework.framework/Chromium Embedded Framework"+ if [ -f "$OTHER_FW" ]; then+ markers() { perl -0777 -ne 'my %s; $s{$1} = 1 while /(Cra\.[A-Za-z]{2,})\0/g; print join(" ", sort keys %s)' "$1"; }+ MINE="$(markers "$FRAMEWORK")" THEIRS="$(markers "$OTHER_FW")"+ if [ "$MINE" != "$THEIRS" ]; then+ echo "Cra.* methods differ from $OTHER's package:" >&2+ echo " $PLATFORM: $MINE" >&2+ echo " $OTHER: $THEIRS" >&2+ exit 1+ fi+ echo "Same Cra.* methods as $OTHER: $MINE"+ fi+ echo "CEF for $cpu in $(( (SECONDS - START) / 60 )) min: $ARCHIVE"+donediff --git a/scripts/update-cef.sh b/scripts/update-cef.shindex 800e78f..30a3f67 100755--- a/scripts/update-cef.sh+++ b/scripts/update-cef.sh@@ -4,7 +4,9 @@# The version comes from the "Chromium engine update" issue that the CEF updates# workflow opens (scripts/check-cef-update.sh). Runs for hours on this Mac; ~80 GB.# fetch-cef.sh is only switched once the build is published, so other builds keep-# working meanwhile. Afterwards: test build/Cra.app, commit, tag a release.+# working meanwhile. The Intel build follows the arm64 one from the same checkout (about+# as long again), so the pin only moves once both are published. Afterwards: test+# build/Cra.app, commit, tag a release.set -euo pipefailcd "$(dirname "$0")/.."@@ -15,6 +17,8 @@ export CEF_VERSION="$NEW"export CEF_RELEASE="cef-${NEW%%+*}-codecs1"scripts/build-cef.shscripts/publish-cef.sh+scripts/rebuild-cef.sh x64+CEF_ARCH=macosx64 scripts/publish-cef.shunset CEF_VERSION CEF_RELEASE# Pin it; a new CEF version starts again at codecs1.