Skip to content

armgit / cra

Public Imported from GitHub
0

The open-source Arc replacement for macOS. Spaces, pinned tabs, split view, Little Windows and Arc's shortcuts on Chromium, with your Arc sidebar imported in one go.

Intel CEF: cross-compile x64 on the arm64 Mac, rebuild-cef.sh for patch rounds

Victor6d8b2f63fdd1
diff --git a/docs/release.md b/docs/release.md
index b9706ce..737bf18 100644
--- a/docs/release.md
+++ b/docs/release.md
@@ -237,6 +237,16 @@ from a different source. To change the build without a CEF version bump, bump th
`codecsN` suffix of `CEF_RELEASE` and publish again. Needs Xcode's Metal toolchain
(`xcodebuild -downloadComponent MetalToolchain`).
+**Patch rounds and Intel.** Once `build-cef.sh` has built a version, a new or changed
+patch goes through `rebuild-cef.sh`, which builds from that checkout for both CPUs
+(minutes each) and never reruns CEF's patcher (rerunning it compiles Web Push out). The
+Intel (`macosx64`) build is cross-compiled on the same Mac from the same sources, in its
+own `out/Release_GN_x64` (~12 GB; the first one compiles everything):
+
+ scripts/rebuild-cef.sh [arm64] [x64] # both when no CPU is given
+ scripts/publish-cef.sh # arm64 asset
+ CEF_ARCH=macosx64 scripts/publish-cef.sh # Intel asset, same release
+
**Staying current.** Chromium only ships security fixes for its newest version. The
"CEF updates" workflow (`.github/workflows/cef-updates.yml`, daily, free Linux runner)
runs `scripts/check-cef-update.sh`: when Spotify's index lists a newer stable CEF for
@@ -244,7 +254,7 @@ macOS arm64, it opens one "Chromium engine update available" issue, assigned to
owner, with the exact command. The build itself runs on the Mac, because GitHub's free
macOS runners are too small (3 cores, 14 GB disk, 6 hour jobs) and larger ones are billed:
- scripts/update-cef.sh <cef_version> # build, publish, pin, rebuild Cra
+ scripts/update-cef.sh <cef_version> # build + publish arm64 and x64, pin, rebuild Cra
A same-major update reuses the checkout: `build-cef.sh` moves it to the new Chromium tag
itself, and ninja recompiles nearly everything (about 2½ hours on the M2 Max). A new Chromium
@@ -255,5 +265,6 @@ issue and tag a release.
- **Homebrew.** A cask (`brew install --cask cra`) pointing at the DMG and
`SHA256SUMS.txt`.
-- **Intel.** The CEF fetch pins `macosarm64`; a universal build needs both CEF
- distributions and `lipo`.
+- **Intel.** CEF is published for `macosx64` too (above). Still missing: an x86_64
+ `swift build` in `scripts/build.sh`, a separate Intel DMG from the release workflow
+ and its own Sparkle feed.
diff --git a/scripts/build-cef.sh b/scripts/build-cef.sh
index 306c82c..505d2ca 100755
--- a/scripts/build-cef.sh
+++ b/scripts/build-cef.sh
@@ -20,9 +20,16 @@ eval "$(grep -E '^CEF_VERSION=' scripts/fetch-cef.sh)"
# 154.0.34+g14c5a08+chromium-154.0.8037.98: CEF commit 14c5a08 on branch 8037.
CEF_COMMIT="$(sed -E 's/.*\+g([0-9a-f]+)\+.*/\1/' <<<"$CEF_VERSION")"
CEF_BRANCH="$(sed -E 's/.*chromium-[0-9]+\.[0-9]+\.([0-9]+)\..*/\1/' <<<"$CEF_VERSION")"
-# Build for the host CPU by default (automate-git/make_distrib take --arm64-build /
-# --x86-build); CEF_TARGET_ARCH (arm64 / x86) overrides.
-CEF_TARGET_ARCH="${CEF_TARGET_ARCH:-$([ "$(uname -m)" = arm64 ] && echo arm64 || echo x86)}"
+# Builds for the host CPU; CEF_TARGET_ARCH=x64 (or arm64) cross-compiles. automate-git
+# and make_distrib take --arm64-build / --x64-build, and CEF's gn_args.py only writes
+# out/Release_GN_<cpu> for the other CPU when CEF_ENABLE_AMD64 / CEF_ENABLE_ARM64 is set.
+# For a second CPU on a checkout that is already built, scripts/rebuild-cef.sh is faster.
+CEF_TARGET_ARCH="${CEF_TARGET_ARCH:-$([ "$(uname -m)" = arm64 ] && echo arm64 || echo x64)}"
+case "$CEF_TARGET_ARCH" in
+ arm64) export CEF_ENABLE_ARM64=1 ;;
+ x64) export CEF_ENABLE_AMD64=1 ;;
+ *) echo "CEF_TARGET_ARCH must be arm64 or x64, not $CEF_TARGET_ARCH" >&2; exit 1 ;;
+esac
ARCH_FLAG="--${CEF_TARGET_ARCH}-build"
echo "Building CEF $CEF_VERSION (branch $CEF_BRANCH, commit $CEF_COMMIT, $CEF_TARGET_ARCH)"
# .noindex keeps Spotlight out of ~40 GB of sources.
@@ -105,12 +112,13 @@ for p in "$PATCHES"/*.patch; do
fi
done
# 4. Build. make_distrib takes the framework from cefclient.app.
-(cd chromium/src && autoninja -C out/Release_GN_arm64 cefclient)
+(cd chromium/src && autoninja -C "out/Release_GN_$CEF_TARGET_ARCH" cefclient)
LC_ALL=C grep -c -a GCM.RegistrationCacheStatus \
- "chromium/src/out/Release_GN_arm64/cefclient.app/Contents/Frameworks/Chromium Embedded Framework.framework/Chromium Embedded Framework" >/dev/null \
+ "chromium/src/out/Release_GN_$CEF_TARGET_ARCH/cefclient.app/Contents/Frameworks/Chromium Embedded Framework.framework/Chromium Embedded Framework" >/dev/null \
|| { echo "GCM is stubbed out (components_gcm_disable.patch won over gcm-enabled.patch)" >&2; exit 1; }
# 5. Package. automate-git does not pass --no-symbols on for branches after 7151,
# and without dSYMs (enable_dsyms=false) make_distrib then fails.
-rm -rf chromium/src/cef/binary_distrib
+# Only this CPU's package: the other CPU's may not be published yet.
+rm -rf chromium/src/cef/binary_distrib/cef_binary_"${CEF_VERSION}"_macos"$([ "$CEF_TARGET_ARCH" = x64 ] && echo x64 || echo arm64)"_minimal*
(cd chromium/src/cef/tools && python3 make_distrib.py --output-dir=../binary_distrib --allow-partial \
--ninja-build "$ARCH_FLAG" --minimal --no-docs --no-symbols)
diff --git a/scripts/publish-cef.sh b/scripts/publish-cef.sh
index 46beeee..f6f56fe 100755
--- a/scripts/publish-cef.sh
+++ b/scripts/publish-cef.sh
@@ -2,7 +2,8 @@
# Publishes the CEF distribution built by scripts/build-cef.sh as a GitHub release
# asset, where scripts/fetch-cef.sh (and CI) download it from. The release is a
# pre-release so releases/latest keeps pointing at the newest Cra.
-# scripts/publish-cef.sh uses CEF_RELEASE from fetch-cef.sh
+# scripts/publish-cef.sh this Mac's CPU, CEF_RELEASE from fetch-cef.sh
+# CEF_ARCH=macosx64 scripts/publish-cef.sh the Intel build, into the same release
set -euo pipefail
cd "$(dirname "$0")/.."
@@ -13,7 +14,7 @@ ARCHIVE="$CEF_BUILD_DIR/chromium/src/cef/binary_distrib/cef_binary_${CEF_VERSION
ASSET="$(mktemp -d)/cef_binary_${ARCH}_minimal.tar.bz2"
ln -s "$ARCHIVE" "$ASSET"
-NOTES="CEF ${CEF_VERSION} for ${ARCH}, built by scripts/build-cef.sh with proprietary codecs (H.264/AAC) and scripts/cef-patches. Downloaded by scripts/fetch-cef.sh; not a Cra release."
+NOTES="CEF ${CEF_VERSION} for macOS, one asset per CPU (macosarm64, macosx64), built by scripts/build-cef.sh with proprietary codecs (H.264/AAC) and scripts/cef-patches. Downloaded by scripts/fetch-cef.sh; not a Cra release."
if gh release view "$CEF_RELEASE" >/dev/null 2>&1; then
gh release upload "$CEF_RELEASE" "$ASSET" --clobber
else
diff --git a/scripts/rebuild-cef.sh b/scripts/rebuild-cef.sh
new file mode 100755
index 0000000..fc707a4
--- /dev/null
+++ b/scripts/rebuild-cef.sh
@@ -0,0 +1,117 @@
+#!/bin/bash
+# Rebuilds Cra's CEF from the checkout scripts/build-cef.sh left, for one or both CPUs.
+# scripts/rebuild-cef.sh arm64 and x64
+# scripts/rebuild-cef.sh x64 one CPU
+# Then publish each: scripts/publish-cef.sh (arm64), CEF_ARCH=macosx64 scripts/publish-cef.sh.
+#
+# For a patch round (a new or changed file in scripts/cef-patches) and for adding the
+# second CPU (update-cef.sh runs it for x64). It skips automate-git and CEF's gclient
+# hook: every patcher run puts components_gcm_disable.patch back over gcm-enabled.patch
+# and compiles Web Push out. A new CEF version goes through scripts/update-cef.sh.
+#
+# Both CPUs share the ~42 GB of sources; each has its own out/Release_GN_<cpu> (~12 GB).
+# The first x64 build compiles everything (hours); later rounds take minutes per CPU.
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+CEF_BUILD_DIR="${CEF_BUILD_DIR:-$HOME/cef-build.noindex}"
+SRC="$CEF_BUILD_DIR/chromium/src"
+PATCHES="$PWD/scripts/cef-patches"
+CPUS=("$@")
+[ ${#CPUS[@]} -gt 0 ] || CPUS=(arm64 x64)
+for cpu in "${CPUS[@]}"; do
+ [[ "$cpu" =~ ^(arm64|x64)$ ]] || { echo "usage: scripts/rebuild-cef.sh [arm64] [x64]" >&2; exit 1; }
+done
+
+eval "$(grep -E '^CEF_VERSION=' scripts/fetch-cef.sh)"
+eval "$(grep -E '^export GN_DEFINES=' scripts/build-cef.sh)"
+CEF_COMMIT="$(sed -E 's/.*\+g([0-9a-f]+)\+.*/\1/' <<<"$CEF_VERSION")"
+CHROMIUM="${CEF_VERSION##*chromium-}"
+
+# The checkout must be the pinned version, built once by build-cef.sh.
+[ -d "$SRC/cef" ] || { echo "No CEF checkout in $CEF_BUILD_DIR; run scripts/build-cef.sh" >&2; exit 1; }
+HAVE_CHROMIUM="$(. "$SRC/chrome/VERSION" && echo "$MAJOR.$MINOR.$BUILD.$PATCH")"
+HAVE_CEF="$(git -C "$SRC/cef" rev-parse HEAD)"
+if [ "$HAVE_CHROMIUM" != "$CHROMIUM" ] || [[ "$HAVE_CEF" != "$CEF_COMMIT"* ]]; then
+ echo "The checkout is Chromium $HAVE_CHROMIUM, CEF ${HAVE_CEF:0:7}; fetch-cef.sh pins $CEF_VERSION." >&2
+ echo "Move it with scripts/update-cef.sh." >&2
+ exit 1
+fi
+# One build at a time: both CPUs share the sources and every core.
+if pgrep -fq 'siso ninja|autoninja'; then
+ echo "Another Chromium build is running; wait for it." >&2
+ exit 1
+fi
+
+export PATH="$CEF_BUILD_DIR/depot_tools:$PATH" # make_distrib needs its clang-format
+export DEPOT_TOOLS_UPDATE=0
+export CEF_ARCHIVE_FORMAT=tar.bz2
+
+# Cra's patches: new ones go on, applied ones stay. A changed patch applies neither way:
+# reverse its old version by hand (git -C chromium/src apply -R) and run again.
+for p in "$PATCHES"/*.patch; do
+ if git -C "$SRC" apply --check --reverse "$p" 2>/dev/null; then
+ continue
+ elif git -C "$SRC" apply --check "$p" 2>/dev/null; then
+ git -C "$SRC" apply "$p" && echo "Applied $(basename "$p")"
+ else
+ echo "$(basename "$p") does not apply; reverse its old version first" >&2
+ exit 1
+ fi
+done
+
+for cpu in "${CPUS[@]}"; do
+ OUT="out/Release_GN_$cpu"
+ if [ ! -f "$SRC/$OUT/build.ninja" ]; then
+ FREE_GB=$(( $(df -k "$CEF_BUILD_DIR" | awk 'NR==2 {print $4}') / 1024 / 1024 ))
+ [ "$FREE_GB" -ge 25 ] || { echo "A first $cpu build needs ~15 GB plus packaging; $FREE_GB GB free" >&2; exit 1; }
+ # The args.gn CEF's gclient_hook.py would write for this CPU (it differs from the
+ # other CPU's only in target_cpu), without running the hook's patcher.
+ echo "Configuring $OUT"
+ mkdir -p "$SRC/$OUT"
+ (cd "$SRC/cef/tools" && CEF_ENABLE_ARM64=1 CEF_ENABLE_AMD64=1 python3 -c "
+import sys, gn_args
+print(gn_args.GetConfigFileContents(gn_args.GetAllPlatformConfigs({}, quiet=True)[sys.argv[1]]), end='')
+" "Release_GN_$cpu") > "$SRC/$OUT/args.gn"
+ (cd "$SRC" && gn gen "$OUT")
+ fi
+
+ echo "Building CEF $CEF_VERSION for $cpu"
+ START=$SECONDS
+ # make_distrib copies the framework out of cefclient.app.
+ (cd "$SRC" && autoninja -C "$OUT" cefsimple cefclient)
+
+ PLATFORM="$([ "$cpu" = x64 ] && echo macosx64 || echo macosarm64)"
+ NAME="cef_binary_${CEF_VERSION}_${PLATFORM}_minimal"
+ rm -rf "$SRC/cef/binary_distrib/$NAME" "$SRC/cef/binary_distrib/$NAME".*
+ # automate-git does not pass --no-symbols on for branches after 7151, and without
+ # dSYMs (enable_dsyms=false) make_distrib then fails.
+ (cd "$SRC/cef/tools" && python3 make_distrib.py --output-dir=../binary_distrib --allow-partial \
+ --ninja-build "--$cpu-build" --minimal --no-docs --no-symbols)
+
+ ARCHIVE="$SRC/cef/binary_distrib/$NAME.tar.bz2"
+ FRAMEWORK="$SRC/cef/binary_distrib/$NAME/Release/Chromium Embedded Framework.framework/Chromium Embedded Framework"
+ [ -f "$ARCHIVE" ] || { echo "make_distrib wrote no $ARCHIVE" >&2; exit 1; }
+ WANT_ARCH="$([ "$cpu" = x64 ] && echo x86_64 || echo arm64)"
+ [ "$(lipo -archs "$FRAMEWORK")" = "$WANT_ARCH" ] || { echo "Framework is $(lipo -archs "$FRAMEWORK"), not $WANT_ARCH" >&2; exit 1; }
+ LC_ALL=C grep -qa "Cra Safe Storage" "$FRAMEWORK" || { echo "Framework names no Cra Safe Storage" >&2; exit 1; }
+ LC_ALL=C grep -qa "GCM.RegistrationCacheStatus" "$FRAMEWORK" || { echo "GCM is compiled out (Web Push)" >&2; exit 1; }
+ # Both CPUs come from the same patched sources, so the Cra.* page methods the patches
+ # add must match the other CPU's package when there is one. Only NUL-terminated
+ # strings count: a literal of 8 bytes or less, like Cra.load, compiles to an immediate,
+ # which arm64 splits up but x86_64's movabs keeps whole ("Cra.loadH" in a plain grep).
+ OTHER="$([ "$cpu" = x64 ] && echo macosarm64 || echo macosx64)"
+ OTHER_FW="$SRC/cef/binary_distrib/cef_binary_${CEF_VERSION}_${OTHER}_minimal/Release/Chromium Embedded Framework.framework/Chromium Embedded Framework"
+ if [ -f "$OTHER_FW" ]; then
+ markers() { perl -0777 -ne 'my %s; $s{$1} = 1 while /(Cra\.[A-Za-z]{2,})\0/g; print join(" ", sort keys %s)' "$1"; }
+ MINE="$(markers "$FRAMEWORK")" THEIRS="$(markers "$OTHER_FW")"
+ if [ "$MINE" != "$THEIRS" ]; then
+ echo "Cra.* methods differ from $OTHER's package:" >&2
+ echo " $PLATFORM: $MINE" >&2
+ echo " $OTHER: $THEIRS" >&2
+ exit 1
+ fi
+ echo "Same Cra.* methods as $OTHER: $MINE"
+ fi
+ echo "CEF for $cpu in $(( (SECONDS - START) / 60 )) min: $ARCHIVE"
+done
diff --git a/scripts/update-cef.sh b/scripts/update-cef.sh
index 800e78f..30a3f67 100755
--- a/scripts/update-cef.sh
+++ b/scripts/update-cef.sh
@@ -4,7 +4,9 @@
# The version comes from the "Chromium engine update" issue that the CEF updates
# workflow opens (scripts/check-cef-update.sh). Runs for hours on this Mac; ~80 GB.
# fetch-cef.sh is only switched once the build is published, so other builds keep
-# working meanwhile. Afterwards: test build/Cra.app, commit, tag a release.
+# working meanwhile. The Intel build follows the arm64 one from the same checkout (about
+# as long again), so the pin only moves once both are published. Afterwards: test
+# build/Cra.app, commit, tag a release.
set -euo pipefail
cd "$(dirname "$0")/.."
@@ -15,6 +17,8 @@ export CEF_VERSION="$NEW"
export CEF_RELEASE="cef-${NEW%%+*}-codecs1"
scripts/build-cef.sh
scripts/publish-cef.sh
+scripts/rebuild-cef.sh x64
+CEF_ARCH=macosx64 scripts/publish-cef.sh
unset CEF_VERSION CEF_RELEASE
# Pin it; a new CEF version starts again at codecs1.