Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

ci: a push checks that packages work; the night checks that they match the editor

esengine91f00dcc5dad
diff --git a/.github/workflows/README.md b/.github/workflows/README.md
index 96045f545..1c767f941 100644
--- a/.github/workflows/README.md
+++ b/.github/workflows/README.md
@@ -8,9 +8,10 @@ checkout.
## The editor submodule (`desktop/`)
The editor is a **private** repository mounted at `desktop/`, so the workflow
-token cannot read it. Three jobs need it — **Editor authoring checks**,
-**Packages (golden + released)** and the release workflow's **electron-builder**
-job — and they fetch it through `./.github/actions/editor-checkout`, which uses a
+token cannot read it. The jobs that drive an editor need it — **Editor authoring
+checks**, **Golden packages**, **Packages (streamed, compiled, released)** and the
+release workflow's **electron-builder** job — and they fetch it through
+`./.github/actions/editor-checkout`, which uses a
**read-only deploy key**:
- public half: a deploy key on `esengine/estella-editor` titled `estella CI (read-only)`
@@ -20,7 +21,7 @@ A deploy key rather than a PAT because it is scoped to that one repository and
cannot write: if it leaks, it reads one private repo and nothing else. It is also
the only option `gh` can issue end to end — GitHub has no API for minting a PAT.
-Without the secret those three jobs **fail at checkout**, deliberately. They
+Without the secret those jobs **fail at checkout**, deliberately. They
drive an editor; reporting green with no editor present would be a lie about what
was checked. Every other job runs fine without it — see `--host engine` in
`tools/verify-render.mjs` and `needs: 'editor'` in `tools/gates.mjs`.
@@ -87,8 +88,10 @@ The Emscripten jobs use [ccache](https://ccache.dev/) to speed up C++ compilatio
- **Action**: `hendrikmuhs/[email protected]`
- **Cache key**: `${{ runner.os }}-emscripten` — shared by `build.yml`'s
`build-emscripten` job and `release-desktop.yml`'s `engine` job, so every
- master push keeps the release engine build's cache warm.
-- **Max cache size**: 500MB
+ master push keeps the release engine build's cache warm. The C++ test jobs keep
+ keys of their own (`emscripten-tests`, `emscripten-asan`): two jobs saving one
+ key overwrite each other.
+- **Max cache size**: 2G for the shared key
### Emscripten-specific Settings
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 5a444413d..75aa069e6 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -36,8 +36,10 @@ env:
test_frame_lifecycle
jobs:
+ # The engine binaries, and nothing else: every gate below waits on this job, so
+ # a step here that no downstream job reads is a delay on all of them.
build-emscripten:
- name: Web (Emscripten) + C++ tests
+ name: Web (Emscripten)
runs-on: ubuntu-latest
timeout-minutes: 40
@@ -79,21 +81,6 @@ jobs:
test -f build/wasm/wechat/esengine.wxgame.js && echo "✓ WeChat JS"
test -f build/wasm/wechat/esengine.wxgame.wasm && echo "✓ WeChat WASM"
- - name: Build C++ test harnesses
- run: |
- emcmake cmake -S . -B build/wasm/web-tests -DCMAKE_BUILD_TYPE=Release -DES_BUILD_WEB=ON -DES_BUILD_TESTS=ON -DES_ENABLE_BOX2D=ON
- # webgpu_engine_bringup is compiled (a link-time smoke) but not run below.
- cmake --build build/wasm/web-tests -j "$(nproc)" --target $CPP_TESTS webgpu_engine_bringup
-
- # The root package.json is ESM, so the emscripten-emitted .js must run as .cjs.
- - name: Run C++ test harnesses
- run: |
- for t in $CPP_TESTS; do
- cp "build/wasm/web-tests/bin/$t.js" "build/wasm/web-tests/bin/$t.cjs"
- echo "=== $t"
- node "build/wasm/web-tests/bin/$t.cjs"
- done
-
# No --continue-on-error: a side-module compile break must FAIL the gate.
# It used to be swallowed here, so a physics-module error lived on master
# for days and only surfaced when a release ran `build -t all` for real.
@@ -117,6 +104,47 @@ jobs:
path: build/wasm/wechat/
retention-days: 3
+ # The C++ harnesses as the release build compiles them, on a runner of their own
+ # so the jobs waiting on the engine binaries are not also waiting on these.
+ cpp-tests:
+ name: C++ tests
+ runs-on: ubuntu-latest
+ timeout-minutes: 40
+
+ steps:
+ - uses: actions/checkout@v7
+
+ - uses: ./.github/actions/engine-submodules
+ - name: Setup Emscripten
+ uses: mymindstorm/setup-emsdk@v16
+ with:
+ version: 5.0.0
+ actions-cache-folder: .emsdk-ci
+ cache-key: emsdk-5.0.0-${{ runner.os }}-${{ runner.arch }}
+
+ # A key of its own: two jobs saving under one key overwrite each other, and
+ # each run restores the other job's objects and misses on its own.
+ - name: Setup ccache
+ uses: hendrikmuhs/[email protected]
+ with:
+ key: emscripten-tests
+ max-size: 1G
+
+ - name: Build C++ test harnesses
+ run: |
+ emcmake cmake -S . -B build/wasm/web-tests -DCMAKE_BUILD_TYPE=Release -DES_BUILD_WEB=ON -DES_BUILD_TESTS=ON -DES_ENABLE_BOX2D=ON
+ # webgpu_engine_bringup is compiled (a link-time smoke) but not run below.
+ cmake --build build/wasm/web-tests -j "$(nproc)" --target $CPP_TESTS webgpu_engine_bringup
+
+ # The root package.json is ESM, so the emscripten-emitted .js must run as .cjs.
+ - name: Run C++ test harnesses
+ run: |
+ for t in $CPP_TESTS; do
+ cp "build/wasm/web-tests/bin/$t.js" "build/wasm/web-tests/bin/$t.cjs"
+ echo "=== $t"
+ node "build/wasm/web-tests/bin/$t.cjs"
+ done
+
# The same harnesses, instrumented. An out-of-bounds read that lands in
# allocation slack (or in the next allocation) is silent in the plain build and
# is exactly what these gates cannot otherwise see. Findings are fatal —
@@ -656,25 +684,34 @@ jobs:
xvfb-run -a --server-args="-screen 0 1920x1080x24" node scripts/editor-mcp-e2e.mjs
xvfb-run -a --server-args="-screen 0 1920x1080x24" node scripts/editor-mcp-editor-e2e.mjs
- verify-packages:
- name: Packages (golden + released)
+ # The golden corpus, packaged and opened the way a player opens it: every
+ # project exports, boots, draws what its points say, and answers the input it
+ # declares. Exporting was checked and the engine was checked; the package never
+ # was — and "it launched" is not the claim, since a build that boots to its
+ # rotate-to-portrait gate launches fine.
+ #
+ # Not held against the editor's frame here. That comparison opens a whole editor
+ # per project on a software rasteriser: it was most of this job's time and every
+ # one of its retries, and it answers drift between two runtimes rather than a
+ # package that broke. The nightly exit criteria make it, with the streamed world,
+ # the released projects and the compiled-versus-interpreted frames.
+ #
+ # A runner per shard, split by the script by what each project asks for;
+ # `fail-fast: false` so every shard answers.
+ verify-golden:
+ name: Golden packages
runs-on: ubuntu-latest
needs: build-emscripten
- # Sized from measurements now, not from room to find out: golden is 14m37s and
- # the streamed world 20m, which at 45 killed the job two minutes AFTER residency
- # had passed 51/51 — and a job killed at its ceiling reports cancelled, not the
- # green step it was standing on. The four steps after it have still never run.
- timeout-minutes: 90
+ strategy:
+ fail-fast: false
+ matrix:
+ shard: [1, 2, 3]
+ timeout-minutes: 30
steps:
- uses: actions/checkout@v7
- with:
- # verify-legacy reads examples out of RELEASE TAGS (git archive v0.20.0
- # …), which a shallow checkout has none of — the step failed on every run
- # that ever reached it.
- fetch-depth: 0
- # The comparison frame is the EDITOR's, so this job needs its checkout.
+ # The package takes the engine the editor host stages.
- uses: ./.github/actions/editor-checkout
with:
ssh-key: ${{ secrets.EDITOR_SSH_KEY }}
@@ -687,71 +724,45 @@ jobs:
# AOT step — and a promise the build cannot keep fails the export rather
# than falling back. This job packages, so this job needs the toolchain.
- name: Setup Emscripten
+ id: toolchain
uses: mymindstorm/setup-emsdk@v16
with:
version: 5.0.0
actions-cache-folder: .emsdk-ci
cache-key: emsdk-5.0.0-${{ runner.os }}-${{ runner.arch }}
- # The golden corpus, packaged and opened the way a player opens it, then held
- # against the editor's own frame of the same game. Exporting was checked and
- # the engine was checked; the package never was — and "it launched" is not the
- # claim, since a build that boots to its rotate-to-portrait gate launches fine.
- # `--jobs`, which this call never passed: the runner has four cores and the
- # projects were packaged and launched one at a time, so the step grew past the
- # job's own cap and the whole matrix was killed for a queue, not for work.
- # The splitter is the script's own and balances by what a project asks for.
- - name: Launch the golden packages and compare them to the editor (PR tier)
- run: node tools/verify-golden.mjs --tier pr --jobs 3
-
- # A parity failure is a claim about two PICTURES, and until now neither left
- # the runner: "the package is not showing what the editor showed" named two
- # paths that were deleted with the machine. A pixel gate that can only fail
- # here has to hand over what it compared, or nobody can tell a real
- # regression from a frame the software rasteriser had not finished.
- - name: Keep the frames a parity failure compared
- if: failure()
- uses: actions/upload-artifact@v7
- with:
- name: golden-parity-frames
- path: |
- .golden/*.png
- .golden/*.log
- include-hidden-files: true
- if-no-files-found: warn
- retention-days: 7
-
- # Which places EXIST in that package, and what leaves with them. A camera
- # cannot answer this: a cell that is not drawn and a cell that is not there
- # look the same, and a body an unload forgot is invisible until someone
- # walks into it.
- - name: Drive a streamed world in its package
- run: node tools/verify-world-residency.mjs
-
- # The same game packaged twice from one tree, compiled and interpreted, and
- # the two frames held against each other — the only check that sees what a
- # compiled system does to a PIXEL.
- - name: Compare a compiled game's frame to an interpreted one (PR tier)
- run: node tools/verify-aot-parity.mjs
+ - name: Launch the golden packages (PR tier)
+ run: node tools/verify-golden.mjs --tier pr --no-parity --shard ${{ matrix.shard }}/3
# Does a real exported WEB package DISPATCH to its compiled systems — the
# claim verify-aot-native makes for the desktop road, which the web road had
# nobody making. A twin that loads and is never called changes no pixel and
- # raises no error, so parity below cannot see it either.
+ # raises no error. Work, not reporting: one shard does it for the matrix.
- name: Check an exported web game runs its systems compiled
+ if: ${{ !cancelled() && matrix.shard == 1 && steps.toolchain.outcome == 'success' }}
run: pnpm run verify:aot
# And the exact half: one fixture packaged both ways, stepped by a fixed dt
# rather than by elapsed time, and the two displacements compared. This is
# the check a twin reading the wrong offset cannot survive.
- name: Check a cooked build's compiled systems against the interpreter
+ if: ${{ !cancelled() && matrix.shard == 1 && steps.toolchain.outcome == 'success' }}
run: pnpm run verify:cooked
- # Projects as they were RELEASED, opened by the editor of today. The golden
- # corpus is re-saved by whoever last touched it, so it can only say that this
- # version opens its own files; a person's project is not re-saved by anybody.
- - name: Open released projects (PR tier)
- run: node tools/verify-legacy.mjs --tier pr
+ # A package whose points missed is a claim about a PICTURE, and the picture
+ # has to leave the runner, or nobody can tell a real regression from a frame
+ # the software rasteriser had not finished.
+ - name: Keep the frames a failure was judged on
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: golden-frames-${{ matrix.shard }}
+ path: |
+ .golden/*.png
+ .golden/*.log
+ include-hidden-files: true
+ if-no-files-found: warn
+ retention-days: 7
# The same scenes on the second backend, on a runner of its own.
#
diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
index 44d9a0154..7f88255c4 100644
--- a/.github/workflows/nightly.yml
+++ b/.github/workflows/nightly.yml
@@ -166,8 +166,11 @@ jobs:
criteria: --only performance-ceilings
- shard: static
criteria: --only static-gates
+ # Four launches of a streamed world, which alone has taken twenty minutes.
+ - shard: streamed
+ criteria: --only streamed-world-comes-and-goes
- shard: rest
- criteria: --except flagship-plays-through,golden-packages,performance-ceilings,static-gates
+ criteria: --except flagship-plays-through,golden-packages,performance-ceilings,static-gates,streamed-world-comes-and-goes
env:
ESTELLA_NATIVE_TEMPLATES: ${{ github.workspace }}/.native-templates
# The longest shard is golden at about a quarter of an hour.
@@ -322,3 +325,45 @@ jobs:
arch: x86_64
disable-animations: true
script: node tools/run-release-gate.mjs --host android
+
+ # A red night that nobody reads is a night that did not run: this workflow was
+ # red five mornings in a row, from three different causes, with nothing saying
+ # so. One issue carries the state — opened or commented on when a scheduled night
+ # is red, and closed when one comes back green.
+ report:
+ name: Say how the night went
+ needs: [build-emscripten, linux-template, linux, macos, android]
+ if: ${{ always() && github.event_name == 'schedule' }}
+ runs-on: ubuntu-latest
+ permissions:
+ actions: read
+ issues: write
+ steps:
+ - name: Open, update or close the nightly issue
+ env:
+ GH_TOKEN: ${{ github.token }}
+ RESULTS: ${{ join(needs.*.result, ' ') }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ run: |
+ set -euo pipefail
+ title="Nightly exit criteria are red"
+ open=$(gh issue list -R "$GITHUB_REPOSITORY" --state open --search "\"$title\" in:title" \
+ --json number,title --jq "map(select(.title == \"$title\")) | .[0].number // empty")
+ case " $RESULTS " in
+ *" failure "*|*" cancelled "*)
+ failed=$(gh run view "$GITHUB_RUN_ID" -R "$GITHUB_REPOSITORY" --json jobs \
+ --jq '.jobs[] | select(.conclusion == "failure" or .conclusion == "cancelled") | "- \(.name) — \(.conclusion)"')
+ body=$(printf 'Night of %s (%s): %s\n\n%s\n' "$(date -u +%F)" "${GITHUB_SHA:0:9}" "$RUN_URL" "$failed")
+ if [ -n "$open" ]; then
+ gh issue comment "$open" -R "$GITHUB_REPOSITORY" --body "$body"
+ else
+ gh issue create -R "$GITHUB_REPOSITORY" --title "$title" --body "$body"
+ fi
+ ;;
+ *)
+ if [ -n "$open" ]; then
+ gh issue close "$open" -R "$GITHUB_REPOSITORY" \
+ --comment "Green again on $(date -u +%F) (${GITHUB_SHA:0:9}): $RUN_URL"
+ fi
+ ;;
+ esac
diff --git a/tools/releaseGate.mjs b/tools/releaseGate.mjs
index 4781c1721..3665d37f6 100644
--- a/tools/releaseGate.mjs
+++ b/tools/releaseGate.mjs
@@ -207,6 +207,12 @@ export const CRITERIA = [
answeredBy: 'node tools/verify-bootlog-crash.mjs --require-device',
needs: ['tools/verify-bootlog-crash.mjs', 'native/host/BootLog.cpp'],
},
+ {
+ id: 'streamed-world-comes-and-goes',
+ says: 'in a packaged streamed world a place exists while someone is near it, and leaving takes everything it brought',
+ answeredBy: 'node tools/verify-world-residency.mjs',
+ needs: ['tools/verify-world-residency.mjs', 'examples/world-streaming-3d/project.esproject'],
+ },
{
id: 'old-projects-open',
says: 'projects released by older versions still open, with nothing dropped',