Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(release): ask a mirror before the origin, and prove it before trusting it

yhh1f0013f7ecaa
diff --git a/.github/workflows/release-desktop.yml b/.github/workflows/release-desktop.yml
index 61f9618f0..132e86306 100644
--- a/.github/workflows/release-desktop.yml
+++ b/.github/workflows/release-desktop.yml
@@ -273,3 +273,67 @@ jobs:
desktop/release/*.exe
desktop/release/*.dmg
retention-days: 3
+
+ # A copy of the release on Cloudflare R2, because GitHub's download speed is the
+ # editor's download speed for anyone far from it — and a runtime template is
+ # hundreds of megabytes.
+ #
+ # The mirror is a shortcut, never an authority: the editor checks every archive
+ # against the size and digest the index states and falls back to this release when
+ # a copy is missing, stale or wrong. That is why this job can run last, fail
+ # without failing the release, and be added to an account that did not exist when
+ # the version was cut.
+ #
+ # Needs five secrets; without them the job skips and the release is GitHub-only:
+ # R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET, and
+ # R2_PUBLIC_BASE — the public URL the bucket is served on, which is what the
+ # editor is pointed at (DEFAULT_RELEASE_MIRROR / ESTELLA_RELEASE_MIRROR).
+ mirror:
+ name: Mirror the release to Cloudflare R2
+ needs: [publish-templates, release]
+ if: github.event_name == 'push' && !cancelled() && needs.release.result == 'success'
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ continue-on-error: true
+ steps:
+ - name: Check for a configured mirror
+ id: cfg
+ env:
+ KEY: ${{ secrets.R2_ACCESS_KEY_ID }}
+ run: echo "configured=${{ env.KEY != '' }}" >> "$GITHUB_OUTPUT"
+
+ - name: Download every asset the release published
+ if: steps.cfg.outputs.configured == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ mkdir -p mirror
+ gh release download "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" --dir mirror --clobber
+
+ # `latest.json` is what the editor's update check reads: one small file at the
+ # root, so asking "is there a newer version?" costs one request and no API.
+ - name: Write latest.json
+ if: steps.cfg.outputs.configured == 'true'
+ run: |
+ version="${GITHUB_REF_NAME#v}"
+ printf '{\n "version": "%s",\n "url": "%s/v%s/"\n}\n' \
+ "$version" "${{ secrets.R2_PUBLIC_BASE }}" "$version" > latest.json
+ cat latest.json
+
+ # The same layout the origin uses — v<version>/ holding that release's assets —
+ # so the editor swaps a base and changes nothing else.
+ - name: Sync to R2
+ if: steps.cfg.outputs.configured == 'true'
+ env:
+ AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
+ AWS_DEFAULT_REGION: auto
+ ENDPOINT: https://${{ secrets.R2_ACCOUNT_ID }}.r2.cloudflarestorage.com
+ BUCKET: ${{ secrets.R2_BUCKET }}
+ run: |
+ version="${GITHUB_REF_NAME#v}"
+ aws s3 sync mirror "s3://$BUCKET/v$version/" --endpoint-url "$ENDPOINT" --no-progress
+ # Written last: an editor that reads latest.json must never be pointed at a
+ # version whose assets are still uploading.
+ aws s3 cp latest.json "s3://$BUCKET/latest.json" \
+ --endpoint-url "$ENDPOINT" --cache-control "max-age=300"
diff --git a/build-tools/utils/nativeTemplate.d.ts b/build-tools/utils/nativeTemplate.d.ts
index b6ce17ca2..219de960c 100644
--- a/build-tools/utils/nativeTemplate.d.ts
+++ b/build-tools/utils/nativeTemplate.d.ts
@@ -74,7 +74,17 @@ export interface PublishedTemplate {
}
export const TEMPLATE_INDEX: string;
+export const RELEASE_MIRROR_ENV: string;
+export const DEFAULT_RELEASE_MIRROR: string;
+export const RELEASE_ORIGIN: string;
+
+/** Mirrors to try before the origin, fastest-first. */
+export function releaseMirrors(env?: NodeJS.ProcessEnv): string[];
+
export function releaseAssetBase(engineVersion: string): string;
+
+/** Every base to try for one version: the mirrors, then the origin. */
+export function releaseAssetBases(engineVersion: string, env?: NodeJS.ProcessEnv): string[];
export function parseTemplateIndex(doc: unknown, engineVersion?: string): PublishedTemplate[] | null;
export const DEFAULT_ICON: string;
/** The pieces an exported Android Studio project is assembled from. */
diff --git a/build-tools/utils/nativeTemplate.js b/build-tools/utils/nativeTemplate.js
index 5b4a05173..c14ce95e2 100644
--- a/build-tools/utils/nativeTemplate.js
+++ b/build-tools/utils/nativeTemplate.js
@@ -238,13 +238,47 @@ export function templateZipName(platform, engineVersion) {
* version, and what each one must hash to. */
export const TEMPLATE_INDEX = 'native-templates.json';
+/**
+ * The mirror a release is copied to, and the environment variable that overrides
+ * it. Empty until one is configured — a build with no mirror simply has one source.
+ *
+ * A mirror is a convenience, never an authority: every archive is checked against
+ * the size and SHA-256 the index states, so the worst a wrong or stale mirror can
+ * do is fail and hand the download back to the origin.
+ */
+export const RELEASE_MIRROR_ENV = 'ESTELLA_RELEASE_MIRROR';
+export const DEFAULT_RELEASE_MIRROR = '';
+
+/** The release origin: where a version is published, and the last word on it. */
+export const RELEASE_ORIGIN = 'https://github.com/esengine/estella/releases/download';
+
+/** Mirrors to try before the origin, fastest-first. `,`-separated in the env. */
+export function releaseMirrors(env = process.env) {
+ const configured = env[RELEASE_MIRROR_ENV] ?? DEFAULT_RELEASE_MIRROR;
+ return configured.split(',').map((s) => s.trim().replace(/\/+$/, '')).filter(Boolean);
+}
+
/**
* Where a release's assets live. Composed rather than stored in the index, so the
* same index works from a mirror, a local directory or a company file share — the
* editor is told a base, not a set of absolute URLs.
*/
export function releaseAssetBase(engineVersion) {
- return `https://github.com/esengine/estella/releases/download/v${engineVersion}`;
+ return `${RELEASE_ORIGIN}/v${engineVersion}`;
+}
+
+/**
+ * Every base to try for one version, in order: the mirrors, then the origin.
+ *
+ * One list, so the download path cannot prefer a mirror the update check has never
+ * heard of. A mirror lays its versions out exactly as the origin does — `v<version>`
+ * holding that release's assets — so a base swap is the whole difference.
+ */
+export function releaseAssetBases(engineVersion, env = process.env) {
+ return [
+ ...releaseMirrors(env).map((base) => `${base}/v${engineVersion}`),
+ releaseAssetBase(engineVersion),
+ ];
}
/**
diff --git a/desktop/electron/nativeTemplates.ts b/desktop/electron/nativeTemplates.ts
index 7924e2503..17b3dd0d7 100644
--- a/desktop/electron/nativeTemplates.ts
+++ b/desktop/electron/nativeTemplates.ts
@@ -21,7 +21,7 @@ import { once } from 'node:events';
import { randomUUID, createHash } from 'node:crypto';
import {
findTemplate, readTemplateManifest, missingTemplateFiles, installedTemplateDir,
- templateStoreDir, iosTemplateSources, releaseAssetBase, parseTemplateIndex,
+ templateStoreDir, iosTemplateSources, releaseAssetBases, parseTemplateIndex,
TEMPLATE_INDEX,
type NativePlatform, type NativeTemplateManifest,
} from '../../build-tools/utils/nativeTemplate.js';
@@ -174,7 +174,28 @@ export async function downloadNativeTemplate(
} = {},
): Promise<InstallResult> {
const fetchImpl = options.fetchImpl ?? fetch;
- const base = options.baseUrl ?? releaseAssetBase(engineVersion);
+ const bases = options.baseUrl ? [options.baseUrl] : releaseAssetBases(engineVersion);
+
+ // Mirrors first, the origin last. A mirror that is stale, truncated, wrong or
+ // simply down fails a check the archive has to pass anyway, and the next source
+ // is tried — so a fast copy is a shortcut and never a risk.
+ let lastError = `no source for v${engineVersion}`;
+ for (const base of bases) {
+ const attempt = await downloadFrom(base, platform, engineVersion, fetchImpl, options.onProgress);
+ if (attempt.ok) return attempt;
+ lastError = attempt.error ?? `download failed from ${base}`;
+ if (bases.length > 1) console.warn(`[templates] ${base}: ${lastError}`);
+ }
+ return { ok: false, error: lastError };
+}
+
+async function downloadFrom(
+ base: string,
+ platform: NativePlatform,
+ engineVersion: string,
+ fetchImpl: typeof fetch,
+ onProgress?: (progress: TemplateDownloadProgress) => void,
+): Promise<InstallResult> {
const staging = path.join(templateStoreDir(), `.download-${randomUUID()}`);
try {
@@ -199,7 +220,7 @@ export async function downloadNativeTemplate(
for await (const chunk of res.body as AsyncIterable<Uint8Array>) {
digest.update(chunk);
received += chunk.byteLength;
- options.onProgress?.({ received, total: entry.bytes });
+ onProgress?.({ received, total: entry.bytes });
if (!sink.write(chunk)) await once(sink, 'drain');
}
await new Promise<void>((resolve, reject) => sink.end((err?: Error) => (err ? reject(err) : resolve())));
diff --git a/desktop/electron/updateCheck.ts b/desktop/electron/updateCheck.ts
index 60cdc07b9..52db4c0fc 100644
--- a/desktop/electron/updateCheck.ts
+++ b/desktop/electron/updateCheck.ts
@@ -8,10 +8,27 @@
* the running version to the latest published release and hand the
* renderer a link. Pure logic — `fetch` is injected so tests need no
* network and main needs no mock.
+ *
+ * Asked of the mirror first, when one is configured: the same copy the
+ * runtime templates come from, so an editor that downloads fast updates
+ * fast. The origin answers when there is no mirror, or when its answer is
+ * missing, malformed or older than what GitHub publishes.
*/
+import { releaseMirrors } from '../../build-tools/utils/nativeTemplate.js';
+
const RELEASES_LATEST = 'https://api.github.com/repos/esengine/estella/releases/latest';
+/**
+ * What a mirror publishes at its root so an editor can ask "what is current?"
+ * without the origin's API: the version, and where its download page or installer
+ * is. Written by the release workflow's mirror step beside the version folders.
+ */
+interface MirrorLatest {
+ version?: string;
+ url?: string;
+}
+
export interface LatestRelease {
/** Normalized version, no leading `v` (e.g. "0.18.0"). */
version: string;
@@ -48,6 +65,40 @@ export function isNewerVersion(candidate: string, current: string): boolean {
export async function checkForUpdate(
currentVersion: string,
fetchImpl: typeof fetch = fetch,
+ env: NodeJS.ProcessEnv = process.env,
+): Promise<LatestRelease | null> {
+ for (const mirror of releaseMirrors(env)) {
+ const found = await checkMirror(mirror, currentVersion, fetchImpl);
+ if (found) return found;
+ }
+ return checkOrigin(currentVersion, fetchImpl);
+}
+
+/** A mirror's `latest.json`. Anything unreadable is "no answer", not an error. */
+async function checkMirror(
+ mirror: string,
+ currentVersion: string,
+ fetchImpl: typeof fetch,
+): Promise<LatestRelease | null> {
+ try {
+ const res = await fetchImpl(`${mirror}/latest.json`, { headers: { accept: 'application/json' } });
+ if (!res.ok) return null;
+ const body = (await res.json()) as MirrorLatest;
+ if (!body.version) return null;
+ const parsed = parseVersion(body.version);
+ if (!parsed || !isNewerVersion(body.version, currentVersion)) return null;
+ const version = parsed.parts.join('.') + (parsed.pre ? `-${parsed.pre}` : '');
+ // The mirror's own copy of that version's assets, so the download is as fast as
+ // the check was. A mirror that names no url still gets the user to the release.
+ return { version, url: body.url ?? `${mirror}/v${version}/` };
+ } catch {
+ return null;
+ }
+}
+
+async function checkOrigin(
+ currentVersion: string,
+ fetchImpl: typeof fetch,
): Promise<LatestRelease | null> {
try {
const res = await fetchImpl(RELEASES_LATEST, {
diff --git a/desktop/tests/release-mirror.test.ts b/desktop/tests/release-mirror.test.ts
new file mode 100644
index 000000000..38a897e48
--- /dev/null
+++ b/desktop/tests/release-mirror.test.ts
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+//
+// A mirror is a shortcut, never an authority. These are the two halves of that
+// sentence: the editor asks a mirror first (so a slow origin is not the download
+// speed anyone gets), and every way a mirror can be wrong — down, missing the
+// version, serving a truncated or substituted archive, lagging behind a release —
+// hands the job back to the origin instead of failing or, worse, succeeding.
+import { describe, it, expect, beforeEach, afterEach } from 'vitest';
+import { mkdtempSync, mkdirSync, rmSync, readFileSync, existsSync } from 'node:fs';
+import { createHash } from 'node:crypto';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+import { downloadNativeTemplate } from '../electron/nativeTemplates';
+import { checkForUpdate } from '../electron/updateCheck';
+import {
+ releaseAssetBase, releaseAssetBases, releaseMirrors, RELEASE_MIRROR_ENV,
+ TEMPLATE_INDEX, TEMPLATE_FORMAT, writeTemplateManifest, installedTemplateDir,
+} from '../../build-tools/utils/nativeTemplate.js';
+import { makeZip } from '../../build-tools/utils/zip.js';
+
+const VERSION = '9.9.9';
+const MIRROR = 'https://mirror.test/estella';
+
+describe('where a release is fetched from', () => {
+ it('is the origin alone when nothing is mirrored', () => {
+ expect(releaseMirrors({})).toEqual([]);
+ expect(releaseAssetBases(VERSION, {})).toEqual([releaseAssetBase(VERSION)]);
+ });
+
+ it('puts each mirror before the origin, and lays a version out the same way', () => {
+ const env = { [RELEASE_MIRROR_ENV]: `${MIRROR}/, https://second.test` };
+ expect(releaseAssetBases(VERSION, env)).toEqual([
+ `${MIRROR}/v${VERSION}`,
+ `https://second.test/v${VERSION}`,
+ releaseAssetBase(VERSION),
+ ]);
+ });
+});
+
+describe('downloading a template with a mirror configured', () => {
+ let store: string;
+
+ beforeEach(() => {
+ store = mkdtempSync(path.join(tmpdir(), 'es-mirror-'));
+ process.env.ESTELLA_NATIVE_TEMPLATES = store;
+ process.env[RELEASE_MIRROR_ENV] = MIRROR;
+ });
+
+ afterEach(() => {
+ delete process.env.ESTELLA_NATIVE_TEMPLATES;
+ delete process.env[RELEASE_MIRROR_ENV];
+ rmSync(store, { recursive: true, force: true });
+ });
+
+ /** A template archive and the index that describes it, as a release publishes them. */
+ function published() {
+ const dir = path.join(store, 'src');
+ mkdirSync(dir, { recursive: true });
+ writeTemplateManifest(dir, { platform: 'ios', engineVersion: VERSION });
+ const zip = makeZip([
+ { name: 'template.json', data: readFileSync(path.join(dir, 'template.json')) },
+ { name: 'Estella.xcframework/Info.plist', data: Buffer.from('<plist/>') },
+ { name: 'App/main.m', data: Buffer.from('int main(){}') },
+ { name: 'App/Info.plist.in', data: Buffer.from('<plist/>') },
+ { name: 'icon.png', data: Buffer.alloc(8, 1) },
+ ]);
+ const index = {
+ kind: 'estella-native-templates',
+ formatVersion: TEMPLATE_FORMAT,
+ engineVersion: VERSION,
+ templates: [{
+ id: 'ios', platform: 'ios', file: `estella-native-ios-${VERSION}.zip`,
+ bytes: zip.length, sha256: createHash('sha256').update(zip).digest('hex'),
+ }],
+ };
+ return { zip, index };
+ }
+
+ /**
+ * A fetch that answers per host: `mirror` decides what the mirror serves, and the
+ * origin always serves the real thing. Records every URL asked for.
+ */
+ function serve(mirrorBehaviour: 'good' | 'down' | 'corrupt') {
+ const { zip, index } = published();
+ const asked: string[] = [];
+ const bodyOf = (data: Buffer) => ({
+ ok: true, status: 200,
+ body: (async function* () { yield data; })(),
+ } as unknown as Response);
+
+ const fetchImpl = (async (url: string) => {
+ asked.push(url);
+ const fromMirror = url.startsWith(MIRROR);
+ if (fromMirror && mirrorBehaviour === 'down') {
+ return { ok: false, status: 503 } as unknown as Response;
+ }
+ if (url.endsWith(TEMPLATE_INDEX)) {
+ return { ok: true, status: 200, json: async () => index } as unknown as Response;
+ }
+ // A substituted archive: right size, wrong bytes — the case a checksum is for.
+ if (fromMirror && mirrorBehaviour === 'corrupt') return bodyOf(Buffer.alloc(zip.length, 0x5a));
+ return bodyOf(zip);
+ }) as unknown as typeof fetch;
+
+ return { fetchImpl, asked };
+ }
+
+ const installed = () => existsSync(path.join(installedTemplateDir(VERSION, 'ios', store), 'template.json'));
+
+ it('takes the mirror when it serves the archive the index describes', async () => {
+ const { fetchImpl, asked } = serve('good');
+
+ const res = await downloadNativeTemplate('ios', VERSION, { fetchImpl });
+
+ expect(res.ok).toBe(true);
+ expect(installed()).toBe(true);
+ expect(asked.every((u) => u.startsWith(MIRROR))).toBe(true);
+ });
+
+ it('falls back to the origin when the mirror is down', async () => {
+ const { fetchImpl, asked } = serve('down');
+
+ const res = await downloadNativeTemplate('ios', VERSION, { fetchImpl });
+
+ expect(res.ok).toBe(true);
+ expect(installed()).toBe(true);
+ expect(asked.some((u) => u.startsWith(MIRROR))).toBe(true);
+ expect(asked.some((u) => u.startsWith('https://github.com/'))).toBe(true);
+ });
+
+ it('falls back when the mirror serves the right size and the wrong bytes', async () => {
+ const { fetchImpl, asked } = serve('corrupt');
+
+ const res = await downloadNativeTemplate('ios', VERSION, { fetchImpl });
+
+ // The checksum is what makes a mirror safe to prefer: a substituted archive is
+ // not installed, and it is not the end of the attempt either.
+ expect(res.ok).toBe(true);
+ expect(installed()).toBe(true);
+ expect(asked.filter((u) => u.startsWith('https://github.com/')).length).toBeGreaterThan(0);
+ });
+});
+
+describe('checking for an editor update with a mirror configured', () => {
+ const env = { [RELEASE_MIRROR_ENV]: MIRROR };
+
+ it('asks the mirror first and points the download at it', async () => {
+ const asked: string[] = [];
+ const fetchImpl = (async (url: string) => {
+ asked.push(url);
+ return { ok: true, status: 200, json: async () => ({ version: '1.2.0' }) } as unknown as Response;
+ }) as unknown as typeof fetch;
+
+ const res = await checkForUpdate('1.1.0', fetchImpl, env);
+
+ expect(res).toEqual({ version: '1.2.0', url: `${MIRROR}/v1.2.0/` });
+ expect(asked).toEqual([`${MIRROR}/latest.json`]);
+ });
+
+ it('asks the origin when the mirror has no answer, or a stale one', async () => {
+ for (const mirrorAnswer of [null, { version: '1.0.0' }]) {
+ const asked: string[] = [];
+ const fetchImpl = (async (url: string) => {
+ asked.push(url);
+ if (url.startsWith(MIRROR)) {
+ return mirrorAnswer
+ ? { ok: true, status: 200, json: async () => mirrorAnswer } as unknown as Response
+ : { ok: false, status: 404 } as unknown as Response;
+ }
+ return {
+ ok: true, status: 200,
+ json: async () => ({ tag_name: 'v1.2.0', html_url: 'https://github.com/r/releases/v1.2.0' }),
+ } as unknown as Response;
+ }) as unknown as typeof fetch;
+
+ const res = await checkForUpdate('1.1.0', fetchImpl, env);
+
+ // The origin publishes; a mirror that has not caught up cannot hide a release.
+ expect(res?.version).toBe('1.2.0');
+ expect(asked.some((u) => u.startsWith('https://api.github.com/'))).toBe(true);
+ }
+ });
+
+ it('says nothing when the running version is current', async () => {
+ const fetchImpl = (async () => (
+ { ok: true, status: 200, json: async () => ({ version: '1.1.0' }) } as unknown as Response
+ )) as unknown as typeof fetch;
+
+ // The mirror agrees it is current, and the origin (same answer) confirms.
+ expect(await checkForUpdate('1.1.0', fetchImpl, env)).toBeNull();
+ });
+});
diff --git a/docs/astro/src/content/docs/guides/mobile.mdx b/docs/astro/src/content/docs/guides/mobile.mdx
index a757bb1c4..c7b2d4df5 100644
--- a/docs/astro/src/content/docs/guides/mobile.mdx
+++ b/docs/astro/src/content/docs/guides/mobile.mdx
@@ -73,6 +73,17 @@ are not the same severity:
Android needs nothing else: the APK is compiled, aligned and signed by the
editor, on whatever OS it is running on.
+ :::tip[Where the download comes from]
+ Releases are mirrored, and the editor asks the mirror before the origin — a
+ template is hundreds of megabytes, and GitHub's speed is not the same
+ everywhere. The mirror is a shortcut and never an authority: every archive is
+ checked against the size and SHA-256 the release's index states, so a copy that
+ is missing, stale or wrong fails that check and the download falls back to
+ GitHub. Point the editor elsewhere — a company share, your own CDN, nothing at
+ all — with `ESTELLA_RELEASE_MIRROR` (comma-separated, tried in order); each one
+ lays a version out exactly as the release does, under `v<version>/`.
+ :::
+
2. **Install the platform toolchain — iOS only.**
- *Android* — none. There is no Android SDK, NDK or JDK in this path.
- *iOS* — **macOS with Xcode**, and nothing else: no CMake, no Ninja, no
diff --git a/docs/astro/src/content/docs/zh-cn/guides/mobile.mdx b/docs/astro/src/content/docs/zh-cn/guides/mobile.mdx
index b1f4d2344..123510d85 100644
--- a/docs/astro/src/content/docs/zh-cn/guides/mobile.mdx
+++ b/docs/astro/src/content/docs/zh-cn/guides/mobile.mdx
@@ -60,6 +60,14 @@ Android 两者都给,是因为它两者都做得到:编辑器既能直接装配
Android 此外别无所需:APK 的编译、对齐与签名都由编辑器在它所运行的任何操作系统上完成。
+ :::tip[下载走的是哪儿]
+ 发布会同步一份到镜像,编辑器**先问镜像再问源站**——一个模板几百 MB,而 GitHub 的
+ 速度并非各地一致。镜像只是捷径,从来不是权威:每个归档都会按发布索引里写的大小与
+ SHA-256 校验,缺失、过期或被替换的副本过不了这一关,下载会自动回落到 GitHub。要指
+ 到别处(公司共享、自建 CDN,或者干脆不用镜像),设 `ESTELLA_RELEASE_MIRROR`(逗号
+ 分隔,按顺序尝试);每个镜像里版本的摆法与发布完全一致,都在 `v<版本号>/` 下。
+ :::
+
2. **装好平台工具链——只有 iOS 需要。**
- *Android* —— 不需要。这条路径上没有 Android SDK、NDK 或 JDK。
- *iOS* —— **装有 Xcode 的 macOS**,仅此而已:不需要 CMake、Ninja、xcodegen,