A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
test(scale): seventeen costs, each with a ceiling it may not pass
esengine21485a826c08
diff --git a/bench/scale/README.md b/bench/scale/README.mdnew file mode 100644index 000000000..2dc7a0348--- /dev/null+++ b/bench/scale/README.md@@ -0,0 +1,109 @@+# The scale corpus and its budgets++Every scale problem this engine has had was reported by someone whose project was+big enough to notice. 36,000 assets where selecting 22 files and pressing Delete+took 52 seconds to open the confirm dialog. A folder of 788 sprites that decoded+26 megapixels to show the twenty on screen. Nothing in this repository was ever+that big — the examples exist to teach, so they are small, and a cost that only+appears at 50,000 assets is invisible here until a user meets it.++So there is now a project that big, and a set of ceilings it has to stay under.++```sh+node tools/perf-budget.mjs # make the corpus if needed, measure, gate+node tools/perf-budget.mjs --table # the last run's table, without re-running+pnpm run scale # the same thing+```++## The corpus++`tools/stress-project.mjs` generates it into `build/stress-project/` — an ordinary+Estella project you can open in the editor and press Play on.++| | |+| --- | --- |+| assets | **50,000** (100,004 files with their `.meta` sidecars, ~42MB) |+| textures | 31,496 — including 2,000 in **one folder**, the shape that broke the Content Browser |+| materials / prefabs / audio / data | 6,000 / 4,000 / 3,000 / 3,000 |+| shaders / tilesets / spine sets / fonts | 700 / 200 / 500 / 60 |+| scenes | 43, of which 7 carry the weight |+| `sprites.esscene` | 10,000 sprites over 8 sorting layers, 4 cameras, 200 parents |+| `ui.esscene` | 5,000 UI nodes, six deep, masks nested inside masks |+| `physics.esscene` | 2,000 bodies with colliders |+| `spine.esscene` | 500 skeletons |+| `prefabs.esscene` | 3,000 instances, most of them overriding something |+| `tilemap.esscene` | a 256×256×4 Tiled map |+| `everything.esscene` | all of the above in one document — 20,707 entries, 7.3MB |++It is **generated, not committed**. 100,000 files would sit in front of every+clone, every `git status` and every editor indexing the tree, to store bytes that+are a pure function of one script. The generator is the corpus: no `Math.random`,+no `Date`, every uuid is `sha1(seed + path)`, so the same spec produces the same+tree byte for byte on any machine. It takes about eight seconds to write, and a+stamp file means it is only written when the generator or the scale changes.++`--scale 0.02` makes a 1/50th corpus for a quick local experiment. Budgets do not+apply to it and the gate refuses to run against one.++## The budgets++Declared next to what they measure, in `desktop/tests/scale/*.scale.ts`, each with+the reason it exists. A metric that goes over fails, and prints that reason.++**Why a budget rather than a snapshot.** `tools/perf-guard.mjs` asserts ratios+between microbenchmarks against a recorded snapshot — the right shape for an+architectural invariant, and the wrong one here, because a snapshot can always be+accepted with `--update`. A feature that takes scene-open from 200ms to 900ms+would pass by rewriting the number it is compared against. A ceiling cannot be+raised by the change that breaks it: raising one is its own commit, with its own+reason, and somebody has to decide to write it.++**Why the numbers are not milliseconds.** They would not survive the CI runner,+which has two shared cores. Each run measures three fixed reference workloads and+every metric is divided by the one whose shape it shares:++| unit | the reference | what it stands in for |+| --- | --- | --- |+| `parse` | `JSON.parse` + walk of a fixed 6,000-row document | scene load, serialize, index reads |+| `loop` | a numeric pass over 2M `Float32` lanes | ECS iteration, transform math |+| `io` | `readdir` over 20 shards, then read+parse 10,000 `.meta` | the asset-database walk |++"Cold scan = 8.3 io" means the whole-project scan costs 8.3× what reading 10,000+metas costs on whatever machine is asking. That reads the same on a laptop and on+a runner. The raw milliseconds are recorded beside it, because that is the number+a human wants; it is not the number the gate uses.++The statistic is `min`, for the reason the perf guard already documents: noise on+a shared runner is one-sided, so the fastest sample is the closest to what the+code costs.++The io reference was first written as "read 2,000 files from one folder". After+the first pass that folder is entirely in the page cache, so it measured syscall+and scheduler cost — a regime the 100,000-file scan is never in. On a busy machine+it slowed by 41% while the scan it denominates slowed by 15%, and the ratio moved+the wrong way. Twenty directories and 10,000 files behave like the thing they+divide: run to run, the scan now reads within 3%.++**How the ceilings were chosen.** Each is about twice what it measures today,+which is the same bargain the perf guard makes: run-to-run noise reaches ~10% on+the io-denominated metrics, so this cannot see a 20% slowdown and is not meant to.+It is meant to catch the 2×-and-up kind — the shape a scale bug actually has. The+memory budgets are tighter because bytes are exact, not sampled.++Anything above 70% of its budget is reported as headroom going, which is the only+warning before it stops being headroom.++## What is not here yet++Everything that needs a renderer: draw calls, GPU uploads, frame time with+submission, editor cold start, project scan measured through the editor rather+than the module it calls, Play startup, build duration. Those need the electron+harness (`ESTELLA_SHOT`, the automation surface) rather than a vitest process, and+they are a second tier of this same contract.++## Changing a budget++Change the number where it is declared, in its own commit, and update the `why`+to say what the new ceiling means. If the corpus spec changes, every budget+measured against it changes with it — that is a change to the contract, not a+rebase.diff --git a/desktop/tests/scale/assets.scale.ts b/desktop/tests/scale/assets.scale.tsnew file mode 100644index 000000000..34827ec2a--- /dev/null+++ b/desktop/tests/scale/assets.scale.ts@@ -0,0 +1,144 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Asset-database budgets against the 50k-asset corpus.+ *+ * These are the costs a user meets before they have done anything: opening the+ * project, saving a file, deleting a folder's worth of sprites. Every one of+ * them was O(whole project) at some point in this repo's history, and the one+ * that shipped was found by someone whose project was big enough to notice —+ * 36k assets, 52 seconds to delete 22 files. The corpus is bigger than that.+ */+import { describe, it, beforeAll, expect } from 'vitest';+import { readFile, writeFile, rm } from 'node:fs/promises';+import path from 'node:path';++import {+ scanAssetDatabase,+ updateAssetIndex,+ readCachedAssetIndex,+ type AssetIndex,+} from '../../electron/assetDb';+import { collectAssetUsagesOfAll } from '../../src/project/assetRefs';+import { budgeted, calibrate, corpusDir } from './harness';++const GROUP = 'Asset database — 50,000 assets';++let root: string;+let index: AssetIndex;++beforeAll(async () => {+ root = corpusDir();+ await calibrate(root);+}, 300_000);++describe(GROUP, () => {+ it('cold scan', async () => {+ const res = await budgeted({+ name: 'assetdb: cold scan of the whole project',+ group: GROUP,+ unit: 'io',+ budget: 18,+ why: 'Every project open pays this once, and nothing is on screen until it lands. '+ + 'It walks the tree, reads every .meta and builds the dependency graph.',+ runs: 3,+ }, () => scanAssetDatabase(root, { write: true }));++ index = res.index;+ expect(res.ok).toBe(true);+ // The corpus is exact by construction; a scan that finds a different+ // number is measuring a tree nobody described.+ expect(index.entries.length).toBe(50_000);+ }, 600_000);++ it('reads the cached index', async () => {+ await budgeted({+ name: 'assetdb: read the cached index',+ group: GROUP,+ unit: 'parse',+ budget: 16,+ why: 'The warm path a second window or a reopened project takes instead of scanning. '+ + 'It is one JSON.parse of 50k entries plus the dep graph.',+ runs: 5,+ }, async () => {+ const cached = await readCachedAssetIndex(root);+ expect(cached?.entries.length).toBe(50_000);+ return cached;+ });+ }, 300_000);++ it('incremental scan after one file changes', async () => {+ const changed = ['assets/textures/_bulk/tex-00001.png'];+ await budgeted({+ name: 'assetdb: incremental scan, one file touched',+ group: GROUP,+ unit: 'io',+ budget: 0.25,+ why: 'What the watcher fires on every save. This exists so a keystroke does not re-walk '+ + 'the project — if it approaches the cold scan, the incremental path has stopped being one.',+ runs: 5,+ }, async () => {+ const res = await updateAssetIndex(root, index, changed, { write: false });+ expect(res.fullRescan).toBe(false);+ return res;+ });+ }, 300_000);++ it('usages of a 22-file selection', async () => {+ const paths = Array.from({ length: 22 }, (_, i) => `assets/textures/_bulk/tex-${String(100 + i).padStart(5, '0')}.png`);+ await budgeted({+ name: 'assetdb: usages of a 22-file selection',+ group: GROUP,+ unit: 'parse',+ budget: 9,+ why: 'The question the delete-confirm dialog asks before it can open. Asked once per '+ + 'selected asset over a freshly built map, 22 files took 52 seconds on a project '+ + 'smaller than this one. It must invert the index once for the batch, not once per file.',+ runs: 5,+ }, () => collectAssetUsagesOfAll(index, paths));+ }, 300_000);++ it('usages of one asset', async () => {+ await budgeted({+ name: 'assetdb: usages of one asset',+ group: GROUP,+ unit: 'parse',+ budget: 9,+ why: 'Find Usages on a single sprite — the interactive case, where the dialog is waiting.',+ runs: 5,+ }, () => collectAssetUsagesOfAll(index, ['assets/textures/_bulk/tex-00100.png']));+ }, 300_000);++ it('incremental scan after 22 files are deleted', async () => {+ // The reported case, restored afterwards so the corpus survives the run:+ // the removal is real (the scan reads the disk, not a list) and the exact+ // bytes go back.+ const rels = Array.from({ length: 22 }, (_, i) => `assets/textures/_bulk/tex-${String(200 + i).padStart(5, '0')}.png`);+ const saved: Array<[string, Buffer]> = [];+ for (const rel of rels) {+ for (const f of [rel, `${rel}.meta`]) {+ saved.push([f, await readFile(path.join(root, f))]);+ await rm(path.join(root, f));+ }+ }+ try {+ await budgeted({+ name: 'assetdb: incremental scan, 22 files deleted',+ group: GROUP,+ unit: 'io',+ budget: 2.6,+ why: 'Deleting a folder\'s worth. A removal cannot make a dangling reference resolve, '+ + 'so only the documents that referenced something removed need re-reading — '+ + 'recomputing the whole dependency graph here is the regression this pins.',+ runs: 3,+ }, async () => {+ const res = await updateAssetIndex(root, index, saved.map(([f]) => f), { write: false });+ expect(res.fullRescan).toBe(false);+ expect(res.index.entries.length).toBe(50_000 - rels.length);+ return res;+ });+ } finally {+ for (const [f, body] of saved) await writeFile(path.join(root, f), body);+ }+ }, 600_000);+});diff --git a/desktop/tests/scale/harness.ts b/desktop/tests/scale/harness.tsnew file mode 100644index 000000000..1dd79d3c5--- /dev/null+++ b/desktop/tests/scale/harness.ts@@ -0,0 +1,282 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file The scale harness: measure one cost against the stress corpus and hold+ * it to a declared budget.+ *+ * WHY A BUDGET AND NOT A SNAPSHOT. The perf snapshot (`tools/perf-guard.mjs`)+ * answers "did this change make it worse", which is the right question for an+ * architectural invariant and the wrong one here. A snapshot can always be+ * accepted with `--update`, so a feature that takes scene-open from 200ms to+ * 900ms passes by rewriting the number it is compared against. A budget cannot+ * be updated by the change that breaks it: it is a ceiling somebody chose, with+ * a reason next to it, and going over is a decision rather than a diff.+ *+ * HOW A BUDGET SURVIVES A DIFFERENT MACHINE. Milliseconds do not — the CI runner+ * has two shared cores and this laptop does not — so a budget is denominated in+ * CALIBRATION UNITS. Each run measures a fixed reference workload and every+ * metric is reported as its own cost divided by that reference. There are three+ * references because there are three kinds of cost here, and one of them cannot+ * stand in for another: `parse` (JSON + object churn), `loop` (tight numeric+ * iteration) and `io` (reading small files off disk). A metric names the one that+ * matches its shape, so "cold scan = 30 io" means "thirty times what reading a+ * fixed pile of files costs on whatever machine this is".+ *+ * THE STATISTIC IS `min`, for the reason the perf guard already documents: noise+ * on a shared runner is one-sided, so the fastest sample is the one closest to+ * what the code costs.+ *+ * Raw milliseconds are recorded too. They are the number a human wants when+ * reading the report; they are not the number the gate uses.+ */+import { expect } from 'vitest';+import { readFile, stat, writeFile, mkdir } from 'node:fs/promises';+import { readdirSync, existsSync } from 'node:fs';+import path from 'node:path';+import { fileURLToPath, URL } from 'node:url';++const REPO = fileURLToPath(new URL('../../../', import.meta.url));+export const REPORT_PATH = path.join(REPO, 'build', 'scale-report.json');++/**+ * A metric's denominator. `MB` and `count` have none: bytes and draw calls are+ * already machine-independent, so their budgets are absolute.+ */+export type Unit = 'parse' | 'loop' | 'io' | 'MB' | 'count';++export interface MetricSpec {+ /** What the metric measures, as it appears in the report. */+ name: string;+ /** The heading it is filed under. */+ group: string;+ unit: Unit;+ /** The ceiling, in `unit`. Going over fails the run. */+ budget: number;+ /** Why this ceiling and what breaking it would mean. Printed on failure. */+ why: string;+ /** Timed metrics only: how many samples to take the minimum of. */+ runs?: number;+ /**+ * Untimed runs first. Needed wherever the first call does less work than the+ * rest — resetting a world that is still empty despawns nothing, and a `min`+ * over those samples would report the one run that skipped half the job.+ */+ warmup?: number;+}++export interface MetricRecord extends MetricSpec {+ measured: number;+ rawMs: number | null;+ pctOfBudget: number;+}++const records: MetricRecord[] = [];++// --- Calibration -----------------------------------------------------------+// Each reference is fixed and shaped like the metrics that divide by it, and+// measured once per process.++const CALIBRATION_RUNS = 7;++let units: Record<'parse' | 'loop' | 'io', number> | null = null;++/** A fixed document to parse — built, never read from disk, so it is identical everywhere. */+function calibrationDocument(): string {+ const rows = [];+ for (let i = 0; i < 6000; i++) {+ rows.push({+ id: i,+ name: `entry-${i}`,+ nested: { a: i * 1.5, b: [i, i + 1, i + 2], c: `${i}` },+ tags: ['alpha', 'beta', 'gamma'],+ });+ }+ return JSON.stringify({ version: 1, rows });+}++function walkSum(v: unknown): number {+ if (typeof v === 'number') return v;+ if (typeof v === 'string') return v.length;+ if (Array.isArray(v)) {+ let s = 0;+ for (const x of v) s += walkSum(x);+ return s;+ }+ if (v && typeof v === 'object') {+ let s = 0;+ for (const x of Object.values(v)) s += walkSum(x);+ return s;+ }+ return 0;+}++function loopWorkload(): number {+ const lanes = new Float32Array(1 << 21);+ for (let i = 0; i < lanes.length; i++) lanes[i] = i * 0.5;+ let acc = 0;+ for (let i = 0; i < lanes.length; i++) acc += lanes[i] * 1.0001 + 1;+ return acc;+}++/**+ * The io reference walks part of the corpus the way the scan walks all of it:+ * `readdir` over twenty directories, then read and parse every `.meta`. One+ * folder instead stays in the page cache and measures syscall cost, a regime+ * the 100,000-file scan is never in — that reference moved the ratio backwards.+ */+async function ioWorkload(corpusDir: string): Promise<number> {+ const files = ioFiles(corpusDir);+ let bytes = 0;+ const LIMIT = 32;+ for (let i = 0; i < files.length; i += LIMIT) {+ const slice = files.slice(i, i + LIMIT);+ const results = await Promise.all(slice.map(async (f) => {+ const [s, body] = await Promise.all([stat(f), readFile(f, 'utf8')]);+ return s.size + JSON.parse(body).uuid.length;+ }));+ for (const r of results) bytes += r;+ }+ return bytes;+}++const IO_REFERENCE_SHARDS = 20;+const IO_REFERENCE_FILES = 10_000;+let ioFileList: string[] | null = null;++function ioFiles(corpusDir: string): string[] {+ if (ioFileList) return ioFileList;+ const textures = path.join(corpusDir, 'assets/textures');+ const shards = readdirSync(textures).filter((d) => d.startsWith('shard-')).sort().slice(0, IO_REFERENCE_SHARDS);+ const out: string[] = [];+ for (const shard of shards) {+ const dir = path.join(textures, shard);+ for (const f of readdirSync(dir).sort()) if (f.endsWith('.meta')) out.push(path.join(dir, f));+ }+ if (out.length !== IO_REFERENCE_FILES) {+ throw new Error(`io calibration expects ${IO_REFERENCE_FILES} files across ${IO_REFERENCE_SHARDS} `+ + `shards of ${textures}, found ${out.length} — the corpus was generated at a reduced `+ + '--scale, and budgets do not apply to it.');+ }+ ioFileList = out;+ return ioFileList;+}++/**+ * Measure the three references. Every suite calls this and only the first one+ * does the work: calibrating again half way through a run would denominate the+ * later metrics in a different machine-state than the earlier ones, and the+ * report is one table.+ */+export async function calibrate(corpusDir: string): Promise<Record<string, number>> {+ if (units) return units;+ const doc = calibrationDocument();+ const parse = minOf(CALIBRATION_RUNS, () => { walkSum(JSON.parse(doc)); });+ const loop = minOf(CALIBRATION_RUNS, () => { loopWorkload(); });+ let io = Infinity;+ for (let i = 0; i < 3; i++) {+ const t = performance.now();+ await ioWorkload(corpusDir);+ io = Math.min(io, performance.now() - t);+ }+ units = { parse, loop, io };+ return units;+}++function minOf(runs: number, fn: () => void): number {+ let best = Infinity;+ for (let i = 0; i < runs; i++) {+ const t = performance.now();+ fn();+ best = Math.min(best, performance.now() - t);+ }+ return best;+}++// ---------------------------------------------------------------------------+// Measurement+// ---------------------------------------------------------------------------++/**+ * Time `fn` and hold the result to `spec.budget`. Returns whatever `fn` returned+ * on its last run, so a metric can assert on the work as well as its cost.+ */+export async function budgeted<T>(spec: MetricSpec, fn: () => Promise<T> | T): Promise<T> {+ if (spec.unit === 'MB' || spec.unit === 'count') {+ throw new Error(`${spec.name}: ${spec.unit} is a value, not a duration — use budgetedValue`);+ }+ if (!units) throw new Error('calibrate() was never called — every budget would be meaningless');++ const runs = spec.runs ?? 3;+ let best = Infinity;+ let last!: T;+ for (let i = 0; i < (spec.warmup ?? 0); i++) last = await fn();+ for (let i = 0; i < runs; i++) {+ const t = performance.now();+ last = await fn();+ best = Math.min(best, performance.now() - t);+ }+ record(spec, best / units[spec.unit], best);+ return last;+}++/** The same contract for a metric that is already a number: bytes, or a count. */+export async function budgetedValue(spec: MetricSpec, fn: () => Promise<number> | number): Promise<number> {+ if (spec.unit !== 'MB' && spec.unit !== 'count') {+ throw new Error(`${spec.name}: ${spec.unit} is a duration — use budgeted`);+ }+ const value = await fn();+ record(spec, value, null);+ return value;+}++function record(spec: MetricSpec, measured: number, rawMs: number | null): void {+ const rec: MetricRecord = {+ ...spec,+ measured: round(measured),+ rawMs: rawMs === null ? null : round(rawMs),+ pctOfBudget: Math.round((measured / spec.budget) * 100),+ };+ records.push(rec);+ void persist();++ // The failure message has to carry the reason, because whoever reads it in a+ // CI log has neither this file nor the budget table in front of them.+ expect(+ measured,+ `\n ${spec.name}\n`+ + ` measured ${rec.measured}${suffix(spec.unit)}, budget ${spec.budget}${suffix(spec.unit)}`+ + `${rawMs === null ? '' : ` (${rec.rawMs}ms on this machine)`}\n`+ + ` ${spec.why}\n`+ + ' If this cost is now correct, raise the budget in the file that declares it —\n'+ + ' deliberately, in its own commit, with the reason updated.\n',+ ).toBeLessThanOrEqual(spec.budget);+}++const suffix = (u: Unit) => (u === 'MB' ? 'MB' : u === 'count' ? '' : ` ${u}`);+const round = (n: number) => Number(n.toPrecision(4));++/**+ * The report is rewritten after every metric rather than at the end, so a run+ * that dies half way still says what it managed to measure.+ */+async function persist(): Promise<void> {+ await mkdir(path.dirname(REPORT_PATH), { recursive: true });+ await writeFile(REPORT_PATH, `${JSON.stringify({+ note: 'Generated by the scale suite — see bench/scale/README.md.',+ units,+ metrics: records,+ }, null, 2)}\n`);+}++// ---------------------------------------------------------------------------+// The corpus+// ---------------------------------------------------------------------------++/** Where the generator put it, unless $ESTELLA_STRESS_PROJECT says otherwise. */+export function corpusDir(): string {+ const dir = process.env.ESTELLA_STRESS_PROJECT ?? path.join(REPO, 'build', 'stress-project');+ if (!existsSync(path.join(dir, 'project.esproject'))) {+ throw new Error(`No stress corpus at ${dir}. Generate it: node tools/stress-project.mjs`);+ }+ return dir;+}diff --git a/desktop/tests/scale/runtime.scale.ts b/desktop/tests/scale/runtime.scale.tsnew file mode 100644index 000000000..82b54d0c3--- /dev/null+++ b/desktop/tests/scale/runtime.scale.ts@@ -0,0 +1,182 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Per-frame and memory budgets at corpus scale.+ *+ * The frame budgets are CPU-side only: a headless app has no renderer, so+ * draw-call generation and GPU upload are not here — they need the electron+ * harness and are a separate tier. What IS here is the half that runs before+ * any of that, and the half a big scene makes expensive.+ *+ * The memory budgets are the ones worth having, because bytes do not care which+ * machine measured them. The wasm heap is exact — it is a byte length, not a+ * sample. The JS heap is not: V8 decides when to give memory back, so its budget+ * is set wide enough to catch a leak and no narrower.+ */+import { describe, it, beforeAll, expect } from 'vitest';+import { readFile } from 'node:fs/promises';+import path from 'node:path';++import {+ createHeadlessApp,+ loadEsengineModule,+ resetWorldTo,+ type App,+ type SceneData,+} from 'esengine/node';++import { WASM_DIR, HAS_WASM } from '../helpers/loadWasm';+import { budgeted, budgetedValue, calibrate, corpusDir } from './harness';++const GROUP = 'Runtime — frames and memory';+const DT = 1 / 60;+const EMPTY: SceneData = { version: 1, name: 'empty', entities: [] };++let app: App;+let module: Awaited<ReturnType<typeof loadEsengineModule>>;+let sprites: SceneData;+let everything: SceneData;++/** One frame of the CPU-side loop: systems, then the C++ transform propagation. */+async function tick(): Promise<void> {+ await app.tick(DT);+ const registry = app.world.getCppRegistry();+ if (registry) module.transform_update(registry);+}++async function ticks(n: number): Promise<void> {+ for (let i = 0; i < n; i++) await tick();+}++/** heapUsed after a collection — the only reading that means anything. */+function settledHeapMB(): number {+ const gc = (globalThis as { gc?: () => void }).gc;+ for (let i = 0; i < 3; i++) gc?.();+ return process.memoryUsage().heapUsed / 1e6;+}++/**+ * Bytes the engine has actually allocated inside wasm.+ *+ * NOT `HEAPU8.byteLength`: emscripten hands the module ~17MB up front and the+ * whole corpus fits inside it, so the heap's length never moves and a budget on+ * it would read zero however much storage a change added.+ */+const wasmBytesMB = (): number => (module.es_getMallocBytes?.() ?? 0) / 1e6;++beforeAll(async () => {+ const root = corpusDir();+ await calibrate(root);+ module = await loadEsengineModule(WASM_DIR);+ app = createHeadlessApp(module);+ sprites = JSON.parse(await readFile(path.join(root, 'assets/scenes/sprites.esscene'), 'utf8'));+ everything = JSON.parse(await readFile(path.join(root, 'assets/scenes/everything.esscene'), 'utf8'));+}, 300_000);++describe.skipIf(!HAS_WASM)(GROUP, () => {+ it('ticks the 10k-sprite scene', async () => {+ resetWorldTo(app.world, sprites);+ await budgeted({+ name: 'frame: 60 update ticks over 10,000 sprites',+ group: GROUP,+ unit: 'loop',+ budget: 10,+ why: 'A second of simulation with nothing moving: system dispatch, query iteration and '+ + 'the C++ transform propagation. Sixty ticks rather than one, because one tick is '+ + 'short enough to be measuring the clock. GPU submission is not in here.',+ runs: 3,+ warmup: 1,+ }, () => ticks(60));+ }, 600_000);++ it('ticks the everything scene', async () => {+ resetWorldTo(app.world, everything);+ await budgeted({+ name: 'frame: 60 update ticks over the everything scene',+ group: GROUP,+ unit: 'loop',+ budget: 16,+ why: 'The same second on the worst scene in the corpus. The gap between this and the '+ + 'sprite scene is what the other component families cost per frame.',+ runs: 3,+ warmup: 1,+ }, () => ticks(60));+ }, 600_000);++ it('holds the 10k-sprite scene in bounded wasm storage', async () => {+ // A module of its own, so this is the cost of the scene and not of+ // whatever an earlier test left allocated.+ const fresh = await loadEsengineModule(WASM_DIR);+ const freshApp = createHeadlessApp(fresh);+ await freshApp.tick(DT);+ const empty = (fresh.es_getMallocBytes?.() ?? 0) / 1e6;+ resetWorldTo(freshApp.world, sprites);+ await freshApp.tick(DT);++ await budgetedValue({+ name: 'heap: wasm bytes for 10,000 sprites',+ group: GROUP,+ unit: 'MB',+ budget: 8,+ why: 'Component storage for 10,000 entities, measured on a module nothing else has '+ + 'touched. Exact bytes, not a sample — the same number on every machine, so it '+ + 'is the strictest budget here.',+ }, () => (fresh.es_getMallocBytes?.() ?? 0) / 1e6 - empty);+ }, 600_000);++ it('holds the 10k-sprite scene in a bounded JS heap', async () => {+ resetWorldTo(app.world, EMPTY);+ await ticks(2);+ const empty = settledHeapMB();+ resetWorldTo(app.world, sprites);+ await ticks(2);+ await budgetedValue({+ name: 'heap: JS growth for 10,000 sprites',+ group: GROUP,+ unit: 'MB',+ budget: 14,+ why: 'The JS-side entity bookkeeping the world keeps beside the wasm storage. Wide by '+ + 'design: V8 decides when to release, so this catches a leak and not a regression.',+ }, () => settledHeapMB() - empty);+ }, 600_000);++ it('gives the scene back when it is closed', async () => {+ // The Play/Stop shape: open a scene, close it, repeat. Measured from+ // cycle 3, so one-time caches are already paid for and what is left is+ // what the cycle itself retains.+ const CYCLES = 20;+ let floorAt3 = 0;+ let wasmAt3 = 0;+ for (let i = 0; i < CYCLES; i++) {+ resetWorldTo(app.world, sprites);+ await ticks(2);+ resetWorldTo(app.world, EMPTY);+ await ticks(2);+ if (i === 2) {+ floorAt3 = settledHeapMB();+ wasmAt3 = wasmBytesMB();+ }+ }++ await budgetedValue({+ name: 'heap: wasm retained over 20 open/close cycles',+ group: GROUP,+ unit: 'MB',+ budget: 2,+ why: 'Opening and closing a scene twenty times must end where it started. Component '+ + 'pools keep their capacity by design — that is a flat floor, reached by cycle '+ + 'three. Anything that keeps climbing after it is storage nobody reclaims.',+ }, () => wasmBytesMB() - wasmAt3);++ await budgetedValue({+ name: 'heap: JS retained over 20 open/close cycles',+ group: GROUP,+ unit: 'MB',+ budget: 8,+ why: 'The same seventeen cycles on the JS side. A per-cycle leak of even a few hundred '+ + 'kilobytes is a session that runs out of memory; a flat line here is the contract.',+ }, () => settledHeapMB() - floorAt3);++ expect(app.world.getAllEntities().length).toBe(0);+ }, 900_000);+});diff --git a/desktop/tests/scale/scene.scale.ts b/desktop/tests/scale/scene.scale.tsnew file mode 100644index 000000000..5b01d8337--- /dev/null+++ b/desktop/tests/scale/scene.scale.ts@@ -0,0 +1,152 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Scene and prefab budgets: what it costs to open, save and instantiate+ * at corpus size.+ *+ * These run against the SHIPPED SDK bundle (`esengine/node`), not `src`. A+ * probe that only exists in source is a probe the users' engine does not have —+ * the soak suite found exactly that, a census that tree-shaking had emptied out+ * of `dist` while every test measuring `src` stayed green.+ *+ * The world here is a headless App: the real wasm core, the real component+ * registry, no renderer. Everything below is CPU-side scene work, which is+ * where the cost of a big scene actually lives.+ */+import { describe, it, beforeAll, expect } from 'vitest';+import { readFile } from 'node:fs/promises';+import path from 'node:path';++import {+ createHeadlessApp,+ loadEsengineModule,+ resetWorldTo,+ serializeScene,+ instantiatePrefab,+ type App,+ type SceneData,+} from 'esengine/node';++import { WASM_DIR, HAS_WASM } from '../helpers/loadWasm';+import { budgeted, calibrate, corpusDir } from './harness';++const GROUP = 'Scenes — 10,000 sprites and the everything scene';++let root: string;+let app: App;+let sprites: SceneData;+let everything: SceneData;+let prefab: unknown;++const readScene = async (rel: string): Promise<SceneData> =>+ JSON.parse(await readFile(path.join(root, rel), 'utf8')) as SceneData;++beforeAll(async () => {+ root = corpusDir();+ await calibrate(root);+ const module = await loadEsengineModule(WASM_DIR);+ app = createHeadlessApp(module);+ sprites = await readScene('assets/scenes/sprites.esscene');+ everything = await readScene('assets/scenes/everything.esscene');+ prefab = JSON.parse(await readFile(path.join(root, 'assets/prefabs/shard-000/prefab-00000.esprefab'), 'utf8'));+}, 300_000);++describe.skipIf(!HAS_WASM)(GROUP, () => {+ it('opens the 10k-sprite scene', async () => {+ // Warmed once so every timed run despawns a full world before loading —+ // the first reset into an empty world does half the work of the rest.+ await budgeted({+ name: 'scene: open 10,000 sprites',+ group: GROUP,+ unit: 'parse',+ budget: 55,+ why: 'Double-clicking a scene in the editor. The entities exist and their components '+ + 'are in wasm storage when this returns; nothing is drawn yet.',+ runs: 3,+ warmup: 1,+ }, () => {+ const map = resetWorldTo(app.world, sprites);+ expect(map.size).toBeGreaterThan(10_000);+ return map;+ });+ }, 600_000);++ it('opens the everything scene', async () => {+ // Prefab instances are entries, not entities, until an asset-backed load+ // expands them; this path spawns the ones that carry components inline.+ // What expanding costs is the instantiate metric below.+ const inline = everything.entities.filter((e) => Array.isArray(e.components)).length;+ await budgeted({+ name: 'scene: open the everything scene',+ group: GROUP,+ unit: 'parse',+ budget: 135,+ why: 'Sprites, UI, physics bodies, spine and a tilemap in one document — the scene '+ + 'nobody authors on purpose and every project eventually has. The worst case for open.',+ runs: 3,+ warmup: 1,+ }, () => {+ const map = resetWorldTo(app.world, everything);+ expect(map.size).toBe(inline);+ return map;+ });+ }, 600_000);++ it('serializes the everything scene', async () => {+ // Left holding the everything scene by the test above; serialize measures+ // the world as it stands.+ await budgeted({+ name: 'scene: serialize the everything scene',+ group: GROUP,+ unit: 'parse',+ budget: 130,+ why: 'Every save, every Play (the editor snapshots the world so Stop can restore it) '+ + 'and every autosave pays this. It reads every component of every entity back out.',+ runs: 3,+ }, () => {+ const data = serializeScene(app.world, 'everything');+ expect(data.entities.length).toBeGreaterThan(17_000);+ return data;+ });+ }, 600_000);++ it('parses the everything scene document', async () => {+ const text = await readFile(path.join(root, 'assets/scenes/everything.esscene'), 'utf8');+ await budgeted({+ name: 'scene: JSON.parse the everything document',+ group: GROUP,+ unit: 'parse',+ budget: 15,+ why: 'The floor under scene open: reading 7MB of JSON off disk into objects. '+ + 'If opening a scene approaches a small multiple of this, the loader is close to free.',+ runs: 5,+ }, () => JSON.parse(text));+ }, 300_000);++ it('instantiates 1000 prefabs', async () => {+ const overrides = [{+ prefabEntityId: '0',+ type: 'property' as const,+ componentType: 'Transform',+ propertyName: 'position',+ value: { x: 1, y: 2, z: 0 },+ }];+ resetWorldTo(app.world, { version: 1, name: 'empty', entities: [] });+ await budgeted({+ name: 'prefab: instantiate 1000 with an override',+ group: GROUP,+ unit: 'parse',+ budget: 26,+ why: 'A spawner\'s first second — bullets, enemies, tiles. Each one flattens its source '+ + 'and replays its overrides, so this is the cost of the prefab pipeline, not of spawn.',+ runs: 3,+ }, async () => {+ for (let i = 0; i < 1000; i++) {+ await instantiatePrefab(app.world, prefab as never, { overrides });+ }+ // Cleared inside the timed region on purpose: leaving 1000 entities per+ // run would make each run measure a bigger world than the last.+ resetWorldTo(app.world, { version: 1, name: 'empty', entities: [] });+ });+ }, 600_000);+});diff --git a/desktop/vitest.scale.config.ts b/desktop/vitest.scale.config.tsnew file mode 100644index 000000000..fe79253f1--- /dev/null+++ b/desktop/vitest.scale.config.ts@@ -0,0 +1,38 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+import { defineConfig } from 'vitest/config';+import { fileURLToPath, URL } from 'node:url';++/**+ * @file Config for the scale suite, which runs against a 50,000-asset corpus.+ *+ * Kept out of `vitest.config.ts` so an ordinary test run neither generates that+ * tree nor waits for it — drive it through `node tools/perf-budget.mjs`, which+ * makes the corpus first. One process and no parallelism, because the+ * calibration every budget divides by is measured once per process and two+ * suites racing for the same two cores would each be measuring the other.+ */+export default defineConfig({+ resolve: {+ alias: {+ '@': fileURLToPath(new URL('./src', import.meta.url)),+ },+ },+ test: {+ environment: 'node',+ include: ['tests/scale/**/*.scale.ts'],+ fileParallelism: false,+ // One module graph for the whole run, so the three suites share one+ // calibration and append to one report instead of each isolating its own+ // copy of the harness and overwriting the others' numbers.+ isolate: false,+ pool: 'forks',+ poolOptions: {+ // --expose-gc: the retained-heap budgets are meaningless without a+ // collection they can force.+ forks: { singleFork: true, execArgv: ['--expose-gc'] },+ },+ testTimeout: 600_000,+ hookTimeout: 600_000,+ },+});diff --git a/package.json b/package.jsonindex ca8f46e39..4e921d45b 100644--- a/package.json+++ b/package.json@@ -19,6 +19,7 @@"migrate-asset-refs": "node tools/migrate-asset-refs.js","verify": "pnpm --filter ./sdk exec tsc --noEmit && pnpm --filter @estella/editor exec tsc --noEmit && node tools/api-surface.mjs --check && node tools/check-cycles.mjs && node tools/check-layers.mjs && node tools/check-project-settings.mjs && node tools/check-workflows.mjs && node tools/check-tool-calls.mjs && node tools/check-inspector-door.mjs && node tools/check-dirty-source.mjs && node tools/check-path-sandbox.mjs && node tools/check-comment-style.mjs && node tools/check-release-metadata.mjs && node tools/component-reference.mjs --check","perf": "node tools/perf-guard.mjs --check",+ "scale": "node tools/perf-budget.mjs","stress-project": "node tools/stress-project.mjs","prepare": "git config core.hooksPath .githooks || true","soak": "pnpm --filter ./sdk exec vitest run tests/soak",diff --git a/tools/perf-budget.mjs b/tools/perf-budget.mjsnew file mode 100644index 000000000..acf4648c4--- /dev/null+++ b/tools/perf-budget.mjs@@ -0,0 +1,144 @@+#!/usr/bin/env node+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file The scale gate — run the budgets against the stress corpus.+ *+ * Generates the 50,000-asset project (or reuses it), runs the scale suite, and+ * prints the contract as a table: what each cost is allowed to be, what it is,+ * and how much of its budget it uses.+ *+ * The gate itself is inside the suite — each metric asserts against its own+ * budget, so a failure names the metric and prints the reason the budget exists.+ * This script exists to make the corpus, to show the whole table at once, and to+ * put that table in the CI job summary where somebody will actually read it.+ *+ * Its counterpart is `tools/perf-guard.mjs`, and they answer different+ * questions. The guard asserts RATIOS between microbenchmarks — architectural+ * invariants like "editing one UI field is far cheaper than adding a node" — and+ * its numbers are a snapshot you accept with `--update`. This asserts CEILINGS+ * on whole operations at a size no example reaches, and its numbers are not+ * acceptable by the change that breaks them.+ *+ * node tools/perf-budget.mjs generate if needed, measure, gate+ * node tools/perf-budget.mjs --table print the last run's table only+ * node tools/perf-budget.mjs --regen force the corpus to be rebuilt+ */++import { existsSync, readFileSync, rmSync, appendFileSync } from 'node:fs';+import { dirname, join, resolve } from 'node:path';+import { fileURLToPath } from 'node:url';+import { spawnSync } from 'node:child_process';++import { generateStressProject, countAssets } from './stress-project.mjs';++const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');+const REPORT = join(ROOT, 'build', 'scale-report.json');++const argv = process.argv.slice(2);++if (argv.includes('--table')) {+ if (!existsSync(REPORT)) {+ console.error('perf-budget: no report yet — run `node tools/perf-budget.mjs` first.');+ process.exit(1);+ }+ printTable(JSON.parse(readFileSync(REPORT, 'utf8')));+ process.exit(0);+}++const corpus = await generateStressProject({+ force: argv.includes('--regen'),+ log: (m) => console.log(m),+});+if (corpus.assets !== countAssets(1)) {+ console.error(`perf-budget: the corpus holds ${corpus.assets} assets, not ${countAssets(1)}.`+ + ' Budgets are calibrated against the full corpus and mean nothing against a smaller one.');+ process.exit(1);+}++// A report left by an earlier run would be read as this one's if the suite died+// before writing anything.+rmSync(REPORT, { force: true });++const run = spawnSync('pnpm', ['exec', 'vitest', 'run', '--config', 'vitest.scale.config.ts'], {+ cwd: join(ROOT, 'desktop'),+ stdio: 'inherit',+ env: { ...process.env, ESTELLA_STRESS_PROJECT: corpus.dir },+});++if (!existsSync(REPORT)) {+ console.error('perf-budget: the scale suite produced no report — it did not reach a single metric.');+ process.exit(1);+}++const report = JSON.parse(readFileSync(REPORT, 'utf8'));+printTable(report);+summarise(report);+process.exit(run.status ?? 1);++// ---------------------------------------------------------------------------++function printTable(report) {+ const { units, metrics } = report;+ console.log('');+ console.log(`Scale budgets — ${countAssets(1).toLocaleString('en-US')} assets`);+ console.log(` one unit on this machine: ${Object.entries(units ?? {})+ .map(([k, v]) => `${k} ${v.toFixed(2)}ms`).join(' ')}`);+ console.log('');++ let group = null;+ for (const m of metrics) {+ if (m.group !== group) {+ group = m.group;+ console.log(` ${group}`);+ }+ const bar = meter(m.pctOfBudget);+ const raw = m.rawMs === null ? '' : ` (${fmt(m.rawMs)}ms)`;+ console.log(` ${status(m.pctOfBudget)} ${m.name.padEnd(52)}`+ + `${String(m.measured).padStart(9)} / ${String(m.budget).padEnd(6)}${unitName(m.unit).padEnd(6)}`+ + ` ${bar} ${String(m.pctOfBudget).padStart(3)}%${raw}`);+ }+ console.log('');++ const over = metrics.filter((m) => m.pctOfBudget > 100);+ const near = metrics.filter((m) => m.pctOfBudget > 70 && m.pctOfBudget <= 100);+ if (over.length) console.log(` ${over.length} over budget.`);+ // Not a failure, and worth saying out loud: a metric drifting into its+ // headroom is the only warning anybody gets before it stops being headroom.+ if (near.length) console.log(` ${near.length} above 70% of budget — headroom is going.`);+ if (!over.length && !near.length) console.log(' All within budget, none above 70%.');+ console.log('');+}++/** The same table as markdown, for the GitHub job summary. */+function summarise(report) {+ const out = process.env.GITHUB_STEP_SUMMARY;+ if (!out) return;+ const lines = [+ `### Scale budgets — ${countAssets(1).toLocaleString('en-US')} assets`,+ '',+ `Calibration on this runner: ${Object.entries(report.units ?? {})+ .map(([k, v]) => `\`${k}\` ${v.toFixed(2)}ms`).join(', ')}`,+ '',+ '| | metric | measured | budget | used | on this runner |',+ '| --- | --- | ---: | ---: | ---: | ---: |',+ ];+ for (const m of report.metrics) {+ lines.push(`| ${m.pctOfBudget > 100 ? '❌' : m.pctOfBudget > 70 ? '⚠️' : '✅'} | ${m.name}`+ + ` | ${m.measured} ${unitName(m.unit)} | ${m.budget} ${unitName(m.unit)}`+ + ` | ${m.pctOfBudget}% | ${m.rawMs === null ? '—' : `${fmt(m.rawMs)}ms`} |`);+ }+ appendFileSync(out, `${lines.join('\n')}\n`);+}++// Declarations, not consts: printTable runs at module top level (above), so an+// arrow bound down here would still be in its temporal dead zone.+function unitName(u) { return u === 'MB' ? 'MB' : u === 'count' ? '' : u; }+function fmt(n) { return n >= 100 ? n.toFixed(0) : n.toFixed(1); }+function status(pct) { return pct > 100 ? 'FAIL' : pct > 70 ? 'near' : ' ok'; }++function meter(pct) {+ const width = 20;+ const filled = Math.min(width, Math.round((pct / 100) * width));+ return `[${'#'.repeat(filled)}${'.'.repeat(width - filled)}]`;+}