A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(aot): a compiled system is paid over what it matches, not over the world
yhh8379ba9f37db
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 750948999..2af5e193b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,6 +16,61 @@ published separately; it ships inside the editor.### Added+- **A compiled system is paid over what it matches, not over the world.** On the+ native road every compiled system was handed EVERY LIVE ENTITY as a candidate+ each frame and resolved every component of each one. The absence of a+ component is the row filter, so the answer was always right — it was the bill+ that was wrong, and it grew with the game rather than with the system. Two+ numbers say it: the same 5,000 movers cost 2.9x more once 45,000 entities no+ query could match were put around them, and at 20,000 entities a system+ matching 200 of them spends 96% of its frame on entities it does not touch.++ A query now walks the shortest column it names. `AotHost.hpp` had said the+ caller "passes the smallest pool it has" since the day it was written; the+ native caller never did, because it had names and `View`'s narrowing takes+ types. `Registry::entitiesWith<T>()` opens that door, EHT emits a second table+ (`engineComponentCandidates`) beside the resolver one, and `ScriptPool` grows+ a slot-to-entity column so a project component can answer the same question —+ which is the half that matters, since the component that narrows a gameplay+ query is usually the project's own.++ Narrowing cannot change which rows a system sees, so the two answers a column+ can give are kept apart: an empty column means nobody has it and the query+ matches nothing, while "this host cannot enumerate that one" means widen. A+ host that answers neither binds and runs exactly as before.++ Measured on the same machine, 20,000 movers in a 200,000-entity world: the+ system's own cost is unchanged from a world with no scenery at all (0.43 ms+ against 0.51 ms), where before the columns existed it was 1.29 ms.++- **`aotCandidates` — what a compiled frame was charged for.** The regression+ above is invisible to everything the engine had: it raises no error, refuses+ no module and changes no pixel. It is also badly served by a clock, because+ the extra work is proportional to the world and on a busy machine that reads+ as a bad minute — measured, the fixed and broken frames above are 1.45 ms and+ 2.31 ms, a 1.6x the noise reaches into. So the host publishes the COUNT+ instead, and the count moves 20,001 to 200,001.++ `bench/aot-native/frame-bench.mjs --gate` asserts it as a second release+ criterion, with scenery in the world — a run whose world IS its matched set+ cannot tell the two apart. Verified in both directions: dropping the candidate+ lookup takes `walked` from 2,000 to 20,001 and the gate goes red.++- **`BENCH_BUILDS`** — export and run one side of the comparison instead of+ both. The interpreted build is where the time goes (seconds per frame at+ 100,000 entities), and a question about the compiled side alone should not pay+ for it.++### Fixed++- **A script component aliased to another entity's row above 2^20 entities.**+ `installNativeAot` spelled its own entity index mask as 20 bits against the+ SDK's real 22, so an entity whose index passed 1,048,575 masked down onto a+ row belonging to a different entity — a wrong component, silently, in a world+ large enough to reach it. It imports the constant now rather than respelling+ it.++- **The no-JIT frame, measured — the number AOT was built for.** Every framenumber this engine had published came from a host with a JIT: V8 on thedesktop, Chromium on a phone. The plan's way to get the other one was a Macdiff --git a/bench/aot-native/frame-bench.mjs b/bench/aot-native/frame-bench.mjsindex 088520864..3c8e73ced 100644--- a/bench/aot-native/frame-bench.mjs+++ b/bench/aot-native/frame-bench.mjs@@ -23,8 +23,14 @@* `bench/aot-frame/project/src/systems.ts` — one source for both benchmarks, so a* ratio measured here and one measured there are about the same code.*+ * and reads the host's own frame clock (native/host/Bench.hpp) off both. It also+ * reads a COUNT off the host — how many entities each compiled system was paid+ * over — because the second thing that can go wrong here is not a slower frame+ * but a frame charged for the whole world (see `--gate`).+ ** node bench/aot-native/frame-bench.mjs* BENCH_ENTITIES=20000 BENCH_BODIES=thin,heavy node bench/aot-native/frame-bench.mjs+ * BENCH_BUILDS=compiled BENCH_BYSTANDERS=95000 node bench/aot-native/frame-bench.mjs*/import { spawnSync } from 'node:child_process';import { copyFileSync, existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from 'node:fs';@@ -38,10 +44,10 @@ const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '.const SYSTEMS = path.join(ROOT, 'bench', 'aot-frame', 'project', 'src', 'systems.ts');/**- * `--gate` turns the benchmark into a release criterion, because this road's- * failure is silent: a system that stops being dispatched to falls back to the- * interpreter by design, raising no error and changing no pixel. A same-machine- * ratio sees that where a millisecond ceiling cannot. See the README.+ * `--gate` makes this two release criteria, for two silent failures. A system+ * that stops being dispatched to falls back to the interpreter BY DESIGN — no+ * error, no pixel, only time — caught by the ratio. One that IS dispatched to+ * can still be paid over the whole world, caught by the count and its scenery.*/const GATE = process.argv.includes('--gate');const ENTITIES = Number(process.env.BENCH_ENTITIES ?? (GATE ? 2000 : 5000));@@ -49,6 +55,14 @@ const FRAMES = Number(process.env.BENCH_FRAMES ?? (GATE ? 60 : 300));const WARMUP = Number(process.env.BENCH_WARMUP ?? (GATE ? 60 : 120));const REPS = Number(process.env.BENCH_REPS ?? (GATE ? 1 : 3));const BODIES = (process.env.BENCH_BODIES ?? 'thin').split(',').map((s) => s.trim()).filter(Boolean);+/**+ * Which of the two builds to export and run. Both by default, because the ratio+ * is the point — but the interpreted one is where the time goes (seconds per+ * frame at 100,000 entities), so a question about the compiled side alone should+ * not pay for it. The report drops the ratio lines when a build is missing.+ */+const BUILDS = (process.env.BENCH_BUILDS ?? 'interpreted,compiled')+ .split(',').map((s) => s.trim()).filter(Boolean);/** Measured at 29x (5000 entities) and ~23x (2000). The bar is what a fallback* cannot pass, not what today happens to reach. */const MIN_RATIO = Number(process.env.BENCH_MIN_RATIO ?? 5);@@ -63,11 +77,12 @@ const KEEP = process.env.BENCH_KEEP === '1';*/const SPRITES = process.env.BENCH_SPRITES !== '0';/**- * Entities carrying only a `Transform`, so no body's query can ever match them.- * They separate the size of the WORLD from the size of the matched set, which is- * the only way to see a dispatcher that scans one to find the other.+ * Entities carrying NO component: the size of the WORLD, separated from the size+ * of the matched set. Bare is load-bearing — a bystander the engine transforms+ * moves the idle floor too, leaving the answer as the difference of two large+ * numbers whose run-to-run spread is bigger than it.*/-const BYSTANDERS = Number(process.env.BENCH_BYSTANDERS ?? 0);+const BYSTANDERS = Number(process.env.BENCH_BYSTANDERS ?? (GATE ? 18000 : 0));/** Which component each body moves, and how to read a position off it. */const BODY = {@@ -156,10 +171,11 @@ const setupSystem = defineSystem([Commands()], (cmds) => {.insert(Pos, { x: 0, y: 0, z: 0 })${SPRITES? "\n .insert(Sprite, { size: { x: 4, y: 4 }, color: { r: 1, g: 1, b: 1, a: 1 } })" : ''};}- // No Mover, so no body's query can match them: they grow the world without- // growing the matched set.+ // NOTHING on them: they grow the world without growing the matched set, and+ // cost the engine's frame nothing. The dispatcher still decides about each —+ // the host's forEachEntity yields every live entity, component or not.for (let i = 0; i < BYSTANDERS; i++) {- cmds.spawn().insert(Transform, { position: { x: 0, y: 0, z: 0 } });+ cmds.spawn();}}, { name: 'BenchSetup' });@@ -275,12 +291,11 @@ try {const project = path.join(out, `${body}-project`);materialize(project, body);const built = {};- for (const compiled of [false, true]) {- const key = compiled ? 'compiled' : 'interpreted';+ for (const key of BUILDS) {process.stdout.write(` exporting ${body} ${key}...\n`);- built[key] = exportApp(project, path.join(out, `${body}-${key}`), compiled);+ built[key] = exportApp(project, path.join(out, `${body}-${key}`), key === 'compiled');}- for (const key of ['interpreted', 'compiled']) {+ for (const key of BUILDS) {// Every rep in its own process, and the FASTEST kept: noise on a desktop// only ever adds time, so the minimum is the closest thing to the cost.let best = null;@@ -304,22 +319,69 @@ console.log(`\nno-JIT frame — QuickJS-ng in the native host, ${ENTITIES} entit// `tick` is the microtask drain, not the `update` call: that one only schedules an// async tick. `draws` is a column because a body can move the render's cost —// `heavy` writes `position.z`, which breaks sprite batching into one draw each.-console.log('body build tick p50 cpu p50 frame p50 draws compiled?');+// `walked` is the entities a compiled system was PAID OVER in one frame, summed+// across its queries. A count and not a clock on purpose — see the ceiling below.+console.log('body build tick p50 cpu p50 frame p50 draws walked compiled?');for (const r of rows) {console.log(`${r.body.padEnd(9)} ${r.key.padEnd(13)} ${`${r.pump.p50.toFixed(3)} ms`.padStart(10)}`+ ` ${`${r.cpu.p50.toFixed(3)} ms`.padStart(10)} ${`${r.frame.p50.toFixed(3)} ms`.padStart(11)}`- + ` ${String(r.draws ?? '?').padStart(7)} ${r.running ?? '—'}`);+ + ` ${String(r.draws ?? '?').padStart(7)} ${String(r.aotCandidates ?? '?').padStart(9)}`+ + ` ${r.running ?? '—'}`);}let bad = 0;console.log('');++// ------------------------------------------------------------------ the ceiling+//+// A compiled system is paid over what it MATCHES, not over the world: the host+// walks the shortest column each query names, or is handed every entity alive.+const WORLD = ENTITIES + BYSTANDERS;+{+ /**+ * How far past the matched set a system may be paid before this is a bug.+ *+ * Slack, not equality: a query names several columns and the shortest is not+ * always the matched set. Wide enough to be a bug and not a tuning knob —+ * 20,000 movers in a 200,000-entity world walked 20,000, or 200,001 unfixed.+ */+ const slack = Number(process.env.BENCH_MAX_WALKED_SLACK ?? 1.5);+ const ceiling = Math.ceil(ENTITIES * slack);+ for (const r of rows) {+ if (r.key !== 'compiled' || r.body === 'idle') continue;+ if (typeof r.aotCandidates !== 'number') continue;+ const line = `${r.body}: walked ${r.aotCandidates} of a ${WORLD}-entity world`+ + ` for the ${ENTITIES} it matches`;+ if (r.aotCandidates > ceiling) {+ console.error(`✗ ${line} — over the ${ceiling} ceiling; a query is not narrowing`);+ bad++;+ } else if (BYSTANDERS > 0) {+ console.log(`${line} — ${(WORLD / Math.max(r.aotCandidates, 1)).toFixed(1)}x fewer`+ + ' entities than the world has');+ }+ }+}// An idle run of the same scene, if one was asked for: the frame minus the system,// so the body's OWN cost can be read off a frame that also renders 5000 sprites.const idleOf = (key) => rows.find((r) => r.body === 'idle' && r.key === key)?.cpu.p50 ?? null;for (const body of [...new Set(rows.map((r) => r.body))]) {const i = rows.find((r) => r.body === body && r.key === 'interpreted');const c = rows.find((r) => r.body === body && r.key === 'compiled');- if (!i || !c) continue;+ // One build alone still has to answer for itself: it must have run compiled+ // if it was the compiled one, and must NOT have if it was the other.+ if (!i || !c) {+ const only = i ?? c;+ // `idle` schedules no system at all, so it has nothing to dispatch to.+ if (only && body !== 'idle' && only.key === 'compiled' && only.running === null) {+ console.error(`✗ ${body}: the compiled build never dispatched to a compiled system`);+ bad++;+ }+ if (only && only.key === 'interpreted' && only.running !== null) {+ console.error(`✗ ${body}: the --no-aot build dispatched to ${only.running}`);+ bad++;+ }+ continue;+ }// Two worlds that computed different things have no ratio between them.if (i.checksum === null || c.checksum === null) {console.error(`✗ ${body}: a run printed no checksum`);diff --git a/docs/ROADMAP.md b/docs/ROADMAP.mdnew file mode 100644index 000000000..7fa95ac5c--- /dev/null+++ b/docs/ROADMAP.md@@ -0,0 +1,152 @@+# Estella Roadmap++> 状态基线:0.58.0(2026-08-27)。这份文件只写**从 CHANGELOG 和提交信息里读不出来的东西**:+> 下一步做什么、为什么是它、以及「做完了」长什么样。逐条改了什么在 CHANGELOG 里。+>+> `docs/REARCH_*.md` 是 gitignore 的,活不过一次 clone;这份是 tracked 的,所以放在这里。++## 现在站在哪++- **AOT 三条路都通了**:web、小游戏、桌面 native。桌面打包的游戏把标了 `@compiled` 的+ 系统当机器码跑,`bench/aot-native/frame-bench.mjs --gate` 让它成为发布准则。+- **无 JIT 的帧第一次被量到**:5,000 实体、一个做三次乘加的系统,**51 ms 解释 → 1.7 ms 编译**,+ 解释器代价在真实帧上是 **61x**(不是裸循环的 154–385x)。+- **3D 地基收口**:render graph 拥有 scene pass、depth 可采样、`render.meshes/triangles/culled`+ 有计数、三条 `*-scale-cost` 有天花板。+- **`examples/celestial-heights/engine-gaps.mjs` 是空的** —— 旗舰游戏没有一处在绕开引擎。+- **子系统定级**:28 个里 6 个 `public`、10 个 `beta`、12 个 `experimental`。++## 1. 编译系统的代价正比于「世界」,不是「它匹配的集合」++**证据。** `native/host/bindings/AotBindings.cpp` 里,每帧、每个编译系统,`es_aot_run`+用 `forEachEntity` 把**所有活实体**推进候选表,`aot::run` 再对每一个解析每个组件。+量到的:同样 5,000 个 mover,世界 5,000 时 0.114 ms,世界 50,000 时 0.333 ms —— **2.9x**,+约 4.9 ns/帧/它根本不碰的实体。解释侧是对照组,几乎不动(49.3 → 51.2 ms),因为 SDK 的+query 会收窄。20k 实体、一个匹配 200 的系统 → 96% 的工作是空转,第二个编译系统再扫一遍。++**为什么它排第一。** 这是唯一一处「编译这条路在某个维度上比解释还差」的地方,而且它随+游戏变大而变糟 —— 正好是 AOT 存在的场景。两处修都不是编译器改动,都在同一个进程里。++**两条 AOT 路要回答的是同一组问题,而 native 这条一个也没答。** `AotDispatch.ts`+(web)的文件头把模型写死了:**一张 row table 是两件事的函数 —— 哪些实体匹配,以及它们的+组件在哪。** 每一件都有一个已经能便宜回答的权威。web 那条路两个都用上了(query cache 的+数组身份 + `World.layoutEpoch`),native 那条路两个都没用 —— 它每帧重扫世界、重打全表。+所以这不是给 native 加一个优化,是把它补齐到 web 已有的那个形状。++**做什么。**++1. **哪些实体匹配 —— 用最短的那一列收窄候选。**〔已落地〕`AotHost::run` 的注释一直写着+ 「caller passes the smallest pool it has」,而 native 这个 caller 从来没做到。+ `Registry::entitiesWith<T>()` 把 `View` 内部一直在用的「取最小池」这件事开成一扇门+ (`View` 拿的是类型,AOT 宿主拿的是名字,所以必须在运行期问);发+ `AotComponents.generated.*` 的那个 generator 顺手发出第二张表。+ script 组件那半需要池自己的 dense 面 —— `ScriptPool` 的 slot 是复用不压缩的,所以+ slot→entity 的表天然就是它,代价是每次 put/delete 一次 O(1) 写。**这一半是必须的**:+ bench 里的 `Mover` 是项目组件,只收窄引擎池什么也买不到。++2. **组件在哪 —— 跨帧复用 row table。**〔待做〕web 那条路量到 27.5 → 不到 2 ns/entity。+ native 侧更值:已经量到「代价不是代码」(thin 20 ns、heavy 21 ns,四次 substep 和四次+ sqrt 与三次乘加同价),也就是说编译系统的代价**几乎全是打表**。++ **开工前必须知道的一件事,否则会写出一个静默错误的缓存。**+ `Registry::layoutEpoch()` 是「地址有没有动」的权威,**不是「谁匹配」的权威** ——+ `SparseSet::emplace` 只在 `components_` 真的 realloc 时才 `++version_`(容量内追加不+ 动已发出的指针,所以不需要 bump)。也就是说**往池里加一个实体可以完全不动 epoch**。+ web 那条路正是因此用了**两个**条件,不是一个。native 侧的对应物:+ - 「谁匹配」= 收窄后那一列的 `(data(), size())` 身份。引擎侧够用:容量内 emplace 会让+ `size()` 变,realloc 会让 `data()` 变或让 epoch 变,`remove`/`clear`/`sort` 无条件+ bump epoch。+ - **script 列不够用。** 删一个 slot 不改 `rows`/`sparse`/`ownerCount`(`next_` 不回退),+ 只在 owners 数组里留一个洞 —— 宿主看到的 span 逐字节同一个。所以 script 那半需要+ `ScriptStorage.layoutEpoch` 真的过界,World 上要开一扇 `scriptLayoutEpoch()`,+ `es_aot_run` 要多收一个参数。+ - 引擎侧的 epoch **不需要**过界:native 宿主就在同一个进程里,直接+ `host().registry->layoutEpoch()`(web 那条路当年还得为此加 embind 绑定)。++ **先做 1 再量 2**,因为 1 做完之后 2 的分母变了。++**完成的样子。** 给 `frame-bench` 加一个 bystander 维度并进 `--gate`:同一台机器、同一个+编译构建,5,000 个 mover 在小世界和大世界里的每实体代价比值回到 ~1.0x。不加这一列,这个+回归和它当初一样是**静默**的 —— 它不报错、不改像素,只是慢。++## 2. AOT 还没到它本来为之而生的平台++**事实。** 拿到 AOT 的是**有 JIT** 的桌面;iOS 和 Android 全解释。这是 2026-08-26+「桌面优先」明确接受的代价(桌面宿主是我们的、能 `dlopen`、导出机就是目标机),但它是+这条路唯一的战略缺口。++**Android 先。** NDK 是个下载,不需要 Apple 硬件;`tools/verify-native-boot.mjs+--platform android` 已经是发布准则,设备那条路已经通了。做的事不是新概念:+`compiler/src/hostCC.ts` 的 seam 从「找宿主编译器」扩成「按 target triple 选编译器」,+`compileTargetFor` 已经是「问宿主要哪条路」的形状,链接标志和模块扩展名那张表也已经在。++**两个已知会咬人的地方**(都是已经付过一次学费的):MSVC 那次说明「导出什么符号」是每个+平台各自的事(`ES_EXPORT` 要按 NDK 再验一次);`LoadLibrary` 那次说明「库从哪里找」也是。++**开工前要定的一件事:iOS 的产物形状。** 桌面走 `dlopen` 一个 `.dll`/`.so`。iOS 上模块+要签名并嵌进 app bundle,导出机必须是 Mac。如果最终形状是「导出产出一个 `.a`,宿主重新+链接」,那 `hostCC` 现在就该按**两种产物**设计,而不是等到 iOS 那步再拆已经绿的桌面路。++**完成的样子。** Android 版的 `verify-aot-native`:「N installed」**和**+「\<name\> is running compiled」两个都要,跟桌面同一个准则 —— installed 不够,一个模块+可以加载成功然后永远不被 dispatch 到。++## 3. 一帧花了多少**时间**,没有任何东西给它天花板++**现状。** 计数天花板有(`sprite-scale-cost` / `mesh-scale-cost` / `shadow-scale-cost`)。+时间的没有:`tools/perf-guard.mjs` 断言的是微基准之间的**比值**(CPU 抵消掉),+`tools/perf-budget.mjs` 断言的是 50k 资产工程上**操作**的上限。两个都不看渲染的一帧。++**为什么是现在。** `WebGPUDevice::setPresentUncapped` 刚好为这件事造出来。在它之前,+Fifo 把每一帧量成刷新间隔的整数倍 —— 51 ms 和 1.7 ms 读作「52 ms」和「16.5 ms」,+30x 被量化成 3x。任何时间门禁在那之前都没有意义。++**形状:同机比值,不是毫秒。** 跟 AOT 那个门禁同一个理由 —— 毫秒天花板在别人的机器上+要么松到抓不住东西,要么随机红。++**顺带还欠一个分母**:阴影图集的图块数对光源数。`shadow-scale-cost` 曾经带过+`render.shadow.tiles` 的上限,被拿掉了,因为那个场景里没有任何东西能把它推过 8 ——+它由 cascade 数界定,不由 caster 数界定。要预算它得换一个分母。++## 4. 重读 `rendering` 的判决,顺手解锁 fog / DOF / SSAO++**`rendering` 的 `why` 已经过期一半。** 它写的是「同 AI —— 而且它背后的 render graph+还在动」。后半句不成立了:scene 是声明出来的 pass、target pool 有生命周期、depth 可采样。+唯一没做的(shadow pass 不声明)已经判过「低价值、真风险、别开」。**要么让它毕业,要么把+`why` 改成还成立的那句** —— 一条过期的理由比没有理由更糟,因为它看起来像是被想过的。++**同一批里最便宜的四个。** `tilemap`、`particles`、`mesh`、`i18n` 的 `why` 都是+「有游戏端到端认证了,但还没被拿来定级」。那是一次**读**,不是一次**建**。12 个+experimental 对创作者读起来是「大半个引擎没做完」,哪怕其中几个下面压着两个出货游戏。++**一个便宜且已标好的改动。** `src/esengine/renderer/frame/FrameConstants.hpp` 只有+`viewProjection`(相机位置是从它反解出来的),没有 near/far。fog 和 DOF 要线性深度。+那个头文件自己写着「后面的成员往后追加不需要 std140 padding」。改完之后 SSAO / fog / DOF+就都是普通 shader 活,不是引擎活。++## 5. 两处小的,但正在削弱「清单是真的」++- **`design-frame` 在这台机器上从来没绿过。** HEAD、加它那个提交的父提交、它自己那个提交+ (`1c8dadaac`)全红。它量的是编辑器窗口大小的函数:`ESTELLA_WIN_W/H` 默认 1480x920 红,+ 1920x1200 绿。把窗口钉成绿的那个尺寸 = 拿环境去迁就断言。要么把 8% 那个门槛从实际视口+ 推出来,要么钉住窗口**并重新标定其它检查**。一个从没绿过的检查不是检查。+- **`tools/releaseGate.mjs` 的 `RELEASE` 还是 `'0.57'`**,而 0.58.0 已经出了。那个文件+ 存在的全部理由就是防止清单漂移。++## 明确不做的(省得重新评估)++- **shadow pass 变成声明的 pass。** 它本来要买的东西(可采样 depth、atlas 的生命周期)+ 已经到手了 —— 前者根本不需要它,后者靠 `rg::TargetPool` 的 per-frame 出借拿到。剩下的+ 是一个声明点加一个 graph 拥有的 atlas,而代价是第二条 draw list + 第二个 transient pool。+ **没有新理由就别开。**+- **本端预测 + 和解、relevancy / 兴趣管理、带宽量化。** RC11 的余项,说好需求驱动。+- **`ai` / `networking` 的定级。** 刻意排在后面:0.50 冻的是「一个游戏由什么构成」,+ 建立在其上的层要等下面这些定下来。++## 建议的顺序++**1 → 5 → 4 → 3 → 2。**++先把量到的浪费修掉(1),顺手清掉两处假的清单项(5),再做一次定级重读 + FrameConstants+(4,这两个是「读」和「一个成员」,不是工程),然后是时间门禁(3),最后开 Android(2,+那是以周计的)。diff --git a/native/host/Bench.cpp b/native/host/Bench.cppindex 9f95383b5..f434815d4 100644--- a/native/host/Bench.cpp+++ b/native/host/Bench.cpp@@ -48,6 +48,10 @@ struct BenchState {long seen = 0; // frames entered, warmup includedSpan update, pump, cpu, frame;unsigned draws = 0, sprites = 0;+ /** This frame's running total, and the last timed frame's — the same shape+ * `draws` has, so the report reads one frame rather than an average that+ * hides a system which stopped being dispatched to halfway through. */+ unsigned aotCandidates = 0, aotCandidatesFrame = 0;};long envLong(const char* name, long fallback) {@@ -113,7 +117,8 @@ void report() {"\"pump\":{\"min\":%.4f,\"p50\":%.4f,\"p90\":%.4f},""\"cpu\":{\"min\":%.4f,\"p50\":%.4f,\"p90\":%.4f},""\"frame\":{\"min\":%.4f,\"p50\":%.4f,\"p90\":%.4f},"- "\"draws\":%u,\"sprites\":%u,\"tickShareOfCpu\":%.4f}",+ "\"draws\":%u,\"sprites\":%u,\"aotCandidates\":%u,"+ "\"tickShareOfCpu\":%.4f}",// The FRAME span's count is the authority on how many frames were// sampled: a frame can be abandoned after it began and reach no other span.s.label.c_str(), s.warmup, s.frame.ms.size(), s.dt,@@ -121,7 +126,7 @@ void report() {at(s.pump.ms, 0.0), p50, at(s.pump.ms, 0.9),at(s.cpu.ms, 0.0), c50, at(s.cpu.ms, 0.9),at(s.frame.ms, 0.0), at(s.frame.ms, 0.5), at(s.frame.ms, 0.9),- s.draws, s.sprites, share);+ s.draws, s.sprites, s.aotCandidatesFrame, share);}} // namespace@@ -138,6 +143,9 @@ double benchDelta(double wall) {void benchFrameBegin() {BenchState& s = state();if (!s.asked || s.reported) return;+ // Reset before the frame, not after: several compiled systems each add to it+ // and the total is what one frame was paid over.+ s.aotCandidates = 0;++s.seen;if (timing()) {s.frame.begin();@@ -157,6 +165,15 @@ void benchNoteDraws(unsigned draws, unsigned sprites) {state().sprites = sprites;}+void benchNoteAotCandidates(unsigned candidates) {+ BenchState& s = state();+ if (!s.asked || s.reported) return;+ s.aotCandidates += candidates;+ // Kept even during warmup: cheap, and it means the report has a number even+ // if the last timed frame happened to dispatch to nothing.+ s.aotCandidatesFrame = s.aotCandidates;+}+bool benchFrameEnd() {BenchState& s = state();if (!s.asked || s.reported) return false;diff --git a/native/host/Bench.hpp b/native/host/Bench.hppindex 95ae30e68..84de1d2f2 100644--- a/native/host/Bench.hpp+++ b/native/host/Bench.hpp@@ -86,6 +86,15 @@ void benchBeforePresent();* a cost that lands somewhere the report cannot see reads as the system's. */void benchNoteDraws(unsigned draws, unsigned sprites);+/**+ * Entities a compiled system was paid over, added to this frame's total.+ *+ * A count beside the clocks: a dispatcher back to offering every entity alive+ * makes a frame dearer in proportion to the WORLD, which on a busy machine reads+ * as noise. This is exact, so a gate can assert it with no tolerance.+ */+void benchNoteAotCandidates(unsigned candidates);+/** Close the frame, and report once the last timed one is in.* @return true when the host was asked to quit and the report is out. */bool benchFrameEnd();diff --git a/native/host/bindings/AotBindings.cpp b/native/host/bindings/AotBindings.cppindex 7498ac4a0..202a1f183 100644--- a/native/host/bindings/AotBindings.cpp+++ b/native/host/bindings/AotBindings.cpp@@ -8,11 +8,14 @@* nothing in QuickJS can hold, so the rows are packed on this side and* JS says only which system to run.*- * What this owns is the naming. `aot::Dispatcher` does the loading and- * the calling; the engine's own components come from the generated- * table; and a project's own live in the pool the SDK allocated out of- * the shared heap, so JS reports where those rows are — by OFFSET,- * which is the one currency both sides already speak (heap.hpp).+ * What this owns is the naming, in both directions. Where one component+ * is for an entity, and WHO holds it — the second is what lets a query+ * narrow to its shortest column instead of being offered the world.+ * `aot::Dispatcher` does the loading and the calling; the engine's own+ * components come from the generated table; and a project's own live in+ * the pool the SDK allocated out of the shared heap, so JS reports where+ * those rows are — by OFFSET, which is the one currency both sides+ * already speak (heap.hpp).** Those reports land in slots with stable addresses, because a pool* reallocates as it grows and a resolver bound at install would keep@@ -26,6 +29,7 @@*/#include "Bindings.hpp"+#include "Bench.hpp"#include "heap.hpp"#include "esn_shim.hpp"@@ -55,7 +59,8 @@ struct AotState {std::deque<aot::RowSpan> scriptSpans;std::unordered_map<std::string, aot::RowSpan*> scriptByName;std::unordered_map<std::string, void*> resources;- /** Candidates for a call, rebuilt per run. Kept so a frame allocates none. */+ /** The fallback list, rebuilt per run for a system that cannot narrow.+ * Kept so a frame that needs it allocates none. */std::vector<std::uint32_t> candidates;bool installed = false;};@@ -88,6 +93,20 @@ aot::ComponentAt componentAt(const char* name) {return found == s.scriptByName.end() ? aot::ComponentAt{} : aot::fromMovingRows(found->second);}+/**+ * WHO has that component — the half of naming that decides what a frame COSTS.+ * Without it a system matching two hundred of twenty thousand pays for 19,800.+ *+ * A script column says nullopt until the SDK reports its owner table: "cannot+ * say", not "empty", so the caller widens rather than running over nothing.+ */+aot::CandidatesOf candidatesFor(const char* name) {+ if (aot::isEngineComponent(name)) return aot::engineComponentCandidates(*host().registry, name);+ AotState& s = state();+ auto found = s.scriptByName.find(name);+ return found == s.scriptByName.end() ? aot::CandidatesOf{} : aot::fromMovingOwners(found->second);+}+void* resourceAt(const char* name) {AotState& s = state();auto found = s.resources.find(name);@@ -111,7 +130,7 @@ JSValue js_install(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv) {AotState& s = state();std::string why;const bool ok = s.dispatcher.install(open, aot::abiHash(ES_ENGINE_ABI_DIGEST),- componentAt, resourceAt, &why);+ componentAt, resourceAt, &why, candidatesFor);if (!ok) ESHOST_LOGE("[aot] %s: %s", path, why.c_str());JS_FreeCString(ctx, path);s.installed = ok;@@ -137,33 +156,47 @@ JSValue js_bound(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv) {}/**- * es_aot_script_rows(name, sparseOff, sparseCount, rowsOff, stride, indexMask)+ * es_aot_script_rows(name, sparseOff, sparseCount, rowsOff, stride, indexMask,+ * ownersOff, ownerCount)*- * Where a project component's rows are NOW. Called again whenever the pool- * moves them; passing 0 for the row offset says the pool is gone.+ * Where a project component's rows are NOW, and who is in them. Re-reported+ * whenever the pool moves them or its membership changes; the SDK keys both on+ * its layout epoch. A 0 row offset says the pool is gone, a 0 owner offset says+ * the column cannot be enumerated and its systems widen.*/JSValue js_scriptRows(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv) {if (argc < 6) return JS_FALSE;const char* name = JS_ToCString(ctx, argv[0]);if (name == nullptr) return JS_FALSE;std::uint32_t sparseOff = 0, sparseCount = 0, rowsOff = 0, stride = 0, mask = 0;+ std::uint32_t ownersOff = 0, ownerCount = 0;JS_ToUint32(ctx, &sparseOff, argv[1]);JS_ToUint32(ctx, &sparseCount, argv[2]);JS_ToUint32(ctx, &rowsOff, argv[3]);JS_ToUint32(ctx, &stride, argv[4]);JS_ToUint32(ctx, &mask, argv[5]);+ if (argc >= 8) {+ JS_ToUint32(ctx, &ownersOff, argv[6]);+ JS_ToUint32(ctx, &ownerCount, argv[7]);+ }// Range-checked through the heap rather than trusted: an offset is a number// JS computed, and a wrong one here is a read outside the arena.void* sparse = heapPtr(sparseOff, static_cast<std::size_t>(sparseCount) * sizeof(std::uint32_t));void* rows = rowsOff == 0 ? nullptr : heapPtr(rowsOff, stride);+ void* owners = ownersOff == 0 ? nullptr+ : heapPtr(ownersOff, static_cast<std::size_t>(ownerCount) * sizeof(std::uint32_t));aot::RowSpan* slot = slotFor(name);JS_FreeCString(ctx, name);if (sparse == nullptr && sparseCount > 0) return JS_FALSE;+ // An owner table that does not check out is DROPPED, not refused: narrowing+ // is an optimisation and must never be the thing that fails a frame.+ if (owners == nullptr) ownerCount = 0;*slot = aot::RowSpan{static_cast<const std::uint32_t*>(sparse), sparseCount,static_cast<unsigned char*>(rows), stride, mask,+ static_cast<const std::uint32_t*>(owners), ownerCount,};return JS_TRUE;}@@ -185,9 +218,10 @@ JSValue js_resource(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv)/*** es_aot_run(i) -> records this host did not know, or -1 when nothing ran.*- * The candidates are every live entity: a row survives only where every- * component resolved, so the completeness check IS the filter. Narrowing it to- * a pool would need the smallest one, and which that is depends on the query.+ * Each query narrows itself to the shortest column it names; the list built here+ * is only the FALLBACK for one that cannot (`aot::narrowest`). When it was the+ * only road a compiled system cost the size of the WORLD — 2.9x for the same+ * 5,000 movers with 45,000 unmatchable entities around them.*/JSValue js_run(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv) {if (argc < 1) return JS_NewInt32(ctx, -1);@@ -197,10 +231,15 @@ JSValue js_run(JSContext* ctx, JSValueConst, int argc, JSValueConst* argv) {if (!s.installed) return JS_NewInt32(ctx, -1);s.candidates.clear();- host().registry->forEachEntity([&s](esengine::Entity e) { s.candidates.push_back(e.id()); });+ if (!s.dispatcher.narrowsAt(i)) {+ host().registry->forEachEntity([&s](esengine::Entity e) { s.candidates.push_back(e.id()); });+ }const auto cmds = s.dispatcher.run(i, s.candidates, resourceAt);// Asked AFTER the run, because that is where a late binding is settled.if (!s.dispatcher.boundAt(i)) return JS_NewInt32(ctx, -1);+ // What this system was paid over, for the bench and for the gate that reads+ // it. A no-op unless a bench was asked for.+ benchNoteAotCandidates(s.dispatcher.candidatesWalked());// After the call, never during it: a despawn invalidates the rows it read.return JS_NewInt32(ctx, static_cast<int>(aot::applyCommands(*host().registry, cmds)));}@@ -225,7 +264,7 @@ void registerAotBindings(HostState& h, JSValue global) {fn("es_aot_install", js_install, 1);fn("es_aot_index", js_index, 1);fn("es_aot_bound", js_bound, 1);- fn("es_aot_script_rows", js_scriptRows, 6);+ fn("es_aot_script_rows", js_scriptRows, 8);fn("es_aot_resource", js_resource, 3);fn("es_aot_run", js_run, 1);fn("es_aot_reset", js_reset, 0);diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.mdindex 0a256a343..5d7662e9e 100644--- a/sdk/etc/index.api.md+++ b/sdk/etc/index.api.md@@ -9322,7 +9322,7 @@ tabIndex: number | undefined@internal resolveHas: (component: AnyComponentDef) => ((entity: Entity) => boolean) | null@internal resolveSetter: (component: AnyComponentDef) => ((entity: Entity, data: unknown) => void) | null@internal resumeIteration: (depth: number) => void-@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; } | undefined+@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; owners: number; ownerCount: number; } | undefined@internal setQueryCostEnabled: (enabled: boolean) => void@internal suspendIteration: () => number@internal useScriptPoolMemory: (memory: PoolMemory) => voiddiff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.mdindex d42437037..539553dd0 100644--- a/sdk/etc/index.native.api.md+++ b/sdk/etc/index.native.api.md@@ -9515,7 +9515,7 @@ tabIndex: number | undefined@internal resolveHas: (component: AnyComponentDef) => ((entity: Entity) => boolean) | null@internal resolveSetter: (component: AnyComponentDef) => ((entity: Entity, data: unknown) => void) | null@internal resumeIteration: (depth: number) => void-@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; } | undefined+@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; owners: number; ownerCount: number; } | undefined@internal setQueryCostEnabled: (enabled: boolean) => void@internal suspendIteration: () => number@internal useScriptPoolMemory: (memory: PoolMemory) => voiddiff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.mdindex cf5231c34..a9d475410 100644--- a/sdk/etc/index.node.api.md+++ b/sdk/etc/index.node.api.md@@ -9317,7 +9317,7 @@ tabIndex: number | undefined@internal resolveHas: (component: AnyComponentDef) => ((entity: Entity) => boolean) | null@internal resolveSetter: (component: AnyComponentDef) => ((entity: Entity, data: unknown) => void) | null@internal resumeIteration: (depth: number) => void-@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; } | undefined+@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; owners: number; ownerCount: number; } | undefined@internal setQueryCostEnabled: (enabled: boolean) => void@internal suspendIteration: () => number@internal useScriptPoolMemory: (memory: PoolMemory) => voiddiff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.mdindex 9b6c8b511..5c3e8c3b1 100644--- a/sdk/etc/minigame.api.md+++ b/sdk/etc/minigame.api.md@@ -9569,7 +9569,7 @@ tabIndex: number | undefined@internal resolveHas: (component: AnyComponentDef) => ((entity: Entity) => boolean) | null@internal resolveSetter: (component: AnyComponentDef) => ((entity: Entity, data: unknown) => void) | null@internal resumeIteration: (depth: number) => void-@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; } | undefined+@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; owners: number; ownerCount: number; } | undefined@internal setQueryCostEnabled: (enabled: boolean) => void@internal suspendIteration: () => number@internal useScriptPoolMemory: (memory: PoolMemory) => voiddiff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.mdindex ccb239b59..259be8f14 100644--- a/sdk/etc/wechat.api.md+++ b/sdk/etc/wechat.api.md@@ -9345,7 +9345,7 @@ tabIndex: number | undefined@internal resolveHas: (component: AnyComponentDef) => ((entity: Entity) => boolean) | null@internal resolveSetter: (component: AnyComponentDef) => ((entity: Entity, data: unknown) => void) | null@internal resumeIteration: (depth: number) => void-@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; } | undefined+@internal scriptSpanOf: (component: AnyComponentDef) => { rows: number; stride: number; sparse: number; sparseCount: number; owners: number; ownerCount: number; } | undefined@internal setQueryCostEnabled: (enabled: boolean) => void@internal suspendIteration: () => number@internal useScriptPoolMemory: (memory: PoolMemory) => voiddiff --git a/sdk/src/ecs/ScriptPool.ts b/sdk/src/ecs/ScriptPool.tsindex 5e722a1fd..62091c9f6 100644--- a/sdk/src/ecs/ScriptPool.ts+++ b/sdk/src/ecs/ScriptPool.ts@@ -67,6 +67,14 @@ export const HEAP_MEMORY: PoolMemory = {/** Absent, in the sparse table: a slot is stored as `slot + 1`. */export const POOL_ABSENT = 0;+/**+ * A slot no entity holds, in the owner table. `Entity::INVALID_RAW` on the C+++ * side, which is the one raw handle the world never hands out — so a host+ * walking the column reads a hole as an entity nothing resolves for, and its+ * existing completeness check drops the row.+ */+export const POOL_NO_OWNER = 0xffffffff;+/*** The fields a pool can back, or null when the defaults are not all scalars.* Anything else — a string, an asset ref, a nested object, an array — has no@@ -101,6 +109,11 @@ export class ScriptPool {* in the same shape, and the reason a host needs no call to resolve a row. */private sparse_: Uint32Array;private sparseLen_: number;+ private ownersBlock_: PoolBlock;+ /** Slot -> the entity holding it, so a host can ask WHO is in this column.+ * The sparse table answers the other direction and cannot answer this one,+ * and without it a compiled system has to be offered every entity alive. */+ private owners_: Uint32Array;private capacity_: number;private readonly slotOf_ = new Map<Entity, number>();private readonly views_ = new Map<Entity, Record<string, unknown>>();@@ -121,9 +134,12 @@ export class ScriptPool {this.rows_ = memory_.alloc(this.capacity_ * this.stride);this.sparseBlock_ = memory_.alloc(this.capacity_ * 4);this.sparseLen_ = this.capacity_;+ this.ownersBlock_ = memory_.alloc(this.capacity_ * 4);this.slots_ = EMPTY_F64;this.sparse_ = EMPTY_U32;+ this.owners_ = EMPTY_U32;this.rebuild_();+ this.owners_.fill(POOL_NO_OWNER);}/**@@ -133,12 +149,20 @@ export class ScriptPool {** Valid until the next growth, which is per call anyway.*/- span(): { rows: number; stride: number; sparse: number; sparseCount: number } {+ span(): {+ rows: number; stride: number; sparse: number; sparseCount: number;+ owners: number; ownerCount: number;+ } {return {rows: this.rows_.byteOffset,stride: this.stride,sparse: this.sparseBlock_.byteOffset,sparseCount: this.sparse_.length,+ owners: this.ownersBlock_.byteOffset,+ // The high-water mark, not the size: slots are reused rather than+ // compacted, so the used prefix has holes and stopping at `size`+ // would stop short of live rows sitting above a freed one.+ ownerCount: this.next_,};}@@ -165,6 +189,8 @@ export class ScriptPool {this.capacity_ * this.fields.length);this.sparse_ = new Uint32Array(this.bufferOf_(this.sparseBlock_),this.sparseBlock_.byteOffset, this.sparseLen_);+ this.owners_ = new Uint32Array(this.bufferOf_(this.ownersBlock_),+ this.ownersBlock_.byteOffset, this.capacity_);}private bufferOf_(block: PoolBlock): ArrayBufferLike {@@ -220,6 +246,8 @@ export class ScriptPool {slot = this.claim_(entity);this.slotOf_.set(entity, slot);this.mark_(entity, slot + 1);+ // After `claim_`, which may have grown the pool and rebuilt this view.+ this.owners_[slot] = entity;this.views_.set(entity, this.makeView_(slot));// A fresh row is seeded whole; an existing one keeps the fields the// caller did not name, which is what `world.set` has always done.@@ -235,6 +263,7 @@ export class ScriptPool {this.slotOf_.delete(entity);this.views_.delete(entity);this.mark_(entity, POOL_ABSENT);+ this.owners_[slot] = POOL_NO_OWNER;this.free_.push(slot);return true;}@@ -262,17 +291,28 @@ export class ScriptPool {private growRows_(): void {const want = this.capacity_ * 2;const block = this.memory_.alloc(want * this.stride);+ // The owner table is indexed by SLOT, so it grows with the rows and on+ // the same schedule — allocated here so one growth is one pair of+ // allocations, and the views below are built once over both.+ const ownerBlock = this.memory_.alloc(want * 4);// Before reading the old rows, not after: allocating may have grown the// heap, and a grown heap detaches the view this is about to copy FROM.this.rebuild_();const wider = new Float64Array(this.bufferOf_(block), block.byteOffset,want * this.fields.length);wider.set(this.slots_);+ const widerOwners = new Uint32Array(this.bufferOf_(ownerBlock), ownerBlock.byteOffset, want);+ widerOwners.fill(POOL_NO_OWNER);+ widerOwners.set(this.owners_);const old = this.rows_;+ const oldOwners = this.ownersBlock_;this.capacity_ = want;this.rows_ = block;this.slots_ = wider;+ this.ownersBlock_ = ownerBlock;+ this.owners_ = widerOwners;this.memory_.release(old);+ this.memory_.release(oldOwners);// Views hold a slot index, not a reference into the old array, so they// survive this. An ADDRESS taken before it does not, which is why// `address` says so.diff --git a/sdk/src/ecs/aot/installNativeAot.ts b/sdk/src/ecs/aot/installNativeAot.tsindex 13f2d2e7b..07871fad0 100644--- a/sdk/src/ecs/aot/installNativeAot.ts+++ b/sdk/src/ecs/aot/installNativeAot.ts@@ -22,7 +22,7 @@ import { getComponent, type AnyComponentDef } from '../component';import { log } from '../../util/logger';import type { World } from '../world';import type { SystemRunner } from '../system';-import type { Entity } from '../../types';+import { ENTITY_INDEX_MASK, type Entity } from '../../types';import { WasmPoolMemory, type WasmHeap } from '../WasmPoolMemory';import { AotSystems, type AotManifest, type AotTwin } from './AotSystems';import { AotResources, type ResourceReader } from './AotResources';@@ -37,7 +37,8 @@ export interface NativeAotBindings {index(name: string): number;bound(index: number): boolean;scriptRows(name: string, sparseOffset: number, sparseCount: number,- rowsOffset: number, stride: number, indexMask: number): boolean;+ rowsOffset: number, stride: number, indexMask: number,+ ownersOffset: number, ownerCount: number): boolean;resource(name: string, offset: number, bytes: number): boolean;run(index: number): number;reset(): void;@@ -69,8 +70,12 @@ function declaredResourceLayouts(manifest: AotManifest): Map<string, readonly streturn out;}-/** The entity index mask a sparse table is addressed by, from the engine's own. */-const ENTITY_INDEX_MASK = 0xfffff;+/**+ * The mask a sparse table is addressed by — the SDK's own, imported rather than+ * spelled again. It was spelled again here, as a 20-bit mask against the real+ * 22-bit one, which aliased every entity above index 2^20 onto another row.+ */+const SPARSE_INDEX_MASK = ENTITY_INDEX_MASK;/** Read the `es_aot_*` globals a native host bound, or null where it bound none. */export function nativeAotBindings(@@ -236,7 +241,7 @@ class NativeAotDispatch implements AotDispatcher {const span = def === undefined ? undefined : this.world.scriptSpanOf(def);if (span === undefined) continue;this.bindings.scriptRows(name, span.sparse, span.sparseCount,- span.rows, span.stride, ENTITY_INDEX_MASK);+ span.rows, span.stride, SPARSE_INDEX_MASK, span.owners, span.ownerCount);}}diff --git a/sdk/src/ecs/world.ts b/sdk/src/ecs/world.tsindex b415e4e4e..64f74914b 100644--- a/sdk/src/ecs/world.ts+++ b/sdk/src/ecs/world.ts@@ -1203,8 +1203,10 @@ export class World {* walks it in its own address space. Undefined for an engine component,* whose column belongs to the registry and is reached by its own door.*/- scriptSpanOf(component: AnyComponentDef):- { rows: number; stride: number; sparse: number; sparseCount: number } | undefined {+ scriptSpanOf(component: AnyComponentDef): {+ rows: number; stride: number; sparse: number; sparseCount: number;+ owners: number; ownerCount: number;+ } | undefined {if (isBuiltinComponent(component)) return undefined;return this.scripts_.poolFor(component._id as symbol)?.span();}diff --git a/sdk/tests/script-pool.test.ts b/sdk/tests/script-pool.test.tsindex 57e6bcd59..2eaeb8048 100644--- a/sdk/tests/script-pool.test.ts+++ b/sdk/tests/script-pool.test.ts@@ -11,7 +11,10 @@* component still reads and writes as an object.*/import { describe, it, expect } from 'vitest';-import { ScriptPool, poolShape, POOL_SLOT_BYTES, POOL_ABSENT, HEAP_MEMORY, type PoolMemory } from '../src/ecs/ScriptPool';+import {+ ScriptPool, poolShape, POOL_SLOT_BYTES, POOL_ABSENT, POOL_NO_OWNER, HEAP_MEMORY,+ type PoolMemory,+} from '../src/ecs/ScriptPool';import { entityIndex, makeEntity } from '../src/types';import type { Entity } from '../src/types';@@ -135,27 +138,27 @@ describe('a row, and the view onto it', () => {* the module has no way to reach it. So the rows come out of one block the host* owns, and an address is an offset into THAT.*/-describe('rows carved out of a host-owned block', () => {- /** Stands in for the wasm heap: one buffer, a bump pointer, nothing freed. */- function heapOf(bytes: number): { memory: PoolMemory; all: Float64Array; used: () => number } {- const block = new ArrayBuffer(bytes);- const all = new Float64Array(block);- let at = 64; // a nonzero start, so an address that forgot byteOffset is wrong- return {- all,- used: () => at,- memory: {- alloc(want) {- const byteLength = (want + 7) & ~7;- const out = { buffer: block, byteOffset: at, byteLength };- at += byteLength;- return out;- },- release: () => { /* a bump allocator frees nothing */ },+/** Stands in for the wasm heap: one buffer, a bump pointer, nothing freed. */+function heapOf(bytes: number): { memory: PoolMemory; all: Float64Array; used: () => number } {+ const block = new ArrayBuffer(bytes);+ const all = new Float64Array(block);+ let at = 64; // a nonzero start, so an address that forgot byteOffset is wrong+ return {+ all,+ used: () => at,+ memory: {+ alloc(want) {+ const byteLength = (want + 7) & ~7;+ const out = { buffer: block, byteOffset: at, byteLength };+ at += byteLength;+ return out;},- };- }+ release: () => { /* a bump allocator frees nothing */ },+ },+ };+}+describe('rows carved out of a host-owned block', () => {it('an address is an offset into the block, not into the pool', () => {const heap = heapOf(1 << 16);const pool = new ScriptPool(poolShape(DEFAULTS)!, 8, heap.memory);@@ -256,3 +259,80 @@ describe('a pool handed over as memory', () => {expect(pool.sparseTable[entityIndex(e(100000))]).toBe(1);});});++/**+ * The other direction. The sparse table says which slot an entity is in; a host+ * running a compiled system needs who is in the column at all, and without it+ * has to offer every entity alive — making a system cost the size of the world.+ * Indexed by SLOT, so it is that table read backwards.+ */+describe('a pool that can say who is in it', () => {+ /** The owner column as the host reads it: from the shared block, at the+ * offset and length `span()` reports. */+ function owners(pool: ScriptPool, all: Float64Array): Uint32Array {+ const { owners: at, ownerCount } = pool.span();+ return new Uint32Array(all.buffer, at, ownerCount);+ }++ it('names the entity in each slot, and only as far as slots were claimed', () => {+ const heap = heapOf(1 << 16);+ const pool = new ScriptPool(poolShape(DEFAULTS)!, 8, heap.memory);+ pool.put(e(5), DEFAULTS);+ pool.put(e(9), DEFAULTS);++ // Two claimed of a capacity of eight: the count is the high-water mark,+ // not the capacity, so a host walks two entries rather than eight.+ expect(pool.span().ownerCount).toBe(2);+ const table = owners(pool, heap.all);+ expect(table[0]).toBe(e(5));+ expect(table[1]).toBe(e(9));+ });++ it('a freed slot becomes a hole rather than shifting the ones above it', () => {+ const heap = heapOf(1 << 16);+ const pool = new ScriptPool(poolShape(DEFAULTS)!, 8, heap.memory);+ for (let i = 1; i <= 3; i++) pool.put(e(i), DEFAULTS);+ pool.delete(e(2));++ // Slots are reused, not compacted — a view holds a slot index and one+ // that moved under it is a bug three frames later. So the column keeps+ // its length and the freed entry says nobody.+ const table = owners(pool, heap.all);+ expect(pool.span().ownerCount).toBe(3);+ expect(table[0]).toBe(e(1));+ expect(table[1]).toBe(POOL_NO_OWNER);+ expect(table[2]).toBe(e(3));++ // And the next put fills the hole, because the free list is LIFO.+ pool.put(e(4), DEFAULTS);+ expect(owners(pool, heap.all)[1]).toBe(e(4));+ expect(pool.span().ownerCount).toBe(3);+ });++ it('survives a growth with every owner intact and the new slots empty', () => {+ const heap = heapOf(1 << 20);+ const pool = new ScriptPool(poolShape(DEFAULTS)!, 2, heap.memory);+ for (let i = 1; i <= 5; i++) pool.put(e(i), DEFAULTS);++ // The column is indexed by slot, so it grows with the rows — read from+ // the block at the NEW offset, since a growth moves it.+ const table = owners(pool, heap.all);+ expect(pool.span().ownerCount).toBe(5);+ for (let i = 1; i <= 5; i++) expect(table[i - 1], `slot ${i - 1}`).toBe(e(i));+ });++ it('the offset it reports is into the same block the rows are in', () => {+ const heap = heapOf(1 << 16);+ const pool = new ScriptPool(poolShape(DEFAULTS)!, 4, heap.memory);+ pool.put(e(3), DEFAULTS, { speed: 12 });++ // What makes the column usable at all: one address space, so the host+ // resolves an owner and then its row with no call back.+ const { owners: at, ownerCount, rows, stride } = pool.span();+ const table = new Uint32Array(heap.all.buffer, at, ownerCount);+ const entity = table[0]!;+ expect(entity).toBe(e(3));+ const slot = pool.sparseTable[entityIndex(entity)]! - 1;+ expect(heap.all[(rows + slot * stride) / POOL_SLOT_BYTES]).toBe(12);+ });+});diff --git a/src/esengine/aot/AotComponents.generated.cpp b/src/esengine/aot/AotComponents.generated.cppindex 48e6ef538..597b7a8e6 100644--- a/src/esengine/aot/AotComponents.generated.cpp+++ b/src/esengine/aot/AotComponents.generated.cpp@@ -5,6 +5,7 @@#include <cstdint>#include <cstring>+#include <span>#include "esengine/ecs/Entity.hpp"#include "esengine/ecs/Registry.hpp"@@ -38,6 +39,24 @@#include "esengine/ecs/components/Velocity.hpp"namespace esengine::aot {+namespace {++/**+ * The dense column of a pool, as raw ids.+ *+ * `Entity` is a four-byte struct whose only member is that id, and+ * Types.hpp static_asserts the size for exactly this — a dense array of+ * them IS an array of ids. Copying it per frame would put the cost back+ * where narrowing took it from.+ */+Candidates denseAsIds(std::span<const Entity> dense) {+ static_assert(sizeof(Entity) == sizeof(std::uint32_t),+ "a dense entity array is only an id array while these agree");+ return std::span<const std::uint32_t>(+ reinterpret_cast<const std::uint32_t*>(dense.data()), dense.size());+}++} // namespaceComponentAt engineComponentAt(ecs::Registry& registry, const char* name) {if (std::strcmp(name, "BitmapText") == 0) {@@ -238,6 +257,205 @@ ComponentAt engineComponentAt(ecs::Registry& registry, const char* name) {return {};}+CandidatesOf engineComponentCandidates(ecs::Registry& registry, const char* name) {+ if (std::strcmp(name, "BitmapText") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::BitmapText>());+ };+ }+ if (std::strcmp(name, "BoxCollider2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::BoxCollider2D>());+ };+ }+ if (std::strcmp(name, "BoxCollider3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::BoxCollider3D>());+ };+ }+ if (std::strcmp(name, "Camera") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Camera>());+ };+ }+ if (std::strcmp(name, "Canvas") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Canvas>());+ };+ }+ if (std::strcmp(name, "CapsuleCollider2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::CapsuleCollider2D>());+ };+ }+ if (std::strcmp(name, "CapsuleCollider3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::CapsuleCollider3D>());+ };+ }+ if (std::strcmp(name, "CharacterController3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::CharacterController3D>());+ };+ }+ if (std::strcmp(name, "Children") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Children>());+ };+ }+ if (std::strcmp(name, "CircleCollider2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::CircleCollider2D>());+ };+ }+ if (std::strcmp(name, "ConvexCollider3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::ConvexCollider3D>());+ };+ }+ if (std::strcmp(name, "Draggable") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Draggable>());+ };+ }+ if (std::strcmp(name, "DragonBonesAnimation") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::DragonBonesAnimation>());+ };+ }+ if (std::strcmp(name, "FlexContainer") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::FlexContainer>());+ };+ }+ if (std::strcmp(name, "Interactable") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Interactable>());+ };+ }+ if (std::strcmp(name, "Light") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Light>());+ };+ }+ if (std::strcmp(name, "MeshCollider3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::MeshCollider3D>());+ };+ }+ if (std::strcmp(name, "MeshRenderer") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::MeshRenderer>());+ };+ }+ if (std::strcmp(name, "MeshSkin") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::MeshSkin>());+ };+ }+ if (std::strcmp(name, "Parent") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Parent>());+ };+ }+ if (std::strcmp(name, "ParticleEmitter") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::ParticleEmitter>());+ };+ }+ if (std::strcmp(name, "ParticleForceField") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::ParticleForceField>());+ };+ }+ if (std::strcmp(name, "RigidBody2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::RigidBody2D>());+ };+ }+ if (std::strcmp(name, "RigidBody3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::RigidBody3D>());+ };+ }+ if (std::strcmp(name, "SegmentCollider2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::SegmentCollider2D>());+ };+ }+ if (std::strcmp(name, "ShadowCaster2D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::ShadowCaster2D>());+ };+ }+ if (std::strcmp(name, "ShapeRenderer") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::ShapeRenderer>());+ };+ }+ if (std::strcmp(name, "SphereCollider3D") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::SphereCollider3D>());+ };+ }+ if (std::strcmp(name, "SpineAnimation") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::SpineAnimation>());+ };+ }+ if (std::strcmp(name, "Sprite") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Sprite>());+ };+ }+ if (std::strcmp(name, "TilemapLayer") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::TilemapLayer>());+ };+ }+ if (std::strcmp(name, "TrailRenderer") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::TrailRenderer>());+ };+ }+ if (std::strcmp(name, "Transform") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Transform>());+ };+ }+ if (std::strcmp(name, "UIInteraction") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::UIInteraction>());+ };+ }+ if (std::strcmp(name, "UIMask") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::UIMask>());+ };+ }+ if (std::strcmp(name, "UINode") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::UINode>());+ };+ }+ if (std::strcmp(name, "UIScroll") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::UIScroll>());+ };+ }+ if (std::strcmp(name, "UIVisual") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::UIVisual>());+ };+ }+ if (std::strcmp(name, "Velocity") == 0) {+ return [®istry]() -> Candidates {+ return denseAsIds(registry.entitiesWith<esengine::ecs::Velocity>());+ };+ }+ return {};+}+bool isEngineComponent(const char* name) {if (std::strcmp(name, "BitmapText") == 0) return true;if (std::strcmp(name, "BoxCollider2D") == 0) return true;diff --git a/src/esengine/aot/AotComponents.generated.hpp b/src/esengine/aot/AotComponents.generated.hppindex 962c95677..2c1e1b7e3 100644--- a/src/esengine/aot/AotComponents.generated.hpp+++ b/src/esengine/aot/AotComponents.generated.hpp@@ -27,6 +27,21 @@ namespace esengine::aot {*/ComponentAt engineComponentAt(ecs::Registry& registry, const char* name);+/**+ * WHO has that component, for a caller narrowing a query to its shortest+ * column.+ *+ * The resolver above answers per entity and can only be asked about one+ * that is already a candidate; this is where candidates come FROM. A host+ * without it must offer every entity, and then what a compiled system+ * costs is the size of the world instead of the size of what it matches.+ *+ * An empty span means nobody has it — a query that matches nothing. The+ * empty FUNCTION means this table does not answer for the name, which is+ * a different answer and the caller must not narrow on it.+ */+CandidatesOf engineComponentCandidates(ecs::Registry& registry, const char* name);+/** Whether this table answers for that name, without building a resolver. */bool isEngineComponent(const char* name);diff --git a/src/esengine/aot/AotDispatcher.hpp b/src/esengine/aot/AotDispatcher.hppindex 7f847c445..a8bbeb506 100644--- a/src/esengine/aot/AotDispatcher.hpp+++ b/src/esengine/aot/AotDispatcher.hpp@@ -44,14 +44,15 @@ public:*/bool install(const char* path, std::uint64_t expected,const ComponentLookup& components, const ResourceLookup& resources,- std::string* why = nullptr) {+ std::string* why = nullptr, const CandidateLookup& candidates = {}) {reset();if (!module_.open(path, expected, why)) return false;components_ = components;resources_ = resources;+ candidates_ = candidates;for (const EsSystemDecl& decl : module_.systems()) {names_.push_back(decl.name != nullptr ? decl.name : "");- bound_.push_back(::esengine::aot::bind(decl, components, resources));+ bound_.push_back(::esengine::aot::bind(decl, components, resources, candidates));}return true;}@@ -65,6 +66,18 @@ public:/** Whether the system at that index resolved. An unbound one runs nothing. */bool boundAt(std::size_t i) const { return i < bound_.size() && bound_[i].fn != nullptr; }+ /**+ * Whether every query of that system names its own candidates, and so+ * whether `run` may be given an EMPTY fallback.+ *+ * Asked per frame, not at install: a script pool exists only once an entity+ * has that component. False when unbound — ignoring it is slow, never wrong.+ */+ bool narrowsAt(std::size_t i) const {+ if (i >= bound_.size() || bound_[i].fn == nullptr) return false;+ return ::esengine::aot::narrows(bound_[i].queries);+ }+/** The index of a declared name, or `npos`. Called at install, not per frame. */std::size_t indexOf(const char* name) const {for (std::size_t i = 0; i < names_.size(); ++i) {@@ -74,11 +87,12 @@ public:}/**- * Run one system over these candidates, and hand back what it wrote.+ * Run one system, and hand back what it wrote.*- * Resources are resolved per call because the contract only promises an- * address for the length of one; the components were resolved at install,- * which is the whole point of binding.+ * `candidates` is the FALLBACK, walked only by a query that could not name+ * its own column — ask {@link narrowsAt} first. Resources are resolved per+ * call because the contract promises an address for the length of one; the+ * components were resolved at install.*/std::span<const EsCmd> run(std::size_t i, std::span<const std::uint32_t> candidates,const ResourceLookup& resources) {@@ -88,18 +102,22 @@ public:// component, so the answer is about the frame, not the module.if (bound_[i].fn == nullptr) {// Qualified: `std::bind` is visible through <functional> and takes anything.- bound_[i] = ::esengine::aot::bind(module_.systems()[i], components_, resources_);+ bound_[i] = ::esengine::aot::bind(module_.systems()[i], components_, resources_, candidates_);}if (bound_[i].fn == nullptr) return {};return runBound(bound_[i], candidates, resources, arena_);}+ /** Entities the last {@link run} was paid over. See CallArena. */+ std::uint32_t candidatesWalked() const { return arena_.candidatesWalked(); }+/** Close the module and forget the bindings — they point into it. */void reset() {bound_.clear();names_.clear();components_ = {};resources_ = {};+ candidates_ = {};module_.close();}@@ -113,6 +131,9 @@ private:* again — the answer changes as the world comes up. */ComponentLookup components_;ResourceLookup resources_;+ /** Optional, and absent on a host that cannot enumerate a column: then every+ * query falls back to the candidates the caller passes. */+ CandidateLookup candidates_;CallArena arena_;};diff --git a/src/esengine/aot/AotHost.hpp b/src/esengine/aot/AotHost.hppindex df11db9dc..bc428b078 100644--- a/src/esengine/aot/AotHost.hpp+++ b/src/esengine/aot/AotHost.hpp@@ -17,12 +17,18 @@* The arena is reused across calls and rebuilt in each one, because* a row array has a one-call lifetime. Nothing here allocates per row* after the first frame at a given size.+ *+ * A row table is a function of two things: which entities match, and+ * where their components are. This answers the first — see+ * {@link narrowest}. The second is open: unlike the web road+ * (sdk/src/ecs/aot/AotDispatch.ts) this repacks every frame.*/#pragma once#include <cstddef>#include <cstdint>#include <functional>+#include <optional>#include <span>#include <utility>#include <vector>@@ -41,11 +47,59 @@ using SystemFn = void (*)(es_addr_t);*/using ComponentAt = std::function<void*(std::uint32_t)>;+/**+ * Which entities a component's storage says it holds, as raw ids.+ *+ * Empty means nobody has it and the query matches nothing; `nullopt` means this+ * host cannot enumerate that column and the caller must WIDEN, not narrow.+ * Reading the second as the first drops every row.+ */+using Candidates = std::optional<std::span<const std::uint32_t>>;++/** A column's members, asked per call because a pool grows between them. */+using CandidatesOf = std::function<Candidates()>;+/** One declared `Query`: its components, in the order the query names them. */struct QuerySpec {std::vector<ComponentAt> comps;+ /**+ * Where each of those components says its members are, in the same order.+ * Empty, or an entry that answers nullopt, simply does not narrow — a+ * resolver is required to run a query and this is not.+ */+ std::vector<CandidatesOf> sources;};+/**+ * The shortest column this query names, or nullopt when none of them could say.+ *+ * An entity missing any one component cannot match, so any column bounds the+ * answer and the shortest bounds it best — {@link ecs::View}'s strategy, reached+ * through names. `run` still filters; this only decides what it is paid over.+ */+inline Candidates narrowest(const QuerySpec& spec) {+ Candidates best;+ for (const CandidatesOf& source : spec.sources) {+ if (!source) continue;+ const Candidates got = source();+ if (!got) continue;+ if (!best || got->size() < best->size()) best = got;+ }+ return best;+}++/**+ * Whether every one of these queries can narrow itself — that is, whether the+ * caller can skip enumerating the world. Asked before it does. No queries at all+ * answers true: nothing would walk the list.+ */+inline bool narrows(std::span<const QuerySpec> queries) {+ for (const QuerySpec& q : queries) {+ if (!narrowest(q)) return false;+ }+ return true;+}+/*** The scratch a call needs. Held by the caller so the vectors keep their* capacity between frames; every `run` overwrites it and nothing survives.@@ -57,6 +111,15 @@ public:/** How many command records the system may write before it must stop. */std::uint32_t commandCapacity() const { return static_cast<std::uint32_t>(cmds_.size()); }+ /**+ * Entities the last `run` was PAID OVER, summed across its queries.+ *+ * This is what a compiled system costs, not the body — measured at 20+ * ns/entity for three multiply-adds and 21 for four substeps with a square+ * root each. A count, because a clock reads a tenfold scan as a busy machine.+ */+ std::uint32_t candidatesWalked() const { return walked_; }+private:friend std::span<const EsCmd> run(SystemFn, std::span<const std::uint32_t>,std::span<const QuerySpec>, std::span<void* const>, CallArena&);@@ -65,21 +128,25 @@ private:std::vector<es_addr_t> resources_;std::vector<EsCmd> cmds_;std::uint32_t count_ = 0;+ std::uint32_t walked_ = 0;EsSysCtx ctx_{};};/*** Materialise, call, and return the commands the system produced. A row is kept- * only when every component resolves for it — `View.hpp`'s strategy exposed, so- * the caller passes the smallest pool it has. Applying the commands is the- * CALLER's job: a despawn here invalidates the rows just handed over.+ * only when every component resolves — an absent one is the filter.+ *+ * `fallback` is walked only by a query that cannot narrow itself; ask+ * {@link narrows} first, since building it is O(world). Applying the commands is+ * the CALLER's job: a despawn here invalidates the rows just handed over.*/inline std::span<const EsCmd> run(SystemFn fn,- std::span<const std::uint32_t> candidates,+ std::span<const std::uint32_t> fallback,std::span<const QuerySpec> queries,std::span<void* const> resources,CallArena& arena) {arena.rows_.clear();+ arena.walked_ = 0;arena.queries_.assign(queries.size(), EsQueryRows{});arena.resources_.assign(resources.size(), es_addr_t{});@@ -91,7 +158,13 @@ inline std::span<const EsCmd> run(SystemFn fn,for (std::size_t q = 0; q < queries.size(); ++q) {starts[q] = arena.rows_.size();const auto& comps = queries[q].comps;- for (std::uint32_t e : candidates) {+ // Its own column where it has one, everybody where it does not. A+ // freed slot in a script column reads as an entity nothing resolves+ // for, so the completeness check below drops it like any other miss.+ const Candidates narrowed = narrowest(queries[q]);+ const std::span<const std::uint32_t> over = narrowed ? *narrowed : fallback;+ arena.walked_ += static_cast<std::uint32_t>(over.size());+ for (std::uint32_t e : over) {const std::size_t mark = arena.rows_.size();arena.rows_.push_back(static_cast<es_addr_t>(e));bool complete = true;@@ -139,8 +212,23 @@ struct RowSpan {/** Entity::Layout::INDEX_MASK, passed rather than included so this header* stays free of the engine and the harness needs no link. */std::uint32_t indexMask = 0;+ /**+ * Slot -> the raw entity holding it, so this column can be asked WHO is in+ * it. The sparse table above answers the other direction only.+ *+ * Null where the owner reported none — the one reason a script component+ * fails to narrow.+ */+ const std::uint32_t* owners = nullptr;+ /** Slots ever claimed. A pool reuses slots rather than compacting, so this+ * is a high-water mark and the array has holes — see {@link NO_OWNER}. */+ std::uint32_t ownerCount = 0;};+/** A slot nobody holds. `Entity::INVALID_RAW`: the one raw value the engine+ * never hands out, so a hole resolves to no row and the filter drops it. */+inline constexpr std::uint32_t NO_OWNER = 0xFFFFFFFFu;+/** One row out of a span. Absent is zero in the table, as it is in a sparse set. */inline void* rowAt(const RowSpan& span, std::uint32_t entity) {const std::uint32_t at = entity & span.indexMask;@@ -168,6 +256,18 @@ inline ComponentAt fromMovingRows(const RowSpan* slot) {};}+/**+ * Who is in that column, read through the same slot for the same reason.+ * nullopt rather than empty when no owner array was reported: "did not say" is+ * not "is empty", and confusing them runs every system over nothing.+ */+inline CandidatesOf fromMovingOwners(const RowSpan* slot) {+ return [slot]() -> Candidates {+ if (slot == nullptr || slot->owners == nullptr) return std::nullopt;+ return std::span<const std::uint32_t>(slot->owners, slot->ownerCount);+ };+}+/*** How a host answers the two questions a declaration asks: where one component* is for an entity, and where one resource is. Both by NAME, because names are@@ -176,6 +276,14 @@ inline ComponentAt fromMovingRows(const RowSpan* slot) {using ComponentLookup = std::function<ComponentAt(const char*)>;using ResourceLookup = std::function<void*(const char*)>;+/**+ * The third question, and the only optional one: who holds that component.+ * Separate from `ComponentLookup` because a host with no answer must still bind+ * and run — narrowing is an optimisation and cannot be allowed to change which+ * rows a system sees.+ */+using CandidateLookup = std::function<CandidatesOf(const char*)>;+/*** A declared system with its resolvers found once. Resources are looked up per* call rather than kept, because a host is free to move one between frames and@@ -194,7 +302,8 @@ struct BoundSystem {*/inline BoundSystem bind(const EsSystemDecl& decl,const ComponentLookup& components,- const ResourceLookup& resources) {+ const ResourceLookup& resources,+ const CandidateLookup& candidates = {}) {BoundSystem out;out.queries.resize(decl.queryCount);for (std::uint32_t q = 0; q < decl.queryCount; ++q) {@@ -203,6 +312,10 @@ inline BoundSystem bind(const EsSystemDecl& decl,ComponentAt at = components(qd.comps[c]);if (!at) return {};out.queries[q].comps.push_back(std::move(at));+ // A column that cannot say leaves a falsy entry rather than none, so+ // `sources` stays index-for-index with `comps` and a reader of one+ // is a reader of the other.+ out.queries[q].sources.push_back(candidates ? candidates(qd.comps[c]) : CandidatesOf{});}}for (std::uint32_t r = 0; r < decl.resourceCount; ++r) {@@ -213,16 +326,17 @@ inline BoundSystem bind(const EsSystemDecl& decl,return out;}-/** Run a bound system, resolving its resources for this call. */+/** Run a bound system, resolving its resources for this call. `fallback` is for+ * the queries that cannot narrow themselves; see {@link run}. */inline std::span<const EsCmd> runBound(const BoundSystem& bound,- std::span<const std::uint32_t> candidates,+ std::span<const std::uint32_t> fallback,const ResourceLookup& resources,CallArena& arena) {if (bound.fn == nullptr) return {};std::vector<void*> addresses;addresses.reserve(bound.resourceNames.size());for (const char* name : bound.resourceNames) addresses.push_back(resources(name));- return run(bound.fn, candidates, bound.queries, addresses, arena);+ return run(bound.fn, fallback, bound.queries, addresses, arena);}/**diff --git a/src/esengine/ecs/Registry.hpp b/src/esengine/ecs/Registry.hppindex 51e4f4cd9..43c9b3a81 100644--- a/src/esengine/ecs/Registry.hpp+++ b/src/esengine/ecs/Registry.hpp@@ -33,6 +33,7 @@#include <any>#include <functional>#include <queue>+#include <span>#include <string>namespace esengine::ecs {@@ -611,6 +612,22 @@ public:pool->rebuildSparse();}+ /**+ * @brief The entities that carry T, densely+ * @return The pool's dense entity array, or an empty span where no pool exists+ *+ * @details What {@link View} narrows to internally, opened for a caller that+ * assembles its query at run time from NAMES, as the AOT host does.+ * Empty means nobody has it — never "ask everybody".+ */+ template<typename T>+ std::span<const Entity> entitiesWith() const {+ const auto* pool = getPool<T>();+ if (pool == nullptr) return {};+ const std::vector<Entity>& dense = pool->entities();+ return std::span<const Entity>(dense.data(), dense.size());+ }+private:// Handed out once per registry and never recycled, so it survives one being// destroyed and the next taking its address. See instanceId().diff --git a/tests/aot/test_aot_host.cpp b/tests/aot/test_aot_host.cppindex 333da6bd4..3f4750179 100644--- a/tests/aot/test_aot_host.cpp+++ b/tests/aot/test_aot_host.cpp@@ -28,6 +28,7 @@#include <cstddef>#include <cstdint>#include <cstring>+#include <span>#include <string>#include <vector>@@ -92,6 +93,10 @@ struct World {std::vector<std::uint32_t> entities;std::vector<MoverRow> movers; // the pool's rowsstd::vector<std::uint32_t> moverSparse; // entity index -> slot + 1+ std::vector<std::uint32_t> moverOwners; // slot -> entity, the pool's dense side+ /** Entities carrying Transform and no Mover: the world around the matched+ * set, which is what a host that offers every entity pays for. */+ std::vector<std::uint32_t> bystanders;TimeRow time{1.0 / 30.0};aot::RowSpan moverSpan() {@@ -101,8 +106,18 @@ struct World {reinterpret_cast<unsigned char*>(movers.data()),static_cast<std::uint32_t>(sizeof(MoverRow)),Entity::Layout::INDEX_MASK,+ moverOwners.data(),+ static_cast<std::uint32_t>(moverOwners.size()),};}++ /** Movers first, then the rest: what a host with nothing to narrow by has+ * to hand over, and what narrowing exists to stop it handing over. */+ std::vector<std::uint32_t> everyEntity() const {+ std::vector<std::uint32_t> all = entities;+ all.insert(all.end(), bystanders.begin(), bystanders.end());+ return all;+ }};constexpr int ENTITY_COUNT = 24;@@ -124,10 +139,20 @@ World makeWorld() {const std::uint32_t at = e.index();if (at >= w.moverSparse.size()) w.moverSparse.resize(at + 1u, 0u);w.moverSparse[at] = static_cast<std::uint32_t>(w.movers.size()); // slot + 1+ w.moverOwners.push_back(e.id());}return w;}+/** Entities with a Transform and no Mover, so no query here can match them. */+void addBystanders(World& w, int count) {+ for (int i = 0; i < count; ++i) {+ Entity e = w.registry.create();+ w.registry.emplace<ecs::Transform>(e);+ w.bystanders.push_back(e.id());+ }+}+/** What move.ts says, written out in C++. The oracle: if the compiled system* disagrees with this, one of them is wrong about the struct. */void moveByHand(World& w) {@@ -160,6 +185,27 @@ aot::ComponentLookup componentsOf(World& w) {};}+/**+ * Who holds each column - the lookup that lets a query narrow itself.+ *+ * `Transform` goes through the GENERATED table, so this exercises the thing a+ * real host uses rather than a second answer written here; `Mover` stands in for+ * a script pool's dense side, which is the half a sparse table cannot give.+ */+aot::CandidateLookup candidatesOf(World& w) {+ return [&w](const char* name) -> aot::CandidatesOf {+ if (std::strcmp(name, "Transform") == 0) {+ return aot::engineComponentCandidates(w.registry, name);+ }+ if (std::strcmp(name, "Mover") == 0) {+ return [&w]() -> aot::Candidates {+ return std::span<const std::uint32_t>(w.moverOwners.data(), w.moverOwners.size());+ };+ }+ return {};+ };+}+aot::ResourceLookup resourcesOf(World& w) {return [&w](const char* name) -> void* {return std::strcmp(name, "Time") == 0 ? &w.time : nullptr;@@ -557,3 +603,178 @@ TEST_CASE("the records a call wrote are applied after it, and unknown ones are cCHECK(w.registry.valid(Entity::fromRaw(victim)) == false);CHECK(w.registry.valid(Entity::fromRaw(bystander)));}++// ---------------------------------------------------------------------------+// Narrowing: which entities a query is paid over.+//+// The completeness check still FILTERS; narrowing only changes the bill. Each+// test asks both halves: identical rows, and no wide list needed to get them.+// ---------------------------------------------------------------------------++TEST_CASE("a narrowed query moves exactly the world the wide one moved") {+ World wide = makeWorld();+ World narrow = makeWorld();+ addBystanders(wide, 200);+ addBystanders(narrow, 200);++ const EsSystemDecl* decl = declOf("MoveSystem");+ REQUIRE(decl != nullptr);+ const aot::BoundSystem plain = aot::bind(*decl, componentsOf(wide), resourcesOf(wide));+ const aot::BoundSystem narrowed = aot::bind(*decl, componentsOf(narrow), resourcesOf(narrow),+ candidatesOf(narrow));+ REQUIRE(plain.fn != nullptr);+ REQUIRE(narrowed.fn != nullptr);+ // The premise: only one of them can say where its rows come from.+ CHECK(aot::narrows(plain.queries) == false);+ CHECK(aot::narrows(narrowed.queries));++ aot::CallArena arena;+ for (int frame = 0; frame < 8; ++frame) {+ const std::vector<std::uint32_t> all = wide.everyEntity();+ aot::runBound(plain, all, resourcesOf(wide), arena);+ // EMPTY, not the world: if narrowing were not doing the work, this frame+ // would move nothing and the comparison below would fail loudly.+ aot::runBound(narrowed, {}, resourcesOf(narrow), arena);+ }++ for (std::size_t i = 0; i < wide.entities.size(); ++i) {+ const auto* a = wide.registry.tryGet<ecs::Transform>(Entity::fromRaw(wide.entities[i]));+ const auto* b = narrow.registry.tryGet<ecs::Transform>(Entity::fromRaw(narrow.entities[i]));+ REQUIRE(a != nullptr);+ REQUIRE(b != nullptr);+ CHECK(a->position.x == b->position.x);+ CHECK(a->position.y == b->position.y);+ }+ // And the entities no query names are untouched by either, which is the+ // claim narrowing makes about them.+ for (std::size_t i = 0; i < narrow.bystanders.size(); ++i) {+ const auto* b = narrow.registry.tryGet<ecs::Transform>(Entity::fromRaw(narrow.bystanders[i]));+ REQUIRE(b != nullptr);+ CHECK(b->position.x == 0.0f);+ CHECK(b->position.y == 0.0f);+ }+}++TEST_CASE("the shortest column is the one walked, and both give the same rows") {+ World w = makeWorld();+ addBystanders(w, 500);++ const EsSystemDecl* decl = declOf("MoveSystem");+ REQUIRE(decl != nullptr);+ const aot::BoundSystem bound = aot::bind(*decl, componentsOf(w), resourcesOf(w), candidatesOf(w));+ REQUIRE(bound.fn != nullptr);+ REQUIRE(bound.queries.size() == 1u);++ // Transform is on every entity here; Mover is on 24 of them. The query names+ // both, so an entity missing either cannot match and the shorter column+ // bounds the answer.+ const aot::Candidates chosen = aot::narrowest(bound.queries[0]);+ REQUIRE(chosen.has_value());+ CHECK(chosen->size() == w.entities.size());+ CHECK(chosen->size() < w.everyEntity().size());+}++TEST_CASE("a column that cannot say does not narrow, and the caller widens") {+ World w = makeWorld();+ addBystanders(w, 50);++ const EsSystemDecl* decl = declOf("MoveSystem");+ REQUIRE(decl != nullptr);+ // A lookup that answers for neither: the shape of a host built before the+ // owner table existed, and of a script pool that has not reported one yet.+ const aot::BoundSystem silent = aot::bind(*decl, componentsOf(w), resourcesOf(w),+ [](const char*) { return aot::CandidatesOf{}; });+ REQUIRE(silent.fn != nullptr);+ CHECK(aot::narrows(silent.queries) == false);+ CHECK(aot::narrowest(silent.queries[0]).has_value() == false);++ // nullopt must not be read as "nobody": handed the world, it still runs.+ // Summed, not sampled: the fixture spreads directions over signs AND zero,+ // so the first entity's dx is 0 and a sample there proves nothing.+ const auto sum = [&w] {+ double total = 0.0;+ for (std::uint32_t e : w.entities) {+ const auto* t = w.registry.tryGet<ecs::Transform>(Entity::fromRaw(e));+ total += static_cast<double>(t->position.x) + static_cast<double>(t->position.y);+ }+ return total;+ };+ aot::CallArena arena;+ const double before = sum();+ const std::vector<std::uint32_t> all = w.everyEntity();+ aot::runBound(silent, all, resourcesOf(w), arena);+ CHECK(before != sum());+}++TEST_CASE("an empty column is a query that matches nothing, not a query that widens") {+ World w = makeWorld();+ const EsSystemDecl* decl = declOf("MoveSystem");+ REQUIRE(decl != nullptr);++ // The difference this whole distinction exists for. `Mover` says it holds+ // nobody; if that were read as "cannot say", the system would fall back to+ // the world and move entities whose component is gone.+ const aot::BoundSystem bound = aot::bind(+ *decl, componentsOf(w), resourcesOf(w),+ [&w](const char* name) -> aot::CandidatesOf {+ if (std::strcmp(name, "Transform") == 0) {+ return aot::engineComponentCandidates(w.registry, name);+ }+ return []() -> aot::Candidates { return std::span<const std::uint32_t>{}; };+ });+ REQUIRE(bound.fn != nullptr);+ const aot::Candidates chosen = aot::narrowest(bound.queries[0]);+ REQUIRE(chosen.has_value());+ CHECK(chosen->empty());+}++TEST_CASE("a freed slot in a column is a hole, and the hole resolves to no row") {+ World w = makeWorld();+ // What a pool that reuses slots rather than compacting leaves behind. The+ // row stays where it is; only the owner is forgotten.+ w.moverOwners[3] = aot::NO_OWNER;+ w.moverOwners[7] = aot::NO_OWNER;++ aot::RowSpan span = w.moverSpan();+ const aot::CandidatesOf of = aot::fromMovingOwners(&span);+ const aot::Candidates got = of();+ REQUIRE(got.has_value());+ // The count is the high-water mark, holes included - narrowing is about the+ // order of magnitude, and skipping them here would cost a pass to save none.+ CHECK(got->size() == w.entities.size());+ CHECK((*got)[3] == aot::NO_OWNER);+ // And a hole reaches no component, so the completeness check drops the row:+ // NO_OWNER is Entity::INVALID_RAW, which the world never hands out.+ CHECK(w.registry.tryGet<ecs::Transform>(Entity::fromRaw(aot::NO_OWNER)) == nullptr);+ CHECK(aot::rowAt(span, aot::NO_OWNER) == nullptr);+}++TEST_CASE("a column with no owner table cannot say, which is not the same as empty") {+ World w = makeWorld();+ aot::RowSpan span = w.moverSpan();+ span.owners = nullptr;+ span.ownerCount = 0;+ CHECK(aot::fromMovingOwners(&span)().has_value() == false);+ // And a null slot is the same answer, for the pool that does not exist yet.+ CHECK(aot::fromMovingOwners(nullptr)().has_value() == false);+}++TEST_CASE("the generated table answers who has a component, and says so by name") {+ World w = makeWorld();+ addBystanders(w, 10);++ const aot::CandidatesOf of = aot::engineComponentCandidates(w.registry, "Transform");+ REQUIRE(static_cast<bool>(of));+ const aot::Candidates got = of();+ REQUIRE(got.has_value());+ CHECK(got->size() == w.entities.size() + w.bystanders.size());++ // A name this table does not answer for leaves an EMPTY function - the+ // caller must not read that as an empty column, or it narrows to nothing.+ CHECK(static_cast<bool>(aot::engineComponentCandidates(w.registry, "Mover")) == false);++ // Live, not a snapshot: the column is asked again per call because a pool+ // grows between them.+ addBystanders(w, 5);+ CHECK(of()->size() == w.entities.size() + w.bystanders.size());+}diff --git a/tools/eht/generators/aot_components.py b/tools/eht/generators/aot_components.pyindex f6d2ecc0e..f9446dc69 100644--- a/tools/eht/generators/aot_components.py+++ b/tools/eht/generators/aot_components.py@@ -14,6 +14,11 @@ Registry.Resolved once per query and called per entity, not looked up per entity: astrcmp chain inside the row loop would be paid on exactly the path AOT existsto make cheap.++The second table answers the other question a query asks — WHO has the+component — because a host that cannot ask it has to offer every entity as a+candidate, and then a system's cost is the size of the world rather than the+size of what it matches."""from typing import List@@ -79,6 +84,21 @@ class AotComponentsGenerator:' */','ComponentAt engineComponentAt(ecs::Registry& registry, const char* name);','',+ '/**',+ ' * WHO has that component, for a caller narrowing a query to its shortest',+ ' * column.',+ ' *',+ ' * The resolver above answers per entity and can only be asked about one',+ ' * that is already a candidate; this is where candidates come FROM. A host',+ ' * without it must offer every entity, and then what a compiled system',+ ' * costs is the size of the world instead of the size of what it matches.',+ ' *',+ ' * An empty span means nobody has it — a query that matches nothing. The',+ ' * empty FUNCTION means this table does not answer for the name, which is',+ ' * a different answer and the caller must not narrow on it.',+ ' */',+ 'CandidatesOf engineComponentCandidates(ecs::Registry& registry, const char* name);',+ '','/** Whether this table answers for that name, without building a resolver. */','bool isEngineComponent(const char* name);','',@@ -94,6 +114,7 @@ class AotComponentsGenerator:'','#include <cstdint>','#include <cstring>',+ '#include <span>','','#include "esengine/ecs/Entity.hpp"','#include "esengine/ecs/Registry.hpp"',@@ -107,6 +128,24 @@ class AotComponentsGenerator:lines += ['','namespace esengine::aot {',+ 'namespace {',+ '',+ '/**',+ ' * The dense column of a pool, as raw ids.',+ ' *',+ ' * `Entity` is a four-byte struct whose only member is that id, and',+ ' * Types.hpp static_asserts the size for exactly this — a dense array of',+ ' * them IS an array of ids. Copying it per frame would put the cost back',+ ' * where narrowing took it from.',+ ' */',+ 'Candidates denseAsIds(std::span<const Entity> dense) {',+ ' static_assert(sizeof(Entity) == sizeof(std::uint32_t),',+ ' "a dense entity array is only an id array while these agree");',+ ' return std::span<const std::uint32_t>(',+ ' reinterpret_cast<const std::uint32_t*>(dense.data()), dense.size());',+ '}',+ '',+ '} // namespace','','ComponentAt engineComponentAt(ecs::Registry& registry, const char* name) {',]@@ -117,6 +156,19 @@ class AotComponentsGenerator:lines.append(f' return registry.tryGet<{full}>(Entity::fromRaw(raw));')lines.append(' };')lines.append(' }')+ lines += [+ ' return {};',+ '}',+ '',+ 'CandidatesOf engineComponentCandidates(ecs::Registry& registry, const char* name) {',+ ]+ for comp in components:+ full = self._full(comp)+ lines.append(f' if (std::strcmp(name, "{comp.name}") == 0) {{')+ lines.append(' return [®istry]() -> Candidates {')+ lines.append(f' return denseAsIds(registry.entitiesWith<{full}>());')+ lines.append(' };')+ lines.append(' }')lines += [' return {};','}',diff --git a/tools/releaseGate.mjs b/tools/releaseGate.mjsindex e9f138eb5..29257b497 100644--- a/tools/releaseGate.mjs+++ b/tools/releaseGate.mjs@@ -115,6 +115,13 @@ export const CRITERIA = ['bench/aot-frame/project/src/systems.ts',],},+ {+ id: 'a-compiled-system-costs-what-it-matches',+ says: 'a compiled system is paid over the entities it matches, not over the size of the world',+ answeredBy: 'node bench/aot-native/frame-bench.mjs --gate',+ needs: ['bench/aot-native/frame-bench.mjs', 'src/esengine/aot/AotHost.hpp',+ 'native/host/bindings/AotBindings.cpp'],+ },{id: 'compiled-systems-compute-what-the-interpreter-computes',says: 'a packaged compiled system moves the world exactly as the interpreted one does',