Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(abi): the entity split has one author, and a handshake

yhh4e2200a78e6a
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f98fe86f0..ac6020314 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -49,6 +49,31 @@ published separately; it ships inside the editor.
### Fixed
+- **The entity split had two authors and no handshake.** A handle crosses the
+ boundary as a bare `u32`, so a side that disagrees about where the index ends
+ does not fail — it decodes to **another entity's** index and the frame is merely
+ wrong. That has happened: `installNativeAot` assembled a 20-bit mask by hand
+ against a 22-bit truth, and every world over 2^20 entities read one component
+ off another entity's row.
+
+ `22` was written in `sdk/src/types.ts` and again as `PackedId<22, 10>` in
+ `core/Types.hpp`, tied only by a test that parses the header. EHT now reads
+ `Entity::Layout` and emits `sdk/src/wasm/entityLayout.generated.ts`, so the C++
+ header is the one author and `types.ts` re-exports. **The split is in the ABI
+ layout hash**, which means a WASM binary and an SDK bundle built from different
+ splits refuse each other at `connect()` instead of decoding each other's
+ handles. `ABI_LAYOUT_HASH` therefore changes: rebuild the engine.
+
+ Not found is a failure, never a default — a fallback would emit a plausible
+ split from a header it could not read, which is the one outcome worse than
+ stopping. And `core/Types.hpp` joins EHT's cache inputs: it is not under the
+ component directory, so without that, editing the split is answered with "no
+ changes detected" and the old split stays in the hash.
+
+ `cpp-contract.test.ts` keeps its pin with a changed job: it reads the header
+ with a second parser, so what it now catches is a checked-in projection gone
+ stale, or an extraction bug — rather than a plausible number nobody re-derived.
+
- **The engine's specified arithmetic was in no digest, so a module built against
a different polynomial loaded.** The AOT subset SPECIFIES `sin` and `cos`,
because ECMAScript does not, and a compiled module carries its own copy of that
diff --git a/build-tools/build.config.js b/build-tools/build.config.js
index a3d5d095b..d9d36dafe 100644
--- a/build-tools/build.config.js
+++ b/build-tools/build.config.js
@@ -213,6 +213,7 @@ export default {
eht: {
inputDir: 'src/esengine/ecs/components',
+ entityHeader: 'src/esengine/core/Types.hpp',
outputDir: 'src/esengine/bindings',
tsOutputDir: 'sdk/src',
script: 'tools/eht.py',
diff --git a/build-tools/tasks/eht.js b/build-tools/tasks/eht.js
index 06f3939e3..8a8804dc0 100644
--- a/build-tools/tasks/eht.js
+++ b/build-tools/tasks/eht.js
@@ -35,7 +35,11 @@ export async function runEht(options = {}) {
const cache = new HashCache(config.paths.cache);
await cache.load();
- const allFiles = [script, ...generatorFiles, ...componentFiles];
+ // The entity split comes from core/Types.hpp, which is NOT under the
+ // component directory — leaving it out means editing the split and being
+ // told "no changes detected", with the old split still in the hash.
+ const entityHeader = path.join(rootDir, config.eht.entityHeader);
+ const allFiles = [script, entityHeader, ...generatorFiles, ...componentFiles];
const currentHash = await hashFiles(allFiles);
if (!await cache.isChanged('eht', currentHash)) {
@@ -117,6 +121,9 @@ async function executeEht(rootDir, script) {
await runCommand(python, [
script,
'--input', path.join(rootDir, config.eht.inputDir),
+ // Passed rather than left to the generator's default: the path the cache
+ // hashes and the path the generator reads must be one path.
+ '--entity-header', path.join(rootDir, config.eht.entityHeader),
'--output', outputDir,
'--ts-output', tsOutputDir,
], {
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 49570bd2a..e26ea0916 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -5,7 +5,7 @@ Symbols: 113 public · 84 beta · 1514 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
-"70afde061183c505"
+"077eebc0e561321f"
```
## ANCHOR_AXES — const @experimental
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 35500b694..234eee152 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -5,7 +5,7 @@ Symbols: 113 public · 89 beta · 1533 experimental · 3 deprecated
## ABI_LAYOUT_HASH — const @experimental
```
-"70afde061183c505"
+"077eebc0e561321f"
```
## ANCHOR_AXES — const @experimental
diff --git a/sdk/src/ecs/component.generated.ts b/sdk/src/ecs/component.generated.ts
index 9fb5000ff..d423da795 100644
--- a/sdk/src/ecs/component.generated.ts
+++ b/sdk/src/ecs/component.generated.ts
@@ -15,7 +15,7 @@ import type { AlignContent, AlignItems, AlignSelf, BodyType, CanvasScaleMode, Cl
* getAbiLayoutHash(); BuiltinBridge.connect() compares them and refuses to
* run on mismatch, because mismatched offsets read the wrong heap bytes.
*/
-export const ABI_LAYOUT_HASH = '70afde061183c505';
+export const ABI_LAYOUT_HASH = '077eebc0e561321f';
/**
* One asset-valued field of a component: which field, and what kind of
diff --git a/sdk/src/types.ts b/sdk/src/types.ts
index a8f3e875a..7645723e2 100644
--- a/sdk/src/types.ts
+++ b/sdk/src/types.ts
@@ -5,6 +5,8 @@
* @brief Core type definitions for ESEngine SDK
*/
+import { ENTITY_INDEX_BITS, ENTITY_GEN_BITS } from './wasm/entityLayout.generated';
+
// =============================================================================
// Entity
// =============================================================================
@@ -29,11 +31,12 @@
export type Entity = number;
/**
- * Bit widths of the packed Entity representation. MUST match the C++ split
- * (`PackedId<22, 10>` via `Entity::Layout` in core/Types.hpp / PackedId.hpp).
+ * Bit widths of the packed Entity representation, from the C++ `Entity::Layout`
+ * that owns them — EHT reads the header, so this is not a second answer. The
+ * split is in the ABI layout hash, so a WASM binary and an SDK bundle built from
+ * different ones refuse each other rather than decoding each other's handles.
*/
-export const ENTITY_INDEX_BITS = 22;
-export const ENTITY_GEN_BITS = 10;
+export { ENTITY_INDEX_BITS, ENTITY_GEN_BITS };
/** Number of representable indices (2^INDEX_BITS) — used for overflow-safe packing. */
const ENTITY_INDEX_COUNT = 2 ** ENTITY_INDEX_BITS;
export const ENTITY_INDEX_MASK = ENTITY_INDEX_COUNT - 1;
diff --git a/sdk/src/wasm/entityLayout.generated.ts b/sdk/src/wasm/entityLayout.generated.ts
new file mode 100644
index 000000000..ec8ed4563
--- /dev/null
+++ b/sdk/src/wasm/entityLayout.generated.ts
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file entityLayout.generated.ts
+ * @brief The packed Entity split, from `Entity::Layout` in core/Types.hpp
+ * @details Generated by EHT - DO NOT EDIT
+ *
+ * A handle crosses as a bare u32, so a split this disagrees with decodes to
+ * another entity rather than failing. It is in the ABI layout hash for that
+ * reason: a WASM binary and an SDK bundle built from different splits refuse
+ * each other at `connect()`.
+ *
+ * No imports on purpose — `types.ts` re-exports these, and everything else
+ * reaches them there.
+ */
+
+/** Low bits of a handle: the registry slot. */
+export const ENTITY_INDEX_BITS = 22;
+
+/** High bits: bumped when a slot is recycled, so a stale handle compares unequal. */
+export const ENTITY_GEN_BITS = 10;
diff --git a/sdk/tests/cpp-contract.test.ts b/sdk/tests/cpp-contract.test.ts
index 071ab9dde..e4c8760f9 100644
--- a/sdk/tests/cpp-contract.test.ts
+++ b/sdk/tests/cpp-contract.test.ts
@@ -149,8 +149,9 @@ describe('C++ contract: particle color LUT (particle/ParticleSystem.hpp)', () =>
describe('C++ contract: packed Entity bit split (core/Types.hpp)', () => {
it('ENTITY_INDEX_BITS / ENTITY_GEN_BITS === PackedId<index, gen>', () => {
- // Entity handles cross the boundary as a raw u32; the wrong split silently
- // mis-decodes index/generation -> stale-handle detection breaks.
+ // The split is GENERATED from this header now, so what this catches is a
+ // checked-in projection gone stale — read with a SECOND parser, so an
+ // extraction bug is a disagreement rather than a plausible number.
const m = /PackedId<\s*(\d+)\s*,\s*(\d+)\s*>/.exec(stripComments(readCpp('core/Types.hpp')));
if (!m) throw new Error('C++ Entity::Layout PackedId<index, gen> not found');
expect([ENTITY_INDEX_BITS, ENTITY_GEN_BITS]).toEqual([Number(m[1]), Number(m[2])]);
diff --git a/src/esengine/bindings/WebBindings.generated.cpp b/src/esengine/bindings/WebBindings.generated.cpp
index 3e36cfde5..77948fc0f 100644
--- a/src/esengine/bindings/WebBindings.generated.cpp
+++ b/src/esengine/bindings/WebBindings.generated.cpp
@@ -3070,7 +3070,7 @@ static_assert(offsetof(esengine::ecs::Velocity, angular) == 12, "ABI offset drif
// ABI Hash -- runtime handshake against the SDK bundle
// =============================================================================
-static const char* kEsAbiLayoutHash = "70afde061183c505";
+static const char* kEsAbiLayoutHash = "077eebc0e561321f";
std::string esengineGetAbiLayoutHash() {
return std::string(kEsAbiLayoutHash);
diff --git a/tools/eht/__main__.py b/tools/eht/__main__.py
index 92d855d95..915531118 100644
--- a/tools/eht/__main__.py
+++ b/tools/eht/__main__.py
@@ -6,6 +6,7 @@ from pathlib import Path
from .parser import CppParser
from .abi import compute_abi_hash
+from .entity_layout import parse_entity_layout, generate_ts as generate_entity_ts
from .generators import (
EmbindGenerator, TypeScriptGenerator, MetadataGenerator,
PtrLayoutGenerator, EditorAPIGenerator, NativeBindingsGenerator, AotComponentsGenerator,
@@ -77,6 +78,9 @@ def main() -> int:
help='Output directory for C++ bindings')
parser.add_argument('--ts-output', type=Path, default=Path('sdk'),
help='Output directory for TypeScript')
+ parser.add_argument('--entity-header', type=Path,
+ default=Path('src/esengine/core/Types.hpp'),
+ help='Header declaring Entity::Layout = PackedId<index, gen>')
parser.add_argument('--verbose', '-v', action='store_true')
# Opt-in native (QuickJS) bindings. Off by default so the standard EHT run and
# its committed *.generated.* files are unchanged; a native build passes this
@@ -163,8 +167,12 @@ def main() -> int:
# Compute pointer layouts and the ABI hash first; both the C++ bindings and
# the TS metadata embed the same hash so connect() can verify they match.
ptr_gen = PtrLayoutGenerator(cpp_parser.components, cpp_parser.enums)
+ # The entity split is a boundary fact with the same failure mode as an
+ # offset — a handle that decodes to another entity — so it is in the hash.
+ entity = parse_entity_layout(args.entity_header)
+ print(f"Entity layout: index={entity[0]} gen={entity[1]}")
abi_hash = compute_abi_hash(
- cpp_parser.components, cpp_parser.enums, ptr_gen.layouts
+ cpp_parser.components, cpp_parser.enums, ptr_gen.layouts, entity
)
print(f"ABI layout hash: {abi_hash}")
@@ -228,6 +236,9 @@ def main() -> int:
meta_gen = MetadataGenerator(cpp_parser.components, cpp_parser.enums, abi_hash=abi_hash)
write_ts('ecs/component.generated.ts', meta_gen.generate())
+ # ── Entity split ──
+ write_ts('wasm/entityLayout.generated.ts', generate_entity_ts(*entity))
+
# ── Pointer Layouts & Accessors ──
write_ts('wasm/ptrLayouts.generated.ts', ptr_gen.generate())
write_ts('ecs/bridge/ptrAccessors.generated.ts', ptr_gen.generate_accessors())
diff --git a/tools/eht/abi.py b/tools/eht/abi.py
index ef4cc4115..fdead2de1 100644
--- a/tools/eht/abi.py
+++ b/tools/eht/abi.py
@@ -7,6 +7,10 @@ SDK compares the two: a mismatch means the loaded WASM binary and the shipped SD
were generated from different schemas, so the pointer offsets the SDK uses would
read the wrong bytes. Refusing to run is the only safe response.
+The entity split is in here for the same reason the offsets are: a handle is a
+bare u32, so two sides that disagree about where the index ends decode each
+other's handles rather than refusing.
+
This is the runtime half of the keystone. The compile-time half is the generated
`static_assert(offsetof(...))` block, which proves EHT's computed offsets equal
the real compiler layout. Together: TS offset == EHT offset == compiler offset,
@@ -14,19 +18,19 @@ and the loaded WASM's schema == the SDK's schema.
"""
import hashlib
-from typing import Dict, List
+from typing import Dict, List, Tuple
from .data import Component, Enum
def _canonical(components: List[Component], enums: List[Enum],
- layouts: List[Dict]) -> str:
+ layouts: List[Dict], entity: Tuple[int, int]) -> str:
"""Build an order-independent canonical description of the boundary ABI.
Components/enums/layouts are sorted by name so the hash is stable regardless
of filesystem glob order (which is not guaranteed across machines).
"""
- parts: List[str] = []
+ parts: List[str] = [f'ENTITY index={entity[0]} gen={entity[1]}']
for enum in sorted(enums, key=lambda e: e.name):
vals = ','.join(enum.values)
@@ -46,7 +50,7 @@ def _canonical(components: List[Component], enums: List[Enum],
def compute_abi_hash(components: List[Component], enums: List[Enum],
- layouts: List[Dict]) -> str:
+ layouts: List[Dict], entity: Tuple[int, int]) -> str:
"""Return a stable 16-hex-char digest of the boundary ABI."""
- canonical = _canonical(components, enums, layouts)
+ canonical = _canonical(components, enums, layouts, entity)
return hashlib.sha1(canonical.encode('utf-8')).hexdigest()[:16]
diff --git a/tools/eht/entity_layout.py b/tools/eht/entity_layout.py
new file mode 100644
index 000000000..0995cc61f
--- /dev/null
+++ b/tools/eht/entity_layout.py
@@ -0,0 +1,73 @@
+"""The packed Entity split, read from the C++ header that owns it.
+
+An entity handle crosses the boundary as a bare u32. The SDK unpacks it with a
+shift and a mask, so a split it disagrees with is not an error — it is a handle
+that decodes to ANOTHER ENTITY'S index, and the frame is merely wrong. That has
+happened: `installNativeAot` assembled a 20-bit mask by hand against a 22-bit
+truth, and every world over 2^20 entities read one component off another entity's
+row.
+
+The split therefore gets what every other boundary fact has: one author (the C++
+`Entity::Layout`), a generated TS projection, and a term in the ABI layout hash
+so a WASM binary and an SDK bundle built from different splits refuse each other
+at `connect()` rather than decoding each other's handles.
+
+Not found is a FAILURE, never a default. A fallback here would emit a plausible
+split from a header it could not read, which is the one outcome worse than
+stopping.
+"""
+
+import re
+from pathlib import Path
+from typing import Tuple
+
+#: `using Layout = PackedId<22, 10>;` — index bits, then generation bits.
+_LAYOUT = re.compile(r'using\s+Layout\s*=\s*PackedId\s*<\s*(\d+)\s*,\s*(\d+)\s*>\s*;')
+
+
+def parse_entity_layout(header: Path) -> Tuple[int, int]:
+ """Return `(index_bits, gen_bits)` from the Entity declaration in `header`."""
+ if not header.is_file():
+ raise SystemExit(f"[FAIL] entity layout: no such header: {header}")
+ text = header.read_text(encoding='utf-8')
+ found = _LAYOUT.findall(text)
+ if not found:
+ raise SystemExit(
+ f"[FAIL] entity layout: no `using Layout = PackedId<i, g>;` in {header}. "
+ "The split is a boundary fact and cannot be guessed.")
+ if len(found) > 1:
+ raise SystemExit(
+ f"[FAIL] entity layout: {len(found)} PackedId layouts in {header}; "
+ "which one is Entity's cannot be decided here.")
+ index_bits, gen_bits = int(found[0][0]), int(found[0][1])
+ if index_bits + gen_bits != 32:
+ raise SystemExit(
+ f"[FAIL] entity layout: PackedId<{index_bits}, {gen_bits}> is not 32 bits. "
+ "The handle is a u32 on both sides.")
+ return index_bits, gen_bits
+
+
+def generate_ts(index_bits: int, gen_bits: int) -> str:
+ """The TS half. No imports: `types.ts` is what everything else reaches."""
+ return f'''// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file entityLayout.generated.ts
+ * @brief The packed Entity split, from `Entity::Layout` in core/Types.hpp
+ * @details Generated by EHT - DO NOT EDIT
+ *
+ * A handle crosses as a bare u32, so a split this disagrees with decodes to
+ * another entity rather than failing. It is in the ABI layout hash for that
+ * reason: a WASM binary and an SDK bundle built from different splits refuse
+ * each other at `connect()`.
+ *
+ * No imports on purpose — `types.ts` re-exports these, and everything else
+ * reaches them there.
+ */
+
+/** Low bits of a handle: the registry slot. */
+export const ENTITY_INDEX_BITS = {index_bits};
+
+/** High bits: bumped when a slot is recycled, so a stale handle compares unequal. */
+export const ENTITY_GEN_BITS = {gen_bits};
+'''
diff --git a/tools/lib/contractFacts.mjs b/tools/lib/contractFacts.mjs
index f52daba79..ad449495a 100644
--- a/tools/lib/contractFacts.mjs
+++ b/tools/lib/contractFacts.mjs
@@ -125,17 +125,15 @@ export const FACTS = [
what: 'How an entity handle packs an index and a generation into one u32.',
surface: 'runtime-wasm-abi',
authors: [
- { path: 'sdk/src/types.ts', probe: /export const ENTITY_INDEX_BITS/, kind: 'semantic' },
{ path: 'src/esengine/core/Types.hpp', probe: /using Layout = PackedId</, kind: 'semantic' },
],
- projections: [],
+ projections: ['sdk/src/wasm/entityLayout.generated.ts'],
verification: [
+ // A SECOND reader of the same header: its own regex against EHT's
+ // parser. Not the projection checking itself — two code paths.
{ path: 'sdk/tests/cpp-contract.test.ts', how: 'test', probe: /packed Entity bit split/ },
],
- digest: null,
- owed: 'no digest covers it: the split crosses as a bare u32, so a mismatch is a decoded'
- + ' handle pointing at another entity rather than a refusal. The test parses the'
- + ' header, which is a binding but not a handshake.',
+ digest: { name: 'eht-abi-layout-hash', path: 'tools/eht/abi.py', probe: /ENTITY index=/ },
},
{
id: 'mathPolicy',