A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
bench(net): the interest grid, kept instead of rebuilt
esengine43fa3006da4b
diff --git a/bench/replication-interest/README.md b/bench/replication-interest/README.mdindex f84995478..5b51832cb 100644--- a/bench/replication-interest/README.md+++ b/bench/replication-interest/README.md@@ -106,7 +106,63 @@ What remains is the radius scan, and a per-sample rebuilt grid takes 100k × 32from 46 cores to 1.8 while keeping arbitrary `position()` support and matchingthe full scan entity for entity. The build is 93% of what is left, so anincremental index is worth designing — but it is now a question about-maintaining a structure, not about whether locality helps.+maintaining a structure, not about whether locality helps. That question is the+third one below, and the answer is 15% of a core.++## Third question: what the grid costs when it is KEPT++The build column is the whole remaining cost, and it is paid every sample+whatever moved. `TransformSystem` knows exactly which entities' composed output+differs — costed in `bench/transform-composition` at 1.3% of a core — so a fourth+arm keeps its grid and moves only those between cells.++| | positions | grid |+|---|---|---|+| **C1** | read once per sample, all of them | rebuilt every sample |+| **D1** | read for the entities that moved | kept; those entities re-celled |++D1's query side is byte for byte C1's — the same cell arrays, the same cache map,+the same `visibleForC1` — so what the two differ by is the build column alone.++| population | connections | C1 | D1 | D1 build | C1 build | D1/B |+|---|---|---|---|---|---|---|+| 10k | 8 | 19% | **1%** | 1,852 | 177,024 | 0.007 |+| 10k | 32 | 19% | **2%** | 1,638 | 170,694 | 0.004 |+| 100k | 8 | 226% | **12%** | 22,026 | 2,179,859 | 0.009 |+| 100k | 32 | 200% | **15%** | 17,034 | 1,834,928 | 0.003 |++Position reads per sample go from 100,000 to **668**. The rest of the mechanism+is unchanged: 864 cells, 83,267 spatial candidates, 83,267 distance tests — the+same numbers C1 produces.++**At 100k × 32 the interest path goes from 5,559% of a core to 15%.** The grid+cost 200% while it was rebuilt and costs 15% while it is kept, and the build+inside that falls by 108×.++Seeding is one full build — 105 ms at 100k — paid once at startup, against the 92+ms C1 pays every sample.++Three properties are asserted rather than assumed:++- **D1 saw exactly what the full scan saw**, every connection, every sample, at+ three points.+- **The kept grid holds what a rebuilt one would.** Compared against a fresh+ build at the end of every point: zero drifted positions, zero drifted cells,+ nothing missing, nothing extra. This is the check that matters, because a stale+ cell only reaches the visible-set differential if it changes somebody's view —+ and at 1% visible, most of the world is nobody's. Stop updating the coordinate+ cache and 74 positions drift; stop re-celling and 74 cells do.+- **Every entity the composition reported was one the grid knew about.**++### What a kept grid still needs, and this does not have++**The changed set is a value-change journal, not a membership journal.** Measured,+not reasoned: a new entity spawned at the origin is not reported, because its+composed output equals what its fields already held; a despawned entity is not+reported at all; a new entity anywhere else is reported only incidentally,+because its value differs from the default. A shipped provider takes membership+from the membership journal it already runs on, and movement from here. This+workload spawns and despawns nothing after startup, so none of that is measured.## Confirmation on the authoritative world-space pathdiff --git a/bench/replication-interest/arm.mjs b/bench/replication-interest/arm.mjsindex 34e6745de..d01380793 100644--- a/bench/replication-interest/arm.mjs+++ b/bench/replication-interest/arm.mjs@@ -11,6 +11,7 @@ import {loadSdk, connectEngine, buildWorld, applyMovement, newStats,visibleForA, visibleForB, visibleForC0, visibleForC1,buildPositionCache, buildGrid, diffAndFilter,+ newLiveGrid, seedLiveGrid, updateLiveGrid, gridDrift,} from './workload.mjs';const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');@@ -37,7 +38,7 @@ const VERIFY = argv.includes('--verify');const REPL_EVERY = Math.round(SIM_HZ / REPL_HZ);const sdk = await loadSdk(ROOT);const app = sdk.App.new();-await connectEngine(sdk, app, ROOT);+const engine = await connectEngine(sdk, app, ROOT);const world = app.world;const ctx = buildWorld(sdk, world, {population: ENTITIES, connections: CONNECTIONS, anchorsPerConn: ANCHORS,@@ -73,6 +74,25 @@ world.ensureTransformsComposed();world.despawn(p);}+// The ids come back as an address into the engine's own memory — a JS array of a+// hundred thousand handles would cost more than the walk that found them. The+// view is rebuilt each call: growing the heap detaches the old one.+const cppRegistry = world.getCppRegistry();+const EMPTY = new Uint32Array(0);+function composeCollecting() {+ const r = engine.transform_composeCollecting(cppRegistry);+ if (!r.ran || r.changed === 0) return EMPTY;+ return new Uint32Array(engine.HEAPU32.buffer, r.ptr, r.changed);+}++const liveGrid = ARM === 'D1' ? newLiveGrid(RADIUS) : null;+let seedUs = 0;+if (liveGrid) {+ const seedStats = newStats();+ seedLiveGrid(world, ctx, seedStats, liveGrid);+ seedUs = Number(seedStats.ns.build) / 1000;+}+const stats = newStats();const previous = new Map();for (let c = 0; c < CONNECTIONS; c++) previous.set(c, new Set());@@ -89,7 +109,7 @@ function sampleOnce(measuring) {// What a server does before an interest snapshot. With no transform mutation// since the last one this is the epoch comparison and nothing else.const et = process.hrtime.bigint();- world.ensureTransformsComposed();+ const changed = ARM === 'D1' ? composeCollecting() : (world.ensureTransformsComposed(), null);if (measuring) stats.ns.ensure += process.hrtime.bigint() - et;const mt = process.hrtime.bigint();const candidates = [...ctx.entities];@@ -105,11 +125,12 @@ function sampleOnce(measuring) {// safe here: nothing is carried between samples to go stale.const shared = measuring ? stats : newStats();const cache = ARM === 'C0' ? buildPositionCache(world, ctx, shared) : null;- const grid = ARM === 'C1' ? buildGrid(world, ctx, shared, RADIUS) : null;+ const grid = ARM === 'C1' ? buildGrid(world, ctx, shared, RADIUS)+ : ARM === 'D1' ? (updateLiveGrid(world, ctx, shared, liveGrid, changed), liveGrid) : null;for (let c = 0; c < CONNECTIONS; c++) {const use = measuring ? stats : newStats();- const visible = ARM === 'C1'+ const visible = ARM === 'C1' || ARM === 'D1'? visibleForC1(ctx, c, candidates, R2, use, ctx.owned, grid, RADIUS): ARM === 'C0'? visibleForC0(ctx, c, candidates, R2, use, ctx.owned, cache)@@ -148,13 +169,18 @@ function run(count, measuring) {run(WARMUP, false);run(MEASURE, true);+// The invariant a KEPT structure lives or dies by. The per-sample visible-set+// check does not reach it: a stale cell shows up there only if it changes+// somebody's view, and at 1% visible most of the world is nobody's.+const drift = liveGrid ? gridDrift(liveGrid, buildGrid(world, ctx, newStats(), RADIUS)) : null;+const simSeconds = MEASURE / SIM_HZ;const us = (ns) => Number(ns) / 1000;const per = (v) => v / simSeconds;process.stdout.write(`${JSON.stringify({arm: ARM, entities: ENTITIES, connections: CONNECTIONS, anchors: ANCHORS,visible: VISIBLE, movement: MOVEMENT, radius: RADIUS, verify: VERIFY,- simSeconds, samples, mismatches, movedTotal,+ simSeconds, samples, mismatches, movedTotal, seedUs, drift,totalUsPerSimSecond: per(us(totalNs)),segmentUsPerSimSecond: Object.fromEntries(Object.entries(stats.ns).map(([k, v]) => [k, per(us(v))]),diff --git a/bench/replication-interest/results.json b/bench/replication-interest/results.jsonindex 70c09efa5..0d1f6ec66 100644--- a/bench/replication-interest/results.json+++ b/bench/replication-interest/results.json@@ -4,13 +4,13 @@"platform": "darwin 25.5.0","cpu": "Apple M4","cores": 10,- "ranAt": "2026-09-05T03:04:45.538Z"+ "ranAt": "2026-09-05T05:32:58.012Z"},"build": {- "gitHead": "272dbb9ed7100693646aea57b8946566b0e9953b",+ "gitHead": "f6225596ebaf8c217750677ce3f146d13c09927c","workingTreeClean": false,- "sdkArtifactSha256": "154b351d2953e6bf",- "sdkBuiltAt": "2026-09-05T03:03:58.785Z"+ "sdkArtifactSha256": "dd0b8cd56acc2a1d",+ "sdkBuiltAt": "2026-09-05T05:31:49.725Z"},"fixed": {"anchors": 1,@@ -35,17 +35,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 1146892.5033333334,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 1347509.4466666665,"segmentUsPerSimSecond": {- "materialize": 109.44333333333333,+ "materialize": 157.08666666666667,+ "ensure": 3132.0833333333335,"build": 0,- "anchor": 254.29000000000002,- "radius": 1144889.8733333333,- "owner": 96.39000000000001,- "enter": 379.4466666666667,- "leave": 335.13,- "removeFilter": 15.683333333333334,- "dirtyFilter": 493.3433333333333+ "anchor": 764.4333333333334,+ "radius": 1340752.78,+ "owner": 259.7066666666667,+ "enter": 523.0600000000001,+ "leave": 379.99666666666667,+ "removeFilter": 94.03666666666666,+ "dirtyFilter": 541.5233333333333},"visitedPerSample": {"candidates": 10000,@@ -56,6 +59,7 @@"cells": 0,"spatialCandidates": 0,"owner": 8,+ "unindexed": 0,"enter": 573.6666666666666,"leave": 573.6666666666666,"removals": 0,@@ -76,17 +80,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 193196.39333333334,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 216748.19,"segmentUsPerSimSecond": {- "materialize": 94.44666666666667,- "build": 150648.47,- "anchor": 39.33,- "radius": 40749.59333333334,- "owner": 305.9666666666667,- "enter": 352.7833333333333,- "leave": 333.90000000000003,- "removeFilter": 10.843333333333334,- "dirtyFilter": 478.4833333333333+ "materialize": 128.47,+ "ensure": 3510.14,+ "build": 162597.77,+ "anchor": 123.05666666666667,+ "radius": 48275.69666666667,+ "owner": 385.55333333333334,+ "enter": 436.38666666666666,+ "leave": 338.34333333333336,+ "removeFilter": 16.106666666666666,+ "dirtyFilter": 499.4466666666667},"visitedPerSample": {"candidates": 10000,@@ -97,6 +104,7 @@"cells": 0,"spatialCandidates": 0,"owner": 8,+ "unindexed": 0,"enter": 573.6666666666666,"leave": 573.6666666666666,"removals": 0,@@ -117,17 +125,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 163987.9166666667,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 185805.14,"segmentUsPerSimSecond": {- "materialize": 95.41666666666667,- "build": 159569.72333333336,- "anchor": 34.57,- "radius": 2865.84,- "owner": 21.946666666666665,- "enter": 358.7533333333334,- "leave": 319.86333333333334,- "removeFilter": 12.093333333333334,- "dirtyFilter": 482.2133333333333+ "materialize": 156.11333333333334,+ "ensure": 2765.976666666667,+ "build": 177023.88333333333,+ "anchor": 47.49666666666667,+ "radius": 3935.696666666667,+ "owner": 22.63,+ "enter": 388.34333333333336,+ "leave": 444.99333333333334,+ "removeFilter": 19.73,+ "dirtyFilter": 559.1700000000001},"visitedPerSample": {"candidates": 10000,@@ -138,6 +149,57 @@"cells": 216,"spatialCandidates": 1839.5,"owner": 8,+ "unindexed": 0,+ "enter": 573.6666666666666,+ "leave": 573.6666666666666,+ "removals": 0,+ "dirty": 800+ },+ "visiblePerConnection": 71.70833333333333+ },+ {+ "arm": "D1",+ "entities": 10000,+ "connections": 8,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 5.641895835477563,+ "verify": false,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 2400,+ "seedUs": 14439.709,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 9113.470000000001,+ "segmentUsPerSimSecond": {+ "materialize": 152.77666666666667,+ "ensure": 3252.5,+ "build": 1851.8033333333335,+ "anchor": 24.456666666666667,+ "radius": 2495.956666666667,+ "owner": 17.776666666666667,+ "enter": 340.42333333333335,+ "leave": 334.0366666666667,+ "removeFilter": 11.67,+ "dirtyFilter": 476.5133333333334+ },+ "visitedPerSample": {+ "candidates": 10000,+ "positionReads": 68,+ "anchor": 8,+ "radius": 0,+ "distanceTests": 1839.5,+ "cells": 216,+ "spatialCandidates": 1839.5,+ "owner": 8,+ "unindexed": 0,"enter": 573.6666666666666,"leave": 573.6666666666666,"removals": 0,@@ -158,17 +220,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 4967617.916666667,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 4965239.173333334,"segmentUsPerSimSecond": {- "materialize": 115,+ "materialize": 124.99666666666668,+ "ensure": 2817.0800000000004,"build": 0,- "anchor": 922.0733333333334,- "radius": 4961684.423333334,- "owner": 208.06333333333333,- "enter": 1401.5466666666666,- "leave": 1172.3366666666668,- "removeFilter": 25.443333333333335,- "dirtyFilter": 1451.246666666667+ "anchor": 2148.7366666666667,+ "radius": 4952658.8,+ "owner": 706.6766666666666,+ "enter": 1716.68,+ "leave": 1340.67,+ "removeFilter": 63.33666666666667,+ "dirtyFilter": 1584.8366666666668},"visitedPerSample": {"candidates": 10000,@@ -179,6 +244,7 @@"cells": 0,"spatialCandidates": 0,"owner": 32,+ "unindexed": 0,"enter": 2942.8333333333335,"leave": 2943,"removals": 0,@@ -199,17 +265,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 318276.39333333337,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 355051.11333333334,"segmentUsPerSimSecond": {- "materialize": 123.61333333333334,- "build": 172926.94,- "anchor": 133.1966666666667,- "radius": 139903.73,- "owner": 804.3133333333334,- "enter": 1341.8100000000002,- "leave": 1172.3833333333332,- "removeFilter": 19.436666666666667,- "dirtyFilter": 1418.36+ "materialize": 170.6966666666667,+ "ensure": 2829.8633333333332,+ "build": 184756.11000000002,+ "anchor": 239.14666666666668,+ "radius": 160636.81666666668,+ "owner": 924.4166666666666,+ "enter": 1569.03,+ "leave": 1407.07,+ "removeFilter": 31.973333333333336,+ "dirtyFilter": 1603.3400000000001},"visitedPerSample": {"candidates": 10000,@@ -220,6 +289,7 @@"cells": 0,"spatialCandidates": 0,"owner": 32,+ "unindexed": 0,"enter": 2942.8333333333335,"leave": 2943,"removals": 0,@@ -240,17 +310,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 180275.97666666665,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 190689.30666666667,"segmentUsPerSimSecond": {- "materialize": 94.72666666666667,- "build": 166073.47,- "anchor": 96.39333333333333,- "radius": 10010.133333333333,- "owner": 52.39666666666667,- "enter": 1224.5533333333333,- "leave": 1090.68,- "removeFilter": 19.6,- "dirtyFilter": 1323.3166666666668+ "materialize": 149.72333333333336,+ "ensure": 2782.6366666666668,+ "build": 170694.44666666668,+ "anchor": 139.04333333333335,+ "radius": 12322.516666666668,+ "owner": 61.373333333333335,+ "enter": 1370.5533333333333,+ "leave": 1189.1466666666668,+ "removeFilter": 21.38,+ "dirtyFilter": 1491.5266666666669},"visitedPerSample": {"candidates": 10000,@@ -261,6 +334,57 @@"cells": 864,"spatialCandidates": 8354.5,"owner": 32,+ "unindexed": 0,+ "enter": 2942.8333333333335,+ "leave": 2943,+ "removals": 0,+ "dirty": 3200+ },+ "visiblePerConnection": 91.96354166666667+ },+ {+ "arm": "D1",+ "entities": 10000,+ "connections": 32,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 5.641895835477563,+ "verify": false,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 2400,+ "seedUs": 12181.75,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 18551.666666666668,+ "segmentUsPerSimSecond": {+ "materialize": 150.83333333333334,+ "ensure": 3215.5533333333337,+ "build": 1637.6366666666668,+ "anchor": 74.76666666666667,+ "radius": 9180.006666666666,+ "owner": 54.03,+ "enter": 1330.95,+ "leave": 1131.6766666666667,+ "removeFilter": 18.586666666666666,+ "dirtyFilter": 1507.0933333333332+ },+ "visitedPerSample": {+ "candidates": 10000,+ "positionReads": 68,+ "anchor": 32,+ "radius": 0,+ "distanceTests": 8354.5,+ "cells": 864,+ "spatialCandidates": 8354.5,+ "owner": 32,+ "unindexed": 0,"enter": 2942.8333333333335,"leave": 2943,"removals": 0,@@ -281,17 +405,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 10939771.246666666,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 13152221.110000001,"segmentUsPerSimSecond": {- "materialize": 2462.7766666666666,+ "materialize": 2941.666666666667,+ "ensure": 28461.530000000002,"build": 0,- "anchor": 753.7633333333333,- "radius": 10925509.293333335,- "owner": 278.73333333333335,- "enter": 3300.1466666666665,- "leave": 2925.1533333333336,- "removeFilter": 20.133333333333333,- "dirtyFilter": 3741.106666666667+ "anchor": 1451.5366666666669,+ "radius": 13105676.663333334,+ "owner": 1006.5233333333333,+ "enter": 3853.4666666666667,+ "leave": 3224.7200000000003,+ "removeFilter": 32.623333333333335,+ "dirtyFilter": 4093.1900000000005},"visitedPerSample": {"candidates": 100000,@@ -302,6 +429,7 @@"cells": 0,"spatialCandidates": 0,"owner": 8,+ "unindexed": 0,"enter": 7020.333333333333,"leave": 7020.666666666667,"removals": 0,@@ -322,17 +450,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 2190655.8333333335,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 2568625.4200000004,"segmentUsPerSimSecond": {- "materialize": 2569.86,- "build": 1661726.3900000001,- "anchor": 156.39666666666668,- "radius": 503567.6533333334,- "owner": 57.766666666666666,- "enter": 15442.490000000002,- "leave": 2798.76,- "removeFilter": 8.75,- "dirtyFilter": 3694.166666666667+ "materialize": 3138.4700000000003,+ "ensure": 28452.920000000002,+ "build": 1863227.2233333334,+ "anchor": 292.7833333333333,+ "radius": 660187.8933333333,+ "owner": 929.5699999999999,+ "enter": 3815.8300000000004,+ "leave": 3167.9133333333334,+ "removeFilter": 22.506666666666668,+ "dirtyFilter": 4069.9766666666665},"visitedPerSample": {"candidates": 100000,@@ -343,6 +474,7 @@"cells": 0,"spatialCandidates": 0,"owner": 8,+ "unindexed": 0,"enter": 7020.333333333333,"leave": 7020.666666666667,"removals": 0,@@ -363,17 +495,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 1751896.11,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 2259926.81,"segmentUsPerSimSecond": {- "materialize": 2676.25,- "build": 1696853.1966666665,- "anchor": 60.71000000000001,- "radius": 26192.373333333337,- "owner": 34.303333333333335,- "enter": 3228.2033333333334,- "leave": 18526.803333333333,- "removeFilter": 9.453333333333333,- "dirtyFilter": 3829.4366666666665+ "materialize": 2682.5,+ "ensure": 28546.523333333334,+ "build": 2179859.1700000004,+ "anchor": 98.18666666666667,+ "radius": 36795.556666666664,+ "owner": 35.14333333333333,+ "enter": 3525.0033333333336,+ "leave": 3207.9133333333334,+ "removeFilter": 11.806666666666667,+ "dirtyFilter": 4301.666666666667},"visitedPerSample": {"candidates": 100000,@@ -384,6 +519,57 @@"cells": 216,"spatialCandidates": 19766.666666666668,"owner": 8,+ "unindexed": 0,+ "enter": 7020.333333333333,+ "leave": 7020.666666666667,+ "removals": 0,+ "dirty": 8000+ },+ "visiblePerConnection": 877.5416666666666+ },+ {+ "arm": "D1",+ "entities": 100000,+ "connections": 8,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 17.841241161527712,+ "verify": false,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 24000,+ "seedUs": 125241.584,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 123480.84000000001,+ "segmentUsPerSimSecond": {+ "materialize": 3056.9500000000003,+ "ensure": 35534.723333333335,+ "build": 22025.976666666666,+ "anchor": 136.80666666666667,+ "radius": 41304.02333333334,+ "owner": 41.806666666666665,+ "enter": 13704.863333333333,+ "leave": 3146.3866666666668,+ "removeFilter": 14.320000000000002,+ "dirtyFilter": 4070.826666666667+ },+ "visitedPerSample": {+ "candidates": 100000,+ "positionReads": 668,+ "anchor": 8,+ "radius": 0,+ "distanceTests": 19766.666666666668,+ "cells": 216,+ "spatialCandidates": 19766.666666666668,+ "owner": 8,+ "unindexed": 0,"enter": 7020.333333333333,"leave": 7020.666666666667,"removals": 0,@@ -404,17 +590,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 46231171.803333335,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 55593067.64,"segmentUsPerSimSecond": {- "materialize": 2744.5833333333335,+ "materialize": 2613.4733333333334,+ "ensure": 29360.83666666667,"build": 0,- "anchor": 3147.6666666666665,- "radius": 46189352.21333334,- "owner": 1179.73,- "enter": 11573.193333333335,- "leave": 9274.45,- "removeFilter": 57.910000000000004,- "dirtyFilter": 10929.273333333334+ "anchor": 4726.673333333333,+ "radius": 55515795.17,+ "owner": 1911.27,+ "enter": 12839.840000000002,+ "leave": 10287.616666666667,+ "removeFilter": 90.53333333333333,+ "dirtyFilter": 10954.146666666667},"visitedPerSample": {"candidates": 100000,@@ -425,6 +614,7 @@"cells": 0,"spatialCandidates": 0,"owner": 32,+ "unindexed": 0,"enter": 29563.5,"leave": 29564.833333333332,"removals": 0,@@ -445,17 +635,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 3333987.5,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 3640731.943333334,"segmentUsPerSimSecond": {- "materialize": 2688.19,- "build": 1539957.5033333334,- "anchor": 310.11,- "radius": 1758727.23,- "owner": 1343.31,- "enter": 10301.960000000001,- "leave": 9186.673333333334,- "removeFilter": 22.76666666666667,- "dirtyFilter": 10439.44+ "materialize": 2784.0233333333335,+ "ensure": 26257.5,+ "build": 1705089.7200000002,+ "anchor": 445.9766666666667,+ "radius": 1871381.7866666666,+ "owner": 1793.32,+ "enter": 10912.390000000001,+ "leave": 9486.956666666667,+ "removeFilter": 36.50333333333334,+ "dirtyFilter": 10877.346666666668},"visitedPerSample": {"candidates": 100000,@@ -466,6 +659,7 @@"cells": 0,"spatialCandidates": 0,"owner": 32,+ "unindexed": 0,"enter": 29563.5,"leave": 29564.833333333332,"removals": 0,@@ -486,17 +680,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 1842971.9466666668,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 1998318.0599999998,"segmentUsPerSimSecond": {- "materialize": 2781.5266666666666,- "build": 1716863.6133333333,- "anchor": 153.1966666666667,- "radius": 77080.54000000001,- "owner": 56.39666666666667,- "enter": 10767.353333333334,- "leave": 9459.976666666667,- "removeFilter": 18.063333333333333,- "dirtyFilter": 25281.53+ "materialize": 3330.1433333333334,+ "ensure": 28725.97333333333,+ "build": 1834928.48,+ "anchor": 175.42333333333335,+ "radius": 83046.51666666668,+ "owner": 58.03333333333334,+ "enter": 10880.153333333334,+ "leave": 9641.793333333333,+ "removeFilter": 15260.946666666667,+ "dirtyFilter": 11494.163333333334},"visitedPerSample": {"candidates": 100000,@@ -507,6 +704,57 @@"cells": 864,"spatialCandidates": 83266.5,"owner": 32,+ "unindexed": 0,+ "enter": 29563.5,+ "leave": 29564.833333333332,+ "removals": 0,+ "dirty": 32000+ },+ "visiblePerConnection": 923.859375+ },+ {+ "arm": "D1",+ "entities": 100000,+ "connections": 32,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 17.841241161527712,+ "verify": false,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 24000,+ "seedUs": 105210.417,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 154640.13333333333,+ "segmentUsPerSimSecond": {+ "materialize": 2511.6633333333334,+ "ensure": 30346.943333333336,+ "build": 17034.31,+ "anchor": 144.02,+ "radius": 71613.47333333334,+ "owner": 52.24666666666667,+ "enter": 10948.623333333333,+ "leave": 9918.633333333335,+ "removeFilter": 18.606666666666666,+ "dirtyFilter": 11573.346666666666+ },+ "visitedPerSample": {+ "candidates": 100000,+ "positionReads": 668,+ "anchor": 32,+ "radius": 0,+ "distanceTests": 83266.5,+ "cells": 864,+ "spatialCandidates": 83266.5,+ "owner": 32,+ "unindexed": 0,"enter": 29563.5,"leave": 29564.833333333332,"removals": 0,@@ -529,17 +777,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 1429582.5,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 1493869.03,"segmentUsPerSimSecond": {- "materialize": 113.05666666666667,- "build": 168227.22666666665,- "anchor": 85.81333333333333,- "radius": 3814.8633333333337,- "owner": 22.076666666666668,- "enter": 434.30333333333334,- "leave": 355.4066666666667,- "removeFilter": 25.16,- "dirtyFilter": 493.8766666666667+ "materialize": 120.27333333333334,+ "ensure": 2837.2200000000003,+ "build": 170708.61000000002,+ "anchor": 104.45,+ "radius": 4317.506666666667,+ "owner": 29.299999999999997,+ "enter": 433.61,+ "leave": 370.12666666666667,+ "removeFilter": 65.42,+ "dirtyFilter": 511.65333333333336},"visitedPerSample": {"candidates": 10000,@@ -550,6 +801,57 @@"cells": 216,"spatialCandidates": 1839.5,"owner": 8,+ "unindexed": 0,+ "enter": 573.6666666666666,+ "leave": 573.6666666666666,+ "removals": 0,+ "dirty": 800+ },+ "visiblePerConnection": 71.70833333333333+ },+ {+ "arm": "D1",+ "entities": 10000,+ "connections": 8,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 5.641895835477563,+ "verify": true,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 2400,+ "seedUs": 11599.542,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 1392836.53,+ "segmentUsPerSimSecond": {+ "materialize": 123.74666666666668,+ "ensure": 3549.996666666667,+ "build": 1989.03,+ "anchor": 94.73,+ "radius": 4409.443333333334,+ "owner": 28.60666666666667,+ "enter": 743.7433333333333,+ "leave": 399.30333333333334,+ "removeFilter": 22.493333333333336,+ "dirtyFilter": 495.8233333333334+ },+ "visitedPerSample": {+ "candidates": 10000,+ "positionReads": 68,+ "anchor": 8,+ "radius": 0,+ "distanceTests": 1839.5,+ "cells": 216,+ "spatialCandidates": 1839.5,+ "owner": 8,+ "unindexed": 0,"enter": 573.6666666666666,"leave": 573.6666666666666,"removals": 0,@@ -570,17 +872,20 @@"samples": 6,"mismatches": 0,"movedTotal": 2400,- "totalUsPerSimSecond": 5040214.72,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 5735095.833333334,"segmentUsPerSimSecond": {- "materialize": 115,- "build": 156925.4166666667,- "anchor": 319.43666666666667,- "radius": 14791.263333333334,- "owner": 57.93666666666667,- "enter": 1445.1233333333332,- "leave": 1181.55,- "removeFilter": 22.79666666666667,- "dirtyFilter": 1521.6299999999999+ "materialize": 121.67333333333335,+ "ensure": 2925.2766666666666,+ "build": 170047.9166666667,+ "anchor": 666.1366666666668,+ "radius": 19846.92666666667,+ "owner": 124.60000000000001,+ "enter": 1806.6533333333334,+ "leave": 1400.15,+ "removeFilter": 56.41666666666667,+ "dirtyFilter": 1671.3933333333334},"visitedPerSample": {"candidates": 10000,@@ -591,6 +896,57 @@"cells": 864,"spatialCandidates": 8354.5,"owner": 32,+ "unindexed": 0,+ "enter": 2942.8333333333335,+ "leave": 2943,+ "removals": 0,+ "dirty": 3200+ },+ "visiblePerConnection": 91.96354166666667+ },+ {+ "arm": "D1",+ "entities": 10000,+ "connections": 32,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 5.641895835477563,+ "verify": true,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 2400,+ "seedUs": 13321.959,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 5864912.64,+ "segmentUsPerSimSecond": {+ "materialize": 113.89000000000001,+ "ensure": 3746.3866666666668,+ "build": 2336.9466666666667,+ "anchor": 638.1800000000001,+ "radius": 22691.496666666666,+ "owner": 140.97,+ "enter": 1833.4633333333334,+ "leave": 1382.6366666666668,+ "removeFilter": 79.86666666666667,+ "dirtyFilter": 1786.63+ },+ "visitedPerSample": {+ "candidates": 10000,+ "positionReads": 68,+ "anchor": 32,+ "radius": 0,+ "distanceTests": 8354.5,+ "cells": 864,+ "spatialCandidates": 8354.5,+ "owner": 32,+ "unindexed": 0,"enter": 2942.8333333333335,"leave": 2943,"removals": 0,@@ -611,17 +967,20 @@"samples": 6,"mismatches": 0,"movedTotal": 24000,- "totalUsPerSimSecond": 15176458.33,+ "seedUs": 0,+ "drift": null,+ "totalUsPerSimSecond": 17427368.75,"segmentUsPerSimSecond": {- "materialize": 2737.6400000000003,- "build": 1789263.61,- "anchor": 243.3366666666667,- "radius": 36898.35666666667,- "owner": 35.7,- "enter": 3444.866666666667,- "leave": 2963.5933333333332,- "removeFilter": 33.333333333333336,- "dirtyFilter": 3809.996666666667+ "materialize": 3229.0233333333335,+ "ensure": 29783.476666666666,+ "build": 1907319.5833333335,+ "anchor": 498.3366666666667,+ "radius": 47509.45,+ "owner": 75.13333333333334,+ "enter": 3900,+ "leave": 3126.5266666666666,+ "removeFilter": 44.016666666666666,+ "dirtyFilter": 4815.256666666667},"visitedPerSample": {"candidates": 100000,@@ -632,6 +991,57 @@"cells": 216,"spatialCandidates": 19766.666666666668,"owner": 8,+ "unindexed": 0,+ "enter": 7020.333333333333,+ "leave": 7020.666666666667,+ "removals": 0,+ "dirty": 8000+ },+ "visiblePerConnection": 877.5416666666666+ },+ {+ "arm": "D1",+ "entities": 100000,+ "connections": 8,+ "anchors": 1,+ "visible": 0.01,+ "movement": 0.01,+ "radius": 17.841241161527712,+ "verify": true,+ "simSeconds": 0.3,+ "samples": 6,+ "mismatches": 0,+ "movedTotal": 24000,+ "seedUs": 105109.917,+ "drift": {+ "position": 0,+ "cell": 0,+ "missing": 0,+ "extra": 0+ },+ "totalUsPerSimSecond": 14596775.283333333,+ "segmentUsPerSimSecond": {+ "materialize": 2678.7533333333336,+ "ensure": 30543.746666666666,+ "build": 17419.306666666667,+ "anchor": 239.44333333333333,+ "radius": 40537.090000000004,+ "owner": 43.616666666666674,+ "enter": 3502.3600000000006,+ "leave": 3061.243333333334,+ "removeFilter": 32.64666666666667,+ "dirtyFilter": 3893.1866666666665+ },+ "visitedPerSample": {+ "candidates": 100000,+ "positionReads": 668,+ "anchor": 8,+ "radius": 0,+ "distanceTests": 19766.666666666668,+ "cells": 216,+ "spatialCandidates": 19766.666666666668,+ "owner": 8,+ "unindexed": 0,"enter": 7020.333333333333,"leave": 7020.666666666667,"removals": 0,diff --git a/bench/replication-interest/run.mjs b/bench/replication-interest/run.mjsindex 5c2df1c25..84b2b8cc0 100644--- a/bench/replication-interest/run.mjs+++ b/bench/replication-interest/run.mjs@@ -23,11 +23,12 @@ const ROOT = path.resolve(HERE, '..', '..');const ARM = path.join(HERE, 'arm.mjs');const QUICK = process.argv.includes('--quick');-const ARMS = ['B', 'C0', 'C1'];+const ARMS = ['B', 'C0', 'C1', 'D1'];const ARM_MEANING = {B: 'as shipped: every connection reads every candidate\'s position',C0: 'positions read once per sample, cached; same full walk',C1: 'grid rebuilt per sample; only nearby cells are walked',+ D1: 'the same grid, KEPT — only the entities the composition says moved',};/** Arm B reads a builtin position per candidate per connection, so 100k x 32 is* already 50 seconds of work per simulated second. Points stay few. */@@ -39,6 +40,8 @@ const FIXED = {simHz: 60, replHz: 20, warmup: QUICK ? 4 : 6, measure: QUICK ? 9 : 18,};const VERIFY_POINTS = QUICK ? [[10000, 8]] : [[10000, 8], [10000, 32], [100000, 8]];+/** The arms whose answer has to be checked against the full scan's, not just timed. */+const VERIFY_ARMS = ['C1', 'D1'];const CACHE = path.join(HERE, `.sweep-${QUICK ? 'quick' : 'matrix'}.jsonl`);const buildId = () => sdkIdentity(ROOT).sdkArtifactSha256;@@ -94,7 +97,7 @@ function measure(arm, entities, connections, verify) {buildSdkOnce();const points = [];-const total = POINTS.length * ARMS.length + VERIFY_POINTS.length;+const total = POINTS.length * ARMS.length + VERIFY_POINTS.length * VERIFY_ARMS.length;let done = 0;for (const [entities, connections] of POINTS) {for (const arm of ARMS) {@@ -106,10 +109,12 @@ for (const [entities, connections] of POINTS) {}const verifies = [];for (const [entities, connections] of VERIFY_POINTS) {- const { result, fresh } = measure('C1', entities, connections, true);- process.stderr.write(`[${++done}/${total}] differential ${entities} ${connections} conn`- + `${fresh ? '' : ' (cached)'}\n`);- verifies.push(result);+ for (const arm of VERIFY_ARMS) {+ const { result, fresh } = measure(arm, entities, connections, true);+ process.stderr.write(`[${++done}/${total}] differential ${arm} ${entities} ${connections} conn`+ + `${fresh ? '' : ' (cached)'}\n`);+ verifies.push(result);+ }}const at = (arm, e, c) => points.find((p) => p.arm === arm && p.entities === e && p.connections === c);@@ -123,10 +128,22 @@ say('');for (const arm of ARMS) say(` ${arm} ${ARM_MEANING[arm]}`);say('');const clean = verifies.every((v) => v.mismatches === 0);+const stranded = verifies.concat(points).reduce((n, v) => n + (v.visitedPerSample?.unindexed ?? 0), 0);say(clean- ? ` DIFFERENTIAL: the grid saw exactly what the full scan saw, every connection,`+ ? ` DIFFERENTIAL: both grids saw exactly what the full scan saw, every connection,`+ ` every sample (${verifies.length} point(s)).`- : ' DIFFERENTIAL: MISMATCHED — the grid changed what a connection sees.');+ : ' DIFFERENTIAL: MISMATCHED — a grid changed what a connection sees.');+say(stranded === 0+ ? ' Every entity the composition reported was one the kept grid knew about.'+ : ` ${stranded} reported entit(ies) were not in the kept grid — it is missing rows.`);+// The kept grid against a rebuilt one, which is the claim it makes: a cell that+// drifted only reaches the visible-set check if it changes somebody's view.+const drifts = points.concat(verifies).map((p) => p.drift).filter(Boolean);+const drifted = drifts.reduce((n, d) => n + d.position + d.cell + d.missing + d.extra, 0);+say(drifts.length === 0 ? ' NO KEPT GRID WAS COMPARED against a rebuilt one.'+ : drifted === 0+ ? ` The kept grid held exactly what a rebuilt one would, at ${drifts.length} point(s).`+ : ` The kept grid DRIFTED from a rebuilt one: ${drifted} entr(ies) across ${drifts.length} point(s).`);say('');say(' us per simulated second, and the share of one core');say(' pop conn arm total 1 core build radius vs B');diff --git a/bench/replication-interest/workload.mjs b/bench/replication-interest/workload.mjsindex 89e1b8ebb..0d4b41d34 100644--- a/bench/replication-interest/workload.mjs+++ b/bench/replication-interest/workload.mjs@@ -45,6 +45,7 @@ export async function connectEngine(sdk, app, root) {const factory = (await import(pathToFileURL(path.join(dir, 'esengine.js')).href)).default;const module = await factory({ locateFile: (f) => path.join(dir, f) });app.connectCpp(new module.Registry(), module, { strict: false });+ return module;}export function defineComponents(sdk) {@@ -63,7 +64,7 @@ export function newStats() {},visited: {candidates: 0, positionReads: 0, anchor: 0, radius: 0, distanceTests: 0,- cells: 0, spatialCandidates: 0, owner: 0,+ cells: 0, spatialCandidates: 0, owner: 0, unindexed: 0,enter: 0, leave: 0, removals: 0, dirty: 0,},visibleTotal: 0,@@ -232,6 +233,90 @@ export function buildGrid(world, ctx, stats, cellSize) {return { cells, placeless, cache, cellSize };}+/**+ * A uniform grid that is KEPT. The price of not rebuilding is an invalidation+ * contract: right only while every author of a world position reaches the+ * composition that reports the change. Buckets are arrays and the cache is the+ * map C1 builds, so the QUERY side is byte for byte C1's.+ */+export function newLiveGrid(cellSize) {+ return { cells: new Map(), placeless: new Set(), cache: new Map(), at: new Map(), cellSize };+}++function place(grid, e, p) {+ if (!p) { grid.placeless.add(e); return; }+ const key = cellKey(p.x, p.y, p.z ?? 0, grid.cellSize);+ let bucket = grid.cells.get(key);+ if (!bucket) { bucket = []; grid.cells.set(key, bucket); }+ grid.at.set(e, { key, idx: bucket.length });+ bucket.push(e);+}++/** Swap with the last and pop, fixing the moved entity's index — a cell is an+ * array so the radius pass walks it exactly as C1's does. */+function displace(grid, e) {+ const at = grid.at.get(e);+ if (!at) { grid.placeless.delete(e); return; }+ const bucket = grid.cells.get(at.key);+ const last = bucket[bucket.length - 1];+ bucket[at.idx] = last;+ grid.at.get(last).idx = at.idx;+ bucket.pop();+ grid.at.delete(e);+}++/** Once, at startup: the same walk C1 pays for every sample. */+export function seedLiveGrid(world, ctx, stats, grid) {+ const t = process.hrtime.bigint();+ for (const e of ctx.entities) {+ stats.visited.positionReads++;+ const p = positionOf(world, ctx.Pos, e);+ grid.cache.set(e, p);+ place(grid, e, p);+ }+ stats.ns.build += process.hrtime.bigint() - t;+}++/** Arm D1's whole build column: only the entities the composition said moved. */+export function updateLiveGrid(world, ctx, stats, grid, changed) {+ const t = process.hrtime.bigint();+ for (const e of changed) {+ if (!grid.cache.has(e)) { stats.visited.unindexed++; continue; }+ stats.visited.positionReads++;+ const p = positionOf(world, ctx.Pos, e);+ grid.cache.set(e, p);+ const at = grid.at.get(e);+ const key = p ? cellKey(p.x, p.y, p.z ?? 0, grid.cellSize) : null;+ // Most movement is within a cell, and then the coordinate cache was the+ // entire update — the bucket it belongs to has not changed.+ if (at && key === at.key) continue;+ displace(grid, e);+ place(grid, e, p);+ }+ stats.ns.build += process.hrtime.bigint() - t;+}++/** What a kept grid holds against what a rebuilt one would: zero on every count,+ * or it has been carrying a lie since whichever update dropped it. */+export function gridDrift(live, fresh) {+ const d = { position: 0, cell: 0, missing: 0, extra: 0 };+ for (const [e, b] of fresh.cache) {+ const a = live.cache.get(e);+ if (!a !== !b || (a && b && (a.x !== b.x || a.y !== b.y || (a.z ?? 0) !== (b.z ?? 0)))) d.position++;+ }+ const held = new Map();+ for (const [key, bucket] of live.cells) for (const e of bucket) held.set(e, key);+ const want = new Map();+ for (const [key, bucket] of fresh.cells) for (const e of bucket) want.set(e, key);+ for (const [e, key] of want) {+ const k = held.get(e);+ if (k === undefined) d.missing++;+ else if (k !== key) d.cell++;+ }+ for (const e of held.keys()) if (!want.has(e)) d.extra++;+ return d;+}+/** Arm C0: cached coordinates, same full walk. */export function visibleForC0(ctx, connId, candidates, r2, stats, ownedIndex, cache) {let t = process.hrtime.bigint();diff --git a/bench/transform-composition/README.md b/bench/transform-composition/README.mdindex 44d409176..f99dd5b15 100644--- a/bench/transform-composition/README.md+++ b/bench/transform-composition/README.md@@ -102,8 +102,12 @@ never move their probe, so the check is live where movement is.- Nothing ships. `composeCollecting` has one caller and it is this bench; theshipped `ensureComposed` path collects nothing and pays one predictable branchper entity, which the A column is what says is affordable.-- The set is COUNTED here, not consumed. What an index costs to update from it —- cell removal, reinsertion, and the ownership of entities that left the world- entirely — is the next measurement, not this one.+- **The set is a value-change journal, not a membership one.** A new entity+ spawned at the origin is not in it — its composed output equals what its fields+ already held — and a despawned entity is never in it. A consumer that maintains+ a structure needs membership from wherever it already gets membership.+- Consuming it is measured next door: `bench/replication-interest` keeps a grid+ from this set and takes the interest path at 100k x 32 from 200% of a core to+ 15%, with the kept grid holding exactly what a rebuilt one would.- One compose per pass. A frame that runs several fixed steps composes severaltimes, and the changed sets would have to be unioned or consumed per step.diff --git a/src/esengine/bindings/WebSDKEntry.cpp b/src/esengine/bindings/WebSDKEntry.cppindex 99ba245b3..2edfafc20 100644--- a/src/esengine/bindings/WebSDKEntry.cpp+++ b/src/esengine/bindings/WebSDKEntry.cpp@@ -660,6 +660,13 @@ EMSCRIPTEN_BINDINGS(esengine_renderer) {out.set("ran", ran);out.set("visited", ts ? ts->visited() : 0u);out.set("changed", static_cast<esengine::u32>(ran ? changed.size() : 0));+ // The ids themselves, as an address into linear memory: a JS array of+ // a hundred thousand handles would cost more than the walk that found+ // them. Valid until the next call, which reuses the vector.+ static_assert(sizeof(esengine::Entity) == sizeof(esengine::u32),+ "the changed set is read as a Uint32Array of raw ids");+ out.set("ptr", static_cast<esengine::u32>(+ reinterpret_cast<uintptr_t>(changed.data())));return out;}), emscripten::allow_raw_pointers());emscripten::function("renderer_setEntityDrawOrder", &esengine::renderer_setEntityDrawOrder);