A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
bench(net): what the interest send path visits, and what it sends
esengine51f664c03ee0
diff --git a/bench/replication-routing/README.md b/bench/replication-routing/README.mdnew file mode 100644index 000000000..b6dfaf178--- /dev/null+++ b/bench/replication-routing/README.md@@ -0,0 +1,117 @@+# The interest send path, once visibility is local++The provider took the spatial question from 5504% of a core to 18%. What is left+in `sampleWithInterest_` is per connection, and two of its passes walk a GLOBAL+list every time:++```+for each connection:+ removals.filter((r) => visible.has(r.entity) && …) // O(C × R)+ for (const d of dirty) if (!visible.has(d.entity)) … // O(C × D)+```++Neither knows that a connection sees 1% of the world. This measures what they+VISIT against what survives them — and, first, whether they are the wall at all.++Decomposition, not optimisation: this drives the real `ReplicationServer` with+32 real connections and changes nothing.++## How it is measured++`node bench/replication-routing/run.mjs`. 100,000 entities, 32 connections, the+shipped `radiusInterestProvider` at 1% visible, and the server sampling every+fixed tick — 60 a second, not 20: `ReplicationSampleSystem` runs in+FixedPostUpdate, so C × D is paid at the simulation rate.++The two axes are swept apart rather than crossed, because the question is which+of them is a wall. Rates are what the bench WRITES; the row counts are what the+server actually has to route, which is not the same number — moving an entity+dirties it, and so does giving a removed component back.++Times are the **fastest** measured sample, not the mean. A neighbouring process+only ever makes a sample slower, and the fastest reproduces to the microsecond+between runs where the mean moves by 5%. Two arms run back to back once differed+by 40% on the mean and not at all on the minimum.++## First: the two filters are not the wall++| workload | µs/sample | one core | over the floor |+|---|---|---|---|+| **nothing moves at all** | 7,690 | 46% | — |+| floor: 1% movement, nothing else | 14,477 | 87% | — |+| dirty 0.1% | 17,524 | 105% | +3,047 |+| **dirty 1%** | 19,343 | 116% | +4,866 |+| dirty 10% | 33,756 | 203% | +19,279 |+| dirty 100% | 90,828 | 545% | +76,351 |+| removals 0.1% | 17,763 | 107% | +3,285 |+| removals 1% | 18,007 | 108% | +3,530 |+| removals 10% | 21,587 | 130% | +7,110 |+| **mixed — 1% dirty, 0.1% removed** | 19,563 | 117% | +5,086 |++At the rates a game actually runs at, **three quarters of the sample is there+before a single extra dirty row exists**. Routing the mixed workload's rows adds+26%. The filters only take over past 10% dirty, where they add 19 ms.++## But what they visit is almost all waste++| workload | D rows | C × D visits | actually sent | ratio |+|---|---|---|---|---|+| 1% movement only | 1,000 | 32,000 | 270 | **119:1** |+| dirty 1% | 1,991 | 63,712 | 567 | **112:1** |+| dirty 10% | 10,902 | 348,853 | 3,201 | 109:1 |+| dirty 100% | 100,000 | 3,200,000 | 29,547 | 108:1 |++| workload | R rows | C × R visits | actually sent | ratio |+|---|---|---|---|---|+| removals 0.1% | 100 | 3,200 | 31 | **102:1** |+| removals 1% | 1,000 | 32,000 | 292 | 110:1 |+| removals 10% | 10,000 | 320,000 | 2,933 | 109:1 |++The ratio is the coverage: 32 connections seeing 923 entities each cover about a+third of a hundred thousand, and a row is asked about 32 times to be sent 0.3+times. It barely moves with the rate, which is what makes it structural rather+than a tuning question.++## What a router would need, and what it would cost to keep++| | spread connections | packed together |+|---|---|---|+| V per connection | 923 | 490 |+| C × V | 29,547 | 15,689 |+| viewers per visible entity | **1.00** | **8.10** |+| enters per sample, all connections | 7 | 3 |+| leaves per sample, all connections | 5 | 4 |++Both candidates are bounded, and by different things:++- **Pull by visible** — for each connection, ask its ~923 visible entities+ whether they are dirty. `O(C × V)` = 29,547, whatever D is. It adds no new+ order: the enter/leave diff is already O(C × V).+- **Push by reverse interest** — for each dirty row, tell the connections that+ can see it. `O(D × viewers)` = 2,089 at the mixed point, 32× less than C × D+ and 14× less than pull.++So the crossover is `D × viewers` against `C × V`: push while fewer than ~29,000+rows are dirty a sample, pull past that. At the mixed workload push wins by 14×;+at 100% dirty pull wins by 3×.++**Maintaining the reverse index is not the tax it looks like.** Twelve enter and+leave events per sample across all 32 connections — the interest sets barely move+once the world is steady, which is the same fact that makes the enter/leave DIFF+(29,547 comparisons to find 12 events) look the way it does.++## What this does not cover++- **Encoding is inside the totals and not separated.** It is proportional to what+ is actually SENT — 598 entries at the mixed point — so it is the floor a+ perfect router leaves behind, not part of the wall. Separating it needs the+ frame writer, which is not on the SDK's entry.+- **The still floor is not explained here.** 7,690 µs with nothing moving is+ about 0.14 µs per (connection × visible entity) plus a constant that does not+ follow the population — a quarter of the entities at the same C × V costs+ 3,675. Whatever that constant is, it is not C × D and not C × R.+- One anchor per connection, and a uniform grid world. `viewers per entity` is+ the number the push/pull choice turns on, and it is a property of how players+ are arranged: 1.00 spread across the map, 8.10 packed into a thousandth of it.+- Client apps are connected but not ticked during measurement. What is timed is+ the server's own sample.diff --git a/bench/replication-routing/arm.mjs b/bench/replication-routing/arm.mjsnew file mode 100644index 000000000..e639133ef--- /dev/null+++ b/bench/replication-routing/arm.mjs@@ -0,0 +1,223 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file What the interest send path costs, on the real server.+ *+ * Visibility is a local question now. What is still global x connections is the+ * two filters after it: every removal is asked about for every connection, and+ * so is every dirty row. This drives `ReplicationServer` itself — no+ * reproduction — and counts what those two passes VISIT against what they send.+ *+ * node bench/replication-routing/arm.mjs --dirty 0.01 --removals 0.001+ */+import path from 'node:path';+import { fileURLToPath, pathToFileURL } from 'node:url';++const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');+const argv = process.argv.slice(2);+const flag = (name, fallback) => {+ const i = argv.indexOf(`--${name}`);+ return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;+};++const ENTITIES = Number(flag('entities', '100000'));+const CONNECTIONS = Number(flag('connections', '32'));+const VISIBLE = Number(flag('visible', '0.01'));+const MOVEMENT = Number(flag('movement', '0.01'));+const DIRTY = Number(flag('dirty', '0'));+const REMOVALS = Number(flag('removals', '0'));+/** How much of the world the anchors are spread over. 1 = the whole map, and each+ * entity is then in one connection's view; smaller packs the connections together+ * and the same entity is seen by many, which is what decides push against pull. */+const CLUSTER = Number(flag('cluster', '1'));+const WARMUP = Number(flag('warmup', '12'));+const MEASURE = Number(flag('measure', '60'));+/** How many measured samples to COUNT. The counts are steady-state and the census+ * is an O(E) rebuild plus C queries, so counting every one would cost more wall+ * clock than the thing being measured. */+const CENSUS = Number(flag('census', '3'));+const STEP = 1 / 60;++const { loadSdk, sdkIdentity } = await import(pathToFileURL(+ path.join(ROOT, 'bench/replication-interest/workload.mjs')).href);+const sdk = await loadSdk(ROOT);+const wasmDir = process.env.ESENGINE_WASM_DIR ?? path.join(ROOT, 'build', 'wasm', 'web');+const factory = (await import(pathToFileURL(path.join(wasmDir, 'esengine.js')).href)).default;+const engine = await factory({ locateFile: (f) => path.join(wasmDir, f) });++/** A second replicated component, so "dirty" and "moved" are separate knobs. */+const Health = sdk.defineComponent('BenchHealth', { hp: 0 }, { replicatedFields: ['hp'] });++const serverApp = sdk.App.new();+serverApp.connectCpp(new engine.Registry(), engine, { strict: false });+serverApp.addPlugin(sdk.replicationPlugin);+const server = serverApp.getResource(sdk.Net).startServer();+const world = serverApp.world;++const side = Math.ceil(Math.sqrt(ENTITIES));+const RADIUS = VISIBLE >= 1 ? side * 2 : Math.sqrt((VISIBLE * ENTITIES) / Math.PI);+server.setInterestProvider(sdk.radiusInterestProvider(RADIUS));++const clients = [];+for (let i = 0; i < CONNECTIONS; i++) {+ const app = sdk.App.new();+ app.connectCpp(new engine.Registry(), engine, { strict: false });+ app.addPlugin(sdk.replicationPlugin);+ const [ta, tb] = sdk.MemoryTransport.pair();+ const id = server.attachConnection(ta);+ await app.getResource(sdk.Net).connect(tb, { interpolationDelayTicks: 0 });+ clients.push({ app, id });+}++const entities = new Array(ENTITIES);+for (let i = 0; i < ENTITIES; i++) {+ const e = world.spawn();+ world.insert(e, sdk.Replicated, { owner: -1 });+ world.insert(e, sdk.Transform, { position: { x: i % side, y: Math.floor(i / side), z: 0 } });+ world.insert(e, Health, { hp: 100 });+ entities[i] = e;+}+/** One anchor per connection, spread so a view is a neighbourhood, not a corner. */+const owned = new Map();+const span = Math.max(1, Math.floor(ENTITIES * CLUSTER));+const stride = Math.max(1, Math.floor(span / CONNECTIONS));+for (let c = 0; c < CONNECTIONS; c++) {+ const e = entities[(c * stride) % ENTITIES];+ world.set(e, sdk.Replicated, { owner: c });+ owned.set(c, [e]);+}++const MOVE = Math.round(ENTITIES * MOVEMENT);+const DIRTY_ROWS = Math.round(ENTITIES * DIRTY);+const REMOVE_ROWS = Math.round(ENTITIES * REMOVALS);+/** The window taken off last tick, put back this one, so the rate is sustained. */+let restoring = [];++/**+ * One tick's worth of world change. Everything that makes an entity dirty ends+ * up in ONE list, because that is what the server has to route: moving an entity+ * dirties it, and so does giving a component back.+ */+function mutate(tick) {+ const dirtied = new Set();+ for (let k = 0; k < MOVE; k++) {+ const e = entities[(tick * 7919 + k * 7919) % ENTITIES];+ world.update(e, sdk.Transform, (d) => { d.position.x += ((tick + k) % 3) - 1; });+ dirtied.add(e);+ }+ // A different stride, so the dirty rows are not the moved ones.+ for (let k = 0; k < DIRTY_ROWS; k++) {+ const e = entities[(tick * 104729 + k * 104729) % ENTITIES];+ // The removal window may have taken this one's component away.+ if (!world.has(e, Health)) continue;+ world.update(e, Health, (d) => { d.hp = (d.hp + 1) & 0xffff; });+ dirtied.add(e);+ }+ for (const e of restoring) {+ if (!world.valid(e)) continue;+ world.insert(e, Health, { hp: 100 });+ dirtied.add(e);+ }+ restoring = [];+ for (let k = 0; k < REMOVE_ROWS; k++) {+ const e = entities[(tick * 15485863 + k * 15485863) % ENTITIES];+ if (!world.has(e, Health)) continue;+ world.remove(e, Health);+ restoring.push(e);+ }+ return { dirtied: [...dirtied], removed: [...restoring] };+}++/**+ * What the server's two filters were asked about, and what came out. Untimed and+ * outside the server: a snapshot provider reading the same canonical positions+ * answers the same question, which is what makes the ratio a fact about the+ * mechanism rather than about this file.+ */+const canonical = (w, e) => {+ if (!w.has(e, sdk.Transform)) return null;+ const t = w.tryGet(e, sdk.Transform);+ return t ? { x: t.worldPosition.x, y: t.worldPosition.y, z: t.worldPosition.z } : null;+};+const oracle = sdk.radiusInterestProvider(RADIUS, { position: canonical });++const totals = {+ samples: 0, visible: 0, dirtyVisits: 0, dirtySent: 0,+ removalVisits: 0, removalSent: 0, dirtyRows: 0, removalRows: 0, viewers: 0, viewed: 0,+ enters: 0, leaves: 0, diffed: 0,+};+/** Last sample's view per connection, so the enter/leave EVENTS can be counted —+ * what a reverse-interest index would have to be maintained by. */+const previous = new Map();++function census(made) {+ world.ensureTransformsComposed();+ const prepared = oracle.prepare({ world, entities, entityCount: entities.length });+ const seenBy = new Map();+ for (let c = 0; c < CONNECTIONS; c++) {+ const answer = prepared.query({ connectionId: c, owned: owned.get(c) });+ const visible = answer === 'all' ? new Set(entities) : answer;+ totals.visible += visible.size;+ for (const e of visible) seenBy.set(e, (seenBy.get(e) ?? 0) + 1);+ const before = previous.get(c);+ if (before) {+ for (const e of visible) if (!before.has(e)) totals.enters++;+ for (const e of before) if (!visible.has(e)) totals.leaves++;+ totals.diffed++;+ }+ previous.set(c, visible);+ // The two filters walk the WHOLE list for every connection; what survives+ // is the intersection with this connection's view.+ for (const e of made.dirtied) if (visible.has(e)) totals.dirtySent++;+ for (const e of made.removed) if (visible.has(e)) totals.removalSent++;+ }+ for (const n of seenBy.values()) { totals.viewers += n; totals.viewed++; }+ totals.dirtyRows += made.dirtied.length;+ totals.removalRows += made.removed.length;+ totals.dirtyVisits += CONNECTIONS * made.dirtied.length;+ totals.removalVisits += CONNECTIONS * made.removed.length;+ totals.samples++;+}++let ns = 0n;+let ticks = 0;+// Kept per tick as well as summed: a neighbouring process only ever makes a+// sample slower, so the fastest one is what this machine can do and the mean is+// what it was doing at the time.+let fastest = Infinity;+for (let t = 0; t < WARMUP; t++) { mutate(t); await serverApp.tick(STEP); }+for (let t = 0; t < MEASURE; t++) {+ const made = mutate(WARMUP + t);+ const t0 = process.hrtime.bigint();+ await serverApp.tick(STEP);+ const dt = Number(process.hrtime.bigint() - t0);+ ns += BigInt(dt);+ fastest = Math.min(fastest, dt / 1000);+ ticks++;+ if (t >= MEASURE - CENSUS) census(made);+}++const us = Number(ns) / 1000;+const per = (v) => v / totals.samples;+const usPerSample = us / ticks;+process.stdout.write(`${JSON.stringify({+ entities: ENTITIES, connections: CONNECTIONS, visible: VISIBLE, movement: MOVEMENT,+ cluster: CLUSTER,+ dirty: DIRTY, removals: REMOVALS, samples: totals.samples,+ usPerSample, usPerSampleMin: fastest,+ oneCorePercent: usPerSample * 60 / 1e4,+ oneCorePercentMin: fastest * 60 / 1e4,+ visiblePerConnection: per(totals.visible) / CONNECTIONS,+ dirtyRows: per(totals.dirtyRows),+ dirtyVisits: per(totals.dirtyVisits),+ dirtySent: per(totals.dirtySent),+ removalRows: per(totals.removalRows),+ removalVisits: per(totals.removalVisits),+ removalSent: per(totals.removalSent),+ viewersPerViewedEntity: totals.viewed > 0 ? totals.viewers / totals.viewed : 0,+ // Per SAMPLE across all connections, which is the rate a reverse index would+ // have to be maintained at.+ entersPerSample: totals.diffed > 0 ? (totals.enters / totals.diffed) * CONNECTIONS : 0,+ leavesPerSample: totals.diffed > 0 ? (totals.leaves / totals.diffed) * CONNECTIONS : 0,+ ...sdkIdentity(ROOT),+})}\n`);diff --git a/bench/replication-routing/run.mjs b/bench/replication-routing/run.mjsnew file mode 100644index 000000000..87fb3c23e--- /dev/null+++ b/bench/replication-routing/run.mjs@@ -0,0 +1,138 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Where the interest SEND path's C x E goes, now that visibility is local.+ *+ * node bench/replication-routing/run.mjs+ * node bench/replication-routing/run.mjs --quick+ *+ * Decomposition, not optimisation: this drives the real server and changes+ * nothing. The two axes are swept apart rather than crossed, because what is+ * being asked is which of them is a wall, not how they interact.+ */+import { spawnSync } from 'node:child_process';+import os from 'node:os';+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { sdkIdentity } from '../replication-interest/workload.mjs';++const HERE = path.dirname(fileURLToPath(import.meta.url));+const ROOT = path.resolve(HERE, '..', '..');+const ARM = path.join(HERE, 'arm.mjs');+const QUICK = process.argv.includes('--quick');++/** The workload the provider was confirmed on, frozen. */+const FIXED = {+ entities: QUICK ? 20000 : 100000,+ connections: QUICK ? 8 : 32,+ visible: 0.01,+ movement: 0.01,+ warmup: QUICK ? 6 : 10,+ measure: QUICK ? 12 : 30,+};++/**+ * The floor first. If what is left with nothing moving is O(C x V), halving what+ * a connection sees halves it — so the shape is asked directly rather than+ * attributed by a timer inside the server.+ */+const FLOOR_POINTS = [+ { name: 'still — nothing moves at all', dirty: 0, removals: 0, movement: 0 },+ { name: 'still, 0.25% visible', dirty: 0, removals: 0, movement: 0, visible: 0.0025 },+ { name: 'still, 0.5% visible', dirty: 0, removals: 0, movement: 0, visible: 0.005 },+ { name: 'still, 2% visible', dirty: 0, removals: 0, movement: 0, visible: 0.02 },+ { name: 'still, 8 connections', dirty: 0, removals: 0, movement: 0, connections: 8 },+ { name: 'still, 16 connections', dirty: 0, removals: 0, movement: 0, connections: 16 },+ { name: 'still, quarter the population', dirty: 0, removals: 0, movement: 0, entities: 25000, visible: 0.036 },+];++const POINTS = [+ { name: 'floor — nothing dirty, nothing removed', dirty: 0, removals: 0 },+ { name: 'dirty 0.1%', dirty: 0.001, removals: 0 },+ { name: 'dirty 1%', dirty: 0.01, removals: 0 },+ { name: 'dirty 10%', dirty: 0.1, removals: 0 },+ { name: 'dirty 100%', dirty: 1, removals: 0 },+ { name: 'removals 0.1%', dirty: 0, removals: 0.001 },+ { name: 'removals 1%', dirty: 0, removals: 0.01 },+ { name: 'removals 10%', dirty: 0, removals: 0.1 },+ { name: 'mixed — 1% dirty, 0.1% removed', dirty: 0.01, removals: 0.001 },+ { name: 'mixed, connections packed together', dirty: 0.01, removals: 0.001, cluster: 0.001 },+];++function run(p) {+ const out = spawnSync(process.execPath, [+ ARM, '--entities', String(p.entities ?? FIXED.entities),+ '--connections', String(p.connections ?? FIXED.connections),+ '--visible', String(p.visible ?? FIXED.visible),+ '--movement', String(p.movement ?? FIXED.movement),+ '--dirty', String(p.dirty), '--removals', String(p.removals),+ '--cluster', String(p.cluster ?? 1),+ '--warmup', String(FIXED.warmup), '--measure', String(FIXED.measure),+ ], { encoding: 'utf8', cwd: ROOT, maxBuffer: 1 << 24 });+ if (out.status !== 0) {+ console.error(out.stderr || out.stdout);+ throw new Error(`${p.name} exited ${out.status}`);+ }+ return JSON.parse(out.stdout);+}++const all = [...FLOOR_POINTS, ...POINTS];+const measured = new Map();+for (const [i, p] of all.entries()) {+ process.stderr.write(`[${i + 1}/${all.length}] ${p.name}\n`);+ measured.set(p, run(p));+}+const floorRows = FLOOR_POINTS.map((p) => ({ p, r: measured.get(p) }));+const rows = POINTS.map((p) => ({ p, r: measured.get(p) }));++const say = (s = '') => console.log(s);+const pad = (v, w) => String(v).padStart(w);+const round = (v) => Math.round(v);+const ratio = (a, b) => (b > 0 ? `${Math.round(a / b)}:1` : '—');++say('');+say(`=== the interest send path (${FIXED.entities} entities, ${FIXED.connections} connections,`+ + ` ${FIXED.visible * 100}% visible) ===`);+say('');+const floor = rows[0].r;+say(' the FLOOR with nothing moving: visibility bookkeeping and the enter/leave diff');+say(' workload us/sample 1 core C x V us per C x V');+for (const { p, r } of floorRows) {+ const cv = r.visiblePerConnection * r.connections;+ say(` ${p.name.padEnd(42)} ${pad(round(r.usPerSampleMin), 8)} ${pad(`${r.oneCorePercentMin.toFixed(0)}%`, 7)}`+ + ` ${pad(round(cv), 8)} ${pad((r.usPerSampleMin / cv).toFixed(3), 15)}`);+}+say('');+// The FASTEST measured sample, not the mean: a neighbouring process only ever+// makes one slower, and the fastest reproduces to the microsecond between runs+// where the mean moves by 5%.+say(' us per sample (fastest), and the share of one core at 60 samples a second');+say(' workload us/sample 1 core over floor');+for (const { p, r } of rows) {+ const over = r === floor ? '—' : `+${round(r.usPerSampleMin - floor.usPerSampleMin)}`;+ say(` ${p.name.padEnd(42)} ${pad(round(r.usPerSampleMin), 8)} ${pad(`${r.oneCorePercentMin.toFixed(0)}%`, 7)} ${pad(over, 11)}`);+}+say('');+say(' what the two filters VISIT, and what survives them (per sample)');+say(' workload D rows C x D sent ratio');+for (const { p, r } of rows) {+ say(` ${p.name.padEnd(42)} ${pad(round(r.dirtyRows), 6)} ${pad(round(r.dirtyVisits), 8)} ${pad(round(r.dirtySent), 7)} ${pad(ratio(r.dirtyVisits, r.dirtySent), 6)}`);+}+say('');+say(' workload R rows C x R sent ratio');+for (const { p, r } of rows) {+ say(` ${p.name.padEnd(42)} ${pad(round(r.removalRows), 6)} ${pad(round(r.removalVisits), 8)} ${pad(round(r.removalSent), 7)} ${pad(ratio(r.removalVisits, r.removalSent), 6)}`);+}+say('');+say(' what a router would have to know');+say(' workload V/conn C x V viewers/e enters leaves');+for (const { p, r } of rows) {+ say(` ${p.name.padEnd(42)} ${pad(round(r.visiblePerConnection), 6)}`+ + ` ${pad(round(r.visiblePerConnection * r.connections), 8)}`+ + ` ${pad(r.viewersPerViewedEntity.toFixed(2), 10)} ${pad(round(r.entersPerSample), 8)} ${pad(round(r.leavesPerSample), 8)}`);+}+say('');+say(' D rows is every row the server has to route, not the knob that was turned:');+say(' moving an entity dirties it, and so does giving a removed component back.');+say('');+say(` ${JSON.stringify({ ...sdkIdentity(ROOT), cpu: os.cpus()[0]?.model })}`);