A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
bench(net): a topology journal against the registry's O(E) reconcile
esengine8a5025785fe7
diff --git a/bench/replication-topology/.gitignore b/bench/replication-topology/.gitignorenew file mode 100644index 000000000..e6d85a8ca--- /dev/null+++ b/bench/replication-topology/.gitignore@@ -0,0 +1,3 @@+results.quick.json+results.big.json+.sweep-*.jsonldiff --git a/bench/replication-topology/README.md b/bench/replication-topology/README.mdnew file mode 100644index 000000000..f8f8f7621--- /dev/null+++ b/bench/replication-topology/README.md@@ -0,0 +1,85 @@+# Registry topology probe++`reconcileRegistry_()` reads every replicated entity and walks the whole+registry, **every sample, whether or not anything entered or left replication**.+With dirty-field discovery now down to candidates (PR7), that scan is what is+left of the O(E) in `sample()`.++This directory measures whether a **topology journal** removes it, and ships+nothing: production still runs the full reconcile.++## The arms++| | membership writes | reconcile |+|---|---|---|+| **A** | nothing recorded | full scan — today's production |+| **B** | journal written | full scan — isolates the journal's write tax |+| **C** | journal written | candidates only |++## The journal is not an add/remove log++It records one thing: *this entity's membership in this component moved at tick+T*. Not what moved, and not in which direction.++That is deliberate, and it is the same rule PR7 established one layer down:++```+field replication: history selects candidates, shadow × world decides truth+registry lifecycle: topology history selects candidates, known × world decides truth+```++So the reducer is four cells, and every out-of-order sequence falls out of it+rather than being special-cased:++| in `known` | live and replicated | result |+|---|---|---|+| no | no | nothing |+| no | yes | register + spawn |+| yes | no | unregister + despawn |+| yes | yes | nothing |++`add → remove` before a sample is one candidate that is neither known nor live:+nothing. `remove → add` is one that is both: nothing. A journal read as a wire+log would emit a spawn and a despawn for each.++## What it costs, and what it saves++`node bench/replication-topology/run.mjs` — 10k and 100k, five churn rates,+three arms, plus a differential at four points. The incremental registry matched+the full scan at **every sample** of every differential point.++`C/A`, and the tax as a share of one core:++| churn/sample | 10k C/A | 10k A | 10k C | 100k C/A | 100k A | 100k C |+|---|---|---|---|---|---|---|+| 0% | 0.00 | 0.8% | 0.0% | 0.00 | 14.3% | 0.0% |+| 0.01% | 0.00 | 1.3% | 0.0% | 0.00 | 21.8% | 0.0% |+| 0.1% | 0.02 | 1.2% | 0.0% | 0.01 | 20.8% | 0.1% |+| 1% | 0.09 | 1.5% | 0.1% | 0.05 | 28.5% | 1.4% |+| 10% | 0.42 | 2.7% | 1.1% | 0.36 | 39.3% | 14.3% |++Entities visited per sample at 100k goes from 200,000 (the population plus the+registry) to **0, 4, 27, 252, 2502**.++**There is no crossover.** Even at 10% of the population changing membership+every sample — far past anything a game does — C is a third of A. The 0% row is+the one that decides it: at 100k, a registry where nothing entered or left still+costs 14.3% of a core to confirm that.++`B/A` sits between 0.90 and 1.19 across the matrix, which is noise: the journal+is written only when a membership moves, never per field write. That is the+whole difference from enrolling a component in change tracking, where every+write pays.++## What this does not cover++- The journal is reproduced here, not implemented in `ChangeTracker`. Its write+ tax is one map lookup and one push, which is what the real one would be, but+ the real one also has to be produced from `recordAdded` / `recordRemovedById`.+- `world.valid()` in the reducer is belt to `has`'s braces — an entity id carries+ its generation, so a stale handle misses the storage map anyway. Sabotaging it+ alone does NOT redden this probe, and a fixture for handle reuse belongs with+ the production change.+- Interest is untouched. With a policy installed, every ready connection still+ builds `[...known_]` and scans it — once the registry scan is gone, that is the+ next O(C × E).diff --git a/bench/replication-topology/arm.mjs b/bench/replication-topology/arm.mjsnew file mode 100644index 000000000..6da4406f2--- /dev/null+++ b/bench/replication-topology/arm.mjs@@ -0,0 +1,151 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file One arm at one workload point, in its own process, printing one JSON+ * object. Own process for the same reasons as the dirty probe: a cold+ * heap, GC history and JIT profile are not shareable between arms.+ *+ * node bench/replication-topology/arm.mjs --arm C --entities 100000 --churn 0.001+ */+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { PerformanceObserver } from 'node:perf_hooks';+import {+ loadSdk, TopologyJournal, Registry, buildPopulation, applyChurn,+ reconcileFull, reconcileIncremental,+} from './workload.mjs';++const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');+const argv = process.argv.slice(2);+const flag = (name, fallback) => {+ const i = argv.indexOf(`--${name}`);+ return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;+};++const ARM = flag('arm', 'A');+const ENTITIES = Number(flag('entities', '10000'));+const CHURN = Number(flag('churn', '0'));+const SIM_HZ = Number(flag('simHz', '60'));+const REPL_HZ = Number(flag('replHz', '20'));+const WARMUP = Number(flag('warmup', '180'));+const MEASURE = Number(flag('measure', '600'));+/** Arm C only: run the full reconcile beside it and compare, every sample. */+const VERIFY = argv.includes('--verify');++const REPL_EVERY = Math.round(SIM_HZ / REPL_HZ);+// Per SAMPLE, spread over the sim ticks between two samples.+const CHURN_PER_SAMPLE = Math.max(0, Math.round(ENTITIES * CHURN));+const JOURNALLED = ARM !== 'A';++let gcCount = 0;+let gcMs = 0;+try {+ const obs = new PerformanceObserver((list) => {+ for (const e of list.getEntries()) { gcCount++; gcMs += e.duration; }+ });+ obs.observe({ entryTypes: ['gc'] });+} catch { /* not every build exposes gc entries */ }++const sdk = await loadSdk(ROOT);+const app = sdk.App.new();+const world = app.world;++const pop = buildPopulation(sdk, world, ENTITIES);+const journal = JOURNALLED ? new TopologyJournal() : null;+const registry = new Registry();+// The oracle keeps its OWN registry, so what it reports is what a full scan+// would have found rather than what the incremental pass left behind.+const oracle = VERIFY ? new Registry() : null;++world.advanceTick();+reconcileFull(sdk, world, registry);+if (oracle) reconcileFull(sdk, world, oracle);++let writeNs = 0n;+let sampleNs = 0n;+let visited = 0;+let samples = 0;+let spawnedTotal = 0;+let despawnedTotal = 0;+let mismatches = 0;+let floor = world.getWorldTick() - 1;++/** How many entities to churn on this sim tick, so a sample's worth lands per sample. */+function churnForTick(t) {+ if (CHURN_PER_SAMPLE === 0) return 0;+ const base = Math.floor(CHURN_PER_SAMPLE / REPL_EVERY);+ const extra = (t % REPL_EVERY) < (CHURN_PER_SAMPLE % REPL_EVERY) ? 1 : 0;+ return base + extra;+}++function runTicks(count, measuring) {+ for (let t = 0; t < count; t++) {+ const tick = measuring ? WARMUP + t : t;+ const w0 = process.hrtime.bigint();+ applyChurn(sdk, world, pop, tick, churnForTick(tick), journal);+ world.advanceTick();+ const w1 = process.hrtime.bigint();+ if (measuring) writeNs += w1 - w0;++ if ((tick + 1) % REPL_EVERY !== 0) continue;++ const s0 = process.hrtime.bigint();+ const result = ARM === 'C'+ ? reconcileIncremental(sdk, world, registry, journal, floor)+ : reconcileFull(sdk, world, registry);+ const s1 = process.hrtime.bigint();++ if (measuring) {+ sampleNs += s1 - s0;+ samples++;+ visited += result.visited;+ spawnedTotal += result.spawned.length;+ despawnedTotal += result.despawned.length;+ }++ if (VERIFY) verify(result);+ const next = world.getWorldTick() - 1;+ journal?.advance(next);+ floor = next;+ }+}++/**+ * The narrow question: does the incremental registry end each sample in the+ * same state a full scan would, and name the same arrivals and departures?+ * netIds are allocated in different orders, so the SETS of entities are what+ * must agree, not the numbers stamped on them.+ */+function verify(result) {+ const truth = reconcileFull(sdk, world, oracle);+ const same = (a, b) => a.length === b.length && a.every((x) => b.includes(x));+ if (!same(result.spawned, truth.spawned)) mismatches++;+ else if (!same(result.despawned.map((d) => d.entity), truth.despawned.map((d) => d.entity))) mismatches++;+ else if (registry.known.size !== oracle.known.size) mismatches++;+ else {+ for (const e of registry.known) if (!oracle.known.has(e)) { mismatches++; break; }+ }+}++const heapBefore = process.memoryUsage().heapUsed;+runTicks(WARMUP, false);+runTicks(MEASURE, true);+const heapAfter = process.memoryUsage().heapUsed;++const simSeconds = MEASURE / SIM_HZ;+const writeUs = Number(writeNs) / 1000;+const sampleUs = Number(sampleNs) / 1000;+process.stdout.write(`${JSON.stringify({+ arm: ARM, entities: ENTITIES, churn: CHURN, verify: VERIFY,+ simSeconds, samples,+ writeTaxUsPerSimSecond: writeUs / simSeconds,+ sampleTaxUsPerSimSecond: sampleUs / simSeconds,+ totalTaxUsPerSimSecond: (writeUs + sampleUs) / simSeconds,+ visitedPerSample: samples > 0 ? visited / samples : 0,+ spawnedTotal, despawnedTotal,+ journalRows: journal?.size ?? 0,+ livePopulation: world.getEntitiesWithComponents([sdk.Replicated]).length,+ mismatches,+ heapDeltaBytes: heapAfter - heapBefore,+ gcCount, gcMs,+})}\n`);diff --git a/bench/replication-topology/results.json b/bench/replication-topology/results.jsonnew file mode 100644index 000000000..59839ff6b--- /dev/null+++ b/bench/replication-topology/results.json@@ -0,0 +1,707 @@+{+ "machine": {+ "node": "v24.15.0",+ "platform": "darwin 25.5.0",+ "cpu": "Apple M4",+ "cores": 10,+ "totalMemGB": 16,+ "ranAt": "2026-09-05T01:50:47.782Z"+ },+ "build": {+ "gitHead": "77661a0ef51b5e6422e68ea4e1e606842b97636a",+ "workingTreeClean": false,+ "sdkArtifactSha256": "0ef414426b66a0a0",+ "sdkBuiltAt": "2026-09-05T01:50:44.906Z"+ },+ "fixed": {+ "simHz": 60,+ "replHz": 20,+ "warmup": 180,+ "measure": 600+ },+ "points": [+ {+ "arm": "A",+ "entities": 10000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 9.7988,+ "sampleTaxUsPerSimSecond": 8279.7462,+ "totalTaxUsPerSimSecond": 8289.545,+ "visitedPerSample": 20000,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 19329888,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 10000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 8.4759,+ "sampleTaxUsPerSimSecond": 8190.3246,+ "totalTaxUsPerSimSecond": 8198.800500000001,+ "visitedPerSample": 20000,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 33542672,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 5.9595,+ "sampleTaxUsPerSimSecond": 3.5332,+ "totalTaxUsPerSimSecond": 9.4927,+ "visitedPerSample": 0,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 245848,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 10000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 94.0201,+ "sampleTaxUsPerSimSecond": 13085.617,+ "totalTaxUsPerSimSecond": 13179.637099999998,+ "visitedPerSample": 19999,+ "spawnedTotal": 100,+ "despawnedTotal": 100,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 27950424,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 10000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 65.5606,+ "sampleTaxUsPerSimSecond": 12231.8663,+ "totalTaxUsPerSimSecond": 12297.4269,+ "visitedPerSample": 19999,+ "spawnedTotal": 100,+ "despawnedTotal": 100,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 22234288,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 28.671100000000003,+ "sampleTaxUsPerSimSecond": 11.8166,+ "totalTaxUsPerSimSecond": 40.487700000000004,+ "visitedPerSample": 1,+ "spawnedTotal": 100,+ "despawnedTotal": 100,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 763592,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 10000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 201.3527,+ "sampleTaxUsPerSimSecond": 12120.1246,+ "totalTaxUsPerSimSecond": 12321.4773,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 13558496,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 10000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 196.7537,+ "sampleTaxUsPerSimSecond": 11825.8549,+ "totalTaxUsPerSimSecond": 12022.6086,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 49744680,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 162.4578,+ "sampleTaxUsPerSimSecond": 25.7132,+ "totalTaxUsPerSimSecond": 188.171,+ "visitedPerSample": 4.25,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 3965200,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 10000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1220.1268,+ "sampleTaxUsPerSimSecond": 13498.949499999999,+ "totalTaxUsPerSimSecond": 14719.0763,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 56094432,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 10000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1281.8494,+ "sampleTaxUsPerSimSecond": 13721.9087,+ "totalTaxUsPerSimSecond": 15003.758100000001,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 59984608,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1296.1192999999998,+ "sampleTaxUsPerSimSecond": 75.087,+ "totalTaxUsPerSimSecond": 1371.2063,+ "visitedPerSample": 26.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 31952968,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 10000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 9694.0955,+ "sampleTaxUsPerSimSecond": 17774.4081,+ "totalTaxUsPerSimSecond": 27468.503600000004,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 45525736,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 10000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 12092.199400000001,+ "sampleTaxUsPerSimSecond": 19557.182999999997,+ "totalTaxUsPerSimSecond": 31649.382400000002,+ "visitedPerSample": 20000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 50963288,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 11001.886999999999,+ "sampleTaxUsPerSimSecond": 487.64129999999994,+ "totalTaxUsPerSimSecond": 11489.5283,+ "visitedPerSample": 251.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 71485152,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 100000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 32.9309,+ "sampleTaxUsPerSimSecond": 143170.9376,+ "totalTaxUsPerSimSecond": 143203.86849999998,+ "visitedPerSample": 200000,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 82096656,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 100000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 31.9464,+ "sampleTaxUsPerSimSecond": 158708.72090000001,+ "totalTaxUsPerSimSecond": 158740.6673,+ "visitedPerSample": 200000,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 78949296,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 5.6073,+ "sampleTaxUsPerSimSecond": 3.1131,+ "totalTaxUsPerSimSecond": 8.720400000000001,+ "visitedPerSample": 0,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 460576,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 100000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 970.7450999999999,+ "sampleTaxUsPerSimSecond": 217363.62960000001,+ "totalTaxUsPerSimSecond": 218334.3747,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 126020176,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 100000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 737.2266999999999,+ "sampleTaxUsPerSimSecond": 208328.9627,+ "totalTaxUsPerSimSecond": 209066.1894,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 117278776,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0.0001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 174.8845,+ "sampleTaxUsPerSimSecond": 31.707900000000002,+ "totalTaxUsPerSimSecond": 206.5924,+ "visitedPerSample": 4.25,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 3875440,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 100000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1799.6756,+ "sampleTaxUsPerSimSecond": 206425.8541,+ "totalTaxUsPerSimSecond": 208225.5297,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 198064112,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 100000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 2390.2197,+ "sampleTaxUsPerSimSecond": 245518.3912,+ "totalTaxUsPerSimSecond": 247908.61090000003,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 185037696,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0.001,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1294.7074,+ "sampleTaxUsPerSimSecond": 111.75399999999999,+ "totalTaxUsPerSimSecond": 1406.4614000000001,+ "visitedPerSample": 26.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 34476800,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 100000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 14052.5258,+ "sampleTaxUsPerSimSecond": 270740.666,+ "totalTaxUsPerSimSecond": 284793.19180000003,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 248030896,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 100000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 14076.482500000002,+ "sampleTaxUsPerSimSecond": 242419.8538,+ "totalTaxUsPerSimSecond": 256496.33630000002,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 291149568,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0.01,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 13507.6256,+ "sampleTaxUsPerSimSecond": 750.3994,+ "totalTaxUsPerSimSecond": 14258.025,+ "visitedPerSample": 251.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 114830592,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "A",+ "entities": 100000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 127208.76389999999,+ "sampleTaxUsPerSimSecond": 266186.9951,+ "totalTaxUsPerSimSecond": 393395.75899999996,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 348970816,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "B",+ "entities": 100000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 126541.8676,+ "sampleTaxUsPerSimSecond": 267089.7416,+ "totalTaxUsPerSimSecond": 393631.6092,+ "visitedPerSample": 200000,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 325989368,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0.1,+ "verify": false,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 133632.94199999998,+ "sampleTaxUsPerSimSecond": 9152.4669,+ "totalTaxUsPerSimSecond": 142785.40889999998,+ "visitedPerSample": 2501.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 76819384,+ "gcCount": 0,+ "gcMs": 0+ }+ ],+ "verifies": [+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0,+ "verify": true,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 8.3029,+ "sampleTaxUsPerSimSecond": 5.6988,+ "totalTaxUsPerSimSecond": 14.0017,+ "visitedPerSample": 0,+ "spawnedTotal": 0,+ "despawnedTotal": 0,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 37377824,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.001,+ "verify": true,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 205.70929999999998,+ "sampleTaxUsPerSimSecond": 29.2212,+ "totalTaxUsPerSimSecond": 234.9305,+ "visitedPerSample": 4.25,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 7577496,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 10000,+ "churn": 0.1,+ "verify": true,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 10937.8985,+ "sampleTaxUsPerSimSecond": 497.8328,+ "totalTaxUsPerSimSecond": 11435.7313,+ "visitedPerSample": 251.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 10000,+ "mismatches": 0,+ "heapDeltaBytes": 38693520,+ "gcCount": 0,+ "gcMs": 0+ },+ {+ "arm": "C",+ "entities": 100000,+ "churn": 0.001,+ "verify": true,+ "simSeconds": 10,+ "samples": 200,+ "writeTaxUsPerSimSecond": 1901.9621,+ "sampleTaxUsPerSimSecond": 147.68720000000002,+ "totalTaxUsPerSimSecond": 2049.6493,+ "visitedPerSample": 26.5,+ "spawnedTotal": 200,+ "despawnedTotal": 200,+ "journalRows": 0,+ "livePopulation": 100000,+ "mismatches": 0,+ "heapDeltaBytes": 136042168,+ "gcCount": 0,+ "gcMs": 0+ }+ ],+ "differentialClean": true+}diff --git a/bench/replication-topology/run.mjs b/bench/replication-topology/run.mjsnew file mode 100644index 000000000..3881fee44--- /dev/null+++ b/bench/replication-topology/run.mjs@@ -0,0 +1,181 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Does a topology journal beat re-reading the world every sample?+ *+ * The dirty probe answered the FIELD question. This one asks the registry+ * question: `reconcileRegistry_()` reads every replicated entity and walks the+ * whole registry, every sample, whether or not anything entered or left.+ *+ * node bench/replication-topology/run.mjs+ * node bench/replication-topology/run.mjs --quick+ *+ * Ships nothing: production still runs the full reconcile.+ */+import { spawnSync } from 'node:child_process';+import { appendFileSync, existsSync, readFileSync, unlinkSync, writeFileSync } from 'node:fs';+import os from 'node:os';+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { sdkIdentity } from './workload.mjs';++const HERE = path.dirname(fileURLToPath(import.meta.url));+const ROOT = path.resolve(HERE, '..', '..');+const ARM = path.join(HERE, 'arm.mjs');+const QUICK = process.argv.includes('--quick');+const BIG = process.argv.includes('--big');++const ARMS = ['A', 'B', 'C'];+const ARM_MEANING = {+ A: 'full reconcile (today\'s production)',+ B: 'journal written, full reconcile (the write tax, alone)',+ C: 'journal + incremental (the candidate architecture)',+};+const ENTITY_COUNTS = QUICK ? [1000] : BIG ? [500000] : [10000, 100000];+/** 0 first, and it is the point: a registry that moved nothing still costs O(E). */+const CHURN_RATES = QUICK ? [0, 0.01] : [0, 0.0001, 0.001, 0.01, 0.1];+const FIXED = { simHz: 60, replHz: 20, warmup: QUICK ? 60 : 180, measure: QUICK ? 180 : 600 };+const VERIFY_POINTS = QUICK ? [[1000, 0.01]]+ : BIG ? [[500000, 0.001]]+ : [[10000, 0], [10000, 0.001], [10000, 0.1], [100000, 0.001]];++const CACHE = path.join(HERE, `.sweep-${QUICK ? 'quick' : BIG ? 'big' : 'matrix'}.jsonl`);+const buildId = () => sdkIdentity(ROOT).sdkArtifactSha256;++function loadCache() {+ const done = new Map();+ if (!existsSync(CACHE)) return done;+ for (const line of readFileSync(CACHE, 'utf8').split('\n').filter(Boolean)) {+ const row = JSON.parse(line);+ if (row.build !== buildId()) { unlinkSync(CACHE); return new Map(); }+ done.set(row.key, row.result);+ }+ return done;+}++function buildSdkOnce() {+ const built = spawnSync('pnpm', ['--filter', './sdk', 'build'],+ { encoding: 'utf8', cwd: ROOT, maxBuffer: 1 << 24 });+ if (built.status !== 0) {+ console.error(built.stderr || built.stdout);+ throw new Error('the SDK build failed; nothing was measured');+ }+}++function runPoint(arm, entities, churn, verify) {+ const args = [+ ARM, '--arm', arm, '--entities', String(entities), '--churn', String(churn),+ '--simHz', String(FIXED.simHz), '--replHz', String(FIXED.replHz),+ '--warmup', String(FIXED.warmup), '--measure', String(FIXED.measure),+ ];+ if (verify) args.push('--verify');+ const run = spawnSync(process.execPath, args, { encoding: 'utf8', cwd: ROOT, maxBuffer: 1 << 24 });+ if (run.status !== 0) {+ console.error(run.stderr || run.stdout);+ throw new Error(`arm ${arm} @ ${entities}/${churn} exited ${run.status}`);+ }+ return JSON.parse(run.stdout);+}++const measured = loadCache();++function measure(arm, entities, churn, verify) {+ const key = `${arm}/${entities}/${churn}/${verify ? 'v' : 'p'}`;+ const cached = measured.get(key);+ if (cached) return { result: cached, fresh: false };+ const result = runPoint(arm, entities, churn, verify);+ appendFileSync(CACHE, `${JSON.stringify({ build: buildId(), key, result })}\n`);+ measured.set(key, result);+ return { result, fresh: true };+}++buildSdkOnce();++const points = [];+const total = ENTITY_COUNTS.length * CHURN_RATES.length * ARMS.length + VERIFY_POINTS.length;+let done = 0;+for (const entities of ENTITY_COUNTS) {+ for (const churn of CHURN_RATES) {+ for (const arm of ARMS) {+ const { result, fresh } = measure(arm, entities, churn, false);+ process.stderr.write(`[${++done}/${total}] arm ${arm} ${entities} `+ + `${churn * 100}% churn${fresh ? '' : ' (cached)'}\n`);+ points.push(result);+ }+ }+}++const verifies = [];+for (const [entities, churn] of VERIFY_POINTS) {+ const { result, fresh } = measure('C', entities, churn, true);+ process.stderr.write(`[${++done}/${total}] differential ${entities} `+ + `${churn * 100}% churn${fresh ? '' : ' (cached)'}\n`);+ verifies.push(result);+}++const at = (arm, entities, churn) =>+ points.find((p) => p.arm === arm && p.entities === entities && p.churn === churn);+const lines = [];+const say = (s = '') => { lines.push(s); console.log(s); };++say('=== replication registry: full reconcile vs topology journal ===');+say('');+for (const arm of ARMS) say(` ${arm} ${ARM_MEANING[arm]}`);+say('');++const clean = verifies.every((v) => v.mismatches === 0);+say(clean+ ? ` DIFFERENTIAL: the incremental registry matched the full scan at every sample`+ + ` (${verifies.length} point(s)).`+ : ` DIFFERENTIAL: MISMATCHED — the incremental registry disagreed with the full scan.`+ + ' It MUST NOT SHIP whatever the timings say.');+say('');++const pad = (v, w) => String(v).padStart(w);+const round = (v) => Math.round(v);+for (const entities of ENTITY_COUNTS) {+ for (const churn of CHURN_RATES) {+ say(` ${entities} entities · ${churn * 100}% membership churn per sample`);+ say(' arm write sample total 1 core B/A C/B C/A visited/sample');+ const a = at('A', entities, churn);+ const b = at('B', entities, churn);+ for (const arm of ARMS) {+ const p = at(arm, entities, churn);+ const ratio = (x, y) => (y > 0 ? (x / y).toFixed(2) : '—');+ const cols = arm === 'A' ? ['—', '—', '—']+ : arm === 'B' ? [ratio(p.totalTaxUsPerSimSecond, a.totalTaxUsPerSimSecond), '—', '—']+ : ['—', ratio(p.totalTaxUsPerSimSecond, b.totalTaxUsPerSimSecond),+ ratio(p.totalTaxUsPerSimSecond, a.totalTaxUsPerSimSecond)];+ const core = `${(p.totalTaxUsPerSimSecond / 1e4).toFixed(1)}%`;+ say(` ${arm} ${pad(round(p.writeTaxUsPerSimSecond), 8)} ${pad(round(p.sampleTaxUsPerSimSecond), 11)}`+ + ` ${pad(round(p.totalTaxUsPerSimSecond), 11)} ${pad(core, 7)}`+ + ` ${pad(cols[0], 6)} ${pad(cols[1], 6)} ${pad(cols[2], 6)}`+ + ` ${pad(round(p.visitedPerSample), 14)}`);+ }+ say('');+ }+}++say('--- WHAT THE THREE RATIOS ASK ---');+say(' B/A what does keeping the journal cost, alone? It is written only when a');+say(' membership moves, never per field write — so this should be near 1.');+say(' C/B having paid that, how much of the O(E) reconcile disappears?');+say(' C/A is the whole thing worth switching production to?');+say('');+say(' The 0% row is the one that decides it: a registry where nothing entered or');+say(' left still reads every replicated entity today.');++const out = path.join(HERE, QUICK ? 'results.quick.json' : BIG ? 'results.big.json' : 'results.json');+const machine = {+ node: process.version,+ platform: `${os.platform()} ${os.release()}`,+ cpu: os.cpus()[0]?.model ?? 'unknown',+ cores: os.cpus().length,+ totalMemGB: Math.round(os.totalmem() / 1024 ** 3),+ ranAt: new Date().toISOString(),+};+writeFileSync(out, `${JSON.stringify({+ machine, build: sdkIdentity(ROOT), fixed: FIXED, points, verifies, differentialClean: clean,+}, null, 2)}\n`);+say('');+say(`raw results: ${path.relative(ROOT, out)}`);diff --git a/bench/replication-topology/workload.mjs b/bench/replication-topology/workload.mjsnew file mode 100644index 000000000..1bf3aac6f--- /dev/null+++ b/bench/replication-topology/workload.mjs@@ -0,0 +1,183 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file The one workload every arm runs, and the two ways of finding which+ * entities entered or left replication.+ *+ * The question is registry lifecycle, not field values: nothing here writes a+ * replicated field. Membership churn is a pure function of the tick, so two+ * processes cannot drift apart.+ */+import { pathToFileURL } from 'node:url';+import { readdirSync, statSync } from 'node:fs';+import path from 'node:path';++export { sdkIdentity } from '../replication-dirty/workload.mjs';++/** The newest hand-written .ts under `dir`, in epoch ms. */+function newestSource(dir) {+ let newest = 0;+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ if (entry.name === 'node_modules' || entry.name === 'dist') continue;+ const full = path.join(dir, entry.name);+ newest = Math.max(newest, entry.isDirectory() ? newestSource(full)+ : entry.name.endsWith('.ts') && !entry.name.endsWith('.generated.ts')+ ? statSync(full).mtimeMs : 0);+ }+ return newest;+}++export async function loadSdk(root) {+ const entry = path.join(root, 'sdk', 'dist', 'index.node.js');+ if (newestSource(path.join(root, 'sdk', 'src')) > statSync(entry).mtimeMs) {+ throw new Error('sdk/dist is older than sdk/src — this run would measure the previous'+ + ' commit. Build it with `pnpm --filter ./sdk build`.');+ }+ return import(pathToFileURL(entry).href);+}++/**+ * For each component, the entities whose MEMBERSHIP moved, and from which tick+ * each reader still needs them. Deliberately not "added" and "removed" as+ * separate records: the registry re-reads the world rather than acting on what+ * history says, so "worth re-checking" is the whole requirement.+ */+export class TopologyJournal {+ #rows = [];+ #retainFrom = 0;++ /** One membership event. This is the write tax the B arm isolates. */+ record(entity, tick) {+ this.#rows.push({ entity, tick });+ }++ /** Entities worth re-checking since `floor` (strict `>`, as the tracker is). */+ since(floor) {+ const out = new Set();+ for (const row of this.#rows) if (row.tick > floor) out.add(row.entity);+ return out;+ }++ /** Give up everything through `lastTick`, as a reader-owned claim does. */+ advance(lastTick) {+ this.#retainFrom = lastTick + 1;+ let w = 0;+ for (let i = 0; i < this.#rows.length; i++) {+ if (this.#rows[i].tick >= this.#retainFrom) this.#rows[w++] = this.#rows[i];+ }+ this.#rows.length = w;+ }++ get size() { return this.#rows.length; }+}++/** The server's view: which entities the clients have been told exist. */+export class Registry {+ known = new Set();+ knownNetIds = new Map();+ #nextNetId = 1;++ register(e) {+ this.known.add(e);+ this.knownNetIds.set(e, this.#nextNetId++);+ }++ unregister(e) {+ const netId = this.knownNetIds.get(e);+ this.known.delete(e);+ this.knownNetIds.delete(e);+ return netId;+ }+}++/**+ * `count` entities carrying Replicated, and a dormant pool the churn moves them+ * into and out of. Dormant entities still EXIST — losing the component and being+ * despawned are different events, and the reducer has to answer for both.+ */+export function buildPopulation(sdk, world, count) {+ const live = [];+ const dormant = [];+ for (let i = 0; i < count; i++) {+ const e = world.spawn();+ world.insert(e, sdk.Replicated, { owner: 0 });+ live.push(e);+ }+ return { live, dormant };+}++/**+ * One tick's membership churn, deterministic in `tick`. Four shapes in rotation+ * so every cell of the reducer is exercised: leaving, returning, dying, and+ * being born already replicated.+ */+export function applyChurn(sdk, world, pop, tick, count, journal) {+ for (let k = 0; k < count; k++) {+ const shape = (tick + k) % 4;+ if (shape === 0 && pop.live.length > 1) {+ const e = pop.live.pop();+ world.remove(e, sdk.Replicated);+ pop.dormant.push(e);+ journal?.record(e, world.getWorldTick());+ } else if (shape === 1 && pop.dormant.length > 0) {+ const e = pop.dormant.pop();+ world.insert(e, sdk.Replicated, { owner: 0 });+ pop.live.push(e);+ journal?.record(e, world.getWorldTick());+ } else if (shape === 2 && pop.live.length > 1) {+ const e = pop.live.pop();+ world.despawn(e);+ journal?.record(e, world.getWorldTick());+ } else {+ const e = world.spawn();+ world.insert(e, sdk.Replicated, { owner: 0 });+ pop.live.push(e);+ journal?.record(e, world.getWorldTick());+ }+ }+}++/**+ * Today's reconcile: read every replicated entity, then walk the registry.+ * O(population) whether anything moved or not.+ */+export function reconcileFull(sdk, world, registry) {+ const current = world.getEntitiesWithComponents([sdk.Replicated]);+ const currentSet = new Set(current);+ const spawned = [];+ const despawned = [];+ for (const e of current) {+ if (!registry.known.has(e)) { registry.register(e); spawned.push(e); }+ }+ for (const e of [...registry.known]) {+ if (!currentSet.has(e) || !world.valid(e)) {+ const netId = registry.unregister(e);+ if (netId !== undefined) despawned.push({ entity: e, netId });+ }+ }+ return { spawned, despawned, visited: current.length + registry.known.size };+}++/**+ * The candidate reducer: the journal says who to re-check, and `known × current+ * world` says what is true. A membership that left and came back before the+ * sample is one candidate that is known and live — so, nothing.+ */+export function reconcileIncremental(sdk, world, registry, journal, floor) {+ const candidates = journal.since(floor);+ const spawned = [];+ const despawned = [];+ for (const e of candidates) {+ const isKnown = registry.known.has(e);+ // `valid` is belt to `has`'s braces — an entity id carries its generation,+ // so a stale handle misses the storage map anyway. Sabotaging it alone+ // does NOT redden this probe; see the README.+ const isLive = world.valid(e) && world.has(e, sdk.Replicated);+ if (!isKnown && isLive) { registry.register(e); spawned.push(e); }+ else if (isKnown && !isLive) {+ const netId = registry.unregister(e);+ if (netId !== undefined) despawned.push({ entity: e, netId });+ }+ }+ return { spawned, despawned, visited: candidates.size };+}