Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

probe(net): the budget again, and what is left is the work

esengineb50b4f8acff2
diff --git a/bench/sample-budget/README.md b/bench/sample-budget/README.md
index a13ed6d4d..b099be438 100644
--- a/bench/sample-budget/README.md
+++ b/bench/sample-budget/README.md
@@ -119,6 +119,96 @@ shadow for each candidate that really did change.
`bench/sample-budget` reports `journalReads` and `populationScans` so a claim
about either projection can say which one it got.
+## N6: the budget re-taken after the spawn split
+
+Same instrumentation, same workloads, three interleaved rounds. Medians; every
+figure a percentage of that column's total.
+
+| phase | still | movement | anchors | mixed |
+|---|---|---|---|---|
+| **visibility query** | 0 | **3,772 · 43%** | **4,435 · 31%** | **4,533 · 31%** |
+| **dirty discovery** | 1 | 1,913 · 22% | 2,165 · 15% | 3,049 · 21% |
+| **spawn payload** | 0 | 50 · 1% | 2,622 · 18% | 2,372 · 16% |
+| visibility diff | 0 | 875 · 10% | 1,392 · 10% | 1,361 · 9% |
+| control send | 2 | 61 · 1% | 1,207 · 8% | 1,061 · 7% |
+| composition | 1 | 895 · 10% | 986 · 7% | 900 · 6% |
+| interest prepare | 4 | 770 · 9% | 972 · 7% | 935 · 6% |
+| routing | 2 | 165 · 2% | 182 · 1% | 502 · 3% |
+| frame encode | 0 | 71 · 1% | 104 · 1% | 281 · 2% |
+| transport send | 0 | 55 · 1% | 43 · 0% | 41 · 0% |
+| other | 11 | 31 · 0% | 121 · 1% | 155 · 1% |
+| **total** | **25** | **8,714** | **14,249** | **14,791** |
+| accounted | 53% | **99.6%** | **99.2%** | **99.2%** |
+
+`still` is 25 µs — the idle fast path survived protocol v4, and it accounts for
+only half of itself because the instrument is 11 of those microseconds.
+
+### The work, beside the time
+
+| per sample | movement | anchors | mixed |
+|---|---|---|---|
+| visibility recomputes | 32 | 32 | 32 |
+| dirty candidates examined | **1,000** | 1,032 | 2,132 |
+| journal reads / population scans | 3 / 0 | 3 / 0 | 3 / 0 |
+| enters / leaves | 6 / 7 | 682 / 686 | 682 / 686 |
+| spawn payloads built | 6 | 682 | 682 |
+| delta rows sent | 175 | 175 | 467 |
+| control bytes | 2,262 | **178,683** | 178,703 |
+| binary bytes | 4,227 | 4,222 | 8,315 |
+
+**Dirty discovery examines exactly the entities that changed** — 1,000 candidates
+for 1,000 movers, and the journal wins the projection choice every time (3 reads,
+0 scans). That phase is now proportional to what happened.
+
+### First place is the query, and it is not overhead
+
+Timed at the provider seam (`--seam`, off by default so the table above is not
+measured through a wrapper that exists to take it apart), `visibility query`
+splits:
+
+| | provider computes | host copies |
+|---|---|---|
+| movement | 3,432 (**94%**) | 216 (6%) |
+| anchors | 4,405 (**92%**) | 399 (8%) |
+| mixed | 4,563 (**95%**) | 260 (5%) |
+
+The host's share — copying the answer into a set it owns and forcing owned
+entities into it — is 5–8%. The rest is the grid walk and the sphere test over
+about 925 entities per connection, at **0.149 µs per (connection × visible
+entity)**. And the number of queries is 32 whatever is done to them: an owned
+anchor is one of its own query's inputs, which `bench/interest-regional/`
+settled. So neither the count nor the unit is carrying obvious waste.
+
+### The spawn, split three ways
+
+| | where it is |
+|---|---|
+| ghost construction | **not on the server at all** — protocol v4 moved it to the client's archetype |
+| baseline serialization | `spawn payload`, 2,622 µs for 682 entities = **3.8 µs each** (14.7 before) |
+| control-message overhead | `control send` 1,207 µs, which the transport's own clock splits 421 encode / 786 transport |
+
+**So N5c stays closed on CPU.** Its re-open condition was spawn encoding
+becoming top-1/top-2 *with baseline encode the bulk of it*; encode is 421 µs of
+a 3,829 µs spawn cost — 11%. Object construction and transport are the rest.
+
+**But note where the bytes are.** The control plane carries 178 KB a sample
+against the binary plane's 4–8 KB: JSON spawn batches outweigh the delta frames
+by twenty to forty times. That is 10 MB/s at this churn rate, and it is not
+visible in the CPU budget at all. If `FrameWriter`-shaped baselines are ever
+worth doing, **bandwidth is the argument, not microseconds** — which is a
+different re-open condition from the one this bench was built to test.
+
+### The judgement
+
+Encode and transport are 1–2% of the sample. Every large phase is now
+proportional to something that actually happened: entities that moved, views
+that had to be re-proved because their own anchor moved, entities that genuinely
+entered a view. `other` is under 1%.
+
+**The remaining cost is mostly necessary work, and this line should stop here.**
+What is left is not scanning to establish that nothing happened — that was the
+whole shape of the first seven cuts, and it is gone.
+
## What this does not cover
- One anchor per connection, a uniform grid world, and clients connected but not
@@ -129,6 +219,8 @@ about either projection can say which one it got.
- `spawn payload` is serialization only. It is timed before the send that carries
it, because inside it the two would be counted twice — which is how the first
version of this table came out at 142% accounted.
+- The N6 table is medians of three interleaved rounds. A machine that drifts 20%
+ across a session will otherwise tell whichever story the run order suggests.
- Phase times are means over the window; the total is reported as a mean and a
minimum. A budget cannot be built from minima, since the phases of one sample
are not the phases of another.
diff --git a/bench/sample-budget/arm.mjs b/bench/sample-budget/arm.mjs
index a8abfc7f2..f9bbbd3c3 100644
--- a/bench/sample-budget/arm.mjs
+++ b/bench/sample-budget/arm.mjs
@@ -25,6 +25,10 @@ const ENTITIES = Number(flag('entities', '100000'));
const CONNECTIONS = Number(flag('connections', '32'));
const VISIBLE = Number(flag('visible', '0.01'));
const SCENARIO = flag('scenario', 'movement');
+/** Time the provider's own answer, to split `visibility query` at that seam.
+ * Off by default: the budget table should not be measured through a wrapper
+ * that only exists to take it apart. */
+const SEAM = argv.includes('--seam');
const WARMUP = Number(flag('warmup', '12'));
const MEASURE = Number(flag('measure', '60'));
const STEP = 1 / 60;
@@ -59,7 +63,50 @@ const world = serverApp.world;
const side = Math.ceil(Math.sqrt(ENTITIES));
const RADIUS = VISIBLE >= 1 ? side * 2 : Math.sqrt((VISIBLE * ENTITIES) / Math.PI);
-server.setInterestProvider(sdk.radiusInterestProvider(RADIUS));
+/**
+ * The provider's own answer, timed at the seam. `visibility query` is that plus
+ * what the host does with it — copying the answer into a set it owns and forcing
+ * the owned entities into it — so the difference says which half it is.
+ */
+const inner = sdk.radiusInterestProvider(RADIUS);
+const provider = { queryNs: 0n, calls: 0 };
+if (!SEAM) server.setInterestProvider(inner);
+else server.setInterestProvider({
+ prepare(view) {
+ const prepared = inner.prepare(view);
+ return {
+ get generation() { return prepared.generation; },
+ query: (q) => {
+ const t0 = process.hrtime.bigint();
+ const r = prepared.query(q);
+ provider.queryNs += process.hrtime.bigint() - t0;
+ provider.calls++;
+ return r;
+ },
+ };
+ },
+ dispose: () => inner.dispose?.(),
+});
+
+/**
+ * What the transport itself cost, and carried. `control send` is the server's
+ * phase around `channel.send`, which is JSON encoding AND this — so encoding is
+ * that phase minus what is measured here, rather than a second guess at it.
+ */
+const wire = { controlNs: 0n, controlBytes: 0, controlMsgs: 0, binaryNs: 0n, binaryBytes: 0 };
+function meter(transport) {
+ const send = transport.send.bind(transport);
+ transport.send = (d) => {
+ const t0 = process.hrtime.bigint();
+ send(d);
+ const dt = process.hrtime.bigint() - t0;
+ if (typeof d === 'string') {
+ wire.controlNs += dt; wire.controlBytes += d.length; wire.controlMsgs++;
+ } else {
+ wire.binaryNs += dt; wire.binaryBytes += d.byteLength ?? 0;
+ }
+ };
+}
const clients = [];
for (let i = 0; i < CONNECTIONS; i++) {
@@ -67,6 +114,7 @@ for (let i = 0; i < CONNECTIONS; i++) {
app.connectCpp(new engine.Registry(), engine, { strict: false });
app.addPlugin(sdk.replicationPlugin);
const [ta, tb] = sdk.MemoryTransport.pair();
+ meter(ta);
const id = server.attachConnection(ta);
await app.getResource(sdk.Net).connect(tb, { interpolationDelayTicks: 0 });
clients.push({ app, id });
@@ -154,6 +202,11 @@ const visible = census();
server.samplePhases.clear();
server.visibilityRecomputes = 0;
server.entersSent = 0; server.leavesSent = 0; server.payloadsBuilt = 0;
+server.journalReads = 0; server.populationScans = 0;
+server.dirtyCandidates = 0; server.deltaRowsSent = 0;
+provider.queryNs = 0n; provider.calls = 0;
+wire.controlNs = 0n; wire.controlBytes = 0; wire.controlMsgs = 0;
+wire.binaryNs = 0n; wire.binaryBytes = 0;
server.profileSample = true;
let tickNs = 0n;
let fastest = Infinity;
@@ -180,6 +233,11 @@ const accounted = LEAVES.reduce((a, k) => a + phases[k], 0);
phases.other = total - accounted;
const tickUs = Number(tickNs) / 1000 / MEASURE;
+const perSample = (v) => Number(v) / MEASURE;
+// Encoding is what the phase cost minus what the transport under it cost. A
+// negative here would mean the two clocks disagree, not that encoding is free.
+const controlTransportUs = Number(wire.controlNs) / 1000 / MEASURE;
+const controlEncodeUs = phases['control send'] - controlTransportUs;
process.stdout.write(`${JSON.stringify({
scenario: SCENARIO, entities: ENTITIES, connections: CONNECTIONS, visible: VISIBLE,
...PROFILE, samples: MEASURE,
@@ -195,5 +253,31 @@ process.stdout.write(`${JSON.stringify({
usOutsideSample: tickUs - total,
accountedPercent: total > 0 ? (accounted / total) * 100 : 0,
phases,
+ // `control send` split by the transport's own clock, not by a second model.
+ controlEncodeUs, controlTransportUs,
+ // `visibility query` split at the provider seam: what it computed, against
+ // what the host spent turning that into a set of its own.
+ seam: SEAM,
+ providerQueryUs: SEAM ? Number(provider.queryNs) / 1000 / MEASURE : null,
+ hostVisibilityUs: SEAM
+ ? phases['visibility query'] - Number(provider.queryNs) / 1000 / MEASURE : null,
+ /**
+ * The work, beside the time. A phase that grows because one of these grew is
+ * doing more of something real; a phase that grows while these hold still is
+ * spending more to establish that nothing happened.
+ */
+ work: {
+ visibilityRecomputes: server.visibilityRecomputes / MEASURE,
+ dirtyCandidates: server.dirtyCandidates / MEASURE,
+ journalReads: server.journalReads / MEASURE,
+ populationScans: server.populationScans / MEASURE,
+ enters: server.entersSent / MEASURE,
+ leaves: server.leavesSent / MEASURE,
+ spawnPayloadsBuilt: server.payloadsBuilt / MEASURE,
+ deltaRowsSent: server.deltaRowsSent / MEASURE,
+ controlMessages: perSample(wire.controlMsgs),
+ controlBytes: perSample(wire.controlBytes),
+ binaryBytes: perSample(wire.binaryBytes),
+ },
...sdkIdentity(ROOT),
})}\n`);
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index 37a664ab1..81355a4cc 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -6653,6 +6653,8 @@ static prototype: ReplicationPlugin
## ReplicationServer — class @experimental
```
+@internal deltaRowsSent: number
+@internal dirtyCandidates: number
@internal entersSent: number
@internal fullScans: number
@internal journalReads: number
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index e3343ec24..25c99020e 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -6846,6 +6846,8 @@ static prototype: ReplicationPlugin
## ReplicationServer — class @experimental
```
+@internal deltaRowsSent: number
+@internal dirtyCandidates: number
@internal entersSent: number
@internal fullScans: number
@internal journalReads: number
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index b8b022c90..6c627351c 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -6648,6 +6648,8 @@ static prototype: ReplicationPlugin
## ReplicationServer — class @experimental
```
+@internal deltaRowsSent: number
+@internal dirtyCandidates: number
@internal entersSent: number
@internal fullScans: number
@internal journalReads: number
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index 05d6e9909..2f291b375 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -6894,6 +6894,8 @@ static prototype: ReplicationPlugin
## ReplicationServer — class @experimental
```
+@internal deltaRowsSent: number
+@internal dirtyCandidates: number
@internal entersSent: number
@internal fullScans: number
@internal journalReads: number
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index 4670fe498..c35b59112 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -6655,6 +6655,8 @@ static prototype: ReplicationPlugin
## ReplicationServer — class @experimental
```
+@internal deltaRowsSent: number
+@internal dirtyCandidates: number
@internal entersSent: number
@internal fullScans: number
@internal journalReads: number
diff --git a/sdk/src/net/replication/server.ts b/sdk/src/net/replication/server.ts
index 00447e31c..bddcc6a27 100644
--- a/sdk/src/net/replication/server.ts
+++ b/sdk/src/net/replication/server.ts
@@ -269,6 +269,12 @@ export class ReplicationServer {
*/
journalReads = 0;
populationScans = 0;
+ /** @internal Entities dirty discovery actually examined against the shadow —
+ * the work, as against the two ways of finding it. */
+ dirtyCandidates = 0;
+ /** @internal Delta rows that reached a connection, summed over connections:
+ * what the wire carried, against what routing considered. */
+ deltaRowsSent = 0;
constructor(world: World) {
this.world_ = world;
@@ -835,6 +841,7 @@ export class ReplicationServer {
const d = dirty[i]!;
frame.entry(d.netId, d.te, d.mask, d.data, this.refs_);
}
+ this.deltaRowsSent += frame.entryCount;
const payload = frame.entryCount > 0 ? frame.finish() : null;
this.since_('frame encode', encoding);
if (payload) {
@@ -1232,6 +1239,7 @@ export class ReplicationServer {
// A Set, so a component lost twice in one window is one candidate.
for (const e of this.world_.getRemovedEntitiesSince(te.def, floor)) candidates.add(e);
+ this.dirtyCandidates += candidates.size;
for (const e of candidates) {
const perComp = this.shadow_.get(e);
if (!perComp) continue;