3D engine that AI agents drive
Cut netcode sample lag to about 95 ms each way, keep injected-latency stats live, expire silent BroadcastChannel peers (#778)
Nguyen Xuan An0b21bcc6f299
diff --git a/changes/778-two-browsers.md b/changes/778-two-browsers.mdindex e59b8546..d4543cae 100644--- a/changes/778-two-browsers.md+++ b/changes/778-two-browsers.md@@ -1 +1,2 @@- Add `createBroadcastChannelTransport` to `67js/addons/netcode`, so tabs of one browser play together with no server, with a two-player sample page, a reference WebSocket relay (`scripts/netcode-relay.mjs`) and a headless Chrome test that plays two pages over both transports with 50 ms injected latency (#778)+- Cut the netcode sample's lag with 50 ms of injected latency to about 95 ms each way (a snapshot every tick, a 40 ms interpolation delay), keep `withInjectedLatency` stats live, close the transport on `pagehide`, and make `createBroadcastChannelTransport` forget peers silent for `peerTimeoutMilliseconds` (#778)diff --git a/docs/netcode.md b/docs/netcode.mdindex 4fa1e735..9e69ef0c 100644--- a/docs/netcode.md+++ b/docs/netcode.md@@ -54,7 +54,7 @@ Other entities are drawn `interpolationDelaySeconds` (default 0.1) behind the neA transport is a plain object `{ id, peers(), send(peerId, message), receive(callback) }`. `peers()` returns the connected peer ids, `send` takes a JSON-compatible object and must not throw for a transient failure, and `receive(callback)` registers `callback(peerId, message)` and returns a function that unregisters it. Write your own for WebRTC or WebTransport.- `createLoopbackNetwork({ seed, latencyMilliseconds, jitterMilliseconds, lossRate })` is an in-memory network for tests. `network.createTransport(id)` adds a peer; `network.advance(milliseconds)` moves time and delivers what is due. Latency, jitter and loss come from the seeded generator, so a seed repeats a run exactly. Messages are copied through JSON and may arrive out of order.-- `createBroadcastChannelTransport(channel, { id })` wraps a `BroadcastChannel` so tabs, windows and workers of one browser play together with no server. Peers find each other with a hello the other side answers once, and a message to one peer reaches only that peer.+- `createBroadcastChannelTransport(channel, { id })` wraps a `BroadcastChannel` so tabs, windows and workers of one browser play together with no server. Peers find each other with a hello the other side answers once, and a message to one peer reaches only that peer. A peer silent for `peerTimeoutMilliseconds` (default 5000) is forgotten until it speaks again, so a closed tab leaves the peer list; call `transport.close()` on `pagehide` to say goodbye at once. `now` swaps the clock for tests.- `createWebSocketTransport(socket, { id, remoteId, multiplexed, maximumMessageBytes })` wraps a socket-like object you opened (`send(text)`, `addEventListener`, `readyState`, `close()`). By default it is a line to one peer, `remoteId`. With `multiplexed: true` it speaks to a relay that wraps messages as `{ to, data }` going out and `{ from, data }` coming in (and `{ from, left: true }` when a peer leaves), so a server or host can hear many clients.## Options and bounds@@ -78,7 +78,7 @@ A transport is a plain object `{ id, peers(), send(peerId, message), receive(cal## Sample game and relay-`scripts/netcode-sample.html` is a two-player sample: one page is the host (`?role=host`, plays `alice`, owns the truth) and the other is a client (`?role=client`, plays `bob` with prediction and reconciliation). Move with the arrow keys or WASD. `?transport=broadcast` (default) uses two tabs of one browser; `?transport=websocket&relay=ws://127.0.0.1:8787/` uses the relay. `?delay=50` adds that many milliseconds to everything a page sends. Serve the repository root (for example with `npx vite`) and open both URLs.+`scripts/netcode-sample.html` is a two-player sample: one page is the host (`?role=host`, plays `alice`, owns the truth) and the other is a client (`?role=client`, plays `bob` with prediction and reconciliation). Move with the arrow keys or WASD. `?transport=broadcast` (default) uses two tabs of one browser; `?transport=websocket&relay=ws://127.0.0.1:8787/` uses the relay. `?delay=50` adds that many milliseconds to everything a page sends. The sample sends a snapshot every tick and draws remote players 40 ms behind (`?snap=` and `?interp=` change them), which shows a move on the other page about 95 ms after the key with 50 ms of injected latency each way. Serve the repository root (for example with `npx vite`) and open both URLs.`node scripts/netcode-relay.mjs --port 8787 [--host 0.0.0.0] [--latency 50]` is a reference relay of about 80 lines. Players connect to `/?id=<name>`; it forwards `{ to, data }` as `{ from, data }`, announces `{ from, left: true }`, refuses a repeated name and keeps no game state, so it runs on any free Node host. It needs the `ws` package, a dev dependency of this repository only; the library itself gains no dependency.diff --git a/scripts/netcode-sample.html b/scripts/netcode-sample.htmlindex d5f57403..95dc43e1 100644--- a/scripts/netcode-sample.html+++ b/scripts/netcode-sample.html@@ -19,6 +19,8 @@transport: query.get('transport') ?? 'broadcast',relayUrl: query.get('relay') ?? 'ws://127.0.0.1:8787/',delayMilliseconds: Number(query.get('delay') ?? 0),+ snapshotEveryTicks: query.has('snap') ? Number(query.get('snap')) : undefined,+ interpolationDelaySeconds: query.has('interp') ? Number(query.get('interp')) : undefined,canvas: document.getElementById('game'),}).then(sample => { window.sample = sample; return sample; });</script>diff --git a/scripts/netcode-sample.js b/scripts/netcode-sample.jsindex 025b5163..ea7a1678 100644--- a/scripts/netcode-sample.js+++ b/scripts/netcode-sample.js@@ -12,6 +12,8 @@ const HOST_ID = 'alice';const CLIENT_ID = 'bob';const WORLD_HALF_SIZE = 8;const FIXED_RATE = 50;+const SNAPSHOT_EVERY_TICKS = 1;+const INTERPOLATION_DELAY_SECONDS = 0.04;const KEY_DIRECTIONS = {ArrowLeft: [-1, 0], KeyA: [-1, 0],ArrowRight: [1, 0], KeyD: [1, 0],@@ -27,10 +29,25 @@ function openWebSocket(relayUrl, playerId) {});}-// Delays what a transport sends, to feel a slow connection on one machine.-function withInjectedLatency(transport, delayMilliseconds) {+// Delays what a transport sends, to feel a slow connection on one machine. Everything else, including the live `stats`+// getter, is read from the real transport, and delayed sends that have not left yet are dropped when it closes.+export function withInjectedLatency(transport, delayMilliseconds) {if (delayMilliseconds <= 0) return transport;- return { ...transport, send: (peerId, message) => { setTimeout(() => transport.send(peerId, message), delayMilliseconds); } };+ const pendingTimers = new Set();+ const delayed = Object.create(transport);+ delayed.send = (peerId, message) => {+ const timer = setTimeout(() => {+ pendingTimers.delete(timer);+ transport.send(peerId, message);+ }, delayMilliseconds);+ pendingTimers.add(timer);+ };+ delayed.close = () => {+ for (const timer of pendingTimers) clearTimeout(timer);+ pendingTimers.clear();+ transport.close();+ };+ return delayed;}async function createSampleTransport(options, playerId, remoteId) {@@ -73,7 +90,8 @@ function drawPlayers(canvas, netcode, localId) {}}-// options: { role: 'host' | 'client', transport: 'broadcast' | 'websocket', relayUrl, delayMilliseconds, canvas }+// options: { role: 'host' | 'client', transport: 'broadcast' | 'websocket', relayUrl, delayMilliseconds, canvas,+// snapshotEveryTicks, interpolationDelaySeconds }export async function startSample(options) {const isHost = options.role === 'host';const playerId = isHost ? HOST_ID : CLIENT_ID;@@ -86,9 +104,16 @@ export async function startSample(options) {window.addEventListener('keydown', event => { if (event.code in KEY_DIRECTIONS) pressedKeys.add(event.code); });window.addEventListener('keyup', event => pressedKeys.delete(event.code));+ // A snapshot every tick (20 ms) and a 40 ms interpolation delay: the drawn remote player trails by two snapshot intervals.+ const timing = {+ snapshotEveryTicks: options.snapshotEveryTicks ?? SNAPSHOT_EVERY_TICKS,+ interpolationDelaySeconds: options.interpolationDelaySeconds ?? INTERPOLATION_DELAY_SECONDS,+ };const netcode = createNetcode(isHost- ? { role: 'authority', authority: 'host', transport, readInput }- : { role: 'client', serverId: HOST_ID, localEntityId: CLIENT_ID, transport, readInput });+ ? { role: 'authority', authority: 'host', transport, readInput, ...timing }+ : { role: 'client', serverId: HOST_ID, localEntityId: CLIENT_ID, transport, readInput, ...timing });+ // A closed page tells the other one it left, so nobody keeps sending to a tab that is gone.+ window.addEventListener('pagehide', () => transport.close());if (isHost) {netcode.addEntity(HOST_ID, { owner: HOST_ID, tag: 'player', state: { position: [-3, 0, 0] } });netcode.addEntity(CLIENT_ID, { owner: CLIENT_ID, tag: 'player', state: { position: [3, 0, 0] } });diff --git a/src/netcode.js b/src/netcode.jsindex 443ea72b..29123098 100644--- a/src/netcode.js+++ b/src/netcode.js@@ -26,6 +26,8 @@ export const NETCODE_LIMITS = Object.freeze({defaultMessageBytes: 65536,});+const DEFAULT_PEER_TIMEOUT_MILLISECONDS = 5000;+const DEFAULTS = Object.freeze({snapshotEveryTicks: 2,inputRedundancy: 3,@@ -832,14 +834,28 @@ export function createBroadcastChannelTransport(channel, options = {}) {}const id = requireName(options.id, 'createBroadcastChannelTransport id');const maximumMessageBytes = boundedInteger(options.maximumMessageBytes ?? NETCODE_LIMITS.defaultMessageBytes, 'createBroadcastChannelTransport maximumMessageBytes', 256, NETCODE_LIMITS.maximumMessageBytes);+ const peerTimeoutMilliseconds = boundedNumber(options.peerTimeoutMilliseconds ?? DEFAULT_PEER_TIMEOUT_MILLISECONDS, 'createBroadcastChannelTransport peerTimeoutMilliseconds', 50, 3600000);+ const now = options.now ?? Date.now;+ if (typeof now !== 'function') throw new TypeError('createBroadcastChannelTransport now must be a function that returns milliseconds, such as performance.now.');const listeners = new Set();- const knownPeers = new Set();+ const knownPeers = new Map();const stats = { sent: 0, received: 0, rejected: 0 };+ // A peer that closed its page without saying goodbye is forgotten once it has been silent for the timeout.+ // Netcode peers speak every tick, so a live one is heard again at once and re-learned if it was dropped.+ function forgetSilentPeers() {+ const oldestAllowed = now() - peerTimeoutMilliseconds;+ for (const [peerId, lastHeardAt] of knownPeers) {+ if (lastHeardAt < oldestAllowed) knownPeers.delete(peerId);+ }+ }+function learnPeer(peerId) {- if (knownPeers.has(peerId)) return true;- if (knownPeers.size >= NETCODE_LIMITS.maximumPeers) return false;- knownPeers.add(peerId);+ if (!knownPeers.has(peerId)) {+ forgetSilentPeers();+ if (knownPeers.size >= NETCODE_LIMITS.maximumPeers) return false;+ }+ knownPeers.set(peerId, now());return true;}@@ -874,7 +890,10 @@ export function createBroadcastChannelTransport(channel, options = {}) {return {id,- peers: () => [...knownPeers],+ peers() {+ forgetSilentPeers();+ return [...knownPeers.keys()];+ },send(peerId, message) {const text = JSON.stringify(message);if (text === undefined || text.length > maximumMessageBytes) {diff --git a/tests/netcode-broadcast.test.js b/tests/netcode-broadcast.test.jsindex 6c3fd23a..5c3f19aa 100644--- a/tests/netcode-broadcast.test.js+++ b/tests/netcode-broadcast.test.js@@ -84,3 +84,40 @@ test('netcode over BroadcastChannel transports syncs a client to a host', () =>assert.ok(Math.abs(hostPosition[0] - clientPosition[0]) < 0.3);assert.ok(hostPosition[0] > 5);});++test('a BroadcastChannel transport forgets a peer that went silent and learns it again when it speaks', () => {+ let clockMilliseconds = 0;+ const openChannel = createFakeBroadcastBus();+ const host = createBroadcastChannelTransport(openChannel(), { id: 'host', peerTimeoutMilliseconds: 1000, now: () => clockMilliseconds });+ const vanished = createBroadcastChannelTransport(openChannel(), { id: 'vanished' });+ const talkative = createBroadcastChannelTransport(openChannel(), { id: 'talkative' });+ assert.deepEqual(host.peers().sort(), ['talkative', 'vanished']);++ clockMilliseconds = 800;+ talkative.send('host', { ping: 1 });+ clockMilliseconds = 1500;+ assert.deepEqual(host.peers(), ['talkative'], 'the silent tab is dropped, the one heard at 800 ms stays');++ vanished.send('host', { back: true });+ assert.deepEqual(host.peers().sort(), ['talkative', 'vanished']);++ clockMilliseconds = 10000;+ assert.deepEqual(host.peers(), [], 'both are silent by now');+ assert.throws(() => createBroadcastChannelTransport(openChannel(), { id: 'bad', peerTimeoutMilliseconds: 1 }), /peerTimeoutMilliseconds/);+ assert.throws(() => createBroadcastChannelTransport(openChannel(), { id: 'bad', now: 5 }), /now must be a function/);+});++test('a full peer table makes room by dropping silent peers before it refuses a new one', () => {+ let clockMilliseconds = 0;+ const openChannel = createFakeBroadcastBus();+ const host = createBroadcastChannelTransport(openChannel(), { id: 'host', peerTimeoutMilliseconds: 1000, now: () => clockMilliseconds });+ const oldChannel = openChannel();+ for (let index = 0; index < 32; index += 1) oldChannel.postMessage({ from: `old-${index}`, hello: true });+ assert.equal(host.peers().length, 32);+ oldChannel.postMessage({ from: 'refused', hello: true });+ assert.equal(host.peers().includes('refused'), false, 'a full table refuses while everyone is fresh');++ clockMilliseconds = 5000;+ oldChannel.postMessage({ from: 'newcomer', hello: true });+ assert.deepEqual(host.peers(), ['newcomer']);+});diff --git a/tests/netcode-sample.test.js b/tests/netcode-sample.test.jsnew file mode 100644index 00000000..d91ad820--- /dev/null+++ b/tests/netcode-sample.test.js@@ -0,0 +1,48 @@+import assert from 'node:assert/strict';+import test from 'node:test';+import { withInjectedLatency } from '../scripts/netcode-sample.js';++function createFakeTransport() {+ const sent = [];+ const counters = { sent: 0 };+ const state = { closed: false };+ const transport = {+ id: 'fake',+ peers: () => ['other'],+ send(peerId, message) {+ sent.push([peerId, message]);+ counters.sent += 1;+ },+ receive: () => () => {},+ close() { state.closed = true; },+ get stats() { return { ...counters }; },+ };+ return { sent, state, transport };+}++const pause = milliseconds => new Promise(resolve => setTimeout(resolve, milliseconds));++test('withInjectedLatency delays sends and keeps stats and peers live', async () => {+ const fake = createFakeTransport();+ const delayed = withInjectedLatency(fake.transport, 30);+ assert.equal(delayed.id, 'fake');+ assert.deepEqual(delayed.peers(), ['other']);+ assert.equal(delayed.stats.sent, 0);++ delayed.send('other', { n: 1 });+ assert.equal(fake.sent.length, 0, 'nothing leaves before the delay');+ await pause(80);+ assert.deepEqual(fake.sent, [['other', { n: 1 }]]);+ assert.equal(delayed.stats.sent, 1, 'the stats getter follows the real transport');+});++test('withInjectedLatency drops sends still waiting when the transport closes, and passes through at zero delay', async () => {+ const fake = createFakeTransport();+ const delayed = withInjectedLatency(fake.transport, 30);+ delayed.send('other', { n: 1 });+ delayed.close();+ await pause(80);+ assert.equal(fake.state.closed, true);+ assert.equal(fake.sent.length, 0);+ assert.equal(withInjectedLatency(fake.transport, 0), fake.transport);+});diff --git a/tests/netcode-two-browsers.test.js b/tests/netcode-two-browsers.test.jsindex 28ddb6ba..8d8d3f5b 100644--- a/tests/netcode-two-browsers.test.js+++ b/tests/netcode-two-browsers.test.js@@ -14,10 +14,11 @@ const REPOSITORY_ROOT = fileURLToPath(new URL('..', import.meta.url));const SERVED_FOLDERS = ['/src/', '/scripts/netcode-sample.'];const MIME_TYPES = { '.js': 'text/javascript', '.html': 'text/html' };const INJECTED_LATENCY_MILLISECONDS = 50;-const MOVE_DISTANCE = 0.25;+// About a pixel on the 30 px per unit sample canvas, so the clock stops at the first visible movement, not 50 ms into it.+const MOVE_DISTANCE = 0.05;const CONVERGENCE_TOLERANCE = 0.05;-// Measured on a loaded 28-thread machine: host to client about 205 ms, client to host about 130 ms, on both transports. The bound leaves room for load.-const MAXIMUM_LAG_MILLISECONDS = 450;+// Measured on a loaded 28-thread machine: about 95 ms each way on both transports. The bound leaves room for load.+const MAXIMUM_LAG_MILLISECONDS = 250;function startStaticServer() {const server = createServer(async (request, response) => {@@ -114,7 +115,7 @@ for (const transport of ['broadcast', 'websocket']) {assert.ok(difference <= CONVERGENCE_TOLERANCE, `${id} axis ${axis} differs by ${difference} between the pages`);}}- assert.ok(Math.abs(positions.host.alice[0] + 3) > 1, 'the host player moved');+ assert.ok(Math.hypot(positions.host.alice[0] + 3, positions.host.alice[2]) > 0.3, 'the host player moved');assert.ok(result.hostToClientLag > INJECTED_LATENCY_MILLISECONDS && result.hostToClientLag < MAXIMUM_LAG_MILLISECONDS, `host to client lag ${result.hostToClientLag} ms`);assert.ok(result.clientToHostLag > INJECTED_LATENCY_MILLISECONDS && result.clientToHostLag < MAXIMUM_LAG_MILLISECONDS, `client to host lag ${result.clientToHostLag} ms`);assert.equal(positions.host.stats.rejectedMessages, 0);