3D engine that AI agents drive
Streaming: keep least-recently-used as the default eviction order, harden the worker loader (part of #776)
Nguyen Xuan An9745e68e41cc
diff --git a/changes/776-streaming-priority-worker.md b/changes/776-streaming-priority-worker.mdindex b2b2abec..8970e340 100644--- a/changes/776-streaming-priority-worker.md+++ b/changes/776-streaming-priority-worker.md@@ -1 +1 @@-- `67js/addons/streaming` evicts by asset `priority`, then distance from the focus, then recency, and adds `createWorkerLoader` to decode assets in a worker (part of #776)+- `67js/addons/streaming` evicts lowest asset `priority` first (new `evictionOrder: "distance"` opts in to farthest-first within a priority; the default stays least recently used) and adds `createWorkerLoader` to decode assets in a worker (part of #776)diff --git a/changes/776-streaming-priority-worker.size.md b/changes/776-streaming-priority-worker.size.mdindex ba5d22cb..e3ccb4d9 100644--- a/changes/776-streaming-priority-worker.size.md+++ b/changes/776-streaming-priority-worker.size.md@@ -1 +1 @@-- entry:streaming: priority-aware eviction and the worker loader let a game keep key assets resident and decode off the main thread; 506 B gzip+- entry:streaming: priority-aware eviction (default order unchanged) and the worker loader let a game keep key assets resident and decode off the main thread; ~600 B gzipdiff --git a/docs/streaming.md b/docs/streaming.mdindex 478dc121..edd0c55c 100644--- a/docs/streaming.md+++ b/docs/streaming.md@@ -1,6 +1,6 @@# Streaming: load and free assets while a game plays-`67js/addons/streaming` is a [runtime](runtime.md) system. It loads assets when the player gets near or the game asks, keeps the bytes in use under a budget, frees low priority, far assets and swaps detail levels. It loads only when imported and adds no dependency. The GLB loader imports the glTF parser on first use.+`67js/addons/streaming` is a [runtime](runtime.md) system. It loads assets when the player gets near or the game asks, keeps the bytes in use under a budget, frees low priority assets and swaps detail levels. It loads only when imported and adds no dependency. The GLB loader imports the glTF parser on first use.```jsimport { createRuntime } from '67js/addons/runtime';@@ -26,7 +26,7 @@ const runtime = createRuntime(renderer, { systems: [input, physics, streaming] }## Memory budget-`budgetBytes` caps loaded bytes plus the `estimatedBytes` of loads in flight. Before a load starts, assets nobody wants are freed until it fits. The order is lowest `priority` first (a number from -100 to 100, default 0, set on the asset), then the farthest from the focus, then the least recently used, then id, so the same walk always frees the same assets. Reading an asset with `getAsset(id)` or keeping it wanted counts as a use. If nothing can be freed, the load waits and update tries again. When a loader reports more bytes than estimated, the system frees unwanted assets to get back under the budget. Wanted and requested assets are never freed, so a budget smaller than what the player needs shows as waiting loads in `stats`.+`budgetBytes` caps loaded bytes plus the `estimatedBytes` of loads in flight. Before a load starts, assets nobody wants are freed until it fits. The order is lowest `priority` first (a number from -100 to 100, default 0, set on the asset), then by `evictionOrder`: `"recency"` (the default, unchanged from before) frees the least recently used first and breaks ties by distance; `"distance"` frees the farthest from the focus first and breaks ties by recency. Last comes the id, so the same walk always frees the same assets. Reading an asset with `getAsset(id)` or keeping it wanted counts as a use. If nothing can be freed, the load waits and update tries again. When a loader reports more bytes than estimated, the system frees unwanted assets to get back under the budget. Wanted and requested assets are never freed, so a budget smaller than what the player needs shows as waiting loads in `stats`.## Detail levels@@ -44,7 +44,7 @@ streaming.registerLoader('terrain-tile', {});```-A loader is `{ load(request), dispose(asset) }`. `load` returns `{ asset, bytes }` and gets an `AbortSignal` that fires when the system is disposed. The format name is lowercase letters, digits and hyphens, and each can be registered once (`createStreaming({ loaders: { 'terrain-tile': loader } })` does the same). `glb` is registered by default through `createGlbLoader({ options, fetchBytes })`; `options` go to `parseGltf`, so its limits and decoders apply. `createWorkerLoader({ createWorker, dispose })` decodes off the main thread: it starts a worker for each load, posts `{ id, format, source, level }`, expects `{ asset, bytes }` or `{ error }` back and terminates the worker when the load ends or the system is disposed, so workers never outnumber `maximumConcurrentLoads`. Splats, point clouds, audio and compressed textures are loaders of their own.+A loader is `{ load(request), dispose(asset) }`. `load` returns `{ asset, bytes }` and gets an `AbortSignal` that fires when the system is disposed. The format name is lowercase letters, digits and hyphens, and each can be registered once (`createStreaming({ loaders: { 'terrain-tile': loader } })` does the same). `glb` is registered by default through `createGlbLoader({ options, fetchBytes })`; `options` go to `parseGltf`, so its limits and decoders apply. `createWorkerLoader({ createWorker, dispose })` decodes off the main thread: it starts a worker for each load, posts `{ id, format, source, level }`, expects `{ asset, bytes }` or `{ error }` back, fails the load when the worker throws, cannot clone the source or sends a reply that cannot be read, and terminates the worker when the load ends or the system is disposed, so workers never outnumber `maximumConcurrentLoads`. Splats, point clouds, audio and compressed textures are loaders of their own.## Reading the statediff --git a/src/streaming.js b/src/streaming.jsindex 47614f32..01178b98 100644--- a/src/streaming.js+++ b/src/streaming.js@@ -97,10 +97,10 @@ export function createWorkerLoader({ createWorker, dispose } = {}) {return {load({ id, format, source, level, signal }) {return new Promise((resolve, reject) => {- const worker = createWorker();+ let worker = null;const finish = settle => value => {signal.removeEventListener('abort', onAbort);- worker.terminate();+ worker?.terminate();settle(value);};const succeed = finish(resolve);@@ -108,23 +108,29 @@ export function createWorkerLoader({ createWorker, dispose } = {}) {function onAbort() {fail(new Error(`Loading "${id}" level ${level} was aborted.`));}- worker.onmessage = event => {- const reply = event?.data;- if (reply === null || typeof reply !== 'object') {- fail(new TypeError(`The worker for "${id}" must answer { asset, bytes } or { error }, got ${describe(reply)}.`));- } else if (reply.error !== undefined) {- fail(new Error(String(reply.error)));- } else {- succeed({ asset: reply.asset, bytes: reply.bytes });- }- };- worker.onerror = event => fail(new Error(event?.message ?? `The worker for "${id}" failed.`));if (signal.aborted) {onAbort();return;}signal.addEventListener('abort', onAbort, { once: true });- worker.postMessage({ id, format, source, level });+ try {+ worker = createWorker();+ worker.onmessage = event => {+ const reply = event?.data;+ if (reply === null || typeof reply !== 'object') {+ fail(new TypeError(`The worker for "${id}" must answer { asset, bytes } or { error }, got ${describe(reply)}.`));+ } else if (reply.error !== undefined) {+ fail(new Error(String(reply.error)));+ } else {+ succeed({ asset: reply.asset, bytes: reply.bytes });+ }+ };+ worker.onmessageerror = () => fail(new Error(`The worker for "${id}" sent a reply that could not be read.`));+ worker.onerror = event => fail(new Error(event?.message ?? `The worker for "${id}" failed.`));+ worker.postMessage({ id, format, source, level });+ } catch (error) {+ fail(error instanceof Error ? error : new Error(String(error)));+ }});},dispose(asset) {@@ -191,6 +197,11 @@ export function createStreaming(options = {}) {throw new TypeError('createStreaming onEvent must be a function (event) or left out.');}+ const evictionOrder = options.evictionOrder ?? 'recency';+ if (evictionOrder !== 'recency' && evictionOrder !== 'distance') {+ throw new RangeError(`createStreaming evictionOrder must be "recency" (least recently used first, the default) or "distance" (farthest first), got ${describe(evictionOrder)} ${String(evictionOrder)}.`);+ }+const loaders = new Map();const records = new Map();const events = [];@@ -243,9 +254,12 @@ export function createStreaming(options = {}) {for (const [level, resident] of record.resident) candidates.push({ record, level, resident });}const distances = new Map(candidates.map(candidate => [candidate.record.id, distanceTo(candidate.record)]));+ const farthestFirst = (first, second) => distances.get(second.record.id) - distances.get(first.record.id);+ const leastRecentFirst = (first, second) => first.record.lastUsed - second.record.lastUsed;+ const [primary, secondary] = evictionOrder === 'distance' ? [farthestFirst, leastRecentFirst] : [leastRecentFirst, farthestFirst];return candidates.sort((first, second) => first.record.priority - second.record.priority- || distances.get(second.record.id) - distances.get(first.record.id)- || first.record.lastUsed - second.record.lastUsed+ || primary(first, second)+ || secondary(first, second)|| compareText(first.record.id, second.record.id)|| first.level - second.level);}diff --git a/tests/streaming.test.js b/tests/streaming.test.jsindex f6af0f4d..4ce48507 100644--- a/tests/streaming.test.js+++ b/tests/streaming.test.js@@ -29,7 +29,7 @@ async function walkTo(streaming, point) {await streaming.settled();}-test('evicts the farthest unwanted asset to stay under the byte budget', async () => {+test('evicts the least recently used far asset to stay under the byte budget', async () => {const loader = fakeLoader();const streaming = createStreaming({ budgetBytes: 250, loaders: { tile: loader } });for (const [id, x] of [['a', 0], ['b', 100], ['c', 200]]) streaming.addAsset(tileDefinition(id, x));@@ -197,17 +197,23 @@ test('eviction frees low priority assets first, then the farthest, whatever theassert.deepEqual(loader.disposed, ['far-plain#0']);});-test('eviction frees the farthest of two equal priority assets even when it was used last', async () => {- const loader = fakeLoader();- const streaming = createStreaming({ budgetBytes: 200, loaders: { tile: loader } });- streaming.addAsset(tileDefinition('west', -100));- streaming.addAsset(tileDefinition('east', 100));- streaming.addAsset(tileDefinition('north', 0, { position: [0, 0, 300] }));- await walkTo(streaming, [100, 0, 0]);- await walkTo(streaming, [-100, 0, 0]);- streaming.getAsset('east');- await walkTo(streaming, [0, 0, 300]);- assert.deepEqual(loader.disposed, ['east#0']);+// west is used first (least recent) and nearer to the final focus; east is used last and farther.+for (const [evictionOrder, expected] of [[undefined, 'west#0'], ['recency', 'west#0'], ['distance', 'east#0']]) {+ test(`eviction order ${evictionOrder ?? 'default'} frees ${expected} among equal priority assets`, async () => {+ const loader = fakeLoader();+ const streaming = createStreaming({ budgetBytes: 200, evictionOrder, loaders: { tile: loader } });+ streaming.addAsset(tileDefinition('west', -100));+ streaming.addAsset(tileDefinition('east', 100));+ streaming.addAsset(tileDefinition('north', 0, { position: [-100, 0, 300] }));+ await walkTo(streaming, [-100, 0, 0]);+ await walkTo(streaming, [100, 0, 0]);+ await walkTo(streaming, [-100, 0, 300]);+ assert.deepEqual(loader.disposed, [expected]);+ });+}++test('an unknown eviction order is rejected with the choices', () => {+ assert.throws(() => createStreaming({ budgetBytes: 100, evictionOrder: 'random' }), /evictionOrder must be "recency".*"distance"/);});test('a priority outside the range is rejected with the range', () => {@@ -279,3 +285,36 @@ test('createWorkerLoader says what to pass when createWorker is missing', () =>assert.throws(() => createWorkerLoader({}), /createWorker: a function/);assert.throws(() => createWorkerLoader({ createWorker: () => ({}), dispose: 3 }), /dispose must be a function/);});++test('a postMessage that throws fails the load and terminates the worker', async () => {+ const { workers, createWorker } = fakeWorkerFactory(() => {});+ const throwing = () => {+ const worker = createWorker();+ worker.postMessage = () => {+ throw new Error('clone');+ };+ return worker;+ };+ const controller = new AbortController();+ const loader = createWorkerLoader({ createWorker: throwing });+ await assert.rejects(loader.load({ id: 'a', format: 'tile', source: 'a.bin', level: 0, signal: controller.signal }), /clone/);+ assert.ok(workers[0].terminated);+ controller.abort();+ assert.equal(workers.length, 1);+});++test('a createWorker that throws fails the load', async () => {+ const loader = createWorkerLoader({ createWorker: () => { throw new Error('no worker'); } });+ await assert.rejects(loader.load({ id: 'a', format: 'tile', source: 'a.bin', level: 0, signal: new AbortController().signal }), /no worker/);+});++test('an unreadable worker reply fails the load and frees the slot', async () => {+ const { workers, createWorker } = fakeWorkerFactory(worker => worker.onmessageerror({}));+ const streaming = createStreaming({ budgetBytes: 1000, loaders: { tile: createWorkerLoader({ createWorker }) } });+ streaming.addAsset(tileDefinition('a', 0));+ await walkTo(streaming, [0, 0, 0]);+ assert.equal(streaming.getState('a').state, 'failed');+ assert.match(streaming.getState('a').failure, /"a" sent a reply that could not be read/);+ assert.ok(workers[0].terminated);+ assert.equal(streaming.stats.activeLoads, 0);+});