3D engine that AI agents drive
Streaming: evict by priority then distance, add a worker loader (part of #776)
Nguyen Xuan Anfc755916d0af
diff --git a/changes/776-streaming-priority-worker.md b/changes/776-streaming-priority-worker.mdnew file mode 100644index 00000000..b2b2abec--- /dev/null+++ b/changes/776-streaming-priority-worker.md@@ -0,0 +1 @@+- `67js/addons/streaming` evicts by asset `priority`, then distance from the focus, then recency, and adds `createWorkerLoader` to decode assets in a worker (part of #776)diff --git a/docs/streaming.md b/docs/streaming.mdindex c8a187c5..478dc121 100644--- a/docs/streaming.md+++ b/docs/streaming.md@@ -1,6 +1,6 @@# Streaming: load and free assets while a game plays-`67js/addons/streaming` is a [runtime](runtime.md) system. It loads assets when the player gets near or the game asks, keeps the bytes in use under a budget, frees the least recently used far assets and swaps detail levels. It loads only when imported and adds no dependency. The GLB loader imports the glTF parser on first use.+`67js/addons/streaming` is a [runtime](runtime.md) system. It loads assets when the player gets near or the game asks, keeps the bytes in use under a budget, frees low priority, far assets and swaps detail levels. It loads only when imported and adds no dependency. The GLB loader imports the glTF parser on first use.```jsimport { createRuntime } from '67js/addons/runtime';@@ -26,7 +26,7 @@ const runtime = createRuntime(renderer, { systems: [input, physics, streaming] }## Memory budget-`budgetBytes` caps loaded bytes plus the `estimatedBytes` of loads in flight. Before a load starts, assets nobody wants are freed, least recently used first, until it fits. Reading an asset with `getAsset(id)` or keeping it wanted counts as a use. If nothing can be freed, the load waits and update tries again. When a loader reports more bytes than estimated, the system frees unwanted assets to get back under the budget. Wanted and requested assets are never freed, so a budget smaller than what the player needs shows as waiting loads in `stats`.+`budgetBytes` caps loaded bytes plus the `estimatedBytes` of loads in flight. Before a load starts, assets nobody wants are freed until it fits. The order is lowest `priority` first (a number from -100 to 100, default 0, set on the asset), then the farthest from the focus, then the least recently used, then id, so the same walk always frees the same assets. Reading an asset with `getAsset(id)` or keeping it wanted counts as a use. If nothing can be freed, the load waits and update tries again. When a loader reports more bytes than estimated, the system frees unwanted assets to get back under the budget. Wanted and requested assets are never freed, so a budget smaller than what the player needs shows as waiting loads in `stats`.## Detail levels@@ -44,7 +44,7 @@ streaming.registerLoader('terrain-tile', {});```-A loader is `{ load(request), dispose(asset) }`. `load` returns `{ asset, bytes }` and gets an `AbortSignal` that fires when the system is disposed. The format name is lowercase letters, digits and hyphens, and each can be registered once (`createStreaming({ loaders: { 'terrain-tile': loader } })` does the same). `glb` is registered by default through `createGlbLoader({ options, fetchBytes })`; `options` go to `parseGltf`, so its limits and decoders apply. Splats, point clouds, audio and compressed textures are loaders of their own.+A loader is `{ load(request), dispose(asset) }`. `load` returns `{ asset, bytes }` and gets an `AbortSignal` that fires when the system is disposed. The format name is lowercase letters, digits and hyphens, and each can be registered once (`createStreaming({ loaders: { 'terrain-tile': loader } })` does the same). `glb` is registered by default through `createGlbLoader({ options, fetchBytes })`; `options` go to `parseGltf`, so its limits and decoders apply. `createWorkerLoader({ createWorker, dispose })` decodes off the main thread: it starts a worker for each load, posts `{ id, format, source, level }`, expects `{ asset, bytes }` or `{ error }` back and terminates the worker when the load ends or the system is disposed, so workers never outnumber `maximumConcurrentLoads`. Splats, point clouds, audio and compressed textures are loaders of their own.## Reading the state@@ -54,4 +54,4 @@ A loader is `{ load(request), dispose(asset) }`. `load` returns `{ asset, bytesAt most 4096 assets, 8 levels per asset and 32 loaders. `budgetBytes` is at least 1. Bad input throws an error that says what to pass instead.-Not in this slice: loading in a worker, culling for instanced groups and a measured frame budget on a laptop (#776).+Not yet: culling for instanced groups and a measured frame budget on a laptop (#776).diff --git a/src/streaming.js b/src/streaming.jsindex 53f6d93c..47614f32 100644--- a/src/streaming.js+++ b/src/streaming.js@@ -1,6 +1,6 @@// Live asset streaming: a runtime system that loads, swaps and frees assets while a game plays.// Assets are requested by distance from a focus point or by an explicit game request, held under a byte budget with-// least-recently-used eviction, and switched between detail levels with hysteresis. Formats load through plugins, so GLB+// priority and distance eviction, and switched between detail levels with hysteresis. Formats load through plugins, so GLB// is only the first loader. Everything is decided inside update() from the focus, a counter and sorted ids, never from// wall time, so the same walk streams in the same order. Loaders are async; update() only starts them. See docs/streaming.md.@@ -14,6 +14,8 @@ export const STREAMING_LIMITS = Object.freeze({defaultHysteresis: 0.1,maximumHysteresis: 1,eventHistory: 256,+ minimumPriority: -100,+ maximumPriority: 100,});export const STREAMING_STATES = Object.freeze(['idle', 'pending', 'loaded', 'evicted', 'failed']);@@ -81,6 +83,56 @@ function baseUrlOf(source) {return /^https?:/.test(source) ? new URL('.', source).href : '';}+/**+ * A loader that decodes off the main thread. `createWorker()` returns a Worker (or anything with postMessage, terminate and+ * onmessage/onerror). The worker receives { id, format, source, level } and answers { asset, bytes } or { error }. A fresh worker+ * serves each load and is terminated when the load ends or is aborted, so the number of workers never exceeds the concurrent+ * load limit. `dispose(asset)` frees what the main thread holds.+ */+export function createWorkerLoader({ createWorker, dispose } = {}) {+ if (typeof createWorker !== 'function') {+ throw new TypeError('createWorkerLoader needs createWorker: a function () => Worker, e.g. () => new Worker(new URL("./decode.js", import.meta.url), { type: "module" }).');+ }+ if (dispose !== undefined && typeof dispose !== 'function') throw new TypeError('createWorkerLoader dispose must be a function (asset) or left out.');+ return {+ load({ id, format, source, level, signal }) {+ return new Promise((resolve, reject) => {+ const worker = createWorker();+ const finish = settle => value => {+ signal.removeEventListener('abort', onAbort);+ worker.terminate();+ settle(value);+ };+ const succeed = finish(resolve);+ const fail = finish(reject);+ function onAbort() {+ fail(new Error(`Loading "${id}" level ${level} was aborted.`));+ }+ worker.onmessage = event => {+ const reply = event?.data;+ if (reply === null || typeof reply !== 'object') {+ fail(new TypeError(`The worker for "${id}" must answer { asset, bytes } or { error }, got ${describe(reply)}.`));+ } else if (reply.error !== undefined) {+ fail(new Error(String(reply.error)));+ } else {+ succeed({ asset: reply.asset, bytes: reply.bytes });+ }+ };+ worker.onerror = event => fail(new Error(event?.message ?? `The worker for "${id}" failed.`));+ if (signal.aborted) {+ onAbort();+ return;+ }+ signal.addEventListener('abort', onAbort, { once: true });+ worker.postMessage({ id, format, source, level });+ });+ },+ dispose(asset) {+ dispose?.(asset);+ },+ };+}+function validateLoader(format, loader) {if (typeof format !== 'string' || !/^[a-z0-9][a-z0-9-]*$/.test(format)) {throw new TypeError(`A loader format must be a lowercase name like "glb" or "terrain-tile", got ${describe(format)} ${String(format)}.`);@@ -123,7 +175,8 @@ function validateDefinition(definition, loaders, hysteresis) {throw new RangeError(`Streamed asset "${id}" level maximumDistance values must increase from the finest level to the coarsest.`);}}- return { id, format, position, loadDistance, unloadDistance: loadDistance * (1 + hysteresis), levels };+ const priority = definition.priority === undefined ? 0 : boundedNumber(definition.priority, `Streamed asset "${id}" priority`, STREAMING_LIMITS.minimumPriority, STREAMING_LIMITS.maximumPriority);+ return { id, format, position, priority, loadDistance, unloadDistance: loadDistance * (1 + hysteresis), levels };}export function createStreaming(options = {}) {@@ -189,7 +242,12 @@ export function createStreaming(options = {}) {if (record.wanted || record.requested) continue;for (const [level, resident] of record.resident) candidates.push({ record, level, resident });}- return candidates.sort((first, second) => first.record.lastUsed - second.record.lastUsed || compareText(first.record.id, second.record.id) || first.level - second.level);+ const distances = new Map(candidates.map(candidate => [candidate.record.id, distanceTo(candidate.record)]));+ return candidates.sort((first, second) => first.record.priority - second.record.priority+ || distances.get(second.record.id) - distances.get(first.record.id)+ || first.record.lastUsed - second.record.lastUsed+ || compareText(first.record.id, second.record.id)+ || first.level - second.level);}function evict({ record, level, resident }, reason) {diff --git a/tests/streaming.test.js b/tests/streaming.test.jsindex d33494da..f6af0f4d 100644--- a/tests/streaming.test.js+++ b/tests/streaming.test.js@@ -1,7 +1,7 @@import assert from 'node:assert/strict';import test from 'node:test';import { createRuntime } from '../src/runtime.js';-import { createGlbLoader, createStreaming, selectDetailLevel } from '../src/streaming.js';+import { createGlbLoader, createStreaming, createWorkerLoader, selectDetailLevel } from '../src/streaming.js';function fakeLoader() {const log = [];@@ -29,7 +29,7 @@ async function walkTo(streaming, point) {await streaming.settled();}-test('evicts the least recently used far asset to stay under the byte budget', async () => {+test('evicts the farthest unwanted asset to stay under the byte budget', async () => {const loader = fakeLoader();const streaming = createStreaming({ budgetBytes: 250, loaders: { tile: loader } });for (const [id, x] of [['a', 0], ['b', 100], ['c', 200]]) streaming.addAsset(tileDefinition(id, x));@@ -180,3 +180,102 @@ test('runs as a runtime system and disposes what it holds', async () => {assert.deepEqual(loader.disposed, ['home#0']);assert.throws(() => streaming.addAsset(tileDefinition('late', 0)), /disposed/);});++test('eviction frees low priority assets first, then the farthest, whatever the use order', async () => {+ const loader = fakeLoader();+ const streaming = createStreaming({ budgetBytes: 300, loaders: { tile: loader } });+ streaming.addAsset(tileDefinition('important', 0, { priority: 10 }));+ streaming.addAsset(tileDefinition('far-plain', 100));+ streaming.addAsset(tileDefinition('near-plain', 200));+ streaming.addAsset(tileDefinition('newcomer', 300));++ await walkTo(streaming, [0, 0, 0]);+ await walkTo(streaming, [100, 0, 0]);+ await walkTo(streaming, [200, 0, 0]);+ await walkTo(streaming, [300, 0, 0]);++ assert.deepEqual(loader.disposed, ['far-plain#0']);+});++test('eviction frees the farthest of two equal priority assets even when it was used last', async () => {+ const loader = fakeLoader();+ const streaming = createStreaming({ budgetBytes: 200, loaders: { tile: loader } });+ streaming.addAsset(tileDefinition('west', -100));+ streaming.addAsset(tileDefinition('east', 100));+ streaming.addAsset(tileDefinition('north', 0, { position: [0, 0, 300] }));+ await walkTo(streaming, [100, 0, 0]);+ await walkTo(streaming, [-100, 0, 0]);+ streaming.getAsset('east');+ await walkTo(streaming, [0, 0, 300]);+ assert.deepEqual(loader.disposed, ['east#0']);+});++test('a priority outside the range is rejected with the range', () => {+ const streaming = createStreaming({ budgetBytes: 100, loaders: { tile: fakeLoader() } });+ assert.throws(() => streaming.addAsset(tileDefinition('a', 0, { priority: 500 })), /priority must be a number from -100 to 100/);+});++function fakeWorkerFactory(behavior) {+ const workers = [];+ const createWorker = () => {+ const worker = {+ terminated: false,+ posted: [],+ postMessage(message) {+ worker.posted.push(message);+ queueMicrotask(() => behavior(worker, message));+ },+ terminate() {+ worker.terminated = true;+ },+ };+ workers.push(worker);+ return worker;+ };+ return { workers, createWorker };+}++test('the worker loader decodes in a worker and terminates it after each load', async () => {+ const { workers, createWorker } = fakeWorkerFactory((worker, message) => worker.onmessage({ data: { asset: { decoded: message.source }, bytes: 64 } }));+ const freed = [];+ const streaming = createStreaming({+ budgetBytes: 1000,+ loaders: { tile: createWorkerLoader({ createWorker, dispose: asset => freed.push(asset.decoded) }) },+ });+ streaming.addAsset(tileDefinition('a', 0));+ streaming.addAsset(tileDefinition('b', 5));+ await walkTo(streaming, [0, 0, 0]);++ assert.deepEqual(streaming.getAsset('a'), { decoded: 'a.bin' });+ assert.equal(streaming.getState('b').bytes, 64);+ assert.equal(workers.length, 2);+ assert.ok(workers.every(worker => worker.terminated));+ assert.deepEqual(workers[0].posted[0], { id: 'a', format: 'tile', source: 'a.bin', level: 0 });+ streaming.dispose();+ assert.deepEqual(freed.sort(), ['a.bin', 'b.bin']);+});++test('a worker error fails the load and a dispose aborts the worker in flight', async () => {+ const failing = fakeWorkerFactory(worker => worker.onmessage({ data: { error: 'bad tile' } }));+ const failingStreaming = createStreaming({ budgetBytes: 1000, loaders: { tile: createWorkerLoader({ createWorker: failing.createWorker }) } });+ failingStreaming.addAsset(tileDefinition('a', 0));+ await walkTo(failingStreaming, [0, 0, 0]);+ assert.equal(failingStreaming.getState('a').state, 'failed');+ assert.equal(failingStreaming.getState('a').failure, 'bad tile');+ assert.ok(failing.workers[0].terminated);++ const silent = fakeWorkerFactory(() => {});+ const silentStreaming = createStreaming({ budgetBytes: 1000, loaders: { tile: createWorkerLoader({ createWorker: silent.createWorker }) } });+ silentStreaming.addAsset(tileDefinition('a', 0));+ silentStreaming.update();+ await new Promise(resolve => setTimeout(resolve, 0));+ silentStreaming.dispose();+ await new Promise(resolve => setTimeout(resolve, 0));+ assert.ok(silent.workers[0].terminated);+ assert.equal(silentStreaming.stats.activeLoads, 0);+});++test('createWorkerLoader says what to pass when createWorker is missing', () => {+ assert.throws(() => createWorkerLoader({}), /createWorker: a function/);+ assert.throws(() => createWorkerLoader({ createWorker: () => ({}), dispose: 3 }), /dispose must be a function/);+});