Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Fall back to the full world when the first view fails, and skip the prepared draw before a world has drawn (Part of #884)

Nguyen Xuan Anfae79a619cc7
diff --git a/docs/hero-loading.md b/docs/hero-loading.md
index 8d783d38..dc7dba67 100644
--- a/docs/hero-loading.md
+++ b/docs/hero-loading.md
@@ -8,10 +8,12 @@ The hero on the home page plays a flight over the Port showcase world. The world
2. When the poster has settled (or after 5 s), `index.html` starts the first view: `scene.json`, `scene.first.glb` (about 2 MB) and `sky.preview.hdr`.
3. When those have settled, it starts the full world: `scene.glb`, `scene.rest.glb` and `sky.hdr`.
4. `fetchWorld(url, { firstView: true })` in `src/hero-lab/loop/world.js` resolves as soon as the first view is in. It holds a `detail` promise for the full world.
-5. The world is built from the first view and draws. `completeDetail` (`src/hero-lab/loop/detail.js`) then waits for `detail`, parses it and calls `world.upgrade(detail)`. The upgrade uploads the textures and the sky while the first view keeps drawing. Only the swap itself (the sky prefilter, one `replaceParts` and a prepared draw) holds the draws.
+5. The world is built from the first view and draws. `completeDetail` (`src/hero-lab/loop/detail.js`) then waits for `detail`, parses it and calls `world.upgrade(detail)`. The upgrade uploads the textures and the sky while the first view keeps drawing. Only the swap itself (the sky prefilter, one `replaceParts` and a prepared draw) holds the draws: about 140 ms on a desktop and a phone viewport (`swapHeldMs` of `scripts/measure-hero-load.mjs`, GPU shared with other jobs). Feeding `replaceParts` in slices was tried and doubled the hold (swap 0.8 s to 1.5 s) without shortening the longest gap between frames (about 110 ms either way).
A link faster than 20 Mbit/s (measured from the resource timing of the first view's own download) skips the first view and builds the full world straight away.
+A first view that is missing, corrupt or fails to build is replaced by the full world (`fetchFirstViewFiles` in `world.js`, `buildWithFallback` in `detail.js`). A world that was never drawn, such as a page opened in a background tab, swaps without a prepared draw; its first real frame does that work.
+
A full world that fails to arrive or to build is asked for again after 4, 10 and 20 s; after that the first view stays. A world that is freed in the middle of a swap ends the swap quietly.
## The first view
@@ -22,7 +24,7 @@ A full world that fails to arrive or to build is asked for again after 4, 10 and
- It splits the rest into six distance bands (edges at 100, 170, 260, 400 and 650 m) and simplifies each band to a screen error of 3 px up to 400 m and 6 px beyond, locking the borders so the bands do not crack apart.
- Textures are 512 px for materials seen within 400 m and 128 px for the rest. Materials that are never seen are dropped.
-The options (`--near-pixels`, `--far-pixels` and others) are read by `parseArguments` in the script. The geometry the first view lacks lies outside the first 5 s of the flight, so `src/hero-lab/loop/hold.js` slows the flight from 2.5 s while the full world is pending and stops it at 5 s, until the full world is in (or has been given up on).
+The options (`--near-pixels`, `--far-pixels` and others) are read by `parseArguments` in the script. The geometry the first view lacks lies outside the first 5 s of the flight, so `src/hero-lab/loop/hold.js` slows the flight from 2.5 s while the full world is pending and stops it at 5 s, until the full world is in (or has been given up on). The cost is that on a very slow link the frame is almost still for most of the wait (at 1.6 Mbit/s the first view appears at about 14 s and the swap lands at about 64 s); the alternative is holes at the edges of the picture, and the page before this change showed nothing at all for that time.
## Measuring
diff --git a/scripts/measure-hero-load.mjs b/scripts/measure-hero-load.mjs
index fd8660f1..5e9e15e2 100644
--- a/scripts/measure-hero-load.mjs
+++ b/scripts/measure-hero-load.mjs
@@ -9,6 +9,7 @@
// up, with brotli for text, as a CDN would. Network time comes from the DevTools throttle, which GPU load does not change.
// posterEndMs responseEnd of the poster image
// firstFrameMs the first draw on a world canvas (the first live 3D frame)
+// swapMs, swapHeldMs, swapLongestGapMs how long the swap to the full world took, how long its draws were held (sky prefilter, replaceParts, prepared draw), and the longest gap between animation frames around it
// detailMs when the full world is drawn (the 'detailed' loop event); without staged loading it equals firstFrameMs
// bytesBeforeFirstFrame, bytesTotal transfer bytes (compressed size on the wire) up to the first live frame, and up to detailMs
import { createServer } from 'node:http';
@@ -87,6 +88,15 @@ async function startServer(root) {
// first live 3D frame a visitor sees. The poster fades out only after that.
function installFrameTimer(marker) {
window.__firstWorldDraw = null;
+ // Gaps between animation frames over 50 ms: the main thread was blocked. The swap to the full world should leave none that is long.
+ window.__frameGaps = [];
+ let previousFrameAt = performance.now();
+ const watchFrameGaps = now => {
+ if (now - previousFrameAt > 50) window.__frameGaps.push([Math.round(previousFrameAt), Math.round(now - previousFrameAt)]);
+ previousFrameAt = now;
+ requestAnimationFrame(watchFrameGaps);
+ };
+ requestAnimationFrame(watchFrameGaps);
new MutationObserver(() => {
if (window.__firstWorldDraw === null && document.querySelector('#stage')?.dataset.live === 'true') {
window.__firstWorldDraw = performance.now();
@@ -143,7 +153,9 @@ async function measureOnce(browser, options, viewportName, conditionName) {
const events = window.__loop?.events ?? [];
const detailed = events.find(event => event.kind === 'detailed');
const failed = events.find(event => event.kind === 'failed');
- return { first: window.__firstWorldDraw, detailed: detailed?.at ?? null, failed: failed?.message ?? null, state: window.__loop?.detail?.() ?? null, mode: window.__heroMode };
+ const swapEnd = detailed?.at ?? Infinity;
+ const swapGaps = (window.__frameGaps ?? []).filter(([at, gap]) => at + gap > swapEnd - (detailed?.ms ?? 0) - 100 && at < swapEnd + 100);
+ return { first: window.__firstWorldDraw, detailed: detailed?.at ?? null, swapMs: detailed?.ms ?? null, swapHeldMs: (window.__phases ?? []).filter(phase => phase[0] === "swap").reduce((total, phase) => total + phase[3] - phase[2], 0), swapLongestGapMs: swapGaps.length ? Math.max(...swapGaps.map(([, gap]) => gap)) : 0, failed: failed?.message ?? null, state: window.__loop?.detail?.() ?? null, mode: window.__heroMode };
});
const deadline = Date.now() + options.timeoutSeconds * 1000;
let status = await finished();
@@ -164,6 +176,9 @@ async function measureOnce(browser, options, viewportName, conditionName) {
posterEndMs: poster,
firstFrameMs,
detailMs,
+ swapMs: status.swapMs,
+ swapLongestGapMs: status.swapLongestGapMs,
+ swapHeldMs: status.swapHeldMs,
bytesBeforeFirstFrame: bytes.atFirstFrame,
bytesAtDetail: detailMs === null ? null : bytesAt(bytes, detailMs),
errors: [...errors, ...(status.failed ? [status.failed] : [])],
@@ -193,6 +208,9 @@ async function runMatrix(browser, options) {
firstFrameMs: median(runs.map(run => run.firstFrameMs)),
firstFrameRangeMs: [Math.min(...runs.map(run => run.firstFrameMs ?? Infinity)), Math.max(...runs.map(run => run.firstFrameMs ?? 0))],
detailMs: median(runs.map(run => run.detailMs)),
+ swapMs: median(runs.map(run => run.swapMs)),
+ swapHeldMs: median(runs.map(run => run.swapHeldMs)),
+ swapLongestGapMs: median(runs.map(run => run.swapLongestGapMs)),
detailRangeMs: [Math.min(...runs.map(run => run.detailMs ?? Infinity)), Math.max(...runs.map(run => run.detailMs ?? 0))],
bytesBeforeFirstFrame: median(runs.map(run => run.bytesBeforeFirstFrame)),
bytesAtDetail: median(runs.map(run => run.bytesAtDetail)),
diff --git a/src/hero-lab/hero-loop.js b/src/hero-lab/hero-loop.js
index 364ff49f..996871e8 100644
--- a/src/hero-lab/hero-loop.js
+++ b/src/hero-lab/hero-loop.js
@@ -10,7 +10,7 @@ import { createFlight } from './loop/camera.js';
import { buildWorld, fetchDetail, fetchWorld } from './loop/world.js';
import { createPlayOnDemand } from './loop/play-on-demand.js';
import { advanceHold } from './loop/hold.js';
-import { completeDetail } from './loop/detail.js';
+import { buildWithFallback, completeDetail } from './loop/detail.js';
const parameters = new URLSearchParams(location.search);
const captureMode = parameters.has('capture');
@@ -97,7 +97,7 @@ function prepare(index, { immediately = false, firstView = false } = {}) {
await entry.permission;
if (entry.skipped) return;
const started = performance.now();
- const world = await buildFrom(download);
+ const { world, builtFrom } = await buildWithFallback(download, buildFrom);
await new Promise(resolve => setTimeout(resolve, 0));
// The world joins the loop only after its first draw has linked every program and allocated every target, so the fade never waits on the driver.
const warmStarted = performance.now();
@@ -108,8 +108,8 @@ function prepare(index, { immediately = false, firstView = false } = {}) {
return;
}
entry.world = world;
- if (download.isFirstView) entry.detail = followDetail(entry, download);
- note('built', { scene: SCENES[index].id, ms: Math.round(performance.now() - started), firstView: Boolean(download.isFirstView) });
+ if (builtFrom.isFirstView) entry.detail = followDetail(entry, builtFrom);
+ note('built', { scene: SCENES[index].id, ms: Math.round(performance.now() - started), firstView: Boolean(builtFrom.isFirstView) });
loopState.peakWorlds = Math.max(loopState.peakWorlds, loopState.worldsAlive());
})
.catch(error => {
diff --git a/src/hero-lab/loop/detail.js b/src/hero-lab/loop/detail.js
index 1658772c..ae2dae1f 100644
--- a/src/hero-lab/loop/detail.js
+++ b/src/hero-lab/loop/detail.js
@@ -41,3 +41,15 @@ export async function completeDetail({
}
}
}
+
+// Builds a downloaded world. A first view that does not parse or upload is not the end of the hero: the full world, which is already
+// on its way, is built instead. Resolves with the world and the download it was built from.
+export async function buildWithFallback(download, build) {
+ try {
+ return { world: await build(download), builtFrom: download };
+ } catch (error) {
+ if (!download.isFirstView) throw error;
+ const fullDownload = await download.detail;
+ return { world: await build(fullDownload), builtFrom: fullDownload };
+ }
+}
diff --git a/src/hero-lab/loop/world.js b/src/hero-lab/loop/world.js
index 9f2c1d44..5f4d94b8 100644
--- a/src/hero-lab/loop/world.js
+++ b/src/hero-lab/loop/world.js
@@ -137,17 +137,58 @@ export function measureLinkMegabits(url) {
return (entry.encodedBodySize * 8) / seconds / 1e6;
}
+async function fetchFirstViewFiles(description) {
+ const paths = [description.glbFirstView];
+ try {
+ const [models, sky] = await Promise.all([Promise.all(paths.map(fetchBlob)), fetchBlob(description.hdriPreview)]);
+ return { paths, models, sky };
+ } catch {
+ return null;
+ }
+}
+
export async function fetchWorld(descriptionUrl, { firstView = false, measureLink = measureLinkMegabits } = {}) {
const description = await readDescription(descriptionUrl);
if (!firstView || !hasFirstView(description)) return fetchDetail(description);
- const paths = [description.glbFirstView];
- const [models, sky] = await Promise.all([Promise.all(paths.map(fetchBlob)), fetchBlob(description.hdriPreview)]);
+ const firstViewFiles = await fetchFirstViewFiles(description);
+ // A first view that is missing or corrupt must not cost the visitor the hero: the full world, which the page serves anyway, takes its place.
+ if (!firstViewFiles) return fetchDetail(description);
+ const { paths, models, sky } = firstViewFiles;
if ((measureLink(description.glbFirstView) ?? 0) > FAST_LINK_MEGABITS) return fetchDetail(description);
const detail = fetchDetail(description);
detail.catch(() => {});
return { description, paths, models, sky, bytes: bytesOf(models, sky), isFirstView: true, detail };
}
+// The part of a swap that holds the draws: the sky prefilter, then the parts. Resolves false when the world was freed on the way.
+// One replaceParts call, not slices: the draws are held either way, and measured on the port the slices made the hold twice as long.
+export async function replaceRendererWorld(renderer, parts, environmentMap, isFreed) {
+ const prefilterStart = performance.now();
+ await renderer.prepareEnvironmentInSlices(environmentMap, createSlicer(SLICE_MILLISECONDS));
+ recordSwapPhase('prefilter', prefilterStart);
+ if (isFreed()) return false;
+ const replaceStart = performance.now();
+ renderer.replaceParts(parts);
+ recordSwapPhase('replaceParts', replaceStart);
+ return !isFreed();
+}
+
+// Leaves the length of one phase of the swap in window.__phases, next to the laps of buildWorld, for the load measurement.
+function recordSwapPhase(name, start) {
+ const phases = globalThis.window?.__phases;
+ if (phases) phases.push(['swap', name, Math.round(start), Math.round(performance.now())]);
+}
+
+// The first draw of a swapped world at the pose of its last frame, so the next frame does not link or allocate. A world that was
+// never drawn (a page opened in a background tab) has no pose, and its first real frame does that work.
+export async function prepareFrameAtLastPose(renderer, lastDraw, frameFor) {
+ if (!lastDraw) return false;
+ const prepareStart = performance.now();
+ await renderer.prepareFrame(frameFor(lastDraw.camera, lastDraw.seconds));
+ recordSwapPhase('prepareFrame', prepareStart);
+ return true;
+}
+
async function parseFiles(download, sharedTextures) {
const { description, paths, models } = download;
const files = [];
@@ -272,9 +313,8 @@ export async function buildWorld(download, canvas, overrides = {}) {
if (disposed) return false;
drawsHeld = true;
try {
- await renderer.prepareEnvironmentInSlices(fullMap, createSlicer(SLICE_MILLISECONDS));
- if (disposed) return false;
- renderer.replaceParts(fullWhole.parts);
+ const replaced = await replaceRendererWorld(renderer, fullWhole.parts, fullMap, () => disposed);
+ if (!replaced) return false;
detachReflection();
detachReflection = attachWaterReflection(renderer, fullWhole.surface);
effects = fullEffects;
@@ -282,8 +322,7 @@ export async function buildWorld(download, canvas, overrides = {}) {
animationOrigin = null;
upgradedMap = fullMap;
isFirstView = false;
- const pose = lastDraw?.camera ?? { eye: frame.eye, target: frame.target, fov: frame.fov };
- await renderer.prepareFrame(frameFor(pose, lastDraw?.seconds ?? 0));
+ await prepareFrameAtLastPose(renderer, lastDraw, frameFor);
return !disposed;
} catch (error) {
// A world freed in the middle of its swap has no renderer left to finish on: that ends the swap, it does not fail it.
diff --git a/tests/hero-progressive.test.js b/tests/hero-progressive.test.js
index 9cfc9c80..5745c3d9 100644
--- a/tests/hero-progressive.test.js
+++ b/tests/hero-progressive.test.js
@@ -1,9 +1,9 @@
import assert from 'node:assert/strict';
import { existsSync, readFileSync, statSync } from 'node:fs';
import { afterEach, beforeEach, test } from 'node:test';
-import { completeDetail } from '../src/hero-lab/loop/detail.js';
+import { buildWithFallback, completeDetail } from '../src/hero-lab/loop/detail.js';
import { advanceHold, FIRST_VIEW_COVERS_SECONDS, HOLD_FROM_SECONDS, HOLD_RATE, holdTargetRate } from '../src/hero-lab/loop/hold.js';
-import { FAST_LINK_MEGABITS, fetchWorld, hasFirstView } from '../src/hero-lab/loop/world.js';
+import { FAST_LINK_MEGABITS, fetchWorld, hasFirstView, prepareFrameAtLastPose, replaceRendererWorld } from '../src/hero-lab/loop/world.js';
const DESCRIPTION_URL = '/showcase/port/scene.json';
const description = {
@@ -248,3 +248,65 @@ test('the first view the port description names exists and stays within its byte
assert.ok(firstViewBytes <= 2_100_000, `first view is ${firstViewBytes} bytes`);
assert.ok(existsSync(new URL(`../public${published.hdriPreview}`, import.meta.url)));
});
+
+test('fetchWorld falls back to the full world when the first view model or sky is missing', async () => {
+ for (const missing of [description.glbFirstView, description.hdriPreview]) {
+ installFetch();
+ const original = globalThis.fetch;
+ globalThis.fetch = url => (url === missing ? Promise.resolve({ ok: false, status: 404 }) : original(url));
+ const download = await fetchWorld(DESCRIPTION_URL, { firstView: true, measureLink: slowLink });
+ assert.equal(download.isFirstView, undefined, missing);
+ assert.deepEqual(download.paths, [description.glb, ...description.glbRest], missing);
+ }
+});
+
+test('a first view that fails to build is replaced by the full world, and a full world that fails to build is not hidden', async () => {
+ const firstViewDownload = { isFirstView: true, detail: Promise.resolve({ id: 'full' }) };
+ const built = [];
+ const build = async download => {
+ built.push(download);
+ if (download.isFirstView) throw new Error('first view does not parse');
+ return { world: download.id };
+ };
+ const result = await buildWithFallback(firstViewDownload, build);
+ assert.deepEqual(result.world, { world: 'full' });
+ assert.equal(result.builtFrom.isFirstView, undefined);
+ assert.equal(built.length, 2);
+ await assert.rejects(buildWithFallback({ id: 'full' }, async () => { throw new Error('full does not parse'); }), /full does not parse/);
+});
+
+function fakeRenderer(calls) {
+ return {
+ prepareEnvironmentInSlices: async () => { calls.push('environment'); },
+ replaceParts: () => { calls.push('replaceParts'); },
+ prepareFrame: async () => { calls.push('prepareFrame'); },
+ };
+}
+
+const fakeParts = [{ geometry: { indices: [0, 1, 2] } }, { geometry: { indices: [0, 1, 2] } }];
+
+test('the swap prefilters the sky before it replaces the parts, and stops when the world is freed on the way', async () => {
+ const calls = [];
+ assert.equal(await replaceRendererWorld(fakeRenderer(calls), fakeParts, {}, () => false), true);
+ assert.equal(calls[0], 'environment');
+ assert.ok(calls.length > 1 && calls.slice(1).every(name => name === 'replaceParts'));
+ const freedCalls = [];
+ assert.equal(await replaceRendererWorld(fakeRenderer(freedCalls), fakeParts, {}, () => true), false);
+ assert.deepEqual(freedCalls, ['environment']);
+});
+
+test('the swap prepares a draw at the last pose, and none when the world was never drawn', async () => {
+ const calls = [];
+ const frames = [];
+ const frameFor = (camera, seconds) => {
+ frames.push([camera, seconds]);
+ return { camera };
+ };
+ assert.equal(await prepareFrameAtLastPose(fakeRenderer(calls), null, frameFor), false);
+ assert.deepEqual(calls, []);
+ assert.equal(frames.length, 0, 'no pose, so no frame is built from a missing eye');
+ const camera = { eye: [1, 2, 3], target: [0, 0, 0], fov: 40 };
+ assert.equal(await prepareFrameAtLastPose(fakeRenderer(calls), { camera, seconds: 7 }, frameFor), true);
+ assert.deepEqual(calls, ['prepareFrame']);
+ assert.deepEqual(frames, [[camera, 7]]);
+});