Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Measure recipe bake presets and frame costs on a quiet machine (part of #170)

Nguyen Xuan Ane3d5dc86dc5f
diff --git a/bench/reference/README.md b/bench/reference/README.md
index cc8e3060..bd8cdff9 100644
--- a/bench/reference/README.md
+++ b/bench/reference/README.md
@@ -582,7 +582,34 @@ What the table shows:
- The 20 s inline limit and the budget agree on large rooms: `temple-hall`, the largest scene (167 parts, 304,480 texels), bakes in 5.7 s at the budget, and the slowest cell above is 6.0 s, so the budget leaves room under the limit on this machine.
- `draft` is a rough preset: one bounce leaves the lit rooms at 0.29 to 0.32, still well under the unbaked 0.51.
-Frame costs from the issue (a baked closed room, and the fog glow mip chain on `dark-shaft`) were not measured: `nvidia-smi` showed 14 to 48 percent utilization and 6 to 7 GB in use over 24 seconds while another worker held the GPU, so a number would be a loaded one. The ranges above in this file stay ranges.
+### Presets again on a quiet machine (2026-11-03, issue #170)
+
+Same command, `npm run reference -- --skip-blender --bake <five ids> --bake-preset <name> --probes <five ids>`, one run per cell, at below-normal priority, one GPU job at a time, against the stored Cycles renders. `nvidia-smi` showed 0 percent GPU and 1,028 MiB (the desktop) before each of the three runs. Whole frame FLIP against Cycles (lower is better), bake seconds from the harness, and `bake.used` (samples, bounces, texels per unit; `reduced` is true when the preset shrank to its budget):
+
+| Scene | `draft` | `standard` and `fine` |
+| --- | --- | --- |
+| `lit-interior` | 0.2975, 0.3 s, 8,000 texels (32, 1, 3) | 0.1342, 3.4 s, 19,360 texels (81, 3, 4.86, reduced) |
+| `brick-room` | 0.3266, 1.2 s, 37,136 texels (32, 1, 3) | 0.1448, 1.7 s, 42,576 texels (32, 3, 3.19, reduced) |
+| `dark-shaft` | 0.0638, 0.7 s, 28,208 texels (32, 1, 3) | 0.0547, 2.8 s, 40,336 texels (40, 3, 3.54, reduced) |
+| `temple-hall` | 0.0656, 3.1 s, 285,376 texels (16, 1, 1.29, reduced) | 0.0655, 3.6 s, 304,480 texels (16, 1, 1.37, reduced) |
+| `metal-room` | 0.3244, 0.6 s, 8,816 texels (32, 1, 3) | 0.1631, 3.0 s, 15,936 texels (64, 3, 4.37, reduced) |
+
+`fine` came out identical to `standard` in every cell (same score, same `used`; the bake seconds differ by 0.1 s, which is noise), so it is still an alias and the table shows one column for both. Every `standard` and `fine` bake was reduced, so an agent that names the preset gets fewer samples and a lower density than the preset asks for on all five rooms, and `bake.reduction` says so. Scores moved by up to 0.008 from the table above (for example `lit-interior` 0.1270 to 0.1342, `dark-shaft` 0.0610 to 0.0547) because `main` changed since then; the order of the presets did not.
+
+### Frame costs on a quiet machine (2026-11-03, issue #170)
+
+`node bench/reference/frame-time.mjs <scene> gpu --bake-cost` (a bake against the plain frame) and `--glow-mips` (the fog's glow mip chain generated every frame against the same frame with `generateMipmap` skipped after warm-up, so the sampling cost is the same and the difference is the chain), both with `--repetitions=5`. Each repetition is the median of 200 frames at 960 x 540, wall clock around `readPixels` (not GPU timer queries: `harness.js` times the frame on the CPU after reading a pixel back), and the two sides alternate order each round. The script sets itself to below-normal priority, so these ran at below-normal, not normal. Idle check before each run, from `nvidia-smi` (GPU utilization, memory, any process over 1 GB) and `Win32_Processor.LoadPercentage`: no process held more than 1 GB, the GPU memory was the desktop's 1.0 GB, utilization 0 percent and CPU load as below. The numbers right after a run (10 to 67 percent GPU) include the run's own tail and are not machine load.
+
+| Measurement | CPU load before | Plain or skipped, median (range) | Baked or generated, median (range) | Cost, paired by round |
+| --- | --- | --- | --- | --- |
+| Baked `lit-interior`, run 1 | 19 percent | 0.835 ms (0.55 to 0.845) | 1.010 ms (0.99 to 1.02) | +0.155 to +0.18 ms in four of five rounds, +0.47 in the first (a 0.55 ms first plain frame) |
+| Baked `lit-interior`, run 2 | 3 percent | 0.855 ms (0.555 to 1.03) | 1.260 ms (0.995 to 2.00) | +0.14 to +0.97 ms, median +0.41 |
+| Glow mip chain, `dark-shaft`, run 1 | 9 percent | 1.70 ms (1.415 to 1.785) | 1.56 ms (1.40 to 2.015) | -0.39 to +0.24 ms, median +0.12 |
+| Glow mip chain, `dark-shaft`, run 2 | 14 percent | 1.72 ms (1.395 to 2.33) | 1.515 ms (1.385 to 2.37) | -0.34 to +0.26 ms, median -0.01 |
+
+- A baked closed room costs about +0.2 to +0.4 ms a frame (all ten paired rounds: +0.14 to +0.97 ms, median +0.29 ms), under the +0.6 to +1.4 ms ranges from a loaded GPU. The first run, with the steadier machine, gives +0.17 ms. Measured.
+- The glow mip chain is not separable from noise: 10 paired rounds went from -0.39 to +0.26 ms, median +0.05 ms, so the earlier +0.5 ms is not reproduced. It was a plain frame of the new commit against numbers from the commit before, on a shared machine; the old commit also changed the glow taps (inferred, not rerun). Within a run the frame time drifts by up to 0.9 ms (1.4 to 2.3 ms on `dark-shaft`, no other process visible), so a cost under about 0.3 ms at this size cannot be read from one run; a bound is that the chain costs less than 0.4 ms at 960 x 540. Measured, with the drift unexplained (GPU clock changes are a guess).
+- The machine was idle by the stated thresholds (GPU under 10 percent, CPU under 20 percent, no process over 1 GB) before every run, apart from CPU at 19 percent in the first bake run.
## Sky visibility in crowns and under the valley rim, what is left (2026-11-03, issue #191)
diff --git a/bench/reference/frame-time.mjs b/bench/reference/frame-time.mjs
index f1222974..3b468771 100644
--- a/bench/reference/frame-time.mjs
+++ b/bench/reference/frame-time.mjs
@@ -27,7 +27,15 @@ try {
// --plain measures only the plain frame, for scenes with more parts than the light bake takes.
// --objects measures the plain frame against the frame with one probe for each reflective part.
// --water measures a frame without water reflection, with the planar one and with the screen-space one.
- const variants = process.argv.includes('--water')
+ // --bake-cost measures the plain frame against the frame with the scene's light bake. --glow-mips measures the frame with the fog's
+ // glow mip chain generated every frame against the same frame with the generation skipped. --repetitions=N sets the rounds (3 by default).
+ const repetitionsArgument = process.argv.find(argument => argument.startsWith('--repetitions='));
+ const rounds = Array.from({ length: repetitionsArgument ? Number(repetitionsArgument.split('=')[1]) : 3 }, (_, index) => index + 1);
+ const variants = process.argv.includes('--bake-cost')
+ ? rounds.flatMap(round => (round % 2 ? [false, true] : [true, false]).map(bake => ({ bake, probes: false })))
+ : process.argv.includes('--glow-mips')
+ ? rounds.flatMap(round => (round % 2 ? [true, false] : [false, true]).map(glowMipChain => ({ bake: false, probes: false, glowMipChain })))
+ : process.argv.includes('--water')
? [1, 2, 3].flatMap(round => (round % 2 ? ['off', 'planar', 'ssr'] : ['ssr', 'planar', 'off']).map(water => ({ bake: false, probes: false, water: water === 'off' ? false : water })))
: process.argv.includes('--plain')
? [1, 2, 3].map(() => ({ bake: false, probes: false }))
@@ -42,11 +50,12 @@ try {
const medianOf = values => values.sort((first, second) => first - second)[Math.floor(values.length / 2)];
const summary = {};
for (const result of results) {
- const key = process.argv.includes('--water') ? (result.water || 'off') : result.baked ? (result.probes ? 'bake+probes' : 'bake') : (result.probes ? 'probes' : 'plain');
+ const key = process.argv.includes('--glow-mips') ? (result.glowMipChain ? 'mips generated' : 'mips skipped') : process.argv.includes('--water') ? (result.water || 'off') : result.baked ? (result.probes ? 'bake+probes' : 'bake') : (result.probes ? 'probes' : 'plain');
(summary[key] ??= []).push(result.medianMilliseconds);
}
const medianByVariant = Object.fromEntries(Object.entries(summary).map(([key, values]) => [key, Number(medianOf(values).toFixed(3))]));
- console.log(JSON.stringify({ sceneId, backend, medianMillisecondsByVariant: medianByVariant, results }, null, 1));
+ const rangeByVariant = Object.fromEntries(Object.entries(summary).map(([key, values]) => [key, [Math.min(...values), Math.max(...values)].map(value => Number(value.toFixed(3)))]));
+ console.log(JSON.stringify({ sceneId, backend, medianMillisecondsByVariant: medianByVariant, rangeMillisecondsByVariant: rangeByVariant, results }, null, 1));
} finally {
await browser.close();
await server.close();
diff --git a/bench/reference/harness.js b/bench/reference/harness.js
index aa0523bc..73cc44e0 100644
--- a/bench/reference/harness.js
+++ b/bench/reference/harness.js
@@ -192,7 +192,7 @@ async function renderReferenceScene(id, { describe, ambientOcclusion = 0, ambien
}
// Frame time of one scene with and without the light bake: a bake only adds an emissive map to each part, so this shows what it costs to draw.
-async function measureReferenceFrameTime(id, { bake, probes = false, cascades = 0, orbit = false, water = WATER_REFLECTION, warmupFrames = 20, measuredFrames = 200 }) {
+async function measureReferenceFrameTime(id, { bake, probes = false, cascades = 0, orbit = false, water = WATER_REFLECTION, glowMipChain = true, warmupFrames = 20, measuredFrames = 200 }) {
const scene = await SCENES[id]();
const baked = bake ? await bakeScene(scene, {}) : null;
const probed = probes ? await probeScene(scene, id, baked ? baked.parts : scene.parts, probes) : null;
@@ -208,6 +208,11 @@ async function measureReferenceFrameTime(id, { bake, probes = false, cascades =
renderer.render(orbit ? { ...frame, eye: orbitedEye(index) } : frame);
finishDrawing();
}
+ // glowMipChain false stops the fog from rebuilding the frame's mip chain after warm-up. The mips stay as the last warm-up frame
+ // left them, so sampling costs the same and the difference is the cost of generating them.
+ if (!glowMipChain) {
+ renderer.context.generateMipmap = () => {};
+ }
const frameMilliseconds = [];
for (let index = 0; index < measuredFrames; index++) {
const started = performance.now();
@@ -218,7 +223,7 @@ async function measureReferenceFrameTime(id, { bake, probes = false, cascades =
renderer.dispose();
frameMilliseconds.sort((first, second) => first - second);
const total = frameMilliseconds.reduce((sum, value) => sum + value, 0);
- return { baked: Boolean(bake), probes: Boolean(probes), water, frames: measuredFrames, meanMilliseconds: total / measuredFrames,
+ return { baked: Boolean(bake), probes: Boolean(probes), water, glowMipChain, frames: measuredFrames, meanMilliseconds: total / measuredFrames,
medianMilliseconds: frameMilliseconds[Math.floor(measuredFrames / 2)], p95Milliseconds: frameMilliseconds[Math.floor(measuredFrames * 0.95)] };
}
diff --git a/bench/reference/run.mjs b/bench/reference/run.mjs
index 42053c4f..8be588f8 100644
--- a/bench/reference/run.mjs
+++ b/bench/reference/run.mjs
@@ -124,7 +124,7 @@ async function renderBackend(backend, serverUrl, graphics) {
const background = await page.evaluate(sceneId => window.renderReferenceScene(sceneId, { describe: false, backgroundOnly: true }), id);
await writeFile(`${WORK_DIRECTORY}/${id}-background.png`, Buffer.from(background.image, 'base64'));
}
- const bakeNote = result.bake?.report.refused ? `, bake REFUSED, drawn unbaked: ${result.bake.report.refused}` : result.bake ? `, bake ${result.bake.seconds.toFixed(1)} s, ${result.bake.report.texels} texels${result.bake.report.reduced ? ` (preset reduced to ${result.bake.report.used.samples} samples, ${result.bake.report.used.bounces} bounces, ${result.bake.report.used.texelsPerUnit} texels per unit)` : ''}` : '';
+ const bakeNote = result.bake?.report.refused ? `, bake REFUSED, drawn unbaked: ${result.bake.report.refused}` : result.bake ? `, bake ${result.bake.seconds.toFixed(1)} s, ${result.bake.report.texels} texels${result.bake.report.used ? ` (used ${result.bake.report.used.samples} samples, ${result.bake.report.used.bounces} bounces, ${result.bake.report.used.texelsPerUnit} texels per unit${result.bake.report.reduced ? ', reduced' : ''})` : ''}` : '';
const probeNote = result.probes ? `, probes ${result.probes.report.placed.length} on ${result.probes.report.partsWithProbe} parts` : '';
console.log(`${id} on ${backend}: ${Math.round(performance.now() - started)} ms${bakeNote}${probeNote}`);
}
diff --git a/changes/issue-170-presets.md b/changes/issue-170-presets.md
new file mode 100644
index 00000000..c9faa86e
--- /dev/null
+++ b/changes/issue-170-presets.md
@@ -0,0 +1 @@
+- Measure the recipe bake presets and the frame costs on a quiet machine (part of issue #170): `draft`, `standard` and `fine` scored against Cycles on the five room scenes with bake seconds and the settings each preset used, and the frame cost of a baked closed room and of the fog's glow mip chain, each with the GPU and CPU load recorded beside it. `frame-time.mjs` gains `--bake-cost`, `--glow-mips` and `--repetitions=N` (and prints the range of the per-run medians); the reference run prints `used` for every preset bake. See `bench/reference/README.md`.