3D engine that AI agents drive
JavaScript 3d3d-engineagentic-aiai-agentsblenderclaude-codegame-enginegaussian-splattinggltfjavascriptmcpmcp-servermodel-context-protocolpath-tracerpath-tracingpbrrenderingthreejs-alternativewebgl2webgpu
Keep the grid texel buffer between frames of a sequence (#1045)
Nguyen Xuan Anfce254a21ee2
diff --git a/changes/1045-upload-buffer.md b/changes/1045-upload-buffer.mdnew file mode 100644index 00000000..17073ec0--- /dev/null+++ b/changes/1045-upload-buffer.md@@ -0,0 +1 @@+- A grid sequence reuses one texel buffer for every frame it uploads, so a changing frame costs about 3 ms less at 1080p (#1045)diff --git a/docs/volumetric.md b/docs/volumetric.mdindex bb8d5522..a44d340f 100644--- a/docs/volumetric.md+++ b/docs/volumetric.md@@ -238,7 +238,7 @@ One grid volume per scene for now. The rasterizer samples two RGBA16F 3D texture`uv run --no-project scripts/vdb-to-grid.py "sim/smoke_*.vdb" blast --sequence --max-resolution 128` converts a numbered VDB sequence in one run: the files are sorted by their numbers (`smoke_2` before `smoke_10`), every frame is put on the union of all the frames' boxes so the resolution is the same, and the files `blast_0000.67vg`, `blast_0001.67vg` and so on are written, each with every channel any frame has (zero where a frame has none, so the shader never reads the density as flame there). A frame with an empty density grid is written empty with a warning that names the file. The byte budget is checked after the first pass and before anything is written. The script prints the `frames` list to paste into the volume.-**Playback cost of a real sequence.** `node bench/reference/volume-grid-vdb-sequence.mjs <grid folder> <output folder> [--width 1920] [--height 1080] [--clip]` plays a converted VDB sequence on a round diorama (a slab on a plinth, a high oblique camera, no sky) and prints milliseconds per frame over the whole sequence, with the frame changing on every draw (one upload each) and with a held frame. Measured on the CC0 "Ground Explosion" by JangaFX (EmberGen), 90 frames converted with `--max-resolution 96` to 82 x 93 x 96 voxels with density and flame, on an RTX 3060 shared with other jobs, 1920 x 1080, five runs: the median of a changing frame was 20.5 to 29.1 ms (30 fps is 33.3 ms), its 95th percentile 30 to 44 ms and its worst frame 40 to 58 ms, and a held frame took a median of 6.8 to 11.0 ms. Before #1045 stopped rebuilding the half float table for every frame, walking the fire light's emission with an array per cell and uploading a zero emission texture for a grid with no temperature, the same sequence took a median of 180 ms a frame at 960 x 540. The 95th percentile is above 33 ms, so a frame now and then misses 30 fps.+**Playback cost of a real sequence.** `node bench/reference/volume-grid-vdb-sequence.mjs <grid folder> <output folder> [--width 1920] [--height 1080] [--clip]` plays a converted VDB sequence on a round diorama (a slab on a plinth, a high oblique camera, no sky) and prints milliseconds per frame over the whole sequence, with the frame changing on every draw (one upload each) and with a held frame. Measured on the CC0 "Ground Explosion" by JangaFX (EmberGen), 90 frames converted with `--max-resolution 96` to 82 x 93 x 96 voxels with density and flame, on an RTX 3060 shared with other jobs, 1920 x 1080, five runs: the median of a changing frame was 20.5 to 29.1 ms (30 fps is 33.3 ms), its 95th percentile 30 to 44 ms and its worst frame 40 to 58 ms, and a held frame took a median of 6.8 to 11.0 ms. Before #1045 stopped rebuilding the half float table for every frame, walking the fire light's emission with an array per cell and uploading a zero emission texture for a grid with no temperature, the same sequence took a median of 180 ms a frame at 960 x 540. The 95th percentile is above 33 ms, so a frame now and then misses 30 fps. The padded texel buffer is now kept between frames of one resolution instead of allocated for each: on a 72 frame sequence of 0.4 m grids (the same bench, three runs) a changing frame went from a median of 11.1 to 17.7 ms (95th percentile 17 to 27 ms) to 9.2 to 11.7 ms (11.5 to 13.5 ms).**Grid file.** `"67VG"`, a uint32 version (1), a uint32 header length, a JSON header `{ "resolution": [x, y, z], "channels": ["density", "flame"], "size": [x, y, z] }` padded to 4 bytes, then each channel as half floats, x fastest, then y, then z, y up. Each axis is 1 to 256 and the data at most 64 MiB; anything past that is refused with the limit and the `--max-resolution` to use.diff --git a/src/fog-shader.js b/src/fog-shader.jsindex 4f5a4d4a..5bfcca36 100644--- a/src/fog-shader.js+++ b/src/fog-shader.js@@ -232,8 +232,8 @@ function bindLocalVolumes(renderer, post, frame, environment) {// temperature, baked on the CPU (bakedGridEmission) so the march reads it with one linear tap and the interpolation happens after the blackbody, as Cycles does// (#1752). Both carry GRID_BORDER_VOXELS empty voxels on every face. A frame with no grid volume gets 1 x 1 x 1 textures, so the samplers are always valid.function localGridTexture(renderer, grid) {+ const { density, temperature, flame } = grid?.channels ?? {};return uploadGridTexture(renderer, 'localGridTextureCache', grid, null, (texels, voxel, texel) => {- const { density, temperature, flame } = grid.channels;texels[texel] = density[voxel];texels[texel + 1] = temperature ? temperature[voxel] : 0;texels[texel + 2] = (flame ?? density)[voxel];@@ -284,7 +284,11 @@ function uploadGridTexture(renderer, cacheName, grid, scale, writeVoxel) {for (const wrap of [context.TEXTURE_WRAP_S, context.TEXTURE_WRAP_T, context.TEXTURE_WRAP_R]) context.texParameteri(context.TEXTURE_3D, wrap, context.CLAMP_TO_EDGE);}const padded = resolution.map(count => count + 2 * GRID_BORDER_VOXELS);- const texels = new Uint16Array(padded[0] * padded[1] * padded[2] * 4);+ // The padded buffer is kept with the texture: its border is never written, and a frame with the same resolution overwrites every interior voxel, so a+ // sequence allocates it once instead of once per frame (#1045).+ const texelCount = padded[0] * padded[1] * padded[2] * 4;+ const texels = reuse && cached.texels ? cached.texels : new Uint16Array(texelCount);+ if (!filled) texels.fill(0);if (filled) {for (let z = 0; z < resolution[2]; z++) {for (let y = 0; y < resolution[1]; y++) {@@ -302,7 +306,7 @@ function uploadGridTexture(renderer, cacheName, grid, scale, writeVoxel) {} else {context.texImage3D(context.TEXTURE_3D, 0, context.RGBA16F, padded[0], padded[1], padded[2], 0, context.RGBA, context.HALF_FLOAT, texels);}- renderer[cacheName] = { grid, scale, texture, resolution, filled };+ renderer[cacheName] = { grid, scale, texture, resolution, filled, texels };return texture;}@@ -805,7 +809,7 @@ function packLights(renderer) {registerShaderFeatureCode('volumeFog', {// The JavaScript half, which the renderer calls for a frame that draws a volume fog.- support: { packLights, bindUniforms, usesDetail: fogUsesDetail, prepareLocalVolumes, bindLocalVolumes, withVolumeFireLights, compositeLocalVolumes, releaseLocalVolumeComposite },+ support: { packLights, localGridTexture, bindUniforms, usesDetail: fogUsesDetail, prepareLocalVolumes, bindLocalVolumes, withVolumeFireLights, compositeLocalVolumes, releaseLocalVolumeComposite },keyDeclarations: KEY_LIGHT_DECLARATIONS,keyFunctions: KEY_LIGHT_FUNCTIONS,localDeclarations: LOCAL_VOLUME_DECLARATIONS,diff --git a/tests/volume-grid-upload.test.js b/tests/volume-grid-upload.test.jsnew file mode 100644index 00000000..c63046de--- /dev/null+++ b/tests/volume-grid-upload.test.js@@ -0,0 +1,72 @@+// Playing a grid sequence uploads one 3D texture per frame (#1045). The padded texel buffer is kept between frames of one resolution, so a sequence allocates it+// once, and each frame still lands in the texture exactly: its voxels inside a border of empty voxels, nothing left over from the frame before.+import test from 'node:test';+import assert from 'node:assert/strict';+import '../src/fog-shader.js';+import { requireFeatureCode } from '../src/shader-registry.js';+import { GRID_BORDER_VOXELS, decodeVolumeGrid, encodeVolumeGrid, floatToHalf } from '../src/volume-grid.js';++const RESOLUTION = [3, 2, 2];+const VOXELS = RESOLUTION[0] * RESOLUTION[1] * RESOLUTION[2];++function gridOf(density, flame) {+ const channels = { density: new Float32Array(VOXELS).fill(density), flame: new Float32Array(VOXELS).fill(flame) };+ return decodeVolumeGrid(encodeVolumeGrid({ resolution: RESOLUTION, size: [1, 1, 1], channels }));+}++function recordingContext() {+ const calls = [];+ const names = ['TEXTURE_3D', 'TEXTURE_MIN_FILTER', 'TEXTURE_MAG_FILTER', 'LINEAR', 'TEXTURE_WRAP_S', 'TEXTURE_WRAP_T', 'TEXTURE_WRAP_R', 'CLAMP_TO_EDGE', 'UNPACK_ALIGNMENT', 'RGBA', 'HALF_FLOAT', 'RGBA16F'];+ const context = Object.fromEntries(names.map((name, index) => [name, index + 1]));+ Object.assign(context, {+ createTexture: () => ({ id: calls.length }),+ deleteTexture() {},+ bindTexture() {},+ texParameteri() {},+ pixelStorei() {},+ texImage3D: (...args) => calls.push({ kind: 'texImage3D', texels: args[9] }),+ texSubImage3D: (...args) => calls.push({ kind: 'texSubImage3D', texels: args[10] }),+ });+ return { context, calls };+}++function texelAt(texels, padded, x, y, z) {+ const border = GRID_BORDER_VOXELS;+ return ((x + border) + padded[0] * ((y + border) + padded[1] * (z + border))) * 4;+}++test('a sequence keeps one texel buffer, and every frame fills the interior and leaves the border empty', () => {+ const { localGridTexture } = requireFeatureCode('volumeFog').support;+ const { context, calls } = recordingContext();+ const renderer = { context };+ const padded = RESOLUTION.map(count => count + 2 * GRID_BORDER_VOXELS);+ localGridTexture(renderer, gridOf(0.5, 2));+ localGridTexture(renderer, gridOf(0.25, 4));+ assert.deepEqual(calls.map(call => call.kind), ['texImage3D', 'texSubImage3D'], 'the second frame overwrites the first one\'s texture');+ assert.equal(calls[1].texels, calls[0].texels, 'the second frame reuses the first frame\'s texel buffer');+ const texels = calls[1].texels;+ assert.equal(texels[texelAt(texels, padded, 1, 1, 1)], floatToHalf(0.25));+ assert.equal(texels[texelAt(texels, padded, 1, 1, 1) + 2], floatToHalf(4));+ assert.equal(texels[texelAt(texels, padded, 0, 0, 0)], floatToHalf(0.25));+ let nonEmptyBorderTexels = 0;+ for (let z = 0; z < padded[2]; z++) {+ for (let y = 0; y < padded[1]; y++) {+ for (let x = 0; x < padded[0]; x++) {+ const interior = [x, y, z].every((value, axis) => value >= GRID_BORDER_VOXELS && value < GRID_BORDER_VOXELS + RESOLUTION[axis]);+ const offset = (x + padded[0] * (y + padded[1] * z)) * 4;+ if (!interior && texels.slice(offset, offset + 4).some(value => value !== 0)) nonEmptyBorderTexels++;+ }+ }+ }+ assert.equal(nonEmptyBorderTexels, 0, 'the border of empty voxels stays empty');+});++test('the same frame twice uploads once', () => {+ const { localGridTexture } = requireFeatureCode('volumeFog').support;+ const { context, calls } = recordingContext();+ const renderer = { context };+ const grid = gridOf(0.5, 2);+ localGridTexture(renderer, grid);+ localGridTexture(renderer, grid);+ assert.equal(calls.length, 1);+});