Skip to content

3D engine that AI agents drive

Check image decode inputs at the boundary (#2454, #2455, #2456)

Nguyen Xuan Ane19ee1e150d2
diff --git a/changes/2454.md b/changes/2454.md
new file mode 100644
index 00000000..860144fc
--- /dev/null
+++ b/changes/2454.md
@@ -0,0 +1 @@
+- `parseRadianceHdr` names the accepted input and refuses a header that claims more pixels than the file holds before allocating them, `keyLightFromHdri` says which bounds to pass, and GLB export checks `maxTextureSize` before reading a texture's pixels (#2454, #2455, #2456)
diff --git a/changes/2454.size.md b/changes/2454.size.md
new file mode 100644
index 00000000..de94627f
--- /dev/null
+++ b/changes/2454.size.md
@@ -0,0 +1 @@
+- entry:hdri: the parser no longer allocates up to 3.2 GB for a 60 byte header, and the two public calls name what to pass instead of a raw TypeError, for about 200 bytes gzip.
diff --git a/docs/agent-look.md b/docs/agent-look.md
index d7cd68d8..e2163655 100644
--- a/docs/agent-look.md
+++ b/docs/agent-look.md
@@ -72,7 +72,7 @@ A recipe names a file; it never carries a path or a URL, and nothing is download
| `teal-orange`, `noir` | `environment.look` | made for Crucible, CC0, 9 points per side |
| 64-character ID | either | a `.hdr` or `.cube` file (an `.hdr` up to 8 MiB, which fits the 2k files from Poly Haven, and a `.cube` up to 4 MiB; a larger file is skipped and listed with a reason, it does not fail the catalog) in the MCP `--assets` directory, listed by `scene_asset_catalog` as kind `environment` or `look` |
-The four shipped files total about 100 KB. For a bigger HDRI, put a 2k or smaller `.hdr` (for example from Poly Haven; the 2k files are about 5 to 7 MiB) in the assets directory and use its ID. Outside MCP, decode with `parseRadianceHdr` or `parseCube` and register with `registerCatalogEnvironment(name, data)` or `registerCatalogLook(name, look)` from `src/look-catalog.js` before building the scene. The look needs the `view-transform-shader` module (`loadAgentSceneModules` loads it). Render jobs light the scene with the same entries.
+The four shipped files total about 100 KB. For a bigger HDRI, put a 2k or smaller `.hdr` (for example from Poly Haven; the 2k files are about 5 to 7 MiB) in the assets directory and use its ID. Outside MCP, decode with `parseRadianceHdr` (an ArrayBuffer or Uint8Array of the whole file; a header that claims more pixels than the file holds is refused) or `parseCube` and register with `registerCatalogEnvironment(name, data)` or `registerCatalogLook(name, look)` from `src/look-catalog.js` before building the scene. The look needs the `view-transform-shader` module (`loadAgentSceneModules` loads it). Render jobs light the scene with the same entries.
## Atomic look commands
diff --git a/docs/export-glb.md b/docs/export-glb.md
index 8ca8b820..111e94ab 100644
--- a/docs/export-glb.md
+++ b/docs/export-glb.md
@@ -94,7 +94,7 @@ A PNG the export writes itself is stored without compression so the export stays
Geometry with textures needs `uvs`.
Without them the export fails and names `uv_unwrap` (a mesh stack node) or `geometry.uvs`.
-An image larger than `maxTextureSize` (default 2048, up to 16384) fails with that option's name.
+An image larger than `maxTextureSize` (default 2048, up to 16384) fails with that option's name, before its pixels are read.
### Pattern materials
diff --git a/docs/lighting.md b/docs/lighting.md
index d299408b..5ce043d2 100644
--- a/docs/lighting.md
+++ b/docs/lighting.md
@@ -51,7 +51,7 @@ const renderer = new CrucibleRenderer(canvas, parts, { hdr: true, environment: {
An HDRI faces the way it does in three.js (`EquirectangularReflectionMapping`) and Blender Cycles (default mapping): at `rotation` 0 the file's column 0.25 is seen toward -Z, the centre column toward +X, and the image turns clockwise seen from above. `loadHdri`, `registerCatalogEnvironment` and so `environment.hdri` turn the rows a quarter of the width as they load (`src/environment-orientation.js`), so the samplers, the path tracers, Cook and the probes keep one convention. Maps built with `createEnvironmentMap` from your own rows are used as given. A scene made before 0.21.0 gets its old look back with `environmentRotation` + `Math.PI / 2`. `tests/environment-orientation.test.js` pins the test spot; the table in the pull request for #1371 lists where it appeared in each renderer.
-`sunFromEnvironmentMap(map)` finds the sun in an outdoor map: the brightest 3 by 3 block, then the luminance-weighted centre of the pixels within 10 degrees of it that reach half its brightness. It returns `{ lightDirection, keyColor, keyIntensity }` with `rotation` applied, or `null` when the peak is under 20 times the mean luminance (overcast). `keyLightFromHdri(map, { min, max })` wraps it for a scene box and returns `{ environment: { lightDirection, keyColor, keyIntensity }, shadowBounds }`, so `new CrucibleRenderer(canvas, parts, { environment: { map, ...lit.environment }, shadowBounds: lit.shadowBounds })` gets sun shadows with no hand-tuned numbers. `keyIntensity` is an estimate (sun irradiance over pi); tune it by eye. On Poly Haven san_giuseppe_bridge_4k with `rotation` pi the direction is 0.04 degrees from the hand-tuned one.
+`sunFromEnvironmentMap(map)` finds the sun in an outdoor map: the brightest 3 by 3 block, then the luminance-weighted centre of the pixels within 10 degrees of it that reach half its brightness. It returns `{ lightDirection, keyColor, keyIntensity }` with `rotation` applied, or `null` when the peak is under 20 times the mean luminance (overcast). `keyLightFromHdri(map, { min, max })` (each corner three finite numbers, else it throws a TypeError) wraps it for a scene box and returns `{ environment: { lightDirection, keyColor, keyIntensity }, shadowBounds }`, so `new CrucibleRenderer(canvas, parts, { environment: { map, ...lit.environment }, shadowBounds: lit.shadowBounds })` gets sun shadows with no hand-tuned numbers. `keyIntensity` is an estimate (sun irradiance over pi); tune it by eye. On Poly Haven san_giuseppe_bridge_4k with `rotation` pi the direction is 0.04 degrees from the hand-tuned one.
Set `environment.map` in the constructor or in `frame.environment`. Maps spread from one created map, such as `{ ...map, rotation: time }`, reuse its GPU textures, so rotation animates for free. Passing a different map prefilters it on first use, and `map: null` returns to studio lighting. With no map the output is pixel-identical to before.
diff --git a/src/export-textures.js b/src/export-textures.js
index 2e381a56..c6750d97 100644
--- a/src/export-textures.js
+++ b/src/export-textures.js
@@ -12,12 +12,7 @@ const VALID_MAG_FILTERS = [9728, 9729];
const VALID_MIN_FILTERS = [9728, 9729, 9984, 9985, 9986, 9987];
-function pixelsOf(image, channel) {
- const width = image?.naturalWidth ?? image?.width;
- const height = image?.naturalHeight ?? image?.height;
- if (!Number.isInteger(width) || !Number.isInteger(height) || width < 1 || height < 1) {
- throw new TypeError(`GLB ${channel} needs an image with a whole width and height.`);
- }
+function pixelsOf(image, channel, { width, height }) {
if (image?.compressed) throw new TypeError(`GLB ${channel} is a GPU-compressed texture, which export cannot read back. Transcode KTX2 to RGBA8 to export it.`);
if (image.data) {
if (image.data.length < width * height * 4) throw new RangeError(`GLB ${channel} image data must hold RGBA8 pixels for every texel.`);
@@ -63,11 +58,12 @@ export function createTextureWriter(writer, maxTextureSize = DEFAULT_MAX_TEXTURE
const known = imageIndices.get(descriptor.image);
if (known?.has(flipRows)) return known.get(flipRows);
const sourceEncoding = flipRows ? undefined : sourceEncodingOf(descriptor.image);
- const decoded = sourceEncoding ? undefined : pixelsOf(descriptor.image, channel);
- const { width, height } = decoded ?? imageSize(descriptor.image, channel);
+ const size = imageSize(descriptor.image, channel);
+ const { width, height } = size;
if (width > maxTextureSize || height > maxTextureSize) {
throw new RangeError(`GLB ${channel} image is ${width}x${height}, above maxTextureSize ${maxTextureSize}. Resize the image, or pass exportGlb(parts, { maxTextureSize }) up to ${ABSOLUTE_MAX_TEXTURE_SIZE} if the file size is acceptable.`);
}
+ const decoded = sourceEncoding ? undefined : pixelsOf(descriptor.image, channel, size);
const bytes = sourceEncoding?.bytes ?? encodePng(decoded.pixels, width, height, { flipRows });
const mimeType = sourceEncoding?.mimeType ?? 'image/png';
document.images.push({ bufferView: writer.addBufferView(bytes), mimeType });
diff --git a/src/hdri.js b/src/hdri.js
index 2a2ab8c5..ba1664f2 100644
--- a/src/hdri.js
+++ b/src/hdri.js
@@ -95,12 +95,28 @@ function startsRunLengthScanline(bytes, position, width) {
&& ((bytes[position + 2] << 8) | bytes[position + 3]) === width;
}
+// Old-format rows are the smallest: one pixel and at most two repeat pixels (255 and 65280 pixels each), and a
+// run-length row never costs less than that, so this bound holds for both.
+function allocatePixels(availableBytes, width, height) {
+ const repeatPixels = Math.min(2, Math.ceil(Math.log(width) / Math.log(256)));
+ if (availableBytes < height * 4 * (1 + repeatPixels)) {
+ throw new RangeError(`Radiance HDR pixel data is truncated: the header says ${width}x${height} but the file holds ${availableBytes} bytes; pass the whole file.`);
+ }
+ return new Float32Array(width * height * 3);
+}
+
+function requireBytes(buffer) {
+ if (buffer instanceof Uint8Array) return buffer;
+ if (buffer instanceof ArrayBuffer) return new Uint8Array(buffer);
+ throw new TypeError('parseRadianceHdr needs an ArrayBuffer or Uint8Array.');
+}
+
// Decodes one scanline per step, so a caller can run it to the end at once or hand the thread back between rows.
function* decodeRadianceRows(buffer) {
- const bytes = buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
+ const bytes = requireBytes(buffer);
if (bytes.length > MAXIMUM_FILE_BYTES) throw new RangeError('Radiance HDR file is larger than 256 MiB.');
const { width, height, dataStart } = parseHeader(bytes);
- const data = new Float32Array(width * height * 3);
+ let data;
const scanline = new Uint8Array(width * 4);
let position = dataStart;
for (let row = 0; row < height; row++) {
@@ -109,6 +125,7 @@ function* decodeRadianceRows(buffer) {
} else {
position = readFlatScanline(bytes, position, width, scanline);
}
+ data ??= allocatePixels(bytes.length - dataStart, width, height);
for (let column = 0; column < width; column++) {
const exponent = scanline[column * 4 + 3];
const target = (row * width + column) * 3;
@@ -244,6 +261,16 @@ export function sunFromEnvironmentMap(map) {
};
}
+function isPoint(point) {
+ return Array.isArray(point) && point.length === 3 && point.every(Number.isFinite);
+}
+
+function requireBounds(bounds) {
+ if (!isPoint(bounds?.min) || !isPoint(bounds?.max)) {
+ throw new TypeError('keyLightFromHdri bounds must be { min, max } of three finite numbers each.');
+ }
+}
+
/**
* Lights a scene from the HDRI sun. Spread the result into the renderer's `environment` and pass its `shadowBounds`:
* `{ environment: { lightDirection, keyColor, keyIntensity }, shadowBounds }`. The shadow view looks at the middle of
@@ -251,6 +278,7 @@ export function sunFromEnvironmentMap(map) {
* so the caller keeps its defaults.
*/
export function keyLightFromHdri(map, bounds) {
+ requireBounds(bounds);
const sun = sunFromEnvironmentMap(map);
if (!sun) return null;
const target = bounds.min.map((minimum, axis) => (minimum + bounds.max[axis]) / 2);
diff --git a/tests/image-decode-hostile-input.test.js b/tests/image-decode-hostile-input.test.js
new file mode 100644
index 00000000..60d9093f
--- /dev/null
+++ b/tests/image-decode-hostile-input.test.js
@@ -0,0 +1,42 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { createModel } from '../src/model.js';
+import { exportGlb } from '../src/export.js';
+import { keyLightFromHdri, parseRadianceHdr } from '../src/hdri.js';
+
+function hdrHeader(width, height) {
+ return new TextEncoder().encode(`#?RADIANCE\nFORMAT=32-bit_rle_rgbe\n\n-Y ${height} +X ${width}\n`);
+}
+
+test('parseRadianceHdr names what to pass when given something that is not bytes', () => {
+ for (const input of ['#?RADIANCE', null, undefined, {}, 12, [35, 63]]) {
+ assert.throws(() => parseRadianceHdr(input), /parseRadianceHdr needs an ArrayBuffer or Uint8Array/, String(input));
+ }
+});
+
+test('parseRadianceHdr refuses a header that claims more pixels than the file can hold, before allocating them', () => {
+ const oneRow = [128, 64, 0, 129, 1, 1, 1, 255, 1, 1, 1, 63];
+ const file = Uint8Array.from([...hdrHeader(16384, 16384), ...oneRow]);
+ assert.throws(() => parseRadianceHdr(file), /truncated: the header says 16384x16384 but the file holds 12 bytes/);
+ assert.throws(() => parseRadianceHdr(file.buffer), /16384x16384/);
+});
+
+test('parseRadianceHdr still reads the smallest valid file', () => {
+ const file = Uint8Array.from([...hdrHeader(1, 1), 128, 64, 0, 129]);
+ assert.deepEqual(Array.from(parseRadianceHdr(file).data), [1, 0.5, 0]);
+});
+
+test('keyLightFromHdri says what bounds to pass instead of a raw TypeError', () => {
+ const sunMap = { width: 4, height: 4, channels: 3, rotation: 0, intensity: 1, data: new Float32Array(48) };
+ for (const bounds of [undefined, null, {}, { min: [0, 0, 0] }, { min: [0, 0], max: [1, 1, 1] }, { min: [0, 0, 0], max: [1, NaN, 1] }]) {
+ assert.throws(() => keyLightFromHdri(sunMap, bounds), /keyLightFromHdri bounds must be \{ min, max \}/, JSON.stringify(bounds));
+ }
+});
+
+test('GLB export refuses an oversized texture by its size before reading its pixels', () => {
+ const part = createModel()[0];
+ const uvs = new Float32Array(part.geometry.positions.length / 3 * 2);
+ const image = { width: 100000, height: 100000, data: new Uint8Array(4) };
+ const textured = { ...part, geometry: { ...part.geometry, uvs }, material: { ...part.material, map: { image } } };
+ assert.throws(() => exportGlb([textured]), /100000x100000, above maxTextureSize 2048/);
+});