3D engine that AI agents drive
JavaScript 3d3d-engineagentic-aiai-agentsblenderclaude-codegame-enginegaussian-splattinggltfjavascriptmcpmcp-servermodel-context-protocolpath-tracerpath-tracingpbrrenderingthreejs-alternativewebgl2webgpu
Read padded glTF matrix columns and name paths in loader errors (#2462, #2464)
Nguyen Xuan And597d96c2580
diff --git a/changes/2462.md b/changes/2462.mdnew file mode 100644index 00000000..dd0f32b8--- /dev/null+++ b/changes/2462.md@@ -0,0 +1 @@+- glTF MAT2 and MAT3 accessors of byte or short components now load the values the file stores instead of shifted ones (#2462)diff --git a/changes/2462.size.md b/changes/2462.size.mdnew file mode 100644index 00000000..6d603648--- /dev/null+++ b/changes/2462.size.md@@ -0,0 +1,3 @@+- scene:gltf-box: valid MAT2 and MAT3 files load shifted values without the column padding code (+231 bytes gzip for it and three path-naming errors)+- scene:gltf-box+lazy: same code as scene:gltf-box+- goal:gltf-box: a wrong value from a valid file is a correctness bug, and the added code is about 230 bytes gzipdiff --git a/changes/2464.md b/changes/2464.mdnew file mode 100644index 00000000..20f9f2c4--- /dev/null+++ b/changes/2464.md@@ -0,0 +1 @@+- glTF errors for a node mesh, an accessor count and a node child name their path, and a string POSITION index no longer loads (#2464)diff --git a/docs/gltf.md b/docs/gltf.mdindex e1e8dc07..4f810e5e 100644--- a/docs/gltf.md+++ b/docs/gltf.md@@ -55,7 +55,7 @@ The importer rejects cyclic hierarchies, multiple parents, and hierarchies deepeIt checks PNG, JPEG and WebP dimensions before browser image decoding.Decoded accessor data has its own budget, `maxDecodedBytes`, and a file over it fails before any accessor is read: `decoded accessor bytes 1200000000, over maxDecodedBytes 1073741824. Fix: pass { maxDecodedBytes: 1200000000 }`.-A file also fails when its meshopt-compressed buffer views would decode to more than `maxDecodedBytes` in total, even a view no accessor uses; the views and the accessors then share that one budget. A meshopt view whose `byteLength`, extension `byteLength`, `count` or `byteStride` is not a non-negative safe integer fails by field name. The check names the total, for example `decoded meshopt view bytes 4800000000, over maxDecodedBytes 1073741824`, and is raised before the decoder runs; when the accessors then push the shared total over, the error reads `decoded accessor and meshopt view bytes`. A scene `nodes` that is not an array fails by field name too. So does any of `nodes`, `meshes`, `accessors`, `bufferViews`, `buffers`, `materials`, `textures`, `images`, `skins` or `animations` that is neither absent, `null` nor an array (`Invalid glTF meshes: expected an array of objects.`), or that holds a `null` or non-object entry (the error names the array and the index of the bad entry); a `null` array is treated as absent. A dangling index (`primitive.material`, a scene node, an accessor `bufferView`, a mesh without `primitives`) fails with the field and the valid range. A buffer view `byteStride` below the element size or not a multiple of 4, an accessor or sparse `byteOffset` that is negative or fractional, and a sparse index type other than 5121, 5123 or 5125, fail by name. A `normalized` flag on a float or 32-bit integer accessor is ignored.+A file also fails when its meshopt-compressed buffer views would decode to more than `maxDecodedBytes` in total, even a view no accessor uses; the views and the accessors then share that one budget. A meshopt view whose `byteLength`, extension `byteLength`, `count` or `byteStride` is not a non-negative safe integer fails by field name. The check names the total, for example `decoded meshopt view bytes 4800000000, over maxDecodedBytes 1073741824`, and is raised before the decoder runs; when the accessors then push the shared total over, the error reads `decoded accessor and meshopt view bytes`. A scene `nodes` that is not an array fails by field name too. So does any of `nodes`, `meshes`, `accessors`, `bufferViews`, `buffers`, `materials`, `textures`, `images`, `skins` or `animations` that is neither absent, `null` nor an array (`Invalid glTF meshes: expected an array of objects.`), or that holds a `null` or non-object entry (the error names the array and the index of the bad entry); a `null` array is treated as absent. A dangling index (`primitive.material`, a scene node, an accessor `bufferView`, a mesh without `primitives`) fails with the field and the valid range. A buffer view `byteStride` below the element size or not a multiple of 4, an accessor or sparse `byteOffset` that is negative or fractional, and a sparse index type other than 5121, 5123 or 5125, fail by name. A `normalized` flag on a float or 32-bit integer accessor is ignored. A MAT2 or MAT3 accessor of byte or short components is read with the padding the glTF spec gives each column (to a multiple of 4 bytes). A node `mesh`, a child of `nodes[i].children`, an accessor `count` or a `POSITION` that is not an integer index fails or skips with its path.Accessors are decoded one primitive at a time while the scene is converted, and each is released after the last node that uses it, so decoding holds the file bytes plus the converted geometry plus one primitive, not every accessor at once. The file bytes stay until the last primitive is read; drop your own reference to them after the call returns.These limits bound asset inputs and geometry, rather than the renderer's total GPU allocation.The renderer retains its separate memory budget.diff --git a/src/gltf-accessors.js b/src/gltf-accessors.jsindex 572c3848..ef20c3bf 100644--- a/src/gltf-accessors.js+++ b/src/gltf-accessors.js@@ -41,10 +41,31 @@ export async function attributeFloatsInSlices(attribute, checkpoint) {return values;}+const MATRIX_COLUMNS = { MAT2: 2, MAT3: 3, MAT4: 4 };++/** Matrix columns as [tight bytes, stored bytes]: the glTF spec pads each MAT2 or MAT3 column to a multiple of 4 bytes. */+function columnBytesOf(accessor) {+ const tight = MATRIX_COLUMNS[accessor.type] * COMPONENT_TYPES[accessor.componentType].BYTES_PER_ELEMENT;+ return [tight, Math.ceil(tight / 4) * 4];+}+function elementBytesOf(accessor) {+ const columns = MATRIX_COLUMNS[accessor.type];+ if (columns) return columns * columnBytesOf(accessor)[1];return COMPONENT_TYPES[accessor.componentType].BYTES_PER_ELEMENT * COMPONENT_COUNTS[accessor.type];}+function removeColumnPadding(packed, accessor) {+ const columns = MATRIX_COLUMNS[accessor.type];+ const [tightBytes, storedBytes] = columnBytesOf(accessor);+ if (!columns || tightBytes === storedBytes) return packed;+ const tight = new Uint8Array(accessor.count * columns * tightBytes);+ for (let column = 0; column < accessor.count * columns; column++) {+ tight.set(packed.subarray(column * storedBytes, column * storedBytes + tightBytes), column * tightBytes);+ }+ return tight;+}+/** The distance between elements of a buffer view: its byteStride, which must hold one element and be a multiple of 4. */function strideOf(view, elementBytes, viewIndex) {if (!view.byteStride) return elementBytes;@@ -167,7 +188,7 @@ export function createAccessorReader(document, bufferBytes, meshoptDecoder) {}function finishAccessor({ accessor, ArrayType, itemSize }, packed) {- const array = new ArrayType(packed.buffer);+ const array = new ArrayType(removeColumnPadding(packed, accessor).buffer);if (accessor.sparse) applySparse(accessor, array, itemSize);return { array, itemSize, count: accessor.count, normalized: accessor.normalized === true, componentType: accessor.componentType };}diff --git a/src/gltf.js b/src/gltf.jsindex 5207e82c..cfa6b7c7 100644--- a/src/gltf.js+++ b/src/gltf.js@@ -434,9 +434,11 @@ function validateDocument(document, limits, allowedExtensions) {overLimit(overruns, 'skin joints', jointCount, 'maxJoints', limits.maxJoints);let poseBytes = 0;let declaredParts = 0;- for (const node of document.nodes ?? []) {+ for (const [nodeIndex, node] of (document.nodes ?? []).entries()) {const mesh = document.meshes?.[node.mesh];- if (node.mesh !== undefined && (!Number.isInteger(node.mesh) || !mesh)) throw new Error('Invalid mesh node index.');+ if (node.mesh !== undefined && (!Number.isInteger(node.mesh) || !mesh)) {+ throw new Error(`Invalid glTF nodes[${nodeIndex}].mesh ${JSON.stringify(node.mesh)}: pass a mesh index from 0 to ${(document.meshes?.length ?? 1) - 1}.`);+ }declaredParts += (mesh?.primitives?.length ?? 0) * nodeInstanceCount(document, node, limits, overruns);validateDefaultWeights(node.weights, morphCount(mesh));morphBytes += morphCount(mesh) * (mesh?.primitives?.length ?? 0) * 16;@@ -499,8 +501,10 @@ function validateDocument(document, limits, allowedExtensions) {// Each skinned primitive keeps its own pose copy, so the pose memory shares this budget. maxJoints counts a skin once above.let decodedBytes = morphBytes + meshoptViewBytes + poseBytes;const components = { SCALAR: 1, VEC2: 2, VEC3: 3, VEC4: 4, MAT2: 4, MAT3: 9, MAT4: 16 };- for (const accessor of document.accessors ?? []) {- if (!Number.isSafeInteger(accessor.count) || accessor.count < 0) throw new Error('Invalid glTF accessor count.');+ for (const [accessorIndex, accessor] of (document.accessors ?? []).entries()) {+ if (!Number.isSafeInteger(accessor.count) || accessor.count < 0) {+ throw new Error(`Invalid glTF accessors[${accessorIndex}].count ${JSON.stringify(accessor.count)}: pass a non-negative integer.`);+ }if (accessor.count > limits.maxVertices * 12) {throwOverruns(overLimit(overruns, 'accessor elements', accessor.count, 'maxVertices', limits.maxVertices, 12));}@@ -572,7 +576,7 @@ function primitiveLabel(document, meshIndex, primitiveIndex) {/** Why a triangle primitive cannot be drawn judging by its accessors alone, or null. Index values and areas need the decoded data. */function accessorProblemOf(document, primitive) {const position = document.accessors?.[primitive.attributes?.POSITION];- if (!position || position.type !== 'VEC3' || !Number.isSafeInteger(position.count) || position.count < 3) {+ if (!Number.isInteger(primitive.attributes?.POSITION) || !position || position.type !== 'VEC3' || !Number.isSafeInteger(position.count) || position.count < 3) {return 'its POSITION accessor is missing, is not VEC3 or has fewer than 3 vertices';}const index = primitive.indices === undefined ? null : document.accessors?.[primitive.indices];@@ -601,9 +605,11 @@ function skipUnusablePrimitives(document) {function validateHierarchy(nodes) {const parents = new Set();- for (const node of nodes) {- for (const child of node.children ?? []) {- if (!Number.isInteger(child) || !nodes[child] || parents.has(child)) throw new Error('Invalid glTF hierarchy child or multiple parents.');+ for (const [nodeIndex, node] of nodes.entries()) {+ for (const [childSlot, child] of (node.children ?? []).entries()) {+ const path = `nodes[${nodeIndex}].children[${childSlot}]`;+ if (!Number.isInteger(child) || !nodes[child]) throw new Error(`Invalid glTF ${path} ${JSON.stringify(child)}: pass an existing node index.`);+ if (parents.has(child)) throw new Error(`Invalid glTF ${path}: node ${child} has two parents; list it once.`);parents.add(child);}}diff --git a/tests/gltf-error-paths.test.js b/tests/gltf-error-paths.test.jsnew file mode 100644index 00000000..83e1214f--- /dev/null+++ b/tests/gltf-error-paths.test.js@@ -0,0 +1,50 @@+import assert from 'node:assert/strict';+import test from 'node:test';+import { parseGltf } from '../src/gltf.js';++function triangleDocument(edit = () => {}) {+ const bytes = new Uint8Array(new Float32Array([0, 0, 0, 1, 0, 0, 0, 1, 0]).buffer);+ const document = {+ asset: { version: '2.0' },+ scene: 0,+ scenes: [{ nodes: [0] }],+ nodes: [{ mesh: 0 }],+ meshes: [{ primitives: [{ attributes: { POSITION: 0 } }] }],+ accessors: [{ bufferView: 0, componentType: 5126, count: 3, type: 'VEC3', min: [0, 0, 0], max: [1, 1, 0] }],+ bufferViews: [{ buffer: 0, byteLength: bytes.length }],+ buffers: [{ byteLength: bytes.length, uri: `data:application/octet-stream;base64,${Buffer.from(bytes).toString('base64')}` }],+ };+ edit(document);+ return JSON.stringify(document);+}++test('a node with a mesh index that does not exist names nodes[i].mesh', async () => {+ const source = triangleDocument(document => { document.nodes.push({ mesh: 5 }); });+ await assert.rejects(() => parseGltf(source), /nodes\[1\]\.mesh 5/);+});++test('an accessor with a negative count names accessors[i].count', async () => {+ const source = triangleDocument(document => {+ document.accessors.push({ componentType: 5126, count: -2, type: 'SCALAR' });+ });+ await assert.rejects(() => parseGltf(source), /accessors\[1\]\.count -2/);+});++test('a child index that does not exist names nodes[i].children[j]', async () => {+ const source = triangleDocument(document => { document.nodes[0].children = [9]; });+ await assert.rejects(() => parseGltf(source), /nodes\[0\]\.children\[0\] 9/);+});++test('a node listed as a child twice names the second listing and the node', async () => {+ const source = triangleDocument(document => {+ document.nodes.push({}, {});+ document.nodes[0].children = [1];+ document.nodes[2].children = [1];+ });+ await assert.rejects(() => parseGltf(source), /nodes\[2\]\.children\[0\].*node 1 has two parents/);+});++test('a POSITION given as a string index is skipped with a POSITION warning instead of loading', async () => {+ const source = triangleDocument(document => { document.meshes[0].primitives[0].attributes.POSITION = '0'; });+ await assert.rejects(() => parseGltf(source), /Skipped primitive 0 of mesh 0: its POSITION accessor is missing/);+});diff --git a/tests/gltf-matrix-column-padding.test.js b/tests/gltf-matrix-column-padding.test.jsnew file mode 100644index 00000000..c128f81a--- /dev/null+++ b/tests/gltf-matrix-column-padding.test.js@@ -0,0 +1,45 @@+import assert from 'node:assert/strict';+import test from 'node:test';+import { createAccessorReader } from '../src/gltf-accessors.js';++const BYTE = 5121;+const SHORT = 5123;++function readMatrices({ type, componentType, bytes, count }) {+ const document = {+ bufferViews: [{ buffer: 0, byteLength: bytes.length }],+ accessors: [{ bufferView: 0, componentType, type, count }],+ };+ return createAccessorReader(document, () => bytes).readAccessor(0);+}++test('MAT2 of unsigned bytes pads each column to 4 bytes and reads the two matrices unpadded', () => {+ const bytes = Uint8Array.from([1, 2, 0, 0, 3, 4, 0, 0, 5, 6, 0, 0, 7, 8, 0, 0]);+ const attribute = readMatrices({ type: 'MAT2', componentType: BYTE, bytes, count: 2 });+ assert.deepEqual([...attribute.array], [1, 2, 3, 4, 5, 6, 7, 8]);+});++test('MAT3 of unsigned bytes pads each column to 4 bytes', () => {+ const bytes = Uint8Array.from([1, 2, 3, 0, 4, 5, 6, 0, 7, 8, 9, 0]);+ const attribute = readMatrices({ type: 'MAT3', componentType: BYTE, bytes, count: 1 });+ assert.deepEqual([...attribute.array], [1, 2, 3, 4, 5, 6, 7, 8, 9]);+});++test('MAT3 of unsigned shorts pads each 6 byte column to 8 bytes and reads two matrices', () => {+ const padded = new Uint16Array([1, 2, 3, 0, 4, 5, 6, 0, 7, 8, 9, 0, 11, 12, 13, 0, 14, 15, 16, 0, 17, 18, 19, 0]);+ const bytes = new Uint8Array(padded.buffer);+ const attribute = readMatrices({ type: 'MAT3', componentType: SHORT, bytes, count: 2 });+ assert.deepEqual([...attribute.array], [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19]);+});++test('MAT2 of shorts and MAT4 of bytes already fill their columns and read as stored', () => {+ const shorts = new Uint8Array(new Uint16Array([1, 2, 3, 4]).buffer);+ assert.deepEqual([...readMatrices({ type: 'MAT2', componentType: SHORT, bytes: shorts, count: 1 }).array], [1, 2, 3, 4]);+ const bytes = Uint8Array.from({ length: 16 }, (_, index) => index + 1);+ assert.deepEqual([...readMatrices({ type: 'MAT4', componentType: BYTE, bytes, count: 1 }).array], [...bytes]);+});++test('a padded MAT3 accessor that needs more bytes than its view holds fails by name', () => {+ const unpaddedBytes = Uint8Array.from([1, 2, 3, 4, 5, 6, 7, 8, 9]);+ assert.throws(() => readMatrices({ type: 'MAT3', componentType: BYTE, bytes: unpaddedBytes, count: 1 }), /exceeds its buffer view/);+});