Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Load spec-gloss, UV-less textured and unlit glTF materials instead of throwing (closes #378)

Nguyen Xuan An0ed131e18962
diff --git a/changes/378.md b/changes/378.md
new file mode 100644
index 00000000..6df252f6
--- /dev/null
+++ b/changes/378.md
@@ -0,0 +1 @@
+- Load glTF files that used to throw (closes #378). `KHR_materials_pbrSpecularGlossiness` converts to metallic-roughness (diffuse as base color, metallic 0, roughness 1 - glossiness; the specular color and `specularGlossinessTexture` are not read). A normal map or other texture on a primitive without `TEXCOORD_0` is dropped from that primitive. Both are named in `warnings`. `KHR_materials_unlit` already loaded with the unlit tag; an unlit texture on a primitive without UVs now loads too. The code is in `src/gltf-fallbacks.js`, which only a file that needs it loads.
diff --git a/changes/378.size.md b/changes/378.size.md
new file mode 100644
index 00000000..02311b5d
--- /dev/null
+++ b/changes/378.size.md
@@ -0,0 +1,3 @@
+- entry:gltf+lazy: parseGltf loads the new gltf-fallbacks.js chunk (spec-gloss conversion, textures without UVs) only for a file that needs it, and the lazy-inclusive bundle counts that chunk (about 1.1 KB gzip).
+- scene:gltf-box: the check that decides whether a file needs gltf-fallbacks.js and the dynamic import (about 210 bytes gzip).
+- scene:gltf-box+lazy: the same gltf-fallbacks.js chunk, counted when lazy modules are included (about 1.1 KB gzip).
diff --git a/docs/gltf.md b/docs/gltf.md
index 3ed1272f..297ef50c 100644
--- a/docs/gltf.md
+++ b/docs/gltf.md
@@ -129,6 +129,7 @@ It also supports `KHR_mesh_quantization` and `KHR_materials_emissive_strength`.
Alpha masking, alpha blending and double-sided materials import onto `alphaMode`, `alphaCutoff`, `opacity` and `doubleSided` (see the material maps guide). Vertex alpha is not drawn: an RGBA `COLOR_0` loads with its RGB, and `warnings` names the meshes whose alpha was below one and was dropped.
It supports base-color, normal, metallic-roughness, occlusion and emissive texture channels.
An optional extension that 67js does not read is ignored and named in `warnings`, as the glTF specification allows. An extension listed in `extensionsRequired` that 67js does not read still fails the load with `Unsupported glTF extension`.
+`KHR_materials_pbrSpecularGlossiness` (also when listed in `extensionsRequired`) is converted to metallic-roughness: the diffuse color and texture become the base color, metallic is 0, roughness is 1 minus `glossinessFactor`. The specular color and `specularGlossinessTexture` are not read, and `warnings` says so. A texture on a primitive without `TEXCOORD_0` is dropped from that primitive with a warning instead of failing the load.
POINTS, LINES, LINE_LOOP and LINE_STRIP primitives are skipped with a warning that names the mesh and their count. Drawing them is not supported. TRIANGLE_STRIP and TRIANGLE_FAN primitives are still refused.
## Extensions
diff --git a/docs/lighting.md b/docs/lighting.md
index eaec7d7d..d943070d 100644
--- a/docs/lighting.md
+++ b/docs/lighting.md
@@ -27,6 +27,8 @@ const renderer = new Renderer67(canvas, parts, {
renderer.render({ eye: [0, 2, 5], target: [0, 1, 0], fov: 40 });
```
+With `world: true` the default `environment.fogDensity` is 0.006 per unit, which hazes a model 100 to 600 units wide into milky haze. Set a lower `fogDensity` for a large scene.
+
## Light controls
`keyIntensity` scales the existing shadowed directional light. `keyColor` optionally replaces its linear RGB radiance color. `fillIntensity` scales the two existing fill/rim lights. `ambientIntensity` scales procedural reflection and hemisphere fill. Each intensity defaults to one, preserving previous lighting when no controls are supplied. Set all three to zero to isolate local lights on standard PBR surfaces.
diff --git a/src/gltf-fallbacks.js b/src/gltf-fallbacks.js
new file mode 100644
index 00000000..08478bc6
--- /dev/null
+++ b/src/gltf-fallbacks.js
@@ -0,0 +1,79 @@
+// Rewrites the parts of a glTF document that common Sketchfab files get wrong, so that they load instead of throwing.
+// parseGltf imports this module only when a file needs it, so it adds nothing to a scene that loads a clean file.
+
+const SPECULAR_GLOSSINESS = 'KHR_materials_pbrSpecularGlossiness';
+
+/** Paths (as arrays of keys) to every texture reference of a material that reads TEXCOORD_0. */
+function collectFirstSetTextureKeys(value, path = [], found = []) {
+ if (!value || typeof value !== 'object') return found;
+ for (const [key, child] of Object.entries(value)) {
+ if (!child || typeof child !== 'object') continue;
+ const isTextureReference = Number.isInteger(child.index) && (child.texCoord ?? 0) === 0;
+ if (isTextureReference) found.push([...path, key]);
+ else collectFirstSetTextureKeys(child, [...path, key], found);
+ }
+ return found;
+}
+
+/**
+ * KHR_materials_pbrSpecularGlossiness to metallic-roughness, the way Blender's importer approximates it: the diffuse
+ * color becomes the base color, the surface is dielectric, and roughness is 1 minus the glossiness factor. The specular
+ * color and the specularGlossinessTexture are dropped, because reading them needs a pass over the image pixels.
+ */
+function convertSpecularGlossiness(document, warnings) {
+ const converted = [];
+ document.materials?.forEach((material, index) => {
+ const specularGlossiness = material?.extensions?.[SPECULAR_GLOSSINESS];
+ if (!specularGlossiness) return;
+ const pbr = { metallicFactor: 0, roughnessFactor: 1 - (specularGlossiness.glossinessFactor ?? 1) };
+ if (specularGlossiness.diffuseFactor) pbr.baseColorFactor = specularGlossiness.diffuseFactor;
+ if (specularGlossiness.diffuseTexture) pbr.baseColorTexture = specularGlossiness.diffuseTexture;
+ material.pbrMetallicRoughness = pbr;
+ delete material.extensions[SPECULAR_GLOSSINESS];
+ if (!Object.keys(material.extensions).length) delete material.extensions;
+ const dropped = specularGlossiness.specularGlossinessTexture ? ' and its specularGlossinessTexture' : '';
+ converted.push(`${material.name ?? `material ${index}`}${dropped}`);
+ });
+ if (!converted.length) return;
+ document.extensionsUsed = document.extensionsUsed?.filter(name => name !== SPECULAR_GLOSSINESS);
+ document.extensionsRequired = document.extensionsRequired?.filter(name => name !== SPECULAR_GLOSSINESS);
+ warnings.push(`${SPECULAR_GLOSSINESS} converted to metallic-roughness (diffuse color as base color, metallic 0, roughness 1 - glossiness, specular color not read) in ${converted.join(', ')}.`);
+}
+
+/**
+ * A primitive without TEXCOORD_0 cannot sample a texture on the first UV set. Such a primitive gets its own copy of the
+ * material without those textures, and the original stays for the primitives that do have UVs.
+ */
+function dropTexturesWithoutUvs(document, warnings) {
+ const copiesByMaterial = new Map();
+ const dropped = new Set();
+ for (const mesh of document.meshes ?? []) {
+ for (const primitive of mesh.primitives ?? []) {
+ const material = document.materials?.[primitive.material];
+ if (primitive.attributes?.TEXCOORD_0 !== undefined || !material) continue;
+ const paths = collectFirstSetTextureKeys(material);
+ if (!paths.length) continue;
+ if (!copiesByMaterial.has(primitive.material)) {
+ const copy = structuredClone(material);
+ for (const path of paths) {
+ const parent = path.slice(0, -1).reduce((node, key) => node[key], copy);
+ delete parent[path.at(-1)];
+ dropped.add(path.at(-1));
+ }
+ document.materials.push(copy);
+ copiesByMaterial.set(primitive.material, document.materials.length - 1);
+ }
+ primitive.material = copiesByMaterial.get(primitive.material);
+ }
+ }
+ if (!dropped.size) return;
+ warnings.push(`Dropped ${[...dropped].sort().join(', ')} from primitives without TEXCOORD_0, which cannot sample a texture.`);
+}
+
+/** Rewrites the document in place and returns the warnings that tell the caller what changed. */
+export function applyGltfFallbacks(document) {
+ const warnings = [];
+ convertSpecularGlossiness(document, warnings);
+ dropTexturesWithoutUvs(document, warnings);
+ return warnings;
+}
diff --git a/src/gltf.js b/src/gltf.js
index 1d082141..ecf04b3f 100644
--- a/src/gltf.js
+++ b/src/gltf.js
@@ -30,6 +30,21 @@ async function loadLayerShadersFor(document) {
if (usesLiquidShader) await import('./liquid-shader.js');
}
+// Spec-gloss materials and textures on primitives without UVs are rewritten by a module that only a file needing it loads.
+function needsFallbacks(document) {
+ const materials = document.materials ?? [];
+ if (materials.some(material => material?.extensions?.KHR_materials_pbrSpecularGlossiness)) return true;
+ return (document.meshes ?? []).some(mesh => (mesh.primitives ?? []).some(primitive => {
+ return primitive.attributes?.TEXCOORD_0 === undefined && JSON.stringify(materials[primitive.material] ?? {}).includes('"index"');
+ }));
+}
+
+async function applyFallbacksTo(document) {
+ if (!needsFallbacks(document)) return [];
+ const { applyGltfFallbacks } = await import('./gltf-fallbacks.js');
+ return applyGltfFallbacks(document);
+}
+
const DEFAULT_MAX_BYTES = 32 * 1024 * 1024;
const DEFAULT_MAX_VERTICES = 2_000_000;
const GLTF_ALPHA_MODES = { OPAQUE: 'opaque', MASK: 'mask', BLEND: 'blend' };
@@ -968,10 +983,11 @@ function convertMaterial(source, limits, maps, definition = {}) {
export async function parseGltf(data, options = {}) {
const limits = limitsFor(options);
const document = documentFrom(data, limits.maxBytes);
+ const fallbackWarnings = await applyFallbacksTo(document);
const decoders = options.decoders ?? {};
const { ignoredExtensions } = validateDocument(document, limits, allowedExtensionsFor(decoders));
await loadLayerShadersFor(document);
- const warnings = [...ignoredExtensions.map(extension => `${extension} is accepted but not rendered.`), ...skippedPrimitiveWarnings(document)];
+ const warnings = [...fallbackWarnings, ...ignoredExtensions.map(extension => `${extension} is accepted but not rendered.`), ...skippedPrimitiveWarnings(document)];
const dataBytes = typeof data === 'string' ? new TextEncoder().encode(data).byteLength : data.byteLength;
const { resourceBytes, textureScaling } = await prepareResources(document, data, dataBytes, options, limits);
if (textureScaling.scaledTextures.length) {
diff --git a/tests/gltf-sketchfab-fallbacks.test.js b/tests/gltf-sketchfab-fallbacks.test.js
new file mode 100644
index 00000000..c72a517e
--- /dev/null
+++ b/tests/gltf-sketchfab-fallbacks.test.js
@@ -0,0 +1,100 @@
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { parseGltf } from '../src/gltf.js';
+
+const PNG_IMAGE = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAYAAABytg0kAAAAEklEQVR4AWI6k2b8H4SZGKAAAAAA//8VS4tGAAAABklEQVQDAEBuBM3uhq5wAAAAAElFTkSuQmCC';
+const TRIANGLE_POSITIONS = new Float32Array([0, 0, 0, 1, 0, 0, 0, 1, 0]);
+const TRIANGLE_UVS = new Float32Array([0, 0, 1, 0, 0, 1]);
+
+function dataUri(...arrays) {
+ const bytes = Buffer.concat(arrays.map(array => Buffer.from(array.buffer)));
+ return `data:application/octet-stream;base64,${bytes.toString('base64')}`;
+}
+
+/** Stands in for the browser image decoder, which Node does not have. */
+async function withImageDecoder(run) {
+ const previousBitmap = globalThis.createImageBitmap;
+ const previousSelf = globalThis.self;
+ globalThis.self = globalThis;
+ globalThis.createImageBitmap = async () => ({ width: 2, height: 2, close() {} });
+ try {
+ return await run();
+ } finally {
+ if (previousBitmap) globalThis.createImageBitmap = previousBitmap;
+ else delete globalThis.createImageBitmap;
+ if (previousSelf) globalThis.self = previousSelf;
+ else delete globalThis.self;
+ }
+}
+
+/** One triangle with one material and one texture, with or without TEXCOORD_0. */
+function triangleDocument(material, { withUvs, extensionsRequired } = {}) {
+ const attributes = { POSITION: 0 };
+ const accessors = [{ bufferView: 0, componentType: 5126, count: 3, type: 'VEC3', min: [0, 0, 0], max: [1, 1, 0] }];
+ const bufferViews = [{ buffer: 0, byteOffset: 0, byteLength: 36 }];
+ if (withUvs) {
+ attributes.TEXCOORD_0 = 1;
+ accessors.push({ bufferView: 1, componentType: 5126, count: 3, type: 'VEC2' });
+ bufferViews.push({ buffer: 0, byteOffset: 36, byteLength: 24 });
+ }
+ const byteLength = withUvs ? 60 : 36;
+ return JSON.stringify({
+ asset: { version: '2.0' },
+ ...(extensionsRequired ? { extensionsUsed: extensionsRequired, extensionsRequired } : {}),
+ scene: 0,
+ scenes: [{ nodes: [0] }],
+ nodes: [{ mesh: 0 }],
+ meshes: [{ primitives: [{ attributes, material: 0 }] }],
+ materials: [material],
+ accessors,
+ bufferViews,
+ buffers: [{ byteLength, uri: withUvs ? dataUri(TRIANGLE_POSITIONS, TRIANGLE_UVS) : dataUri(TRIANGLE_POSITIONS) }],
+ textures: [{ source: 0 }],
+ images: [{ uri: PNG_IMAGE }],
+ });
+}
+
+test('a required KHR_materials_pbrSpecularGlossiness material loads as metallic-roughness', async () => {
+ const extension = { diffuseFactor: [0.5, 0.25, 0.125, 1], glossinessFactor: 0.25, diffuseTexture: { index: 0 }, specularFactor: [1, 1, 1] };
+ const document = triangleDocument({ name: 'old paint', extensions: { KHR_materials_pbrSpecularGlossiness: extension } }, { withUvs: true, extensionsRequired: ['KHR_materials_pbrSpecularGlossiness'] });
+ const { parts, warnings } = await withImageDecoder(() => parseGltf(document));
+ const material = parts[0].material;
+ assert.deepEqual(material.color.map(value => Number(value.toFixed(3))), [0.5, 0.25, 0.125]);
+ assert.equal(material.metallic, 0);
+ assert.equal(material.roughness, 0.75);
+ assert.ok(material.map, 'the diffuse texture becomes the base color map');
+ assert.ok(warnings.some(warning => /KHR_materials_pbrSpecularGlossiness converted to metallic-roughness.*old paint/.test(warning)));
+});
+
+test('a KHR_materials_unlit material with a base color map loads with the unlit tag and no lit maps', async () => {
+ const document = triangleDocument({
+ pbrMetallicRoughness: { baseColorFactor: [1, 0.5, 0, 1], baseColorTexture: { index: 0 } },
+ normalTexture: { index: 0 },
+ extensions: { KHR_materials_unlit: {} },
+ }, { withUvs: true, extensionsRequired: ['KHR_materials_unlit'] });
+ const { parts, warnings } = await withImageDecoder(() => parseGltf(document));
+ const material = parts[0].material;
+ assert.equal(material.tag, 'unlit');
+ assert.deepEqual(material.color, [1, 0.5, 0]);
+ assert.equal(material.emission, 0);
+ assert.ok(material.map);
+ assert.equal(material.normalMap, undefined);
+ assert.deepEqual(warnings, []);
+});
+
+test('a normal map on a primitive without TEXCOORD_0 is dropped with a warning', async () => {
+ const document = triangleDocument({ pbrMetallicRoughness: { baseColorFactor: [0.2, 0.4, 0.6, 1] }, normalTexture: { index: 0 } }, { withUvs: false });
+ const { parts, warnings } = await withImageDecoder(() => parseGltf(document));
+ const material = parts[0].material;
+ assert.equal(material.normalMap, undefined);
+ assert.deepEqual(material.color.map(value => Number(value.toFixed(3))), [0.2, 0.4, 0.6]);
+ assert.ok(warnings.some(warning => /Dropped normalTexture from primitives without TEXCOORD_0/.test(warning)));
+});
+
+test('an unlit material with a base color map and no UVs loads without the map', async () => {
+ const document = triangleDocument({ pbrMetallicRoughness: { baseColorTexture: { index: 0 } }, extensions: { KHR_materials_unlit: {} } }, { withUvs: false });
+ const { parts, warnings } = await withImageDecoder(() => parseGltf(document));
+ assert.equal(parts[0].material.tag, 'unlit');
+ assert.equal(parts[0].material.map, undefined);
+ assert.ok(warnings.some(warning => /Dropped baseColorTexture/.test(warning)));
+});
diff --git a/tests/gltf.test.js b/tests/gltf.test.js
index b967fda8..27bd6839 100644
--- a/tests/gltf.test.js
+++ b/tests/gltf.test.js
@@ -461,7 +461,6 @@ test('map preflight rejects unsupported UVs, sampler states, invalid scales and
[document => { document.samplers[0].minFilter = 4; }, /filtering/],
[document => { document.textures[0].source = 4; }, /image index/],
[document => { document.textures[0].sampler = 4; }, /sampler index/],
- [document => { delete document.meshes[0].primitives[0].attributes.TEXCOORD_0; }, /TEXCOORD_0/],
[document => { document.materials[0].emissiveTexture.extensions = { KHR_texture_transform: { offset: [0.2, 0] } }; }, /KHR_texture_transform/],
];
for (const [change, expected] of changes) {