Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(import): separate metalness, roughness and occlusion maps are packed the way glTF packs them

yhh070d17610ce3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7edc70c10..9f2e70d81 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,6 +18,7 @@ published separately; it ships inside the editor.
- **Contact shadows and screen-space bounce light.** Contact Shadows and Screen Space GI effects add small sharp shadows at contacts and one bounce of colour between surfaces, and the weather softens the sun's shadows. `contactShadows` marches a short ray toward the scene's sun in the depth buffer; `ssgi` gathers eight rays a pixel over the G-buffer's normals at half size into a float target and averages the neighbours on the same surface before laying the light, times the receiver's colour, over the picture. The Weather component raises the sun's `sourceAngle` and lowers its `shadowStrength` as the sky clouds over.
- **Terrain and Foliage.** A Terrain component makes ground from a heightmap or noise, paints it with grass, dirt, snow and rock layers and draws it in chunks that coarsen with distance; a Foliage component scatters grass or a mesh over it, swaying in the wind and thinning out with distance.
+- **Separate material maps packed on import.** Importing an FBX with separate metalness, roughness and occlusion images now packs them into the one map the Model shader samples, instead of dropping them. Occlusion goes in red, roughness in green and metal in blue, each from its image's red channel (a glossiness map is inverted); a channel with no image stays at full so the material's own value speaks. Only PNG and DDS can be read.
- **Skin and hair.** Two new shaders for characters: Skin softens diffuse light by colour, warms the terminator and lets light through thin places without blurring the picture, and Hair lights strands along a flow map with two shifted highlights. Normal maps on stretched UVs (a face unwrapped into an atlas) now keep their shape, and mipmapped textures are sampled with anisotropic filtering where the device has it. Both read a normal map and an occlusion/roughness map as data textures (import them with sRGB off); the Skin shader shades each pixel with three lighting passes, so it is for characters in view. Hair is shaded as strands but a hair model is still cards or ribbons.
- **Anisotropic filtering.** Mipmapped textures are now sampled with up to 16x anisotropic filtering where the device has it (WebGL2 with the extension, WebGPU always), so a floor, a hair card or a sleeve seen at a slant keeps its detail instead of washing out to grey.
- **Wilds example.** Wilds is a new 3D example: a lake under mountains at dusk, made from a generated heightmap with the Terrain, Water and Foliage features and the sky's cloud shadows. Everything in it is made from numbers by `examples/wilds/tools/build.mjs`, so there is no art to credit.
diff --git a/docs/astro/src/content/docs/assets/models.mdx b/docs/astro/src/content/docs/assets/models.mdx
index b316bb974..2e20fa69d 100644
--- a/docs/astro/src/content/docs/assets/models.mdx
+++ b/docs/astro/src/content/docs/assets/models.mdx
@@ -272,11 +272,14 @@ exponent, and it reports **no** metalness rather than a made-up zero-metal surfa
Two things about FBX materials are worth knowing before you author for it:
-- **Metalness and roughness maps are only carried when they are one image**, packed the way
- glTF packs them (roughness in green, metal in blue) — which is what the engine's Model
- shader samples. Exporters normally write two separate images; when they do, the import
- says so and the constants are used. Pack them into one image, or set the map by hand on
- the `.esmaterial`.
+- **Separate metalness, roughness and occlusion maps are packed for you.** The engine's Model
+ shader samples one image packed the way glTF packs it (occlusion in red, roughness in
+ green, metal in blue). Exporters normally write a separate image for each, so the import
+ reads them (the value is the red channel; a glossiness map is inverted), packs them into
+ one `<material>_orm.png` beside the model, and binds it to both slots. A channel with no
+ map stays at full, so the material's own metallic or roughness value still says it, and
+ maps of different sizes are lined up on the largest. Only PNG and DDS can be read there:
+ with another format the import says so and uses the constants.
- **An opacity map is not carried.** Alpha comes from the base colour image's own channel.
## Compressed geometry (glTF)
diff --git a/docs/astro/src/content/docs/zh-cn/assets/models.mdx b/docs/astro/src/content/docs/zh-cn/assets/models.mdx
index 7692ea8b6..b6f810494 100644
--- a/docs/astro/src/content/docs/zh-cn/assets/models.mdx
+++ b/docs/astro/src/content/docs/zh-cn/assets/models.mdx
@@ -240,9 +240,12 @@ Physical、Blender 的 Principled 最终都落到[材质](#材质)那一节列
为 FBX 创作之前,有两件关于它材质的事值得先知道:
-- **金属度与粗糙度贴图只有在是同一张图时才带得过来**,且要按 glTF 的方式打包(粗糙度在绿
- 通道、金属度在蓝通道)——引擎的 Model shader 采样的就是这一张。导出器通常写成两张独立的
- 图;那种情况导入会说出来,并改用常量。要么把它们打包成一张,要么在 `.esmaterial` 上手工指定。
+- **分开的金属度、粗糙度和遮蔽贴图会替你打包。** 引擎的 Model shader 采样的是一张按 glTF
+ 方式打包的图(遮蔽在红通道、粗糙度在绿通道、金属度在蓝通道)。导出器通常给每个量写一张独立的图,
+ 所以导入会把它们读进来(取红通道的值;光泽度贴图会被反相),打包成模型旁边的一张
+ `<材质名>_orm.png`,并绑到两个槽上。没有贴图的通道保持为满值,所以材质自己的金属度或粗糙度
+ 值仍然生效;尺寸不同的贴图会按最大的那张对齐。这里只能读 PNG 和 DDS:别的格式导入会说出来,
+ 并改用常量。
- **不透明度贴图不会带过来。** alpha 来自 base color 图自己的通道。
## 压缩几何(glTF)
diff --git a/pipeline/src/assets/fbxImport.ts b/pipeline/src/assets/fbxImport.ts
index dfc93af38..0b264ffbc 100644
--- a/pipeline/src/assets/fbxImport.ts
+++ b/pipeline/src/assets/fbxImport.ts
@@ -327,16 +327,26 @@ function readMaterial(ctx: TextureContext, source: FbxMaterial, index: number):
? readTexture(ctx, source.specularColor.texture, label) : null;
const metalMap = source.metalness?.texture ?? null;
const roughMap = source.roughness?.texture ?? null;
- // The engine samples ONE map packed glTF's way (roughness in green, metal in
- // blue), and FBX exporters normally write two separate images. There is no
- // single image to bind when they do, so the import says so.
+ // The engine samples ONE map packed glTF's way (roughness in green, metal in blue), and FBX exporters normally write a separate
+ // image for each; those are handed on below to be packed into one.
const packed = metalMap && roughMap && metalMap.file === roughMap.file
? readTexture(ctx, metalMap, label) : null;
const metallicRoughnessTexture = orm ?? packed;
- if (!metallicRoughnessTexture && (metalMap || roughMap)) {
- ctx.warnings.push(`${label}: its separate metalness/roughness maps are not imported —`
- + ' the engine samples one image packed as glTF does (roughness in green, metal in'
- + ` blue); the constants ${metallic.toFixed(2)}/${roughness.toFixed(2)} are used instead`);
+ // Separate images are handed on as they are: the image step packs them into one, and says so if it cannot read them.
+ const glossMap = !roughMap ? source.glossiness?.texture ?? null : null;
+ let separateMaps: NonNullable<ImportedMaterial['separateMaps']> | null = null;
+ if (!metallicRoughnessTexture && (metalMap || roughMap || glossMap)) {
+ const metalness = metalMap ? readTexture(ctx, metalMap, label) : null;
+ const rough = roughMap ?? glossMap;
+ const roughnessRef = rough ? readTexture(ctx, rough, label) : null;
+ const occlusionRef = source.occlusion?.texture ? readTexture(ctx, source.occlusion.texture, label) : null;
+ if (metalness || roughnessRef) {
+ separateMaps = {
+ ...(metalness ? { metalness } : {}),
+ ...(roughnessRef ? { roughness: roughnessRef, roughnessIsGlossiness: !roughMap } : {}),
+ ...(occlusionRef ? { occlusion: occlusionRef } : {}),
+ };
+ }
}
const emissionFactor = scalar(source.emissionFactor, 1);
@@ -368,6 +378,7 @@ function readMaterial(ctx: TextureContext, source: FbxMaterial, index: number):
metallic: orm ? 1 : metallic,
roughness: orm ? 1 : roughness,
...(metallicRoughnessTexture ? { metallicRoughnessTexture } : {}),
+ ...(separateMaps ? { separateMaps } : {}),
};
}
diff --git a/pipeline/src/assets/modelImages.ts b/pipeline/src/assets/modelImages.ts
index 9f3fc0085..480eb00f0 100644
--- a/pipeline/src/assets/modelImages.ts
+++ b/pipeline/src/assets/modelImages.ts
@@ -115,11 +115,60 @@ export function prepareModelImages(result: ModelImportResult,
return entry;
};
+ /**
+ * Metalness, roughness and occlusion written as separate images, packed into the one map the engine samples: occlusion in red,
+ * roughness in green, metal in blue, each from its source's red channel (roughness inverted when the source gave glossiness).
+ * A channel with no image stays at full, so the material's own factor says it. Only PNG and DDS can be read here.
+ */
+ const packSeparateMaps = (material: ImportedMaterial, label: string): void => {
+ const separate = material.separateMaps;
+ if (!separate) return;
+ delete material.separateMaps;
+ const read = (ref: ImportedImageRef | undefined) => (ref ? load(ref, label).pixels : null);
+ const metal = read(separate.metalness), rough = read(separate.roughness), occlusion = read(separate.occlusion);
+ const wanted = [separate.metalness, separate.roughness].filter(Boolean).length;
+ if (wanted === 0 || [metal, rough].filter(Boolean).length < wanted) {
+ result.warnings.push(`${label}: its separate metalness/roughness maps could not be packed (only PNG and DDS can be read) —`
+ + ` the constants ${(material.metallic ?? 0).toFixed(2)}/${(material.roughness ?? 1).toFixed(2)} are used instead`);
+ return;
+ }
+ const sources = [metal, rough, occlusion].filter((p): p is Pixels => !!p);
+ const width = Math.max(...sources.map(p => p.width)), height = Math.max(...sources.map(p => p.height));
+ // Nearest-neighbour from each source into the largest's grid: maps of different sizes still line up in uv.
+ const sample = (p: Pixels | null, x: number, y: number): number | null => {
+ if (!p) return null;
+ const sx = Math.min(p.width - 1, Math.floor((x + 0.5) * p.width / width));
+ const sy = Math.min(p.height - 1, Math.floor((y + 0.5) * p.height / height));
+ return p.rgba[(sy * p.width + sx) * 4]!;
+ };
+ const packed = new Uint8Array(width * height * 4);
+ for (let y = 0; y < height; y++) {
+ for (let x = 0; x < width; x++) {
+ const at = (y * width + x) * 4;
+ const ao = sample(occlusion, x, y), r = sample(rough, x, y), m = sample(metal, x, y);
+ packed[at] = ao ?? 255;
+ packed[at + 1] = r === null ? 255 : separate.roughnessIsGlossiness ? 255 - r : r;
+ packed[at + 2] = m ?? 255;
+ packed[at + 3] = 255;
+ }
+ }
+ const name = `${material.name.replace(/[^A-Za-z0-9_-]+/g, '_')}_orm.png`;
+ const texture = { name, bytes: encodeRgbaPng(width, height, packed) };
+ result.textures.push(texture);
+ embedded.set(name, texture);
+ const ref: ImportedImageRef = { file: name, external: false };
+ material.metallicRoughnessTexture = ref;
+ if (metal) material.metallic = 1;
+ if (rough) material.roughness = 1;
+ if (occlusion) { material.occlusionTexture = ref; material.occlusionStrength = 1; }
+ };
+
const noticed = new Set<string>();
const materials = new Set<ImportedMaterial>();
for (const mesh of result.meshes) if (mesh.material) materials.add(mesh.material);
for (const material of materials) {
const label = `material "${material.name}"`;
+ packSeparateMaps(material, label);
for (const slot of SLOTS) {
const ref = material[slot];
if (!ref) continue;
diff --git a/pipeline/src/assets/modelImport.ts b/pipeline/src/assets/modelImport.ts
index 38feb93aa..2023305bf 100644
--- a/pipeline/src/assets/modelImport.ts
+++ b/pipeline/src/assets/modelImport.ts
@@ -84,6 +84,13 @@ export interface ImportedMaterial {
roughness?: number;
/** glTF's packing: roughness in green, metal in blue, multiplying the factors. */
metallicRoughnessTexture?: ImportedImageRef;
+ /**
+ * Metalness, roughness and occlusion as the source wrote them: a separate image each, the value in its red channel (roughness
+ * inverted when the source gave glossiness). Image preparation packs them into the one map above, glTF's way, and removes this.
+ */
+ separateMaps?: {
+ metalness?: ImportedImageRef; roughness?: ImportedImageRef; roughnessIsGlossiness?: boolean; occlusion?: ImportedImageRef;
+ };
/**
* What the surface has beyond a metal/rough pair, one field per glTF extension that says it — the values are
* the extension's own factors, absent where the source says nothing (the engine's defaults are glTF's).
diff --git a/pipeline/tests/model-images.test.ts b/pipeline/tests/model-images.test.ts
index 9a3323898..7dff2125f 100644
--- a/pipeline/tests/model-images.test.ts
+++ b/pipeline/tests/model-images.test.ts
@@ -50,6 +50,53 @@ describe('preparing a model\'s images', () => {
expect([...decoded.data.subarray(0, 4)]).toEqual([10, 20, 30, 255]);
});
+ it('packs separate metalness, roughness and occlusion maps the way glTF does', () => {
+ const files: Record<string, Uint8Array> = {
+ 'metal.png': png(() => [200, 0, 0, 255]),
+ 'rough.png': png(() => [60, 0, 0, 255]),
+ 'ao.png': png(() => [150, 0, 0, 255]),
+ };
+ const result = model({
+ metallic: 0.3, roughness: 0.5,
+ separateMaps: {
+ metalness: { file: 'metal.png', external: true }, roughness: { file: 'rough.png', external: true },
+ occlusion: { file: 'ao.png', external: true },
+ },
+ }, ['metal.png', 'rough.png', 'ao.png']);
+ prepareModelImages(result, uri => files[uri] ?? null);
+ const material = result.meshes[0]!.material!;
+ expect(material.separateMaps).toBeUndefined();
+ expect(material.metallicRoughnessTexture).toEqual({ file: 'm_orm.png', external: false, settings: { sRGB: false } });
+ expect(material.occlusionTexture?.file).toBe('m_orm.png');
+ // The factors step aside, since the maps now say it.
+ expect(material.metallic).toBe(1);
+ expect(material.roughness).toBe(1);
+ const packed = PNG.sync.read(Buffer.from(result.textures.find(t => t.name === 'm_orm.png')!.bytes));
+ expect([...packed.data.subarray(0, 4)]).toEqual([150, 60, 200, 255]);
+ });
+
+ it('inverts a glossiness map, and leaves a channel with no image at full so the factor speaks', () => {
+ const files: Record<string, Uint8Array> = { 'gloss.png': png(() => [230, 0, 0, 255]) };
+ const result = model({ metallic: 0.3, roughness: 0.5, separateMaps: { roughness: { file: 'gloss.png', external: true }, roughnessIsGlossiness: true } },
+ ['gloss.png']);
+ prepareModelImages(result, uri => files[uri] ?? null);
+ const material = result.meshes[0]!.material!;
+ expect(material.metallic).toBe(0.3);
+ expect(material.roughness).toBe(1);
+ const packed = PNG.sync.read(Buffer.from(result.textures[0]!.bytes));
+ expect([...packed.data.subarray(0, 4)]).toEqual([255, 25, 255, 255]);
+ });
+
+ it('says so when a map cannot be read, and keeps the constants', () => {
+ const files: Record<string, Uint8Array> = { 'metal.png': png(() => [200, 0, 0, 255]), 'rough.jpg': new Uint8Array([1, 2, 3]) };
+ const result = model({ metallic: 0.3, roughness: 0.5, separateMaps: { metalness: { file: 'metal.png', external: true }, roughness: { file: 'rough.jpg', external: true } } },
+ ['metal.png', 'rough.jpg']);
+ prepareModelImages(result, uri => files[uri] ?? null);
+ const material = result.meshes[0]!.material!;
+ expect(material.metallicRoughnessTexture).toBeUndefined();
+ expect(result.warnings.join('\n')).toMatch(/could not be packed/);
+ });
+
it('drops an occlusion map that is zero everywhere, and says so', () => {
const result = model({ occlusionTexture: { file: 'orm.png', external: true }, occlusionStrength: 1 });
prepareModelImages(result, () => png(i => [0, 100 + i, 0, 255]));
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 505c19813..7fd99ae82 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2521,6 +2521,8 @@ export const NOTED = {
{"note":"Two new shaders for characters: Skin softens diffuse light by colour, warms the terminator and lets light through thin places without blurring the picture, and Hair lights strands along a flow map with two shifted highlights. Normal maps on stretched UVs (a face unwrapped into an atlas) now keep their shape, and mipmapped textures are sampled with anisotropic filtering where the device has it."},
"chore(editor-checks): hair-look and skin-look checks":
{"internal":"the editor checks for the Skin and Hair entry above"},
+ "feat(import): separate metalness, roughness and occlusion maps are packed the way glTF packs them":
+ {"note":"Importing an FBX with separate metalness, roughness and occlusion images now packs them into the one map the Model shader samples, instead of dropping them."},
"chore(tools): the gates run in parallel with --jobs, and the reds a full run gathered are cleared":
{"internal":"tooling for the static gates, and fixes to checks that had drifted from the code they guard"},
"fix(render): a wire narrower than a pixel is no longer painted over by the sky in dashes":