Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(import): a model can give its skin, hair and eye materials the character shaders

yhhbef6d1bdf8de
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8d8a7c472..c7f9b450a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -35,6 +35,7 @@ published separately; it ships inside the editor.
- **Contact shadows and screen-space bounce light.** Contact Shadows and Screen Space GI effects add small sharp shadows at contacts and one bounce of colour between surfaces, and the weather softens the sun's shadows. `contactShadows` marches a short ray toward the scene's sun in the depth buffer; `ssgi` gathers eight rays a pixel over the G-buffer's normals at half size into a float target and averages the neighbours on the same surface before laying the light, times the receiver's colour, over the picture. The Weather component raises the sun's `sourceAngle` and lowers its `shadowStrength` as the sky clouds over.
- **Terrain and Foliage.** A Terrain component makes ground from a heightmap or noise, paints it with grass, dirt, snow and rock layers and draws it in chunks that coarsen with distance; a Foliage component scatters grass or a mesh over it, swaying in the wind and thinning out with distance.
- **Separate material maps packed on import.** Importing an FBX with separate metalness, roughness and occlusion images now packs them into the one map the Model shader samples, instead of dropping them. Occlusion goes in red, roughness in green and metal in blue, each from its image's red channel (a glossiness map is inverted); a channel with no image stays at full so the material's own value speaks. Only PNG and DDS can be read.
+- **Character materials on import.** Importing a model can now give its character materials the right shader: turn on Character Materials in the model's import settings and a material whose name says skin, face, head, hair or eye gets the Skin, Hair or Eye shader with its normal and occlusion/roughness maps bound, and hair is cut out by its alpha and feathered by coverage. Off by default.
- **Curvature-shaped skin scattering.** The Skin shader can take its wrap past the terminator per colour and shape it by curvature: `u_preIntegrate` (red travels furthest, every colour further where the surface turns sharply) and `u_curvature`, so a nose or an ear glows at its edge while a flat cheek stays crisp. Off by default.
- **Skin detail normal layer.** The Skin shader has a detail normal layer: `u_detailMap` is a fine normal map tiled over the surface (`u_detailTile`) for pores and creases, laid on the sharp normal the shine reads by `u_detailStrength` (0 leaves it out) and not on the soft diffuse normals.
- **Cutout shadows for hair cards and leaves.** A material whose alpha is coverage (`u_alphaCutoff` above 0, now also on the Hair shader) casts its cut-out shape into shadow maps instead of the rectangle it is drawn on, and Hair cards feather their edges with alpha-to-coverage.
diff --git a/desktop b/desktop
index 16c82aa6b..50a6ccd99 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 16c82aa6b1b23c5758266dc8a4f1f931032fc753
+Subproject commit 50a6ccd99bd77f2a799922439e69412f2319af16
diff --git a/docs/astro/src/content/docs/assets/models.mdx b/docs/astro/src/content/docs/assets/models.mdx
index 7e8960c0e..8ae17e544 100644
--- a/docs/astro/src/content/docs/assets/models.mdx
+++ b/docs/astro/src/content/docs/assets/models.mdx
@@ -212,6 +212,10 @@ product that left them out would describe a surface the source did not.
The material also restates the draw's blending, depth and culling, because a material
replaces them — a model told to occlude itself keeps doing so through its material.
+## Character materials
+
+A model's materials import on the Model shader. Turn on **Character Materials** (`characterMaterials: auto`) in the model's import settings and a material whose name says skin, face, head, hair or eye gets the Skin, Hair or Eye shader instead, with its normal map and its occlusion/roughness map bound to them. A hair material is cut out by its alpha (`u_alphaCutoff`, 0.5 unless the source says otherwise) and drawn without blending, so its edges are feathered by coverage. The name is read word by word, so `BodySkin` and `Hair_A` match while `Eyelashes` and `Headband` do not, and the colour texture's file name counts too. A name is a convention, not a fact: look at what it chose in the materials the import writes, and change a shader by hand where it guessed wrong. Eyes need their iris maps (`u_depthMap`, `u_irisNormalMap`) assigned by hand, as the source has no slot for them. It is off by default, so re-importing never changes a material you did not ask about.
+
## Animations
Each animation in the source becomes an `.estimeline` beside the meshes, and the prefab's
diff --git a/docs/astro/src/content/docs/zh-cn/assets/models.mdx b/docs/astro/src/content/docs/zh-cn/assets/models.mdx
index 74e204335..63b2955a5 100644
--- a/docs/astro/src/content/docs/zh-cn/assets/models.mdx
+++ b/docs/astro/src/content/docs/zh-cn/assets/models.mdx
@@ -183,6 +183,10 @@ chart,每片投影到自己的平面,再按**同一个**世界-纹素比打包
材质还会把这次 draw 的混合、深度和剔除重述一遍 —— 因为材质会**替换**它们,
这样一个被要求自我遮挡的模型,在有了材质之后仍然自我遮挡。
+## 角色材质
+
+模型的材质默认导入到 Model 着色器。在模型的导入设置里打开**角色材质**(`characterMaterials: auto`)后,名字里带 skin、face、head、hair 或 eye 的材质会改用 Skin、Hair 或 Eye 着色器,并绑上它的法线贴图和遮蔽/粗糙度贴图。头发材质按 alpha 裁剪(`u_alphaCutoff`,源文件没说就取 0.5),并且不做混合绘制,边缘由覆盖率来柔化。名字是按单词读的,所以 `BodySkin`、`Hair_A` 会命中,`Eyelashes`、`Headband` 不会,颜色贴图的文件名也算。名字只是约定,不是事实:请看导入写出的材质选了什么,猜错的地方手动改着色器。眼睛的虹膜贴图(`u_depthMap`、`u_irisNormalMap`)需要手动指定,源文件里没有对应的位置。默认关闭,所以重新导入不会改动你没要求改的材质。
+
## 动画
源里的每个动画会在 mesh 旁边写出一个 `.estimeline`,prefab 的根挂一个 `TimelinePlayer`
diff --git a/pipeline/bin/estella.mjs b/pipeline/bin/estella.mjs
index 0f7ada7fd..39f2f662a 100644
--- a/pipeline/bin/estella.mjs
+++ b/pipeline/bin/estella.mjs
@@ -739,10 +739,14 @@ if (opts.command === 'import-model' || opts.command === 'import-gltf') {
// named once the project root is known; without one they are bare names.
const root = opts.project ?? findProjectRoot(sourceDir);
const projectRef = (abs) => path.relative(root, abs).split(path.sep).join('/');
+ let characters = false;
+ try {
+ characters = JSON.parse(readFileSync(`${opts.source}.meta`, 'utf8')).importer?.characterMaterials === 'auto';
+ } catch { /* no meta: the schema's default */ }
const refs = root
? { prefix: projectRef(dir) ? `${projectRef(dir)}/` : '',
- external: (uri) => projectRef(path.resolve(sourceDir, uri)) }
- : {};
+ external: (uri) => projectRef(path.resolve(sourceDir, uri)), characters }
+ : { characters };
const { meshes, textures, nodes, animations, warnings } = await reader.readModelSource(
new Uint8Array(readFileSync(opts.source)), stem,
diff --git a/pipeline/src/assets/modelImport.ts b/pipeline/src/assets/modelImport.ts
index 55c075f45..363ba848c 100644
--- a/pipeline/src/assets/modelImport.ts
+++ b/pipeline/src/assets/modelImport.ts
@@ -522,6 +522,8 @@ export interface ProductRefs {
prefix?: string;
/** An external image uri (relative to the source) → the ref a component carries. */
external?: (uri: string) => string;
+ /** Give a skin, hair or eye material the engine's character shader instead of the Model shader (see {@link characterShading}). */
+ characters?: boolean;
}
/** How the products go together. */
@@ -588,6 +590,51 @@ function needsMaterial(material: ImportedMaterial | undefined): material is Impo
|| MATERIAL_LAYER_PROPERTIES.some(([field]) => material[field] !== undefined));
}
+/** The character shader a material's name asks for, when it names skin, hair or an eye. */
+export type CharacterShading = 'skin' | 'hair' | 'eye';
+
+const CHARACTER_TOKENS: readonly [CharacterShading, readonly string[]][] = [
+ ['eye', ['eye', 'eyes', 'eyeball', 'eyeballs', 'iris', 'cornea']],
+ ['hair', ['hair', 'hairs', 'ponytail', 'bangs', 'fringe']],
+ ['skin', ['skin', 'face', 'head', 'neck']],
+];
+
+/**
+ * What a material is, read from its name and its colour texture's file name, split into words (`T_C004_BodySkin` is c, body, skin):
+ * a word has to BE `eye` or `hair`, so `Eyelashes` and `Headband` are neither. A name is a convention and not a fact, which is why
+ * the import asks for this by setting and never assumes it.
+ */
+export function characterShading(material: ImportedMaterial): CharacterShading | undefined {
+ const file = material.baseColorTexture?.file ?? '';
+ const words = `${material.name} ${file.slice(file.lastIndexOf('/') + 1)}`
+ .replace(/([a-z])([A-Z])/g, '$1 $2').toLowerCase().split(/[^a-z]+/);
+ for (const [shading, names] of CHARACTER_TOKENS) {
+ if (words.some((word) => names.includes(word))) return shading;
+ }
+ return undefined;
+}
+
+/** Whether a material gets a product: it says something a MeshRenderer cannot, or its name asks for a character shader. */
+function hasProduct(material: ImportedMaterial, refs: ProductRefs): boolean {
+ return needsMaterial(material) || (!!refs.characters && characterShading(material) !== undefined);
+}
+
+/** What the character shaders read, from what the source says: the maps they share, and the cutout hair needs. */
+function characterProperties(shading: CharacterShading, material: ImportedMaterial, properties: Record<string, unknown>,
+ bind: (name: string, image?: ImportedImageRef) => void): void {
+ // Skin and hair read the glTF packing (occlusion, roughness, metalness) as their `u_ormMap`; their roughness is a SCALE on it,
+ // so a factor of 0 (what an FBX without one writes) would be a mirror, and is left out.
+ const rough = material.roughness !== undefined && material.roughness > 0 ? material.roughness : undefined;
+ for (const key of ['u_metallic', 'u_roughness', 'u_metallicRoughnessMap', 'u_occlusionMap', 'u_occlusionStrength']) delete properties[key];
+ if (shading !== 'eye') {
+ bind('u_ormMap', material.metallicRoughnessTexture);
+ if (rough !== undefined && !material.metallicRoughnessTexture) properties.u_roughness = rough;
+ }
+ if (shading === 'hair') properties.u_alphaCutoff = material.alphaCutoff ?? 0.5;
+ else delete properties.u_alphaCutoff;
+ if (shading === 'eye') delete properties.u_normalMap;
+}
+
/** `<stem>_m<source material index>` — the product a material becomes. */
function materialName(stem: string, material: ImportedMaterial): string {
return `${stem}_m${material.index}`;
@@ -626,8 +673,9 @@ export function materialProducts(meshes: ImportedMesh[], stem: string,
const seen = new Set<number>();
for (const mesh of meshes) {
const material = mesh.material;
- if (!needsMaterial(material) || seen.has(material.index)) continue;
+ if (!material || !hasProduct(material, refs) || seen.has(material.index)) continue;
seen.add(material.index);
+ const shading = refs.characters ? characterShading(material) : undefined;
const images: ImportedImageRef[] = [];
const properties: Record<string, unknown> = {};
@@ -652,7 +700,8 @@ export function materialProducts(meshes: ImportedMesh[], stem: string,
properties.u_occlusionStrength = material.occlusionStrength;
}
if (material.alphaCutoff !== undefined) properties.u_alphaCutoff = material.alphaCutoff;
- for (const [field, parameter] of MATERIAL_LAYER_PROPERTIES) {
+ if (shading) characterProperties(shading, material, properties, bind);
+ else for (const [field, parameter] of MATERIAL_LAYER_PROPERTIES) {
const value = material[field];
if (value === undefined) continue;
properties[parameter] = Array.isArray(value)
@@ -664,10 +713,11 @@ export function materialProducts(meshes: ImportedMesh[], stem: string,
data: {
version: MATERIAL_FORMAT_VERSION,
type: 'material',
- shader: 'builtin:model',
- blendMode: material.opaque ? BlendMode.None : BlendMode.Normal,
- depthTest: material.opaque,
- depthWrite: material.opaque,
+ shader: shading ? `builtin:${shading}` : 'builtin:model',
+ // Hair is cut out and feathered by coverage, which only an unblended draw gets.
+ blendMode: material.opaque || shading === 'hair' ? BlendMode.None : BlendMode.Normal,
+ depthTest: material.opaque || shading === 'hair',
+ depthWrite: material.opaque || shading === 'hair',
cull: material.cullBackfaces ? CullMode.Back : CullMode.None,
properties,
},
@@ -698,7 +748,7 @@ function meshComponent(mesh: ImportedMesh, stem: string, refs: ProductRefs): Com
// Shading moves WHOLE to the material when there is one: a material shader
// reads its own samplers, so a normal map left on the component would be a
// map nothing samples.
- const material = needsMaterial(mesh.material)
+ const material = mesh.material && hasProduct(mesh.material, refs)
? `${prefix}${materialName(stem, mesh.material)}.esmaterial` : null;
// A normal map needs normals to perturb; without them the engine draws the
// unlit variant and the map would be a reference to nothing.
diff --git a/pipeline/src/project/importSettings.ts b/pipeline/src/project/importSettings.ts
index dd04110fe..895c4901e 100644
--- a/pipeline/src/project/importSettings.ts
+++ b/pipeline/src/project/importSettings.ts
@@ -231,6 +231,14 @@ const MODEL: ImporterFieldSpec[] = [
tooltip: 'A mesh with fewer triangles than this gets no levels: there is too little to '
+ 'take out for a swap to pay.',
},
+ {
+ key: 'characterMaterials', label: 'Character Materials', type: 'enum', default: 'off',
+ category: 'Import', advanced: true,
+ options: [{ label: 'Off', value: 'off' }, { label: 'By Material Name', value: 'auto' }],
+ tooltip: 'Give materials whose names say skin, face, head, hair or eye the engine\u0027s Skin, Hair and Eye '
+ + 'shaders instead of the Model shader, with the normal and occlusion/roughness maps bound to them, and hair '
+ + 'cut out and feathered by its alpha. A name is a convention, so check what it chose.',
+ },
{
key: 'specularMap', label: 'Specular Map', type: 'enum', default: 'color',
category: 'Import', advanced: true,
diff --git a/pipeline/tests/character-materials.test.ts b/pipeline/tests/character-materials.test.ts
new file mode 100644
index 000000000..e83d7e28a
--- /dev/null
+++ b/pipeline/tests/character-materials.test.ts
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+import { describe, expect, it } from 'vitest';
+import { BlendMode } from 'esengine';
+import { characterShading, materialProducts, type ImportedImageRef, type ImportedMaterial, type ImportedMesh } from '../src/assets/modelImport';
+
+const image = (file: string, data = false): ImportedImageRef => ({ file, external: true, ...(data ? { settings: { sRGB: false } } : {}) } as ImportedImageRef);
+
+function material(index: number, name: string, extra: Partial<ImportedMaterial> = {}): ImportedMaterial {
+ return { index, name, baseColor: [1, 1, 1, 1], opaque: true, cullBackfaces: true, ...extra };
+}
+const meshOf = (m: ImportedMaterial): ImportedMesh => ({ material: m } as ImportedMesh);
+
+describe('characterShading', () => {
+ it('reads skin, hair and eyes from the words of a name, and nothing from a word that merely contains one', () => {
+ expect(characterShading(material(0, 'M_BodySkin'))).toBe('skin');
+ expect(characterShading(material(0, 'Face_005'))).toBe('skin');
+ expect(characterShading(material(0, 'C004_Hair_A'))).toBe('hair');
+ expect(characterShading(material(0, 'EyeLeft'))).toBe('eye');
+ expect(characterShading(material(0, 'Eyes'))).toBe('eye');
+ expect(characterShading(material(0, 'Eyelashes'))).toBeUndefined();
+ expect(characterShading(material(0, 'Headband'))).toBeUndefined();
+ expect(characterShading(material(0, 'Body1'))).toBeUndefined();
+ });
+
+ it('also reads the colour texture\'s file name', () => {
+ expect(characterShading(material(0, 'Material.003', { baseColorTexture: image('assets/T_Hair_D.png') }))).toBe('hair');
+ });
+});
+
+describe('materialProducts with character materials', () => {
+ const skin = material(0, 'Face', { normalTexture: image('n.png', true), metallicRoughnessTexture: image('orm.png', true), metallic: 0, roughness: 0 });
+ const hair = material(1, 'Hair', { normalTexture: image('hn.png', true), metallicRoughnessTexture: image('horm.png', true), opaque: false });
+ const eye = material(2, 'Eye', { roughness: 0.2 });
+ const cloth = material(3, 'Jacket', { normalTexture: image('jn.png', true) });
+ const meshes = [skin, hair, eye, cloth].map(meshOf);
+
+ it('keeps every material on the Model shader unless asked', () => {
+ const products = materialProducts(meshes, 'nitro');
+ expect(products.map((p) => p.data.shader)).toEqual(['builtin:model', 'builtin:model', 'builtin:model', 'builtin:model']);
+ });
+
+ it('binds the shaders the names ask for, with their maps and a cutout for hair', () => {
+ const products = materialProducts(meshes, 'nitro', { characters: true });
+ const by = Object.fromEntries(products.map((p) => [p.name, p.data]));
+ expect(by.nitro_m0!.shader).toBe('builtin:skin');
+ expect(by.nitro_m0!.properties).toMatchObject({ u_normalMap: expect.any(String), u_ormMap: expect.any(String) });
+ expect(by.nitro_m0!.properties).not.toHaveProperty('u_roughness');
+ expect(by.nitro_m1!.shader).toBe('builtin:hair');
+ expect(by.nitro_m1!.properties).toMatchObject({ u_alphaCutoff: 0.5 });
+ expect(by.nitro_m1!.blendMode).toBe(BlendMode.None);
+ expect(by.nitro_m2!.shader).toBe('builtin:eye');
+ expect(by.nitro_m3!.shader).toBe('builtin:model');
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 7983e0fbf..d2a57641b 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2551,6 +2551,8 @@ export const NOTED = {
{"note":"The Skin shader can take its wrap past the terminator per colour and shape it by curvature: `u_preIntegrate` (red travels furthest, every colour further where the surface turns sharply) and `u_curvature`, so a nose or an ear glows at its edge while a flat cheek stays crisp. Off by default."},
"feat(render): the Eye shader takes a lid shadow, authored or from the surface facing up":
{"note":"The Eye shader shades the eye the lids sit on: `u_lidMap` is an authored occlusion map, and `u_lidShadow` darkens, and fades the cornea's reflections, where the surface faces up from `u_lidStart`, which is the upper lid on an upright head."},
+ "feat(import): a model can give its skin, hair and eye materials the character shaders":
+ {"note":"Importing a model can now give its character materials the right shader: turn on Character Materials in the model's import settings and a material whose name says skin, face, head, hair or eye gets the Skin, Hair or Eye shader with its normal and occlusion/roughness maps bound, and hair is cut out by its alpha and feathered by coverage. Off by default."},
"chore(tools): the gates run in parallel with --jobs, and the reds a full run gathered are cleared":
{"internal":"tooling for the static gates, and fixes to checks that had drifted from the code they guard"},
"fix(render): a wire narrower than a pixel is no longer painted over by the sky in dashes":