A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(render): imported meshes carry a tangent frame, so a normal map is read against the frame it was baked in
yhhcf313034c087
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 8ebddee1a..beec65fe6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -16,6 +16,7 @@ published separately; it ships inside the editor.### Fixed+- **Normal maps read against the frame they were baked in.** Imported meshes now carry a tangent at each vertex (a glTF's own direction, or one made from the UVs), and normal maps on them are read against that frame instead of one derived per pixel, which was wrong at UV seams and on stretched textures. Re-import a model to get them; one imported before keeps its old shading. A mesh whose vertices carry a tangent is drawn with an extra vertex attribute and one more varying; a shader of your own that calls `perturbNormal` picks the frame up with no change.- **New Project keeps the wrong template name.** The New Project page no longer keeps the previous template's name when you pick another: the suggested name follows the template, and a name you typed stays.### Changed@@ -33,6 +34,7 @@ published separately; it ships inside the editor.- **Contact shadows and screen-space bounce light.** Contact Shadows and Screen Space GI effects add small sharp shadows at contacts and one bounce of colour between surfaces, and the weather softens the sun's shadows. `contactShadows` marches a short ray toward the scene's sun in the depth buffer; `ssgi` gathers eight rays a pixel over the G-buffer's normals at half size into a float target and averages the neighbours on the same surface before laying the light, times the receiver's colour, over the picture. The Weather component raises the sun's `sourceAngle` and lowers its `shadowStrength` as the sky clouds over.- **Terrain and Foliage.** A Terrain component makes ground from a heightmap or noise, paints it with grass, dirt, snow and rock layers and draws it in chunks that coarsen with distance; a Foliage component scatters grass or a mesh over it, swaying in the wind and thinning out with distance.- **Separate material maps packed on import.** Importing an FBX with separate metalness, roughness and occlusion images now packs them into the one map the Model shader samples, instead of dropping them. Occlusion goes in red, roughness in green and metal in blue, each from its image's red channel (a glossiness map is inverted); a channel with no image stays at full so the material's own value speaks. Only PNG and DDS can be read.+- **Eye shader.** A new Eye shader (`builtin:eye`) draws an eye from a mask and a dome normal map: the iris sits behind a refracting cornea so it moves as the eye turns, with a pupil, fibres and a limbal ring, a white lit with wrapped light, and a near-mirror cornea that shows every light as a catchlight.- **Skin and hair.** Two new shaders for characters: Skin softens diffuse light by colour, warms the terminator and lets light through thin places without blurring the picture, and Hair lights strands along a flow map with two shifted highlights. Normal maps on stretched UVs (a face unwrapped into an atlas) now keep their shape, and mipmapped textures are sampled with anisotropic filtering where the device has it. Both read a normal map and an occlusion/roughness map as data textures (import them with sRGB off); the Skin shader shades each pixel with three lighting passes, so it is for characters in view. Hair is shaded as strands but a hair model is still cards or ribbons.- **Anisotropic filtering.** Mipmapped textures are now sampled with up to 16x anisotropic filtering where the device has it (WebGL2 with the extension, WebGPU always), so a floor, a hair card or a sleeve seen at a slant keeps its detail instead of washing out to grey.- **Wilds example.** Wilds is a new 3D example: a lake under mountains at dusk, made from a generated heightmap with the Terrain, Water and Foliage features and the sky's cloud shadows. Everything in it is made from numbers by `examples/wilds/tools/build.mjs`, so there is no art to credit.diff --git a/docs/astro/src/content/docs/assets/models.mdx b/docs/astro/src/content/docs/assets/models.mdxindex 2e20fa69d..7e8960c0e 100644--- a/docs/astro/src/content/docs/assets/models.mdx+++ b/docs/astro/src/content/docs/assets/models.mdx@@ -86,6 +86,15 @@ Re-importing replaces every product. Edits made to a generated `.esprefab` are lplace an instance in a scene and edit that instead.</Aside>+## Tangents++A mesh that has normals and UVs is written with a **tangent** at each vertex, so a normal map is read against the frame it was+baked in rather than one derived per pixel from screen-space derivatives (which is right on average and wrong at a UV seam or on+a stretched texture). A glTF's own `TANGENT` keeps its direction; a mesh without one gets it from its UVs, weighted by the angle each+triangle makes at the vertex. The bitangent's side is always taken from the geometry as stored, so it is right whichever way the+importer turned V. A mesh imported before this carries none and is shaded as it was; re-import it to get them. A shader of your own+that calls `perturbNormal` gets the mesh's frame without any change.+## ScaleModels are authored in **metres** — an FBX authored in centimetres is converted on the way@@ -245,7 +254,7 @@ glTF states none, and pushing past 1 or below 0 is a shape an author may want.|---|---|| 8 shapes blended at once | A budget on what is LIVE, not on what a mesh carries — a face holds a handful of expressions open while its mesh has a hundred. The heaviest weights are taken; the rest read as zero, which is what an untouched weight already means. || Normals deform only where the geometry has them | A mesh with no `NORMAL` channel is shaded off a constant normal, which no delta can bend. |-| Tangents are not carried | The tangent frame is derived per pixel, from the surface as deformed. |+| Morph targets carry no tangents | The mesh keeps the tangent frame it has at rest; a shape moves the surface but not the frame its normal map was baked in. || An FBX's in-between shapes | A channel that passes through shapes on its way to the last one carries only the last; the import says which it dropped. |## FBXdiff --git a/docs/astro/src/content/docs/graphics/materials.mdx b/docs/astro/src/content/docs/graphics/materials.mdxindex be131a07e..dd55cfe21 100644--- a/docs/astro/src/content/docs/graphics/materials.mdx+++ b/docs/astro/src/content/docs/graphics/materials.mdx@@ -86,6 +86,8 @@ one under it (`u_secondaryTint`). `u_roughness` sets how tight the lines are. Li`u_sharpen` keeps the fine strands a card is covered in from washing out at a slant, and `u_rootDark` darkens the roots by the occlusion map.A hair model is usually cards or ribbons, not strands; this shader gives the shading of a strand to what the geometry gives it.+**Eye** (`builtin:eye`) builds an eye from two maps and a few numbers. `u_depthMap` is white over the iris and the dome of the cornea, and `u_irisNormalMap` holds that dome's normals (import both with sRGB off). The view ray is bent into the cornea (`u_ior`) and carried down to the iris (`u_irisDepth`), so the iris moves behind the surface as the eye turns. The iris itself is drawn from `u_irisColor` to `u_irisColor2` with fibres, a dark limbal ring (`u_limbal`) and a pupil (`u_pupilSize`), against the white (`u_scleraColor`), which is lit with light that wraps past the terminator (`u_scleraWrap`). The cornea is a near-mirror (`u_corneaRoughness`, `u_specular`), so every light and the sky show as a catchlight. A painted eye texture on the mesh multiplies the result.+Both read the same lighting core as every Lit shader, so shadows, image-based light and the other effects apply.## Shading layersdiff --git a/docs/astro/src/content/docs/zh-cn/assets/models.mdx b/docs/astro/src/content/docs/zh-cn/assets/models.mdxindex b6f810494..74e204335 100644--- a/docs/astro/src/content/docs/zh-cn/assets/models.mdx+++ b/docs/astro/src/content/docs/zh-cn/assets/models.mdx@@ -76,6 +76,13 @@ Transform 给,和导入的模型一样。重新导入会替换每一个产物。手工改过的 `.esprefab` 会丢失;正确做法是在场景里放一个实例、改那个实例。</Aside>+## 切线++有法线和 UV 的网格会在每个顶点写入一个**切线**,这样法线贴图读的是它烘焙时用的那个基,而不是逐像素从屏幕空间导数推出来的+(后者平均意义上是对的,在 UV 接缝或被拉伸的贴图上就是错的)。glTF 自带的 `TANGENT` 保留它的方向;没有的网格由 UV 推出,+按每个三角形在顶点处的夹角加权。副切线朝哪一侧永远取自存下来的几何,所以不管导入器翻没翻 V 都是对的。这次之前导入的网格+没有切线,仍按原来的方式着色;重新导入就有了。自己写的、调用 `perturbNormal` 的着色器不用改就会拿到网格的这个基。+## Scale模型以**米**为单位——以厘米创作的 FBX 会在进来的路上换算好。一个世界单位是一个**设计像素**。@@ -216,7 +223,7 @@ target 对每个顶点的 delta,prefab 给实体挂一个 `MeshMorph`,它的|---|---|| 同时最多混合 8 个形状 | 限制的是**活跃**数量,不是 mesh 能携带的数量——一张脸同时张开的表情就那么几个,而它的 mesh 可以有上百个形状。取权重最大的那些,其余按 0 读,这和作者没动过的权重是同一件事。 || 只有几何自带法线时法线才变形 | 没有 `NORMAL` 通道的 mesh 按一个常量法线着色,任何 delta 都弯不动它。 |-| 不带切线 | 切线基是逐像素从变形后的表面推出来的。 |+| 形变目标不带切线 | 网格保留静止姿态下的切线基;形变目标移动表面,但不改变法线贴图烘焙时用的那个基。 || FBX 的 in-between 形状 | 一个通道途经多个形状时只带最后一个;导入时会说明丢掉了哪些。 |## FBXdiff --git a/docs/astro/src/content/docs/zh-cn/graphics/materials.mdx b/docs/astro/src/content/docs/zh-cn/graphics/materials.mdxindex 0b1688a0c..0170201d2 100644--- a/docs/astro/src/content/docs/zh-cn/graphics/materials.mdx+++ b/docs/astro/src/content/docs/zh-cn/graphics/materials.mdx@@ -67,6 +67,8 @@ std140 uniform 块,于是 CPU 能按名字设置、并安全地送到 GPU。一**头发**(`builtin:hair`)把发丝当发丝来打光。`u_flowMap` 说明发丝朝哪个方向(红、绿是贴图上的一个方向),漫反射是 Kajiya-Kay(发丝被光照亮的程度取决于光离它的走向有多远),高光是两条沿发丝的窄高光,由 `u_shiftMap` 把它们错开(`u_shiftAmount`、`u_shiftTile`):一条白的靠近表面,一条带颜色的在下面(`u_secondaryTint`)。`u_roughness` 决定线条有多紧。光会从发丝背面透过来(`u_translucency`),`u_sharpen` 让发片上密布的细发丝在斜着看时不被冲淡,`u_rootDark` 按遮蔽图把发根压暗。头发模型通常是发片或发带而不是真正的发丝;这个着色器把发丝的明暗给了几何本身给不了的部分。+**眼睛**(`builtin:eye`)用两张贴图和几个数值做出一只眼睛。`u_depthMap` 在虹膜和角膜穹顶上为白色,`u_irisNormalMap` 是这个穹顶的法线(两张都要以关闭 sRGB 的方式导入)。视线在角膜处折射(`u_ior`)再下行到虹膜(`u_irisDepth`),所以眼球转动时虹膜在表面后面移动。虹膜本身由 `u_irisColor` 到 `u_irisColor2` 绘制,带纤维纹理、深色角膜缘(`u_limbal`)和瞳孔(`u_pupilSize`),衬在眼白(`u_scleraColor`)上,眼白的光会绕过明暗交界线(`u_scleraWrap`)。角膜近似镜面(`u_corneaRoughness`、`u_specular`),所以每盏灯和天空都会变成一个眼神光。网格上画的眼睛贴图会与结果相乘。+两者读的是和所有 Lit 着色器相同的光照核心,所以阴影、基于图像的光照和其他特效都适用。## 着色层diff --git a/fixtures/scenes/mesh-normalmap-tangent.esscene b/fixtures/scenes/mesh-normalmap-tangent.esscenenew file mode 100644index 000000000..9c0a5c4d9--- /dev/null+++ b/fixtures/scenes/mesh-normalmap-tangent.esscene@@ -0,0 +1,71 @@+{+ "version": 4,+ "name": "MeshNormalMapTangent",+ "entities": [+ {+ "id": 0,+ "name": "Camera",+ "parent": null,+ "children": [],+ "visible": true,+ "components": [+ {+ "type": "Transform",+ "data": {+ "position": {+ "x": 0,+ "y": 0,+ "z": 10+ }+ }+ },+ {+ "type": "Camera",+ "data": {+ "projectionType": 1,+ "orthoSize": 300,+ "isActive": true,+ "priority": 0+ }+ }+ ]+ },+ {+ "id": 1,+ "name": "Sun",+ "parent": null,+ "children": [],+ "visible": true,+ "components": [+ {+ "type": "Transform",+ "data": {+ "position": {+ "x": 0,+ "y": 0,+ "z": 0+ }+ }+ },+ {+ "type": "Light",+ "data": {+ "type": 1,+ "color": {+ "r": 1,+ "g": 1,+ "b": 1,+ "a": 1+ },+ "intensity": 1,+ "direction": {+ "x": 1,+ "y": 0+ },+ "enabled": true+ }+ }+ ]+ }+ ]+}diff --git a/fixtures/scenes/mesh-normalmap-tangent.textures.json b/fixtures/scenes/mesh-normalmap-tangent.textures.jsonnew file mode 100644index 000000000..0967ef424--- /dev/null+++ b/fixtures/scenes/mesh-normalmap-tangent.textures.json@@ -0,0 +1 @@+{}diff --git a/fixtures/scenes/normalmap-quad-tangent.esmesh b/fixtures/scenes/normalmap-quad-tangent.esmeshnew file mode 100644index 000000000..ca897f1f4Binary files /dev/null and b/fixtures/scenes/normalmap-quad-tangent.esmesh differdiff --git a/fixtures/scenes/normalmap-quad-tangent.esmesh.meta b/fixtures/scenes/normalmap-quad-tangent.esmesh.metanew file mode 100644index 000000000..5034651c2--- /dev/null+++ b/fixtures/scenes/normalmap-quad-tangent.esmesh.meta@@ -0,0 +1,6 @@+{+ "uuid": "ea3c909d-81b6-4a85-a612-9cf712288c5c",+ "version": "2.0",+ "type": "mesh",+ "importer": {}+}diff --git a/fixtures/scenes/normalmap-quad-tangent.esprefab b/fixtures/scenes/normalmap-quad-tangent.esprefabnew file mode 100644index 000000000..784a9b2e7--- /dev/null+++ b/fixtures/scenes/normalmap-quad-tangent.esprefab@@ -0,0 +1,37 @@+{+ "version": "2",+ "name": "normalmap-quad-tangent",+ "rootEntityId": "n0",+ "entities": [+ {+ "prefabEntityId": "n0",+ "name": "Quad",+ "parent": null,+ "children": [],+ "visible": true,+ "components": [+ {+ "type": "Transform",+ "data": {}+ },+ {+ "type": "MeshRenderer",+ "data": {+ "mesh": "scenes/normalmap-quad-tangent.esmesh",+ "lit": true,+ "material": "scenes/normalmap-quad_m0.esmaterial",+ "color": {+ "r": 1,+ "g": 1,+ "b": 1,+ "a": 1+ },+ "opaque": true,+ "cullBackfaces": true,+ "enabled": true+ }+ }+ ]+ }+ ]+}diff --git a/fixtures/scenes/normalmap-quad-tangent.esprefab.meta b/fixtures/scenes/normalmap-quad-tangent.esprefab.metanew file mode 100644index 000000000..d5a6f47d8--- /dev/null+++ b/fixtures/scenes/normalmap-quad-tangent.esprefab.meta@@ -0,0 +1,8 @@+{+ "uuid": "685eb607-06e2-4ef5-af8a-8fb9f19d63f8",+ "version": "2.0",+ "type": "prefab",+ "importer": {+ "autoMigrate": true+ }+}diff --git a/pipeline/src/assets/fbxImport.ts b/pipeline/src/assets/fbxImport.tsindex 0b264ffbc..93d00d68f 100644--- a/pipeline/src/assets/fbxImport.ts+++ b/pipeline/src/assets/fbxImport.ts@@ -14,6 +14,7 @@* is the other half of that contract.*/import { MESH_MAX_BONES, MeshChannel, MeshChannelType, packChannels } from 'esengine';+import { generateTangents } from './meshTangents';import {alignQuaternionSigns, animationProductName, ANIMATED_PATHS, disambiguateNodes,nodeChildPaths, nodeNameFor, samplerKeyframes, timelineDocument,@@ -480,12 +481,17 @@ function buildMesh(part: FbxMeshPart, name: string, payload: Uint8Array,}const skinned = !!(joints && weights && bones > 0 && bones <= MESH_MAX_BONES);+ // The frame the normal map was baked in, from the UVs as stored.+ const tangents = normals && uvs ? generateTangents(positions, normals, uvs, indices) : null;+const { channels, vertexStride } = packChannels([{ semantic: MeshChannel.Position, components: 3, type: MeshChannelType.Float32 },{ semantic: MeshChannel.TexCoord0, components: 2, type: MeshChannelType.Float32 },{ semantic: MeshChannel.Color, components: 4, type: MeshChannelType.UNorm8 },...(normals ? [{ semantic: MeshChannel.Normal, components: 3,type: MeshChannelType.Float32 }] : []),+ ...(tangents ? [{ semantic: MeshChannel.Tangent, components: 4,+ type: MeshChannelType.Float32 }] : []),// Ahead of the skinning pair, which the two lines below find by position....(uvs1 ? [{ semantic: MeshChannel.TexCoord1, components: 2,type: MeshChannelType.Float32 }] : []),@@ -497,6 +503,7 @@ function buildMesh(part: FbxMeshPart, name: string, payload: Uint8Array,const jointsChannel = skinned ? channels[channels.length - 2]! : null;const weightsChannel = skinned ? channels[channels.length - 1]! : null;const uv1Channel = channels.find(c => c.semantic === MeshChannel.TexCoord1) ?? null;+ const tangentChannel = channels.find(c => c.semantic === MeshChannel.Tangent) ?? null;const vertices = new Uint8Array(vertexCount * vertexStride);const dv = new DataView(vertices.buffer);@@ -529,6 +536,9 @@ function buildMesh(part: FbxMeshPart, name: string, payload: Uint8Array,dv.setFloat32(at + channels[3].offset + c * 4, normals[i * 3 + c] ?? 0, true);}}+ if (tangentChannel && tangents) {+ for (let c = 0; c < 4; c++) dv.setFloat32(at + tangentChannel.offset + c * 4, tangents[i * 4 + c] ?? 0, true);+ }if (jointsChannel && weightsChannel) {for (let c = 0; c < 4; c++) {dv.setUint16(at + jointsChannel.offset + c * 2, joints![i * 4 + c] ?? 0, true);diff --git a/pipeline/src/assets/gltfImport.ts b/pipeline/src/assets/gltfImport.tsindex c90a5fe12..808b6f775 100644--- a/pipeline/src/assets/gltfImport.ts+++ b/pipeline/src/assets/gltfImport.ts@@ -14,6 +14,7 @@import { MESH_MAX_BONES, MeshChannel, MeshChannelType, packChannels,type MeshMorphTargets } from 'esengine';import { MeshoptDecoder } from 'meshoptimizer/decoder';+import { generateTangents } from './meshTangents';import {ANIMATED_PATHS, alignQuaternionSigns, animationProductName, disambiguateNodes,nodeChildPaths, nodeNameFor, samplerKeyframes, timelineDocument,@@ -1193,6 +1194,10 @@ export async function importGltfMeshes(catch { warnings.push(`${label}: TEXCOORD_1 is declared and missing from the Draco blob`); }const normalIndex = prim.attributes.NORMAL;const normals = attribute('NORMAL', normalIndex);+ // The baker's own tangents, where the file has them: their direction is kept, the handedness is re-derived.+ let suppliedTangents: Float32Array | null = null;+ try { suppliedTangents = attribute('TANGENT', prim.attributes.TANGENT); }+ catch { warnings.push(`${label}: TANGENT is declared and missing from the Draco blob`); }const colorsRaw = attribute('COLOR_0', colorIndex);const colorComps = colorIndex !== undefined? TYPE_COMPONENTS[json.accessors?.[colorIndex]?.type ?? 'VEC4'] ?? 4 : 4;@@ -1227,12 +1232,19 @@ export async function importGltfMeshes(+ ' — imported static');}+ // The frame the normal map was baked in, from the UVs as stored (V flipped below).+ const tangents = normals && uvs && uvs.length >= vertexCount * 2+ ? generateTangents(positions, normals, Float32Array.from({ length: vertexCount * 2 },+ (_, k) => (k & 1 ? 1 - (uvs[k] ?? 0) : uvs[k] ?? 0)), indices, suppliedTangents) : null;+const { channels, vertexStride } = packChannels([{ semantic: MeshChannel.Position, components: 3, type: MeshChannelType.Float32 },{ semantic: MeshChannel.TexCoord0, components: 2, type: MeshChannelType.Float32 },{ semantic: MeshChannel.Color, components: 4, type: MeshChannelType.UNorm8 },...(normals ? [{ semantic: MeshChannel.Normal, components: 3,type: MeshChannelType.Float32 }] : []),+ ...(tangents ? [{ semantic: MeshChannel.Tangent, components: 4,+ type: MeshChannelType.Float32 }] : []),// Ahead of the skinning pair, which the two lines below find by position....(uvs1 ? [{ semantic: MeshChannel.TexCoord1, components: 2,type: MeshChannelType.Float32 }] : []),@@ -1244,6 +1256,7 @@ export async function importGltfMeshes(const jointsChannel = skinned ? channels[channels.length - 2]! : null;const weightsChannel = skinned ? channels[channels.length - 1]! : null;const uv1Channel = channels.find(c => c.semantic === MeshChannel.TexCoord1) ?? null;+ const tangentChannel = channels.find(c => c.semantic === MeshChannel.Tangent) ?? null;const vertices = new Uint8Array(vertexCount * vertexStride);const dv = new DataView(vertices.buffer);@@ -1282,6 +1295,9 @@ export async function importGltfMeshes(dv.setFloat32(at + channels[3].offset + c * 4, normals[i * 3 + c] ?? 0, true);}}+ if (tangentChannel && tangents) {+ for (let c = 0; c < 4; c++) dv.setFloat32(at + tangentChannel.offset + c * 4, tangents[i * 4 + c] ?? 0, true);+ }if (jointsChannel && weightsChannel) {for (let c = 0; c < 4; c++) {dv.setUint16(at + jointsChannel.offset + c * 2, joints![i * 4 + c] ?? 0, true);diff --git a/pipeline/src/assets/meshTangents.ts b/pipeline/src/assets/meshTangents.tsnew file mode 100644index 000000000..1b5188d24--- /dev/null+++ b/pipeline/src/assets/meshTangents.ts@@ -0,0 +1,87 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file meshTangents.ts+ * @brief The tangent frame an imported mesh carries: xyz along +u, w the handedness of its bitangent.+ *+ * @details A normal map is a vector in a frame — the one the baker used. Deriving that frame per pixel from screen-space+ * derivatives is right on average and wrong at every UV seam and wherever a texture is stretched; carrying it with+ * the vertices reads the map as it was baked. A tangent the source supplies keeps its direction (it is the baker's);+ * one it does not is made from the triangles' own UV derivatives, weighted by the angle each makes at the vertex.+ *+ * The bitangent's SIDE is always taken from the geometry as stored: the shader builds it as `cross(N, T) * w`, and+ * `w` is chosen so that matches the direction the stored v grows. That makes it correct under whatever flip the+ * importer applied to v, which a handedness copied from the source would not be.+ */++/**+ * @param positions 3 floats per vertex.+ * @param normals 3 floats per vertex, unit length.+ * @param uvs 2 floats per vertex, exactly as stored in the mesh (after any v flip).+ * @param indices A triangle list.+ * @param supplied The source's own tangents (3 or 4 floats per vertex), or null to derive them.+ * @returns 4 floats per vertex.+ */+export function generateTangents(+ positions: ArrayLike<number>, normals: ArrayLike<number>, uvs: ArrayLike<number>,+ indices: ArrayLike<number>, supplied: ArrayLike<number> | null = null, suppliedStride = 4,+): Float32Array {+ const count = Math.floor(positions.length / 3);+ const tan = new Float64Array(count * 3);+ const bit = new Float64Array(count * 3);++ for (let t = 0; t + 2 < indices.length; t += 3) {+ const a = indices[t]!, b = indices[t + 1]!, c = indices[t + 2]!;+ if (a >= count || b >= count || c >= count) continue;+ const e1x = positions[b * 3]! - positions[a * 3]!, e1y = positions[b * 3 + 1]! - positions[a * 3 + 1]!, e1z = positions[b * 3 + 2]! - positions[a * 3 + 2]!;+ const e2x = positions[c * 3]! - positions[a * 3]!, e2y = positions[c * 3 + 1]! - positions[a * 3 + 1]!, e2z = positions[c * 3 + 2]! - positions[a * 3 + 2]!;+ const du1 = uvs[b * 2]! - uvs[a * 2]!, dv1 = uvs[b * 2 + 1]! - uvs[a * 2 + 1]!;+ const du2 = uvs[c * 2]! - uvs[a * 2]!, dv2 = uvs[c * 2 + 1]! - uvs[a * 2 + 1]!;+ const det = du1 * dv2 - du2 * dv1;+ if (Math.abs(det) < 1e-20) continue; // a triangle with no UV area says nothing about the frame+ const r = 1 / det;+ const tx = (e1x * dv2 - e2x * dv1) * r, ty = (e1y * dv2 - e2y * dv1) * r, tz = (e1z * dv2 - e2z * dv1) * r;+ const bx = (e2x * du1 - e1x * du2) * r, by = (e2y * du1 - e1y * du2) * r, bz = (e2z * du1 - e1z * du2) * r;+ const tl = Math.hypot(tx, ty, tz), bl = Math.hypot(bx, by, bz);+ if (!(tl > 0) || !(bl > 0)) continue;+ // The angle the triangle makes at each of its corners: a vertex shared by a fan of slivers and one big face is+ // shaped by the face, as its normal is.+ const corner = [a, b, c];+ for (let k = 0; k < 3; k++) {+ const i = corner[k]!, j = corner[(k + 1) % 3]!, l = corner[(k + 2) % 3]!;+ const ux = positions[j * 3]! - positions[i * 3]!, uy = positions[j * 3 + 1]! - positions[i * 3 + 1]!, uz = positions[j * 3 + 2]! - positions[i * 3 + 2]!;+ const vx = positions[l * 3]! - positions[i * 3]!, vy = positions[l * 3 + 1]! - positions[i * 3 + 1]!, vz = positions[l * 3 + 2]! - positions[i * 3 + 2]!;+ const ul = Math.hypot(ux, uy, uz), vl = Math.hypot(vx, vy, vz);+ if (!(ul > 0) || !(vl > 0)) continue;+ const angle = Math.acos(Math.max(-1, Math.min(1, (ux * vx + uy * vy + uz * vz) / (ul * vl))));+ tan[i * 3] += tx / tl * angle; tan[i * 3 + 1] += ty / tl * angle; tan[i * 3 + 2] += tz / tl * angle;+ bit[i * 3] += bx / bl * angle; bit[i * 3 + 1] += by / bl * angle; bit[i * 3 + 2] += bz / bl * angle;+ }+ }++ const out = new Float32Array(count * 4);+ for (let i = 0; i < count; i++) {+ const nx = normals[i * 3]!, ny = normals[i * 3 + 1]!, nz = normals[i * 3 + 2]!;+ let tx = tan[i * 3]!, ty = tan[i * 3 + 1]!, tz = tan[i * 3 + 2]!;+ if (supplied) {+ const sx = supplied[i * suppliedStride]!, sy = supplied[i * suppliedStride + 1]!, sz = supplied[i * suppliedStride + 2]!;+ if (Math.hypot(sx, sy, sz) > 1e-6) { tx = sx; ty = sy; tz = sz; }+ }+ // Square to the normal (Gram-Schmidt): a tangent that leans on it is a shear, not a frame.+ const along = tx * nx + ty * ny + tz * nz;+ tx -= nx * along; ty -= ny * along; tz -= nz * along;+ let len = Math.hypot(tx, ty, tz);+ if (!(len > 1e-8)) {+ // No usable direction (no UV area here): any axis square to the normal keeps the frame well-formed.+ if (Math.abs(nx) < 0.9) { tx = 0; ty = -nz; tz = ny; } else { tx = nz; ty = 0; tz = -nx; }+ len = Math.hypot(tx, ty, tz) || 1;+ }+ tx /= len; ty /= len; tz /= len;+ // The side the stored v grows toward, against cross(N, T): that sign is the handedness.+ const cx = ny * tz - nz * ty, cy = nz * tx - nx * tz, cz = nx * ty - ny * tx;+ const side = cx * bit[i * 3]! + cy * bit[i * 3 + 1]! + cz * bit[i * 3 + 2]!;+ out[i * 4] = tx; out[i * 4 + 1] = ty; out[i * 4 + 2] = tz;+ out[i * 4 + 3] = side < 0 ? -1 : 1;+ }+ return out;+}diff --git a/pipeline/src/assets/modelImport.ts b/pipeline/src/assets/modelImport.tsindex 2023305bf..55c075f45 100644--- a/pipeline/src/assets/modelImport.ts+++ b/pipeline/src/assets/modelImport.ts@@ -71,7 +71,7 @@ export interface ImportedMaterial {/** The tint multiplied into the vertex colors. */baseColor: [number, number, number, number];baseColorTexture?: ImportedImageRef;- /** Tangent-space normal map; the engine derives its tangent frame per pixel. */+ /** Tangent-space normal map, read against the mesh's own tangent frame where it has one (see meshTangents.ts). */normalTexture?: ImportedImageRef;/** Light the surface makes, unaffected by the scene's lights. */emissive?: [number, number, number];diff --git a/pipeline/tests/fbx-import.test.ts b/pipeline/tests/fbx-import.test.tsindex 7a68337ca..92fd4f52e 100644--- a/pipeline/tests/fbx-import.test.ts+++ b/pipeline/tests/fbx-import.test.ts@@ -61,6 +61,15 @@ describe('fbx geometry', () => {expect(attribute(meshes[0]!, MeshChannel.Normal, 0, 3)).toEqual([0, 0, 1]);});+ it('writes the tangent frame the uvs imply, beside the normals', async () => {+ const { meshes } = await importFbxMeshes(texturedTriangle(), 'tri', 'tri.fbx');+ // u grows along +X and v along +Y on a surface facing +Z: T = +X, and cross(N, T) = +Y is the way v grows.+ for (let i = 0; i < 3; i++) {+ const [x, y, z, w] = attribute(meshes[0]!, MeshChannel.Tangent, i, 4) as [number, number, number, number];+ expect([Math.round(x), Math.round(y), Math.round(z), w]).toEqual([1, 0, 0, 1]);+ }+ });+it('triangulates a quad', async () => {const { meshes } = await importFbxMeshes(skinnedBar(), 'bar', 'bar.fbx');expect(meshes[0]!.triangleCount).toBe(2);diff --git a/pipeline/tests/mesh-tangents.test.ts b/pipeline/tests/mesh-tangents.test.tsnew file mode 100644index 000000000..5e5dcc4fe--- /dev/null+++ b/pipeline/tests/mesh-tangents.test.ts@@ -0,0 +1,69 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file The tangent frame an import writes beside the normals.+ *+ * What it must be is geometry, so these are small meshes with a known answer rather than a render: a normal map read against+ * the wrong frame lights the wrong side of every bump, and nothing about that looks like a failure.+ */+import { describe, it, expect } from 'vitest';+import { generateTangents } from '../src/assets/meshTangents';++/** A unit square in the XY plane, facing +Z, two triangles. */+const quad = (uvOf: (x: number, y: number) => [number, number]) => {+ const corners: Array<[number, number]> = [[0, 0], [1, 0], [1, 1], [0, 1]];+ return {+ positions: corners.flatMap(([x, y]) => [x, y, 0]),+ normals: corners.flatMap(() => [0, 0, 1]),+ uvs: corners.flatMap(([x, y]) => uvOf(x, y)),+ indices: [0, 1, 2, 0, 2, 3],+ };+};++const at = (t: Float32Array, i: number) => [...t.subarray(i * 4, i * 4 + 4)].map((v) => Math.round(v * 1e5) / 1e5);++describe('generated tangents', () => {+ it('point along the direction u grows, with a right-handed bitangent where v grows along cross(N, T)', () => {+ const m = quad((x, y) => [x, y]);+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices);+ for (let i = 0; i < 4; i++) expect(at(t, i)).toEqual([1, 0, 0, 1]);+ });++ it('flip the handedness when v grows the other way, as it does after the glTF importer flips v', () => {+ const m = quad((x, y) => [x, 1 - y]);+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices);+ for (let i = 0; i < 4; i++) expect(at(t, i)).toEqual([1, 0, 0, -1]);+ });++ it('follow a rotated unwrap: u growing along +Y gives a +Y tangent', () => {+ const m = quad((x, y) => [y, x]);+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices);+ // u = y, v = x: T = dP/du = +Y; cross(N, T) = (-1, 0, 0) while v grows along +X, so the handedness is -1.+ for (let i = 0; i < 4; i++) expect(at(t, i)).toEqual([0, 1, 0, -1]);+ });++ it("keep a supplied tangent's direction and still take the handedness from the geometry", () => {+ const m = quad((x, y) => [x, 1 - y]);+ const supplied = m.positions.map((_, k) => [1, 0, 0, 1][k % 4]!).slice(0, 16);+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices, supplied, 4);+ for (let i = 0; i < 4; i++) expect(at(t, i)).toEqual([1, 0, 0, -1]);+ });++ it('square a supplied tangent to the normal', () => {+ const m = quad((x, y) => [x, y]);+ const leaning = [1, 0, 0.5, 1, 1, 0, 0.5, 1, 1, 0, 0.5, 1, 1, 0, 0.5, 1];+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices, leaning, 4);+ for (let i = 0; i < 4; i++) expect(at(t, i)).toEqual([1, 0, 0, 1]);+ });++ it('stay a unit frame square to the normal where the UVs give no area to read', () => {+ const m = quad(() => [0.5, 0.5]);+ const t = generateTangents(m.positions, m.normals, m.uvs, m.indices);+ for (let i = 0; i < 4; i++) {+ const [x, y, z, w] = at(t, i) as [number, number, number, number];+ expect(Math.hypot(x, y, z)).toBeCloseTo(1, 4);+ expect(z).toBeCloseTo(0, 5);+ expect(Math.abs(w)).toBe(1);+ }+ });+});diff --git a/sdk/src/render/builtinShaders.ts b/sdk/src/render/builtinShaders.tsindex cb43545b8..8508ef680 100644--- a/sdk/src/render/builtinShaders.ts+++ b/sdk/src/render/builtinShaders.ts@@ -587,6 +587,9 @@ fn packNormal(n : vec3f) -> vec2f {g_probeSlot = v.v_probeSlot;#ifdef MESH_LIGHTMAPg_lightmap = v.v_lightmap;+#endif+#ifdef MESH_TANGENTS+ g_worldTangent = v.v_worldTangent;#endifvar base = textureSampleGrad(t0, s0, v.v_texCoord, duvx, duvy) * v.v_color * mc.u_tint;// Before any discard: a derivative needs every fragment of its quad.@@ -1480,6 +1483,9 @@ fn packNormal(n : vec3f) -> vec2f {g_probeSlot = v.v_probeSlot;#ifdef MESH_LIGHTMAPg_lightmap = v.v_lightmap;+#endif+#ifdef MESH_TANGENTS+ g_worldTangent = v.v_worldTangent;#endifvar base = textureSampleBias(t0, s0, v.v_texCoord, mc.u_sharpen) * v.v_color * mc.u_tint;let flowTex = textureSampleGrad(u_flowMap, u_flowMap_s, v.v_texCoord, duvx, duvy).rg * 2.0 - vec2f(1.0);@@ -1656,6 +1662,9 @@ fn packNormal(n : vec3f) -> vec2f {g_probeSlot = v.v_probeSlot;#ifdef MESH_LIGHTMAPg_lightmap = v.v_lightmap;+#endif+#ifdef MESH_TANGENTS+ g_worldTangent = v.v_worldTangent;#endiflet base = textureSample(t0, s0, v.v_texCoord) * v.v_color * mc.u_tint;let orm = textureSample(u_ormMap, u_ormMap_s, v.v_texCoord);@@ -1712,6 +1721,201 @@ fn packNormal(n : vec3f) -> vec2f {#pragma end`;+const EYE = `#pragma shader "Eye"+#pragma version 300 es+#pragma domain Lit+#pragma param u_tint color default(1,1,1,1)+// The iris is built from two maps and the numbers below: a mask that is white over the iris and the dome of the cornea+// (u_depthMap), and that dome's normals (u_irisNormalMap). A painted eye texture on the mesh multiplies the result.+#pragma param u_depthMap texture default(black) texel(data)+#pragma param u_irisNormalMap texture default(flatnormal) texel(data)+#pragma param u_scleraColor color default(0.92,0.9,0.87,1)+// The iris from the pupil's edge out to the limbus, and how dark its rim is.+#pragma param u_irisColor color default(0.22,0.4,0.52,1)+#pragma param u_irisColor2 color default(0.08,0.17,0.22,1)+#pragma param u_irisSize float default(0.42) range(0.1,0.6) ui(slider)+#pragma param u_pupilSize float default(0.3) range(0,0.8) ui(slider)+#pragma param u_limbal float default(0.7) range(0,1) ui(slider)+// How far behind the cornea the iris lies (in uv), and the cornea's index of refraction, which bend what is seen of it.+#pragma param u_irisDepth float default(0.06) range(0,0.2) ui(slider)+#pragma param u_ior float default(1.336) range(1,1.6) ui(slider)+// The wet cornea is nearly a mirror; the white of the eye is a duller sheen with light that wraps past the terminator.+#pragma param u_corneaRoughness float default(0.04) range(0.01,0.3) ui(slider)+#pragma param u_scleraRoughness float default(0.45) range(0.1,1) ui(slider)+#pragma param u_specular float default(1) range(0,2) ui(slider)+#pragma param u_scleraWrap float default(0.35) range(0,1) ui(slider)++#pragma fragment+precision highp float;++in vec4 v_color;+in vec2 v_texCoord;+in highp vec2 v_worldPos;+#ifdef MESH_NORMALS+in highp vec3 v_worldNormal;+in highp vec3 v_worldXYZ;+#endif+uniform sampler2D u_textures[1];++#ifdef ES_GBUFFER+layout(location = 0) out vec4 fragColor;+layout(location = 1) out vec4 fragSurface;++vec2 packNormal(vec3 n) {+ n /= (abs(n.x) + abs(n.y) + abs(n.z));+ vec2 sgn = vec2(n.x >= 0.0 ? 1.0 : -1.0, n.y >= 0.0 ? 1.0 : -1.0);+ vec2 e = n.z >= 0.0 ? n.xy : (1.0 - abs(n.yx)) * sgn;+ return e * 0.5 + 0.5;+}+#else+out vec4 fragColor;+#endif++void main() {+ vec2 uv = v_texCoord;+ // Every sample that takes part in a derivative is taken here, before any branch.+ float h = texture(u_depthMap, uv).r;+ vec3 dome = normalize(texture(u_irisNormalMap, uv).xyz * 2.0 - 1.0);+#ifdef MESH_NORMALS+ highp vec3 Ng = normalize(v_worldNormal);+ highp vec3 P = v_worldXYZ;+ if (!gl_FrontFacing) Ng = -Ng;+ highp vec3 T = perturbNormal(Ng, P, uv, vec3(1.0, 0.0, 0.0));+ highp vec3 B = perturbNormal(Ng, P, uv, vec3(0.0, 1.0, 0.0));+ highp vec3 Ndome = perturbNormal(Ng, P, uv, dome);+#else+ highp vec3 P = vec3(v_worldPos, 0.0);+ highp vec3 Ng = vec3(0.0, 0.0, 1.0);+ highp vec3 T = vec3(1.0, 0.0, 0.0);+ highp vec3 B = vec3(0.0, 1.0, 0.0);+ highp vec3 Ndome = Ng;+#endif+ vec3 viewDir = viewDirection(P);+ // The ray from the eye, in the surface's own frame, bent into the cornea and carried down to the iris: that is where on+ // the iris this pixel looks, which moves as the eye turns.+ vec3 vt = normalize(vec3(dot(viewDir, T), dot(viewDir, B), max(dot(viewDir, Ng), 0.05)));+ vec3 R = refract(-vt, dome, 1.0 / u_ior);+ vec2 uvI = uv + R.xy * (u_irisDepth * h / max(-R.z, 0.2));+ float m = texture(u_depthMap, uvI).r;+ float irisMask = smoothstep(0.15, 0.85, m);+ float surfaceMask = smoothstep(0.15, 0.85, h);++ vec2 c = uvI - vec2(0.5);+ float rr = length(c) / max(u_irisSize, 1e-3);+ float ang = atan(c.y, c.x);+ float fibers = 0.82 + 0.18 * sin(ang * 70.0 + sin(ang * 13.0 + rr * 9.0) * 2.0);+ vec3 iris = mix(u_irisColor.rgb, u_irisColor2.rgb, smoothstep(0.2, 0.95, rr)) * fibers;+ iris *= 1.0 - u_limbal * smoothstep(0.78, 1.0, rr);+ iris *= smoothstep(u_pupilSize - 0.05, u_pupilSize + 0.03, rr);+ vec4 painted = texture(u_textures[0], uv) * v_color * u_tint;+ vec3 albedo = mix(u_scleraColor.rgb, iris, irisMask) * painted.rgb;++ float ao = 1.0;+ vec3 Nd = normalize(mix(Ng, Ndome, 0.5 * surfaceMask));+ SurfaceExt body = surfaceExtNone(Nd);+ body.scatter = vec4(vec3(0.9, 0.5, 0.4) * albedo, u_scleraWrap * (1.0 - surfaceMask));+ vec3 diffuse = applyLightingSurface(albedo, Nd, P, viewDir, 0.0, 1.0, 0.0, ao, body);++ // The cornea: a clear dome, so its normal is the dome's, and it shows every light and the sky as a small bright shape.+ vec3 Nc = normalize(mix(Ng, Ndome, surfaceMask));+ SurfaceExt shine = surfaceExtNone(Nc);+ shine.coat = vec4(0.25 * (1.0 - surfaceMask), u_scleraRoughness, 0.0, 0.0);+ shine.coatN = vec4(Ng, 0.0);+ vec3 spec = applyLightingSurface(vec3(0.0), Nc, P, viewDir, 0.0, u_corneaRoughness, u_specular, ao, shine);++ fragColor = vec4(diffuse + spec, 1.0);+#ifdef ES_GBUFFER+ fragSurface = vec4(packNormal(normalize(Nc)), u_corneaRoughness, 0.75);+#endif+}+#pragma end++#pragma fragment wgsl+#ifdef ES_GBUFFER+struct FragOut {+ @location(0) color : vec4f,+ @location(1) surface : vec4f,+};++fn packNormal(n : vec3f) -> vec2f {+ let a = n / (abs(n.x) + abs(n.y) + abs(n.z));+ let sgn = vec2f(select(-1.0, 1.0, a.x >= 0.0), select(-1.0, 1.0, a.y >= 0.0));+ var e = a.xy;+ if (a.z < 0.0) { e = (vec2f(1.0) - abs(a.yx)) * sgn; }+ return e * 0.5 + 0.5;+}++@fragment fn fs_main(v : VSOut, @builtin(front_facing) front : bool) -> FragOut {+#else+@fragment fn fs_main(v : VSOut, @builtin(front_facing) front : bool) -> @location(0) vec4f {+#endif+ g_probeSlot = v.v_probeSlot;+#ifdef MESH_LIGHTMAP+ g_lightmap = v.v_lightmap;+#endif+#ifdef MESH_TANGENTS+ g_worldTangent = v.v_worldTangent;+#endif+ let uv = v.v_texCoord;+ // Every sample is taken here, before any branch.+ let h = textureSample(u_depthMap, u_depthMap_s, uv).r;+ let dome = normalize(textureSample(u_irisNormalMap, u_irisNormalMap_s, uv).xyz * 2.0 - vec3f(1.0));+#ifdef MESH_NORMALS+ var Ng = normalize(v.v_worldNormal);+ let P = v.v_worldXYZ;+ if (!front) { Ng = -Ng; }+ let T = perturbNormal(Ng, P, uv, vec3f(1.0, 0.0, 0.0));+ let B = perturbNormal(Ng, P, uv, vec3f(0.0, 1.0, 0.0));+ let Ndome = perturbNormal(Ng, P, uv, dome);+#else+ let P = vec3f(v.v_worldPos.x, 0.0, v.v_worldPos.y);+ let Ng = vec3f(0.0, 0.0, 1.0);+ let T = vec3f(1.0, 0.0, 0.0);+ let B = vec3f(0.0, 1.0, 0.0);+ let Ndome = Ng;+#endif+ let viewDir = viewDirection(P);+ let vt = normalize(vec3f(dot(viewDir, T), dot(viewDir, B), max(dot(viewDir, Ng), 0.05)));+ let R = refract(-vt, dome, 1.0 / mc.u_ior);+ let uvI = uv + R.xy * (mc.u_irisDepth * h / max(-R.z, 0.2));+ let m = textureSample(u_depthMap, u_depthMap_s, uvI).r;+ let irisMask = smoothstep(0.15, 0.85, m);+ let surfaceMask = smoothstep(0.15, 0.85, h);++ let c = uvI - vec2f(0.5);+ let rr = length(c) / max(mc.u_irisSize, 1e-3);+ let ang = atan2(c.y, c.x);+ let fibers = 0.82 + 0.18 * sin(ang * 70.0 + sin(ang * 13.0 + rr * 9.0) * 2.0);+ var iris = mix(mc.u_irisColor.rgb, mc.u_irisColor2.rgb, smoothstep(0.2, 0.95, rr)) * fibers;+ iris = iris * (1.0 - mc.u_limbal * smoothstep(0.78, 1.0, rr));+ iris = iris * smoothstep(mc.u_pupilSize - 0.05, mc.u_pupilSize + 0.03, rr);+ let painted = textureSample(t0, s0, uv) * v.v_color * mc.u_tint;+ let albedo = mix(mc.u_scleraColor.rgb, iris, irisMask) * painted.rgb;++ let ao = 1.0;+ let Nd = normalize(mix(Ng, Ndome, 0.5 * surfaceMask));+ var body = surfaceExtNone(Nd);+ body.scatter = vec4f(vec3f(0.9, 0.5, 0.4) * albedo, mc.u_scleraWrap * (1.0 - surfaceMask));+ let diffuse = applyLightingSurface(albedo, Nd, P, viewDir, 0.0, 1.0, 0.0, ao, body);++ let Nc = normalize(mix(Ng, Ndome, surfaceMask));+ var shine = surfaceExtNone(Nc);+ shine.coat = vec4f(0.25 * (1.0 - surfaceMask), mc.u_scleraRoughness, 0.0, 0.0);+ shine.coatN = vec4f(Ng, 0.0);+ let spec = applyLightingSurface(vec3f(0.0), Nc, P, viewDir, 0.0, mc.u_corneaRoughness, mc.u_specular, ao, shine);++#ifdef ES_GBUFFER+ var frag : FragOut;+ frag.color = vec4f(diffuse + spec, 1.0);+ frag.surface = vec4f(packNormal(normalize(Nc)), mc.u_corneaRoughness, 0.75);+ return frag;+#else+ return vec4f(diffuse + spec, 1.0);+#endif+}+#pragma end+`;+export const BUILTIN_SHADER_TEMPLATES: readonly BuiltinShaderTemplate[] = [template('sprite-unlit', 'Unlit', 'Texture × vertex color × tint, no lighting.', SPRITE_UNLIT),template('sprite-lit', 'Lit', 'Lit by the scene\'s 2D lights; optional normal map.', SPRITE_LIT),@@ -1737,6 +1941,8 @@ export const BUILTIN_SHADER_TEMPLATES: readonly BuiltinShaderTemplate[] = ['Strand lighting: Kajiya-Kay diffuse, two shifted highlights along a flow map, and light through the strands.', HAIR),template('skin', 'Skin','Soft diffuse by colour, a warm terminator, light through thin places, two shine lobes and a fuzz — without blurring the picture.', SKIN),+ template('eye', 'Eye',+ 'A wet cornea over an iris that moves as the eye turns: refracted depth, a pupil, a dark limbal ring, and catchlights of every light.', EYE),];export function builtinShaderTemplate(id: string): BuiltinShaderTemplate | undefined {diff --git a/src/esengine/data/shaders/mesh.esshader b/src/esengine/data/shaders/mesh.esshaderindex ac54f04ab..c1df31196 100644--- a/src/esengine/data/shaders/mesh.esshader+++ b/src/esengine/data/shaders/mesh.esshader@@ -32,6 +32,9 @@// the morph weights below — because a vertex attribute a shader does not declare// may not appear in the layout it is drawn through, which WebGPU rejects.#pragma feature MESH_LIGHTMAP+// A tangent frame the geometry carries (xyz along +u, w the handedness of the bitangent), for a lit mesh with normals: the+// normal map is then read against the frame it was baked in rather than one derived per pixel from screen-space derivatives.+#pragma feature MESH_TANGENTS// The shadow-map pass: the same geometry, from the light, writing depth as colour// instead of shading. A feature and not a second shader, because what has to be@@ -65,6 +68,9 @@ layout(location = 6) in vec4 a_weights;#ifdef MESH_LIGHTMAPlayout(location = 7) in vec2 a_texCoord1;#endif+#ifdef MESH_TANGENTS+layout(location = 4) in vec4 a_tangent;+#endif// The per-object record comes from the frame's texture, not from attributes:// esInstanceModel / esInstanceTint / esInstanceNormalMatrix /@@ -127,6 +133,9 @@ out highp vec3 v_worldNormal;// The full position, not just the plane's: a tangent frame is derived from how// it changes across a fragment, which xy alone cannot say for a tilted surface.out highp vec3 v_worldPos;+#ifdef MESH_TANGENTS+out highp vec4 v_worldTangent;+#endif#endif#ifdef SHADOW_DEPTH// The clip position, and not the depth it divides down to. z/w is not affine across a@@ -243,6 +252,15 @@ void main() {v_worldNormal = vec3(0.0, 0.0, 1.0);#endifv_worldPos = world.xyz;+#ifdef MESH_TANGENTS+#ifdef SKINNED+ v_worldTangent = vec4(mat3(skin) * a_tangent.xyz, a_tangent.w);+#else+ mat3 tangentModel = mat3(esInstanceModel());+ // A mirrored placement turns the bitangent the other way round.+ v_worldTangent = vec4(tangentModel * a_tangent.xyz, determinant(tangentModel) < 0.0 ? -a_tangent.w : a_tangent.w);+#endif+#endif#endif}#pragma end@@ -368,6 +386,9 @@ struct VSIn {#endif#ifdef MESH_LIGHTMAP@location(7) a_texCoord1 : vec2f,+#endif+#ifdef MESH_TANGENTS+ @location(4) a_tangent : vec4f,#endif// What the GLSL twin reads as gl_VertexID: a morph delta is addressed by it.@builtin(vertex_index) vertexIndex : u32,@@ -379,6 +400,9 @@ struct VSOut {#ifdef LIT@location(2) v_worldNormal : vec3f,@location(3) v_worldPos : vec3f,+#ifdef MESH_TANGENTS+ @location(8) v_worldTangent : vec4f,+#endif#endif#ifdef SHADOW_DEPTH// The clip position, not the depth it divides down to — see the GLSL twin.@@ -505,6 +529,15 @@ fn esPrevBone(joint : u32) -> mat4x4f {out.v_worldNormal = vec3f(0.0, 0.0, 1.0);#endifout.v_worldPos = world.xyz;+#ifdef MESH_TANGENTS+#ifdef SKINNED+ out.v_worldTangent = vec4f(mat3x3f(pose[0].xyz, pose[1].xyz, pose[2].xyz) * v.a_tangent.xyz, v.a_tangent.w);+#else+ let tangentModel = mat3x3f(esInstanceModel()[0].xyz, esInstanceModel()[1].xyz, esInstanceModel()[2].xyz);+ out.v_worldTangent = vec4f(tangentModel * v.a_tangent.xyz,+ select(v.a_tangent.w, -v.a_tangent.w, determinant(tangentModel) < 0.0));+#endif+#endif#endifreturn out;}@@ -547,6 +580,9 @@ struct VSOut {#ifdef LIT@location(2) v_worldNormal : vec3f,@location(3) v_worldPos : vec3f,+#ifdef MESH_TANGENTS+ @location(8) v_worldTangent : vec4f,+#endif#endif#ifdef SHADOW_DEPTH@location(4) v_shadowClip : vec4f,@@ -588,6 +624,11 @@ fn packNormal(n : vec3f) -> vec2f {#ifdef MESH_LIGHTMAPg_lightmap = v.v_lightmap;#endif+#ifdef MESH_TANGENTS+#ifdef LIT+ g_worldTangent = v.v_worldTangent;+#endif+#endif#ifdef ES_VELOCITY// See the GLSL twin. This device's uv runs down the rows, so the sign of y is the reader's to flip.let now = v.v_velCur.xy / v.v_velCur.w;diff --git a/src/esengine/renderer/RenderTypePlugin.hpp b/src/esengine/renderer/RenderTypePlugin.hppindex 2b92e9663..e07b9eb8b 100644--- a/src/esengine/renderer/RenderTypePlugin.hpp+++ b/src/esengine/renderer/RenderTypePlugin.hpp@@ -153,6 +153,8 @@ struct MeshProgramFacts {* set, and is not posed by bones. Whether it HAS one is the atlas rectangle* in its record, which is a number rather than a program. */bool lightmapped = false;+ /** Whether the geometry carries a tangent frame (with its normals): the program then declares and reads it. */+ bool tangents = false;u32 materialId = 0;};@@ -184,8 +186,8 @@ struct RenderPrewarmResult {/** @brief What a material program is asked for, as one comparable value. */static u64 materialKey(u32 materialId, bool normals, bool skinned, bool envMapped,- bool lightmapped) {- return (static_cast<u64>(materialId) << 4) | (lightmapped ? 8u : 0u) | (normals ? 4u : 0u)+ bool lightmapped, bool tangents = false) {+ return (static_cast<u64>(materialId) << 5) | (tangents ? 16u : 0u) | (lightmapped ? 8u : 0u) | (normals ? 4u : 0u)| (skinned ? 2u : 0u) | (envMapped ? 1u : 0u);}diff --git a/src/esengine/renderer/plugins/MeshPlugin.cpp b/src/esengine/renderer/plugins/MeshPlugin.cppindex cf9ff458f..1c5d5f83a 100644--- a/src/esengine/renderer/plugins/MeshPlugin.cpp+++ b/src/esengine/renderer/plugins/MeshPlugin.cpp@@ -97,6 +97,7 @@ MeshProgramFacts meshFactsFromDocument(RenderFrameContext& ctx, const MeshDocume}facts.skinned = resident && skinnedBy(*resident, row.jointCount, warnedBones) > 0;facts.lightmapped = resident && resident->hasLightmapUV;+ facts.tangents = resident && resident->hasTangents;return facts;}@@ -248,11 +249,11 @@ void reportLodAuthoring(const LodLevels& levels, bool& warned) {u32 meshVariant(bool normals, bool lit, bool normalMapped, bool skinned, bool depthOnly,bool envMapped, bool lightmapped, bool weightHeat = false, bool gbuffer = false,- bool motion = false) {+ bool motion = false, bool tangents = false) {return (normals ? 1u : 0u) | (lit ? 2u : 0u) | (normalMapped ? 4u : 0u)| (skinned ? 8u : 0u) | (depthOnly ? 16u : 0u) | (envMapped ? 32u : 0u)| (lightmapped ? 64u : 0u) | (weightHeat ? 128u : 0u) | (gbuffer ? 256u : 0u)- | (motion ? 512u : 0u);+ | (motion ? 512u : 0u) | (tangents ? 1024u : 0u);}/**@@ -303,9 +304,7 @@ struct MeshDrawKeys {u32 meshVariantFrom(const MeshProgramFacts& facts, bool envMapped, bool shadowDepth, bool weightHeat = false,bool gbuffer = false) {- // `lightmapped` survives the shadow pass, unlike the three describing- // shading: it declares two vertex attributes, and a layout may not carry one- // the program ignores. The depth fragment simply never reads the atlas.+ // `lightmapped` and `tangents` survive the shadow pass, unlike the three describing shading: each declares a vertex attribute.// Weights are read off the surface of a skinned draw and nothing else's: the colour is// the shade, so it asks for no light, no normal map and no depth pass of its own.const bool heat = weightHeat && facts.skinned && !shadowDepth;@@ -313,7 +312,7 @@ u32 meshVariantFrom(const MeshProgramFacts& facts, bool envMapped, bool shadowDe// The surface record is for what takes light: an unlit surface has no normal worth recording.return meshVariant(facts.hasNormals, lit,facts.normalTextureId != 0 && !shadowDepth && !heat, facts.skinned, shadowDepth,- envMapped, facts.lightmapped, heat, gbuffer && lit);+ envMapped, facts.lightmapped, heat, gbuffer && lit, false, facts.tangents);}/** @brief The facts a LIVE entity answers with: its components and its mesh. */@@ -332,6 +331,7 @@ MeshProgramFacts meshFactsFromEntity(RenderFrameContext& ctx, ecs::Registry& reg}facts.skinned = resident && skinJointCount(registry, entity, *resident, warnedBones) > 0;facts.lightmapped = resident && resident->hasLightmapUV;+ facts.tangents = resident && resident->hasTangents;return facts;}@@ -381,6 +381,7 @@ u32 MeshPlugin::meshProgram(RenderFrameContext& ctx, u32 variant) {const bool depthOnly = (variant & 16u) != 0, envMapped = (variant & 32u) != 0;const bool lightmapped = (variant & 64u) != 0, weightHeat = (variant & 128u) != 0;const bool gbuffer = (variant & 256u) != 0, motion = (variant & 512u) != 0;+ const bool tangents = (variant & 1024u) != 0;if (mesh_compiled_[variant]) return mesh_programs_[variant];mesh_compiled_[variant] = true;// Counted because the cost of this function is entirely the times it does@@ -396,6 +397,7 @@ u32 MeshPlugin::meshProgram(RenderFrameContext& ctx, u32 variant) {if (normalMapped) features.emplace_back("NORMAL_MAP");if (skinned) features.emplace_back("SKINNED");if (lightmapped) features.emplace_back("MESH_LIGHTMAP");+ if (tangents && normals) features.emplace_back("MESH_TANGENTS");if (weightHeat) features.emplace_back("WEIGHT_HEAT");if (depthOnly) features.emplace_back("SHADOW_DEPTH");if (motion) features.emplace_back("ES_VELOCITY");@@ -469,9 +471,9 @@ RenderPrewarmResult MeshPlugin::prewarmFacts(RenderFrameContext& ctx,++out.materialAsks;out.addMaterialKey(RenderPrewarmResult::materialKey(facts[i].materialId, facts[i].hasNormals, facts[i].skinned, envMapped,- facts[i].lightmapped));+ facts[i].lightmapped, facts[i].tangents));ctx.materials->meshProgram(facts[i].materialId, ctx.resources, facts[i].hasNormals,- facts[i].skinned, envMapped, facts[i].lightmapped);+ facts[i].skinned, envMapped, facts[i].lightmapped, false, facts[i].tangents);}}if (ctx.materials) {@@ -708,7 +710,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {meshFactsFromEntity(ctx, registry, entity, mesh, resident, warned_bones_);// A palette of a different size than last frame's has no joint-for-joint past to follow.if (facts.skinned && pose->second.bonesPrevious.size() != pose->second.bonesCurrent.size()) continue;- keys = { meshVariant(facts.hasNormals, false, false, facts.skinned, false, false, false, false, false, true), 0 };+ keys = { meshVariant(facts.hasNormals, false, false, facts.skinned, false, false, false, false, false, true, facts.tangents), 0 };} else {keys = meshDrawFor(ctx, registry, entity, mesh, resident, shadowDepth, warned_bones_, heated);}@@ -909,7 +911,8 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {resident->hasNormals, skinned,key.envTextureId != 0,resident->hasLightmapUV,- ctx.gbuffer && !shadowDepth);+ ctx.gbuffer && !shadowDepth,+ resident->hasTangents);if (materialProgram == 0) {if (!warned_material_) {warned_material_ = true;@@ -924,7 +927,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {// material's hook, and the draw keeps the material so its params are bound.if (shadowDepth && material && ctx.materials && ctx.materials->hasVertexHook(mesh.material)) {const u32 bentDepth = ctx.materials->shadowProgram(mesh.material, ctx.resources,- resident->hasNormals, skinned);+ resident->hasNormals, skinned, resident->hasTangents);if (bentDepth != 0) {key.shaderId = bentDepth;key.materialId = mesh.material;diff --git a/src/esengine/renderer/rhi/GfxEnums.hpp b/src/esengine/renderer/rhi/GfxEnums.hppindex 432a385c3..57c20deb4 100644--- a/src/esengine/renderer/rhi/GfxEnums.hpp+++ b/src/esengine/renderer/rhi/GfxEnums.hpp@@ -140,11 +140,10 @@ static constexpr u32 MORPH_TEXTURE_WIDTH = 2048;/// Beyond it the geometry draws unmorphed — a texture the device refuses is a/// mesh that does not draw at all.static constexpr u32 MORPH_MAX_TEXELS = MORPH_TEXTURE_WIDTH * MORPH_TEXTURE_WIDTH;-/// Programs the resident-mesh shader has: one per combination of {normals, lit,-/// normal-mapped, skinned, depth-only, env-mapped, lightmapped, weight-heat,-/// writes-the-surface-record, writes-the-motion}. Ten bits, not one: the geometry, the draw, the frame-/// and the pass each answer a different.-static constexpr u32 MESH_VARIANTS = 1024;+/// Programs the resident-mesh shader has: one per combination of {normals, lit, normal-mapped, skinned, depth-only,+/// env-mapped, lightmapped, weight-heat, writes-the-surface-record, writes-the-motion, tangents}: eleven bits, because+/// the geometry, the draw, the frame and the pass each answer a different one.+static constexpr u32 MESH_VARIANTS = 2048;/*** @brief What a mesh vertex channel MEANS, and the attribute location it binds to.* @details One vocabulary for the file format, the layout and the shaders: a@@ -167,13 +166,13 @@ enum class MeshChannel : u8 {/*** @brief Whether the mesh shaders READ a channel, as opposed to the file carrying it.- * @details An attribute nothing reads spends a slot and a fetch per vertex, so a- * channel kept for its information stays out of the LAYOUT. TexCoord1- * left this list when a bake gave it a reader.- * check-mesh-shader-inputs holds this both ways.+ * @details An attribute nothing reads spends a slot and a fetch per vertex, so a channel kept for its information+ * stays out of the LAYOUT. Every channel the format names is read now; the predicate stays so the next one can+ * wait for its reader. check-mesh-shader-inputs holds this both ways.*/constexpr bool meshShaderReads(MeshChannel c) {- return c != MeshChannel::Tangent;+ (void)c;+ return true;}/**diff --git a/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp b/src/esengine/renderer/rhi/ShaderEmbeds.generated.hppindex fbad7880e..feb15dafa 100644--- a/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp+++ b/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp@@ -509,6 +509,9 @@ inline constexpr const char* MESH = R"esshader(#pragma shader "Mesh"#pragma feature MESH_LIGHTMAP+#pragma feature MESH_TANGENTS++@@ -540,6 +543,9 @@ layout(location = 6) in vec4 a_weights;#ifdef MESH_LIGHTMAPlayout(location = 7) in vec2 a_texCoord1;#endif+#ifdef MESH_TANGENTS+layout(location = 4) in vec4 a_tangent;+#endif@@ -602,6 +608,9 @@ out highp vec3 v_worldNormal;out highp vec3 v_worldPos;+#ifdef MESH_TANGENTS+out highp vec4 v_worldTangent;+#endif#endif#ifdef SHADOW_DEPTH@@ -718,6 +727,15 @@ void main() {v_worldNormal = vec3(0.0, 0.0, 1.0);#endifv_worldPos = world.xyz;+#ifdef MESH_TANGENTS+#ifdef SKINNED+ v_worldTangent = vec4(mat3(skin) * a_tangent.xyz, a_tangent.w);+#else+ mat3 tangentModel = mat3(esInstanceModel());++ v_worldTangent = vec4(tangentModel * a_tangent.xyz, determinant(tangentModel) < 0.0 ? -a_tangent.w : a_tangent.w);+#endif+#endif#endif}#pragma end@@ -844,6 +862,9 @@ struct VSIn {#ifdef MESH_LIGHTMAP@location(7) a_texCoord1 : vec2f,#endif+#ifdef MESH_TANGENTS+ @location(4) a_tangent : vec4f,+#endif@builtin(vertex_index) vertexIndex : u32,};@@ -854,6 +875,9 @@ struct VSOut {#ifdef LIT@location(2) v_worldNormal : vec3f,@location(3) v_worldPos : vec3f,+#ifdef MESH_TANGENTS+ @location(8) v_worldTangent : vec4f,+#endif#endif#ifdef SHADOW_DEPTH@@ -980,6 +1004,15 @@ fn esPrevBone(joint : u32) -> mat4x4f {out.v_worldNormal = vec3f(0.0, 0.0, 1.0);#endifout.v_worldPos = world.xyz;+#ifdef MESH_TANGENTS+#ifdef SKINNED+ out.v_worldTangent = vec4f(mat3x3f(pose[0].xyz, pose[1].xyz, pose[2].xyz) * v.a_tangent.xyz, v.a_tangent.w);+#else+ let tangentModel = mat3x3f(esInstanceModel()[0].xyz, esInstanceModel()[1].xyz, esInstanceModel()[2].xyz);+ out.v_worldTangent = vec4f(tangentModel * v.a_tangent.xyz,+ select(v.a_tangent.w, -v.a_tangent.w, determinant(tangentModel) < 0.0));+#endif+#endif#endifreturn out;}@@ -1022,6 +1055,9 @@ struct VSOut {#ifdef LIT@location(2) v_worldNormal : vec3f,@location(3) v_worldPos : vec3f,+#ifdef MESH_TANGENTS+ @location(8) v_worldTangent : vec4f,+#endif#endif#ifdef SHADOW_DEPTH@location(4) v_shadowClip : vec4f,@@ -1063,6 +1099,11 @@ fn packNormal(n : vec3f) -> vec2f {#ifdef MESH_LIGHTMAPg_lightmap = v.v_lightmap;#endif+#ifdef MESH_TANGENTS+#ifdef LIT+ g_worldTangent = v.v_worldTangent;+#endif+#endif#ifdef ES_VELOCITYlet now = v.v_velCur.xy / v.v_velCur.w;diff --git a/src/esengine/renderer/store/MaterialStore.cpp b/src/esengine/renderer/store/MaterialStore.cppindex 79e3042a8..ee4d3e6a9 100644--- a/src/esengine/renderer/store/MaterialStore.cpp+++ b/src/esengine/renderer/store/MaterialStore.cpp@@ -67,7 +67,7 @@ void MaterialStore::clear() {u32 MaterialStore::meshProgram(u32 materialId, resource::ResourceManager& resources,bool withNormals, bool skinned, bool envMapped,- bool lightmapped, bool gbuffer) const {+ bool lightmapped, bool gbuffer, bool tangents) const {// Only for a shader that has something to write: one that never mentions the feature would// compile the same program twice and then be declared to write an output it does not have.if (gbuffer) {@@ -81,11 +81,13 @@ u32 MaterialStore::meshProgram(u32 materialId, resource::ResourceManager& resour| (skinned ? (1ull << 33) : 0ull)| (envMapped ? (1ull << 34) : 0ull)| (lightmapped ? (1ull << 36) : 0ull)- | (gbuffer ? (1ull << 38) : 0ull);+ | (gbuffer ? (1ull << 38) : 0ull)+ | (tangents && withNormals ? (1ull << 39) : 0ull);std::vector<std::string> features{ "MESH" };if (withNormals) features.push_back("MESH_NORMALS");if (skinned) features.push_back("SKINNED");if (lightmapped) features.push_back("MESH_LIGHTMAP");+ if (tangents && withNormals) features.push_back("MESH_TANGENTS");// Resident geometry carries the frame's shadow map on its own slot 2 and the// reflection on slot 3; the batch variant of the same material must not, which// is why these ride the mesh path.@@ -101,10 +103,11 @@ u32 MaterialStore::meshProgram(u32 materialId, resource::ResourceManager& resour}u32 MaterialStore::shadowProgram(u32 materialId, resource::ResourceManager& resources,- bool withNormals, bool skinned) const {+ bool withNormals, bool skinned, bool tangents) const {const MaterialRecord* rec = find(materialId);if (!rec || !rec->shaderRef.isValid()) return 0;- const u64 shape = (1ull << 37) | (withNormals ? (1ull << 32) : 0ull) | (skinned ? (1ull << 33) : 0ull);+ const u64 shape = (1ull << 37) | (withNormals ? (1ull << 32) : 0ull) | (skinned ? (1ull << 33) : 0ull)+ | (tangents && withNormals ? (1ull << 39) : 0ull);auto& variants = mesh_programs_[rec->shaderRef.id()];auto cached = variants.find(shape);if (cached != variants.end()) {@@ -132,6 +135,7 @@ u32 MaterialStore::shadowProgram(u32 materialId, resource::ResourceManager& resostd::vector<std::string> features;if (withNormals) features.emplace_back("MESH_NORMALS");+ if (tangents && withNormals) features.emplace_back("MESH_TANGENTS");if (skinned) features.emplace_back("SKINNED");features.emplace_back("SHADOW_DEPTH");features.emplace_back("ES_MATERIAL_VERTEX");diff --git a/src/esengine/renderer/store/MaterialStore.hpp b/src/esengine/renderer/store/MaterialStore.hppindex b49ba2849..ec7b1d5a6 100644--- a/src/esengine/renderer/store/MaterialStore.hpp+++ b/src/esengine/renderer/store/MaterialStore.hpp@@ -195,7 +195,8 @@ public:/// caller falls back rather than drawing it wrong.u32 meshProgram(u32 materialId, resource::ResourceManager& resources,bool withNormals = false, bool skinned = false,- bool envMapped = false, bool lightmapped = false, bool gbuffer = false) const;+ bool envMapped = false, bool lightmapped = false, bool gbuffer = false,+ bool tangents = false) const;/// Whether @p programId is a variant that writes the surface record's second output — set when/// it was compiled for a pass that has one AND the material's shader has something to write@@ -226,7 +227,7 @@ public:/// shadow of the bent shape. 0 when the source was never kept or the variant fails — the caller/// then casts the unbent shape, which is the old behaviour.u32 shadowProgram(u32 materialId, resource::ResourceManager& resources,- bool withNormals, bool skinned) const;+ bool withNormals, bool skinned, bool tangents = false) const;/// The program that draws @p materialId on PARTICLE geometry — the instanced/// quad an emitter streams. Same source, same author's fragment, a differentdiff --git a/src/esengine/resource/Mesh.hpp b/src/esengine/resource/Mesh.hppindex 898cd18ad..1f6dcda3e 100644--- a/src/esengine/resource/Mesh.hpp+++ b/src/esengine/resource/Mesh.hpp@@ -88,6 +88,10 @@ public:* the geometry owns decides the record's shape and the variant. */bool hasLightmapUV = false;+ /** Whether the vertices carry a tangent (xyz along +u, w the bitangent's handedness) beside the normal — the frame a+ * normal map was baked in. Without one the fragment derives a frame from screen-space derivatives. */+ bool hasTangents = false;+/** Local-space bounds of the vertices, for the frustum cull. */glm::vec3 localMin{0.0f, 0.0f, 0.0f};glm::vec3 localMax{0.0f, 0.0f, 0.0f};diff --git a/src/esengine/resource/ResourceManager.cpp b/src/esengine/resource/ResourceManager.cppindex 230572d3d..005b56143 100644--- a/src/esengine/resource/ResourceManager.cpp+++ b/src/esengine/resource/ResourceManager.cpp@@ -572,10 +572,12 @@ bool ResourceManager::realizeMesh(Mesh& mesh, ConstSpan<u8> vertexBytes, ConstSpbool hasNormals = false;bool skinned = false;bool lightmapUV = false;+ bool tangentChannel = false;for (const GfxVertexAttribute& c : channels) {if (c.location == static_cast<u32>(MeshChannel::Normal)) hasNormals = true;if (c.location == static_cast<u32>(MeshChannel::Joints)) skinned = true;if (c.location == static_cast<u32>(MeshChannel::TexCoord1)) lightmapUV = true;+ if (c.location == static_cast<u32>(MeshChannel::Tangent)) tangentChannel = true;}skinned = skinned && !inverseBind.empty();// Bones move a character and a bake cannot hold one still, so a skinned mesh@@ -646,6 +648,8 @@ bool ResourceManager::realizeMesh(Mesh& mesh, ConstSpan<u8> vertexBytes, ConstSpmesh.indexCount = static_cast<u32>(indices.size());mesh.hasNormals = hasNormals;mesh.hasLightmapUV = lightmapped;+ // A tangent frame is a frame OF the normal: one without a normal to square it to is not read.+ mesh.hasTangents = tangentChannel && hasNormals;mesh.localMin = localMin;mesh.localMax = localMax;mesh.inverseBind.clear();diff --git a/src/esengine/resource/ShaderParser.cpp b/src/esengine/resource/ShaderParser.cppindex 96e33b885..4eae76aaf 100644--- a/src/esengine/resource/ShaderParser.cpp+++ b/src/esengine/resource/ShaderParser.cpp@@ -213,6 +213,10 @@ std::string wgslCanonicalVSOut(bool lit) {" @location(3) v_worldNormal : vec3f,\n"" @location(4) v_worldXYZ : vec3f,\n""#endif\n"+ // The geometry's own tangent frame (xyz along +u, w the bitangent's handedness), when the mesh carries one.+ "#ifdef MESH_TANGENTS\n"+ " @location(7) v_worldTangent : vec4f,\n"+ "#endif\n"// mesh.esshader's twin of the same varying, carried through a material's// variant so the LAYOUT matches. What a material's fragment makes of it// is the material's, as all of its shading is.@@ -273,6 +277,9 @@ std::string canonicalVertexStageWGSL(bool lit, const std::string& hook = {}) {"#ifdef MESH_LIGHTMAP\n"" @location(7) a_texCoord1 : vec2f,\n""#endif\n"+ "#ifdef MESH_TANGENTS\n"+ " @location(4) a_tangent : vec4f,\n"+ "#endif\n""#ifdef SKINNED\n"" @location(5) a_joints : vec4u,\n"" @location(6) a_weights : vec4f,\n"@@ -355,6 +362,15 @@ std::string canonicalVertexStageWGSL(bool lit, const std::string& hook = {}) {" out.v_worldNormal = esInstanceNormalMatrix() * localNormal;\n""#endif\n"" out.v_worldXYZ = world.xyz;\n"+ "#ifdef MESH_TANGENTS\n"+ "#ifdef SKINNED\n"+ " out.v_worldTangent = vec4f(mat3x3f(pose[0].xyz, pose[1].xyz, pose[2].xyz) * v.a_tangent.xyz, v.a_tangent.w);\n"+ "#else\n"+ " let tangentModel = mat3x3f(esInstanceModel()[0].xyz, esInstanceModel()[1].xyz, esInstanceModel()[2].xyz);\n"+ " out.v_worldTangent = vec4f(tangentModel * v.a_tangent.xyz,\n"+ " select(v.a_tangent.w, -v.a_tangent.w, determinant(tangentModel) < 0.0));\n"+ "#endif\n"+ "#endif\n""#endif\n""#ifdef MESH_LIGHTMAP\n"" let lmRect = esInstanceLightmapRect();\n"@@ -453,6 +469,9 @@ std::string canonicalVertexStage(bool lit, const std::string& hook = {}) {"#ifdef MESH_LIGHTMAP\n""layout(location = 7) in vec2 a_texCoord1;\n""#endif\n"+ "#ifdef MESH_TANGENTS\n"+ "layout(location = 4) in vec4 a_tangent;\n"+ "#endif\n""#ifdef SKINNED\n""layout(location = 5) in uvec4 a_joints;\n""layout(location = 6) in vec4 a_weights;\n"@@ -492,6 +511,9 @@ std::string canonicalVertexStage(bool lit, const std::string& hook = {}) {"#ifdef MESH_NORMALS\n""out highp vec3 v_worldNormal;\n""out highp vec3 v_worldXYZ;\n"+ "#ifdef MESH_TANGENTS\n"+ "out highp vec4 v_worldTangent;\n"+ "#endif\n""#endif\n""#ifdef MESH_LIGHTMAP\n""out highp vec3 v_lightmap;\n"@@ -564,6 +586,15 @@ std::string canonicalVertexStage(bool lit, const std::string& hook = {}) {" v_worldNormal = esInstanceNormalMatrix() * localNormal;\n""#endif\n"" v_worldXYZ = world.xyz;\n"+ "#ifdef MESH_TANGENTS\n"+ "#ifdef SKINNED\n"+ " v_worldTangent = vec4(mat3(skin) * a_tangent.xyz, a_tangent.w);\n"+ "#else\n"+ " mat3 tangentModel = mat3(esInstanceModel());\n"+ // A mirrored placement turns the bitangent the other way round.+ " v_worldTangent = vec4(tangentModel * a_tangent.xyz, determinant(tangentModel) < 0.0 ? -a_tangent.w : a_tangent.w);\n"+ "#endif\n"+ "#endif\n""#endif\n""#ifdef MESH_LIGHTMAP\n"" highp vec4 lmRect = esInstanceLightmapRect();\n"@@ -1225,6 +1256,9 @@ struct PlanarConstants { planarProjection : mat4x4f, planarParams : vec4f };var<private> g_probeSlot : f32 = 0.0;// The GLSL twin's v_lightmap, handed over by the entry point like the slot above.var<private> g_lightmap : vec3f = vec3f(0.0);+// The mesh's own tangent frame, handed over by the entry point like the slot above; zero where the shader never sets it, which+// is what sends perturbNormal back to deriving a frame per pixel.+var<private> g_worldTangent : vec4f = vec4f(0.0);fn packDepth(d : f32) -> vec3f {let enc = fract(d * vec3f(1.0, 255.0, 65025.0));return enc - enc.yzz * vec3f(1.0 / 255.0, 1.0 / 255.0, 0.0);@@ -1386,6 +1420,7 @@ fn sampleNormalGrad(map : texture_2d<f32>, samp : sampler, uv : vec2f,return vec3f(xy, sqrt(max(0.0, 1.0 - dot(xy, xy))));}fn perturbNormal(N : vec3f, worldPos : vec3f, uv : vec2f, tangentNormal : vec3f) -> vec3f {+ // The derivatives first: they are only valid in uniform control flow, and the choice below depends on a variable.let dp1 = dpdx(worldPos);// Negated: this backend's framebuffer y runs the other way, so dpdy answers// with the opposite sign to GLSL's dFdy. Left alone it flips T and B, and a@@ -1393,6 +1428,12 @@ fn perturbNormal(N : vec3f, worldPos : vec3f, uv : vec2f, tangentNormal : vec3f)let dp2 = -dpdy(worldPos);let duv1 = dpdx(uv);let duv2 = -dpdy(uv);+ // The frame the normal map was baked in, when the geometry carries it: the tangent made square to the interpolated normal.+ if (dot(g_worldTangent.xyz, g_worldTangent.xyz) > 1e-8) {+ let Tg = normalize(g_worldTangent.xyz - N * dot(N, g_worldTangent.xyz));+ let Bg = cross(N, Tg) * g_worldTangent.w;+ return normalize(mat3x3f(Tg, Bg, N) * tangentNormal);+ }let dp2perp = cross(dp2, N);let dp1perp = cross(N, dp1);let T = dp2perp * duv1.x + dp1perp * duv2.x;@@ -2070,6 +2111,12 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p&& std::find(features.begin(), features.end(), "ES_LINEAR") == features.end()) {features.push_back("ES_LINEAR");}+ // A tangent frame arrives through a varying only a vertex stage that knows about it writes: the engine's own canonical one,+ // or a shader that declares the feature. Any other keeps reading its frame from derivatives, as it always did.+ if (!parsed.vertexIsCanonical+ && std::find(parsed.features.begin(), parsed.features.end(), "MESH_TANGENTS") == parsed.features.end()) {+ features.erase(std::remove(features.begin(), features.end(), std::string("MESH_TANGENTS")), features.end());+ }if (target == ShaderTargetLanguage::WGSL) {return assembleWGSLStage(parsed, stage, features);}@@ -2441,8 +2488,17 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p// Tangent frame from screen-space derivatives, so a tangent-space normal// lands in world space without the geometry carrying a tangent channel —// which most exported models do not. Degenerate UVs return N unchanged.+ "#ifdef MESH_TANGENTS\n"+ "in highp vec4 v_worldTangent;\n"+ "#endif\n""highp vec3 perturbNormal(in highp vec3 N, in highp vec3 worldPos,\n"" in highp vec2 uv, in highp vec3 tangentNormal) {\n"+ "#ifdef MESH_TANGENTS\n"+ // The frame the normal map was baked in: the tangent made square to the interpolated normal.+ " highp vec3 Tg = normalize(v_worldTangent.xyz - N * dot(N, v_worldTangent.xyz));\n"+ " highp vec3 Bg = cross(N, Tg) * v_worldTangent.w;\n"+ " return normalize(mat3(Tg, Bg, N) * tangentNormal);\n"+ "#else\n"" highp vec3 dp1 = dFdx(worldPos);\n"" highp vec3 dp2 = dFdy(worldPos);\n"" highp vec2 duv1 = dFdx(uv);\n"@@ -2457,6 +2513,7 @@ ShaderParser::AssembledStage ShaderParser::assembleStageEx(const ParsedShader& p" highp float scale = max(dot(dp1, dp1), dot(dp2, dp2)) * max(dot(duv1, duv1), dot(duv2, duv2));\n"" if (!(m > scale * 1e-6)) return N;\n"" return normalize(mat3(T * inversesqrt(max(dot(T, T), 1e-20)), B * inversesqrt(max(dot(B, B), 1e-20)), N) * tangentNormal);\n"+ "#endif\n""}\n"// Where a world point lands in this camera's part of a screen-space mask.// Measured on the Z = 0 plane, which is the plane a 2D occluder hides thingsdiff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex ba1984f80..9c1ee6305 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -2537,6 +2537,10 @@ export const NOTED = {{"note":"A WeChat or Douyin package whose scenes and scripts use no particle emitter, post-process volume or tilemap now ships a lite engine, the same engine without those three subsystems and about 200 KB smaller, so a UI-only game has room under the 4 MB main-package limit."},"fix(launcher): switching template renames the suggested project name, unless the user typed one":{"note":"The New Project page no longer keeps the previous template's name when you pick another: the suggested name follows the template, and a name you typed stays."},+ "feat(render): imported meshes carry a tangent frame, so a normal map is read against the frame it was baked in":+ {"note":"Imported meshes now carry a tangent at each vertex (a glTF's own direction, or one made from the UVs), and normal maps on them are read against that frame instead of one derived per pixel, which was wrong at UV seams and on stretched textures. Re-import a model to get them; one imported before keeps its old shading."},+ "feat(render): an Eye shader draws the iris behind a refracting cornea, with a pupil, a limbal ring and a catchlight":+ {"note":"A new Eye shader (`builtin:eye`) draws an eye from a mask and a dome normal map: the iris sits behind a refracting cornea so it moves as the eye turns, with a pupil, fibres and a limbal ring, a white lit with wrapped light, and a near-mirror cornea that shows every light as a catchlight."},"chore(tools): the gates run in parallel with --jobs, and the reds a full run gathered are cleared":{"internal":"tooling for the static gates, and fixes to checks that had drifted from the code they guard"},"fix(render): a wire narrower than a pixel is no longer painted over by the sky in dashes":diff --git a/tools/renderScenes.mjs b/tools/renderScenes.mjsindex a6297ff9b..a0194df57 100644--- a/tools/renderScenes.mjs+++ b/tools/renderScenes.mjs@@ -755,6 +755,9 @@ export const SCENES = [// both halves take the same light. Its texels point -X and +X, so the lit half// and the unlit one are the tangent frame, derived per pixel from derivatives.{ id: "mesh-normalmap", tier: "pr", webgpu: true, env: { ESTELLA_VERIFY_PREFAB: "/scenes/normalmap-quad.esprefab", ESTELLA_VERIFY_SCENE: "/scenes/mesh-normalmap.esscene", ESTELLA_VERIFY_MANIFEST: "/scenes/mesh-normalmap.textures.json", ESTELLA_VERIFY_W: "256", ESTELLA_VERIFY_H: "256", ESTELLA_VERIFY_STEPS: "4", ESTELLA_VERIFY_EXPECT: "[{\"x\":0.375,\"y\":0.5,\"rgb\":[180,180,180],\"tol\":45},{\"x\":0.625,\"y\":0.5,\"rgb\":[0,0,0],\"tol\":30}]" } },+ // The quad of mesh-normalmap with a tangent a quarter turn from the one its UVs imply: read against the frame the mesh+ // carries, both halves come out alike (33), where the derived frame lights them 180 and 0.+ { id: "mesh-normalmap-tangent", tier: "pr", webgpu: true, env: { ESTELLA_VERIFY_PREFAB: "/scenes/normalmap-quad-tangent.esprefab", ESTELLA_VERIFY_SCENE: "/scenes/mesh-normalmap-tangent.esscene", ESTELLA_VERIFY_MANIFEST: "/scenes/mesh-normalmap-tangent.textures.json", ESTELLA_VERIFY_W: "256", ESTELLA_VERIFY_H: "256", ESTELLA_VERIFY_STEPS: "4", ESTELLA_VERIFY_EXPECT: "[{\"x\":0.375,\"y\":0.5,\"rgb\":[33,33,33],\"tol\":12},{\"x\":0.625,\"y\":0.5,\"rgb\":[33,33,33],\"tol\":12}]" } },// The same quad ten thousand times smaller, seen as large. A tangent frame from// derivatives shrinks with the fourth power of scale, so a degenerate-UV floor// fixed in world units switched normal maps off on anything small up close.