Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): model highlights widen with how fast the vertex normal turns, so they stop sparkling

yhhe90c90a6aba7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 76ba770ab..f26620cb1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- **Steadier highlights on curved and bevelled surfaces.** Lit models no longer sparkle on rails, bevels and rounded trim. Where a surface's normal turns faster than a pixel can resolve, the highlight is the sum of many normals the pixel cannot tell apart, and shading it with one made it flicker as the camera moved. The Model shader now widens a highlight by how fast the vertex normal turns across the pixel — a steady, slightly rougher highlight in place of a flickering sharp one — on WebGL2 and WebGPU alike. Measured on a street scene shifted by sub-pixel steps, frame-to-frame shimmer on rails and lamp trim fell by about 5–8%; a surface that is flat or smooth is not touched.
- **A Neutral output transform.** The output transform has a Neutral option that keeps hue and saturation. The output transform is the one curve a finished frame goes through on its way to the display; None only encodes and ACES rolls highlights off a filmic shoulder, but ACES also desaturates as it goes, so a brightly lit red drifts toward cream. Neutral (Khronos PBR Neutral) rolls highlights off in the same way and leaves colour alone, so a sunlit red awning stays red. Pick it in Settings → Project → Rendering, or set `outputTransform: 'neutral'`; it applies to the viewport, Play and every export, on WebGL2 and WebGPU alike.
- **Static batching.** Static meshes that share a material can be drawn as one. Add `StaticBatch` to a mesh that will not move, and the meshes that share its material, texture, colour and layer are merged while the game runs into one mesh per area, so a level built from many different props costs a few draws instead of one per distinct mesh. The renderer already drew copies of the same mesh as one instanced draw; this is for different meshes. The marked entities stay as authored — scripts find them, colliders collide, the editor picks them — and only their drawing moves to the merged mesh. A mesh is left out, and the reason recorded, when it is skinned, morphing, a level-of-detail group, blended, moving (a dynamic body or a character), lit by a bake or a probe volume, or alone in its group. Merging copies each part's vertices, so it spends memory to save draws; `StaticBatching` reports each merged mesh's size and what drawing its parts cost before. In the editor the outliner's menu marks the selected meshes, a marked mesh's Details say whether it merged and why not, and the profiler lists every merged mesh and the meshes left out.
- **Camera shots and nested sequences on the timeline.** Timelines have camera and sequence tracks. A camera track is a row of shots: each bar is the camera that is the view for its span, bars that touch cut, and a bar dragged over the previous one blends across the overlap with its own curve. A sequence track plays another timeline in a window of this one, on its own clock, with a speed, a loop, and a node to bind to so one sequence can play on different characters. A timeline cannot play itself, directly or through another, and nesting stops at four levels. The Sequencer shows both as bars above the object tracks, with the shot or clip in Details, and the viewport's camera preview follows the shot while you scrub. Which shot and which nested frame show is a function of time alone, so scrubbing, jumping and playing backwards agree.
diff --git a/desktop b/desktop
index fa526a489..320482486 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit fa526a48954a22aa2e87f9bf9219494f520fb1f9
+Subproject commit 320482486a70b9706b35367eca2e7c1d3e9fe6ed
diff --git a/sdk/src/render/builtinShaders.ts b/sdk/src/render/builtinShaders.ts
index cb3cc73cf..a6d9b607f 100644
--- a/sdk/src/render/builtinShaders.ts
+++ b/sdk/src/render/builtinShaders.ts
@@ -381,11 +381,30 @@ uniform sampler2D u_textures[1];
out vec4 fragColor;
+// Geometric specular anti-aliasing (Tokuyoshi & Kaplanyan 2019, as Filament spells it).
+// Where the vertex normal turns quickly across a pixel — a bevel, a curved rail, a
+// bolt-head — the highlight is the sum of many normals the pixel cannot resolve, and
+// shading it with ONE normal makes it sparkle from frame to frame. The turn rate is the
+// screen-space derivative of the normal; it widens the lobe by that variance, so the
+// sparkle becomes a steady, slightly rougher highlight. Taken from the vertex normal, not
+// the perturbed one: a normal map's detail is the texture filter's to average.
+float specularAA(highp vec3 dndx, highp vec3 dndy, float roughness) {
+ float variance = 0.25 * (dot(dndx, dndx) + dot(dndy, dndy));
+ float kernel = min(2.0 * variance, 0.18);
+ float alpha = roughness * roughness;
+ return sqrt(sqrt(clamp(alpha * alpha + kernel, 0.0, 1.0)));
+}
+
// Base colour rides the draw itself (slot 0 x vertex colour) — glTF's baseColor
// is what a MeshRenderer already carries. Emission is added AFTER lighting (it is light
// this surface makes) and occlusion scales only the ambient term.
// Metal-roughness packing is glTF's: roughness in green, metal in blue.
void main() {
+#ifdef MESH_NORMALS
+ // Before any branch: a derivative needs every fragment of its quad.
+ highp vec3 dndx = dFdx(v_worldNormal);
+ highp vec3 dndy = dFdy(v_worldNormal);
+#endif
vec4 base = texture(u_textures[0], v_texCoord) * v_color * u_tint;
// A cutout's alpha becomes coverage: sharpened to about a texel around the
// cutoff, so alpha-to-coverage feathers the edge instead of stair-stepping it.
@@ -413,17 +432,33 @@ void main() {
highp vec3 P = vec3(v_worldPos, 0.0);
#endif
// specular 1: a glTF material reflects unless KHR_materials_specular says less.
+ float roughness = u_roughness * mr.g;
+#ifdef MESH_NORMALS
+ roughness = specularAA(dndx, dndy, roughness);
+#endif
vec3 lit = applyLightingPBR(base.rgb, N, P, viewDirection(P),
- u_metallic * mr.b, u_roughness * mr.g, 1.0, ao);
+ u_metallic * mr.b, roughness, 1.0, ao);
fragColor = vec4(lit + u_emissive.rgb * texture(u_emissiveMap, v_texCoord).rgb, alpha);
}
#pragma end
#pragma fragment wgsl
+// The GLSL twin's geometric specular anti-aliasing, spelled the same.
+fn specularAA(dndx : vec3f, dndy : vec3f, roughness : f32) -> f32 {
+ let variance = 0.25 * (dot(dndx, dndx) + dot(dndy, dndy));
+ let kernel = min(2.0 * variance, 0.18);
+ let alpha = roughness * roughness;
+ return sqrt(sqrt(clamp(alpha * alpha + kernel, 0.0, 1.0)));
+}
+
@fragment fn fs_main(v : VSOut, @builtin(front_facing) front : bool) -> @location(0) vec4f {
// Taken before any branch: a derivative needs every fragment of its quad.
let duvx = dpdx(v.v_texCoord);
let duvy = dpdy(v.v_texCoord);
+#ifdef MESH_NORMALS
+ let dndx = dpdx(v.v_worldNormal);
+ let dndy = dpdy(v.v_worldNormal);
+#endif
// WGSL has no module varying: the entry point hands the injected header the
// run its GLSL twin reads straight off one. Without this every material-shaded
// draw read instance 0's indirect light — one probe volume for the lot.
@@ -454,9 +489,13 @@ void main() {
let P = v.v_worldXYZ;
#else
let P = vec3f(v.v_worldPos, 0.0);
+#endif
+ var roughness = mc.u_roughness * mr.g;
+#ifdef MESH_NORMALS
+ roughness = specularAA(dndx, dndy, roughness);
#endif
let lit = applyLightingPBR(base.rgb, N, P, viewDirection(P),
- mc.u_metallic * mr.b, mc.u_roughness * mr.g, 1.0, ao);
+ mc.u_metallic * mr.b, roughness, 1.0, ao);
let emit = mc.u_emissive.rgb
* textureSampleGrad(u_emissiveMap, u_emissiveMap_s, v.v_texCoord, duvx, duvy).rgb;
return vec4f(lit + emit, alpha);
diff --git a/sdk/tests/model-shader-specular-aa.test.ts b/sdk/tests/model-shader-specular-aa.test.ts
new file mode 100644
index 000000000..845e3ff92
--- /dev/null
+++ b/sdk/tests/model-shader-specular-aa.test.ts
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file The Model shader filters roughness by how fast the vertex normal turns, in both languages.
+ *
+ * The GLSL and WGSL stages are written apart and have to agree: a highlight that is filtered on
+ * WebGL2 and sparkles on WebGPU is the same bug in the other backend. This holds the two to one
+ * shape — the derivative is taken before any branch (a quad needs every fragment), only where the
+ * mesh has normals, and what the lighting reads is the FILTERED roughness.
+ */
+import { describe, it, expect } from 'vitest';
+import { builtinShaderTemplate } from '../src/render/builtinShaders';
+
+const source = builtinShaderTemplate('model')!.source;
+const [glsl, wgsl] = [source.slice(0, source.indexOf('#pragma fragment wgsl')), source.slice(source.indexOf('#pragma fragment wgsl'))];
+
+describe('the Model shader filters specular by the normal derivative', () => {
+ it('widens the lobe by the variance of the vertex normal, capped, in each language', () => {
+ for (const stage of [glsl, wgsl]) {
+ expect(stage).toMatch(/specularAA/);
+ expect(stage).toMatch(/0\.18/); // the cap: a lobe never widens past a point the art did not ask for
+ expect(stage).toMatch(/0\.25 \* \(dot\(dndx, dndx\) \+ dot\(dndy, dndy\)\)/);
+ }
+ });
+
+ it('takes the derivative of the interpolated vertex normal before the cutout can discard', () => {
+ expect(glsl.indexOf('dFdx(v_worldNormal)')).toBeGreaterThan(0);
+ expect(glsl.indexOf('dFdx(v_worldNormal)')).toBeLessThan(glsl.indexOf('discard'));
+ expect(wgsl.indexOf('dpdx(v.v_worldNormal)')).toBeGreaterThan(0);
+ expect(wgsl.indexOf('dpdx(v.v_worldNormal)')).toBeLessThan(wgsl.indexOf('discard'));
+ });
+
+ it('lights with the filtered roughness, not the authored one', () => {
+ expect(glsl).toMatch(/roughness = specularAA\(dndx, dndy, roughness\)/);
+ expect(glsl).toMatch(/applyLightingPBR\([^;]*u_metallic \* mr\.b, roughness,/);
+ expect(wgsl).toMatch(/roughness = specularAA\(dndx, dndy, roughness\)/);
+ expect(wgsl).toMatch(/applyLightingPBR\([^;]*mc\.u_metallic \* mr\.b, roughness,/);
+ });
+});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 911272721..74fc35ada 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2465,4 +2465,6 @@ export const NOTED = {
{"note":"The output transform has a Neutral option that keeps hue and saturation."},
"feat(editor): the project's Output transform offers Neutral":
{"note":"The output transform has a Neutral option that keeps hue and saturation."},
+ "feat(render): model highlights widen with how fast the vertex normal turns, so they stop sparkling":
+ {"note":"Lit models no longer sparkle on rails, bevels and rounded trim."},
};