Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): Temporal Anti-Aliasing follows meshes that move by themselves, from a motion pass that draws only what moved

yhh8bb626e7e5fd
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5644fe4ef..0b5a0ddb8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,7 +22,7 @@ published separately; it ships inside the editor.
- **Auto Exposure and Depth of Field.** Auto Exposure and Depth of Field post-process effects are available. `autoExposure` measures the average log luminance of the picture (leaving out pixels darker than 2^-8, so a black backdrop does not open the exposure to its limit), turns it into the exposure that puts it at middle grey, and moves toward that over time from where the last frame left it — brighter faster than darker, per second, within `minEV` and `maxEV`, with a `bias` in stops. To keep the settled value between frames a post-process shader can now declare a `u_feedback` texture: the engine keeps that pass's last output, at the pass's own size, and binds it there, without shifting the picture the way `u_history` does. `depthOfField` blurs by distance from `focusDistance`, with a sharp slab of `focusRange`, a blur radius that grows with how far a surface is from focus up to `maxBlur`, and a gather that counts a neighbour only where its own blur reaches back, so a sharp object does not bleed into a blurred background. Verified in the editor on both backends: a dim scene brightens (mean 6.9 to 16.5) and a bright one darkens (30.7 to 20.4), a drop in light is first dark and then recovered over real time (5.2 then 15.8), and focusing close takes the squared edge energy down to 21% of the sharp frame.
- **Height Fog and Light Shafts.** Height Fog and Light Shafts post-process effects add atmosphere. `heightFog` is exponential height fog with a closed-form integral — no marching — so a valley fills and a hilltop clears, the horizon turns hazy while the zenith stays clear, and looking toward the sun the haze takes the sun's colour; sky pixels run through the same formula with a very long ray, and `skyFog` sets how much haze the sky keeps. `lightShafts` marches each pixel toward the sun's place on screen and counts how much of the way is open sky, leaving rays through whatever stands in front of the sun. The sun's direction is set on the effects (azimuth and elevation), not read from a light. Neither is a shadowed volume: fog is lit evenly. Both run on WebGL2 and WebGPU; with an output transform of None, a floor fully taken by the fog reads exactly the authored colour on both.
- **Screen Space Reflections.** A Screen Space Reflections post-process effect mirrors props in floors. Add `ssr` to a `PostProcessVolume`: with no G-buffer, the surface normal is rebuilt from the depth buffer, the reflected ray is marched through the world and projected back to the screen at every step, and a step landing just behind the visible surface is a hit, refined by bisection. The reflection is blended over the pixel by a Schlick Fresnel term — faint face-on, strong at a grazing angle — and blurred by how far the ray travelled, which stands in for roughness. `intensity`, `maxDistance`, `thickness` and `floorBias` are the settings; `floorBias` at 1 lets only up-facing surfaces reflect, since the chain does not know a material's roughness. What is off screen or in the sky is not reflected; the pixel is left as it was. It reads the same depth buffer as ambient occlusion, runs on WebGL2 and WebGPU alike, and its per-pixel start offset is meant to be averaged away by `taa`. Verified in the editor on a lit scene with both backends: the props appear in the floor they stand on and nothing else on screen changes.
-- **Temporal Anti-Aliasing.** A Temporal Anti-Aliasing post-process effect smooths edges across frames. Add `taa` to a `PostProcessVolume` and the engine shifts the scene's picture by a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's last output as its history, and blends the two — reaching what multisampling cannot: shader aliasing and thin geometry. Where a pixel was last frame comes from the depth buffer and the previous camera, so a moving camera keeps its history; the history is clipped to the colours around it, so what no longer matches is dropped rather than smeared, and each camera keeps a history of its own. One `blend` setting says how much of each frame is new. Measured in the editor on a lit scene with multisampling off, in-between edge pixels went from 2% of edge pixels to 43% on WebGL2 and 44% on WebGPU, still frames held still, and a picture taken mid-slide of the camera matched the plain one to within a fraction of a level. Objects that move by themselves are not tracked — they are handled by the clip, so fast movers can look slightly soft.
+- **Temporal Anti-Aliasing.** A Temporal Anti-Aliasing post-process effect smooths edges across frames. Add `taa` to a `PostProcessVolume` and the engine shifts the scene's picture by a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's last output as its history, and blends the two — reaching what multisampling cannot: shader aliasing and thin geometry. Where a pixel was last frame comes from the depth buffer and the previous camera, so a moving camera keeps its history; the history is clipped to the colours around it, so what no longer matches is dropped rather than smeared, and each camera keeps a history of its own. One `blend` setting says how much of each frame is new. Measured in the editor on a lit scene with multisampling off, in-between edge pixels went from 2% of edge pixels to 43% on WebGL2 and 44% on WebGPU, still frames held still, and a picture taken mid-slide of the camera matched the plain one to within a fraction of a level. Objects that move by themselves are not tracked — they are handled by the clip, so fast movers can look slightly soft. Temporal Anti-Aliasing now follows meshes that move by themselves, from a motion pass that draws only what moved, so a thin fast object keeps its contrast. Moving meshes are drawn a second time into a target of their own that records how far each pixel travelled (an RGBA32F target, only on devices with float render targets), and the effect follows that where it is the surface the scene shows; `Object Motion` at 0 trusts the camera alone. Skinned meshes are not followed yet.
- **Smaller meshes on disk.** Newly imported meshes are about 9% smaller on disk. A mesh whose indices all fit in 16 bits is now written with 16-bit indices (`.esmesh` version 4), half the index bytes, and read back as 32-bit so nothing downstream changes: the renderer, the lightmap and decal bakers and static batching see the same indices as before. A mesh that needs 32 bits is written exactly as before, on the version every earlier reader knows, and files written before this change still load. Measured on the Bistro meshes (1143 files, 116 MB), re-encoding saves 10.4 MB; existing `.esmesh` files are not rewritten until the model is re-imported.
- **Levels of detail generated at import.** Imported models can generate cheaper levels of detail automatically. Set **Generate LODs** to 1–3 levels in a model's import settings (or pass `--lods <n>` to `import-model`) and each mesh is collapsed into copies with about 50%, 20% and 7% of its triangles, written as `<mesh>_lod<n>.esmesh`, with a `LODGroup` on the entity that swaps them in as it shrinks on screen. The collapse bounds how far the surface may move and will not pull apart a UV seam or a hard edge; vertex colours and the lightmap UV come along unchanged. Skinned and morphing meshes, and meshes under a minimum triangle count, keep one level. Measured on the Bistro exterior (1143 meshes, 2.83M triangles), the 621 meshes large enough to get levels hold 1.35M, 0.44M and 0.10M triangles at levels 1, 2 and 3; the levels are extra files, so they make a large scene cheaper to draw rather than smaller to ship.
- **Steadier highlights on curved and bevelled surfaces.** Lit models no longer sparkle on rails, bevels and rounded trim. Where a surface's normal turns faster than a pixel can resolve, the highlight is the sum of many normals the pixel cannot tell apart, and shading it with one made it flicker as the camera moved. The Model shader now widens a highlight by how fast the vertex normal turns across the pixel — a steady, slightly rougher highlight in place of a flickering sharp one — on WebGL2 and WebGPU alike. Measured on a street scene shifted by sub-pixel steps, frame-to-frame shimmer on rails and lamp trim fell by about 5–8%; a surface that is flat or smooth is not touched.
diff --git a/desktop b/desktop
index 32eda9eb2..9c0b54685 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 32eda9eb27ce8e285507bee8c2753d5a47bade0e
+Subproject commit 9c0b54685410de9175fac2a6f26b1de3482ce3cd
diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdx
index 785dd4e67..08a2decfa 100644
--- a/docs/astro/src/content/docs/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx
@@ -29,7 +29,7 @@ several can stack in one frame at no extra target cost beyond the passes.
| `depthOfField` | Blur by distance from a focus | `focusDistance`, `focusRange`, `aperture`, `maxBlur` |
| `rain` | Falling rain, wet ground, puddles, ripples and splashes | `intensity`, `wetness`, `wind`, `speed`, `puddles`, `puddleScale` |
| `ssr` | Screen-space reflections: props mirrored in the floor they stand on | `intensity`, `maxDistance`, `thickness`, `floorBias`, `skyReflection` |
-| `taa` | Temporal anti-aliasing: edges smoothed over several frames | `blend` |
+| `taa` | Temporal anti-aliasing: edges smoothed over several frames, moving meshes followed by their own motion | `blend`, `objectMotion` |
| `lensDistortion` | Barrel / pincushion warp | `strength`, `zoom` |
| `pixelate` | Block quantization | `pixelSize` |
| `outline` | Ink-edge outlines (Sobel edge darkening) | `intensity`, `threshold`, `thickness` |
@@ -158,11 +158,21 @@ frame's camera, so a moving camera keeps its history. `blend` is the share of ea
frame that is new: lower is smoother and slower to follow a change, 1 is no accumulation.
Put it **first** in the chain, so everything after it sees the settled image, and keep
-`msaaSamples` on — the two add. Each camera keeps its own history. What it does not
-know is how an object moves by itself: an object that moves is a different surface
-from its history, which is clipped to the colours around it, so fast movers can look
-slightly soft. It needs scene depth like SSAO does, so on WebGPU it cannot run with
-multisampling on a scene that has no depth resolve.
+`msaaSamples` on — the two add. Each camera keeps its own history.
+
+The camera's motion says where history was for everything that stood still. An object
+that moves by itself is a different case, so while the effect is in the chain the frame
+draws the **moving meshes a second time** into a target of its own, writing how far each
+pixel travelled since last frame, and the effect follows that instead where it is the
+surface the scene shows. This is why a thin, fast mover keeps its contrast instead of
+fading into the floor behind it. It costs one extra draw of only what moved (nothing,
+for a frame where nothing did) and an RGBA32F target the size of the scene, so it needs
+float render targets, and is skipped on a device without them. `objectMotion` at 0
+turns it off, which leaves the camera's answer alone. What it does not follow yet is a
+skinned mesh, whose bones are not remembered from last frame, a blended surface, and
+motion a material's vertex hook adds — those fall back to the camera's answer, clipped
+into the colours around the pixel. TAA needs scene depth like SSAO does, so on WebGPU
+it cannot run with multisampling on a scene that has no depth resolve.
The uniform names above are the friendly labels; **in code they are `u_`-prefixed**
(`u_intensity`, `u_exposure`, …). Most effects are identity at their defaults
diff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
index fe91aa4c4..211a9095e 100644
--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
@@ -28,7 +28,7 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
| `depthOfField` | 按到焦点的距离模糊 | `focusDistance`、`focusRange`、`aperture`、`maxBlur` |
| `rain` | 下雨、潮湿的地面、水坑、涟漪与水花 | `intensity`、`wetness`、`wind`、`speed`、`puddles`、`puddleScale` |
| `ssr` | 屏幕空间反射:道具映在它们所站的地面上 | `intensity`、`maxDistance`、`thickness`、`floorBias`、`skyReflection` |
-| `taa` | 时间性抗锯齿:边缘跨多帧平滑 | `blend` |
+| `taa` | 时间性抗锯齿:边缘跨多帧平滑,移动网格按它自己的运动跟随 | `blend`、`objectMotion` |
| `lensDistortion` | 桶形 / 枕形畸变 | `strength`、`zoom` |
| `pixelate` | 块状量化 | `pixelSize` |
| `outline` | 墨线描边(Sobel 边缘压暗) | `intensity`、`threshold`、`thickness` |
@@ -119,9 +119,15 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
是每帧里新画面的占比:越低越平滑、对变化跟得越慢,1 就是不累积。
把它放在链路**最前**,后面的效果看到的就是稳定后的画面,同时保留 `msaaSamples`——两者叠加。
-每个相机有自己的历史。它不知道物体自己怎么动:自己动的物体与它的历史是不同的表面,历史会被
-夹进周围的颜色里,所以快速移动的物体可能略软。它和 SSAO 一样要读场景深度,因此在 WebGPU 上,
-没有深度解析的多重采样场景里无法运行。
+每个相机有自己的历史。
+
+相机的运动能说明所有静止物体的历史在哪。自己在动的物体是另一回事,所以效果在链路里时,这一帧会把
+**移动的网格再画一遍**到它自己的目标里,写下每个像素自上一帧走了多远;在那里就是场景所显示的表面时,
+效果改用这个位移。所以又细又快的移动物体会保住对比度,而不是淡进身后的地面。它只多画一遍真正动了的
+东西(一帧里没有物体动就什么都不画),外加一个和场景同尺寸的 RGBA32F 目标,因此需要浮点渲染目标,
+没有的设备会跳过。`objectMotion` 设为 0 就关掉它,只剩相机的答案。目前还不跟随的有:蒙皮网格
+(上一帧的骨骼没有保存)、混合表面,以及材质顶点钩子加上的运动——它们退回相机的答案,并被夹进像素周围
+的颜色里。TAA 和 SSAO 一样要读场景深度,因此在 WebGPU 上,没有深度解析的多重采样场景里无法运行。
上面是友好标签;**在代码里它们带 `u_` 前缀**(`u_intensity`、`u_exposure`……)。多数效果
在默认值下是恒等(`colorGrade`、`blur`、`chromaticAberration`、`fxaa`、`lensDistortion`、
diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.ts
index cacf0ad70..cda3054cd 100644
--- a/sdk/src/postprocess/effects.ts
+++ b/sdk/src/postprocess/effects.ts
@@ -410,6 +410,9 @@ registerEffect({
// Lower is smoother and slower to follow a change; 1 is no accumulation at all,
// which is what a blend toward "no effect" fades to.
{ name: 'u_blend', label: 'Blend', min: 0.02, max: 1, step: 0.01, defaultValue: 0.1, neutralValue: 1 },
+ // 1 follows a surface that moved by itself by its own displacement; 0 trusts the camera's alone, which
+ // is what every pixel gets where nothing moved — and what a moving one smears with.
+ { name: 'u_objectMotion', label: 'Object Motion', min: 0, max: 1, step: 1, defaultValue: 1, neutralValue: 1 },
],
});
diff --git a/sdk/src/postprocess/postProcessEffects.ts b/sdk/src/postprocess/postProcessEffects.ts
index 3d1f6ba15..710d15c2d 100644
--- a/sdk/src/postprocess/postProcessEffects.ts
+++ b/sdk/src/postprocess/postProcessEffects.ts
@@ -298,8 +298,11 @@ fn sampleLut(c : vec3f) -> vec3f {
*
* Where last frame's point was comes from the depth buffer: un-project the pixel to
* the world, project it with last frame's camera. That is exact for everything the
- * camera moves past and wrong for an object that moved by itself — there the history
- * is a different surface, so it is clipped into the colours the neighbourhood holds,
+ * camera moves past and wrong for an object that moved by itself. Those objects are
+ * drawn again in a pass of their own that writes how far each pixel moved
+ * (`u_velocity`), and where that pass is what the scene shows, the pixel follows its
+ * own displacement instead. What neither explains — a skinned surface, a thing the
+ * motion pass did not draw — is clipped into the colours the neighbourhood holds,
* which hides the mismatch instead of smearing it. The blend is done in a space where
* a bright pixel weighs less, or one firefly outlives the frame it appeared in.
*/
@@ -327,12 +330,14 @@ fn expand(c : vec3f) -> vec3f { return c / max(1.0 - max(max(c.r, c.g), c.b), 1e
#pragma version 300 es
#pragma domain PostProcess
#pragma param u_blend float default(0.1) range(0.02,1)
+#pragma param u_objectMotion float default(1) range(0,1)
#pragma param u_historyValid float default(0) range(0,1)
#pragma param u_prevViewProjection0 color default(1,0,0,0)
#pragma param u_prevViewProjection1 color default(0,1,0,0)
#pragma param u_prevViewProjection2 color default(0,0,1,0)
#pragma param u_prevViewProjection3 color default(0,0,0,1)
#pragma param u_history texture default(black)
+#pragma param u_velocity texture default(black)
#pragma fragment
precision highp float;
@@ -371,7 +376,17 @@ void main() {
vec4 prev = mat4(u_prevViewProjection0, u_prevViewProjection1,
u_prevViewProjection2, u_prevViewProjection3) * vec4(world, 1.0);
vec2 puv = prev.xy / max(prev.w, 1e-6) * 0.5 + 0.5;
- bool seen = u_historyValid > 0.5 && prev.w > 1e-6
+ bool inFront = prev.w > 1e-6;
+ // A surface that moved by itself: the frame drew its displacement in a pass of its own, and where what it
+ // drew is the surface the scene shows here (its depth agrees), that displacement says where this pixel was
+ // — the camera's does not know about it. Anywhere else the camera's answer stands.
+ ivec2 vsize = textureSize(u_velocity, 0);
+ vec4 vel = texelFetch(u_velocity, screenTexel(v_texCoord, vsize), 0);
+ if (u_objectMotion > 0.5 && vel.a > 0.5 && abs(vel.b - d) <= 2e-5 + (1.0 - d) * 0.02) {
+ puv = v_texCoord + vel.xy;
+ inFront = true;
+ }
+ bool seen = u_historyValid > 0.5 && inFront
&& puv.x >= 0.0 && puv.x <= 1.0 && puv.y >= 0.0 && puv.y <= 1.0;
if (!seen) { fragColor = cur; return; }
@@ -422,8 +437,17 @@ ${WGSL}
let prev = vp * vec4f(world, 1.0);
// This device's uv runs down the rows, so clip y is flipped on the way back.
let pw = max(prev.w, 1e-6);
- let puv = vec2f(prev.x / pw * 0.5 + 0.5, 0.5 - prev.y / pw * 0.5);
- let seen = mc.u_historyValid > 0.5 && prev.w > 1e-6
+ var puv = vec2f(prev.x / pw * 0.5 + 0.5, 0.5 - prev.y / pw * 0.5);
+ var inFront = prev.w > 1e-6;
+ // See the GLSL twin: a surface that moved by itself, from the frame's motion pass. This device's uv
+ // runs down the rows, so the displacement's y is flipped on the way in.
+ let vsize = vec2i(textureDimensions(u_velocity, 0));
+ let vel = textureLoad(u_velocity, screenTexel(v.v_texCoord, vsize), 0);
+ if (mc.u_objectMotion > 0.5 && vel.a > 0.5 && abs(vel.b - d) <= 2e-5 + (1.0 - d) * 0.02) {
+ puv = v.v_texCoord + vec2f(vel.x, -vel.y);
+ inFront = true;
+ }
+ let seen = mc.u_historyValid > 0.5 && inFront
&& puv.x >= 0.0 && puv.x <= 1.0 && puv.y >= 0.0 && puv.y <= 1.0;
if (!seen) { return cur; }
diff --git a/src/esengine/data/shaders/mesh.esshader b/src/esengine/data/shaders/mesh.esshader
index 8978e06e3..0860a3f7b 100644
--- a/src/esengine/data/shaders/mesh.esshader
+++ b/src/esengine/data/shaders/mesh.esshader
@@ -137,6 +137,14 @@ out highp vec3 v_worldPos;
// there is worth several world units, which shadows the caster against itself.
out highp vec4 v_shadowClip;
#endif
+#ifdef ES_VELOCITY
+// Where this point is in the picture now and where it was one frame ago, as clip positions so the
+// fragment divides them: z/w is not affine across a triangle, and neither is xy/w. BOTH carry the shift
+// their own frame was drawn with: what the previous picture holds at a pixel is what stood at its
+// SHIFTED place, so the difference is exactly where in that picture to read.
+out highp vec4 v_velCur;
+out highp vec4 v_velPrev;
+#endif
// A material's own vertex offset, for the shadow program built from this source on its behalf: the
// caster has to bend where the lit surface does, or its shadow stands still. The assembler puts the
@@ -187,6 +195,18 @@ void main() {
#endif
#ifdef SHADOW_DEPTH
v_shadowClip = gl_Position;
+#endif
+#ifdef ES_VELOCITY
+ {
+ // This record is the motion pass's own: the tint texel carries the shift LAST frame was drawn with, and
+ // the four texels after the model are last frame's view-projection times last frame's model, as rows.
+ highp vec4 L = vec4(local, 1.0);
+ highp vec4 before = esInstanceTint();
+ v_velCur = gl_Position;
+ highp vec4 prev = vec4(dot(esInstanceTexel(4), L), dot(esInstanceTexel(5), L),
+ dot(esInstanceTexel(6), L), dot(esInstanceTexel(7), L));
+ v_velPrev = vec4(prev.xy + before.xy * prev.w, prev.zw);
+ }
#endif
// Only the vertex stage has an instance index, and every instance of a
// merged run reads its own coefficients.
@@ -217,6 +237,10 @@ in highp vec3 v_worldPos;
#ifdef SHADOW_DEPTH
in highp vec4 v_shadowClip;
#endif
+#ifdef ES_VELOCITY
+in highp vec4 v_velCur;
+in highp vec4 v_velPrev;
+#endif
uniform sampler2D u_texture;
#ifdef NORMAL_MAP
@@ -242,7 +266,14 @@ out vec4 fragColor;
#endif
void main() {
-#ifdef SHADOW_DEPTH
+#ifdef ES_VELOCITY
+ // How far this pixel's surface moved in the picture since last frame, in uv, and the depth it sits at
+ // so a reader can tell whether it is the surface that is visible there. A point that was behind the
+ // camera has no previous place: alpha 0 says "nothing known", which reads as the camera's own motion.
+ highp vec2 now = v_velCur.xy / v_velCur.w;
+ highp vec2 was = v_velPrev.xy / max(v_velPrev.w, 1e-6);
+ fragColor = vec4((was - now) * 0.5, gl_FragCoord.z, v_velPrev.w > 1e-6 ? 1.0 : 0.0);
+#elif defined(SHADOW_DEPTH)
// The expression the receiver evaluates on u_shadowMatrix, evaluated here on the
// same matrix and at the same point: whatever each backend's clip z means, both
// sides mean it alike.
@@ -331,6 +362,10 @@ struct VSOut {
// The clip position, not the depth it divides down to — see the GLSL twin.
@location(4) v_shadowClip : vec4f,
#endif
+#ifdef ES_VELOCITY
+ @location(4) v_velCur : vec4f,
+ @location(7) v_velPrev : vec4f,
+#endif
#ifdef MESH_LIGHTMAP
@location(5) v_lightmap : vec3f,
#endif
@@ -399,6 +434,17 @@ struct VSOut {
#endif
#ifdef SHADOW_DEPTH
out.v_shadowClip = out.pos;
+#endif
+#ifdef ES_VELOCITY
+ {
+ // See the GLSL twin: the tint texel is last frame's shift, texels 4..7 last frame's matrix as rows.
+ let L = vec4f(local, 1.0);
+ let before = esInstanceTint();
+ out.v_velCur = out.pos;
+ let prev = vec4f(dot(esInstanceTexel(4), L), dot(esInstanceTexel(5), L),
+ dot(esInstanceTexel(6), L), dot(esInstanceTexel(7), L));
+ out.v_velPrev = vec4f(prev.xy + before.xy * prev.w, prev.zw);
+ }
#endif
// The GLSL twin's v_probeSlot fill.
out.v_probeSlot = f32(inst);
@@ -460,6 +506,10 @@ struct VSOut {
#ifdef SHADOW_DEPTH
@location(4) v_shadowClip : vec4f,
#endif
+#ifdef ES_VELOCITY
+ @location(4) v_velCur : vec4f,
+ @location(7) v_velPrev : vec4f,
+#endif
#ifdef MESH_LIGHTMAP
@location(5) v_lightmap : vec3f,
#endif
@@ -493,7 +543,12 @@ fn packNormal(n : vec3f) -> vec2f {
#ifdef MESH_LIGHTMAP
g_lightmap = v.v_lightmap;
#endif
-#ifdef SHADOW_DEPTH
+#ifdef ES_VELOCITY
+ // See the GLSL twin. This device's uv runs down the rows, so the sign of y is the reader's to flip.
+ let now = v.v_velCur.xy / v.v_velCur.w;
+ let was = v.v_velPrev.xy / max(v.v_velPrev.w, 1e-6);
+ return vec4f((was - now) * 0.5, v.pos.z, select(0.0, 1.0, v.v_velPrev.w > 1e-6));
+#elif defined(SHADOW_DEPTH)
// The expression the receiver evaluates on u_shadowMatrix, evaluated here on the
// same matrix and at the same point: whatever each backend's clip z means, both
// sides mean it alike.
diff --git a/src/esengine/renderer/RenderTypePlugin.hpp b/src/esengine/renderer/RenderTypePlugin.hpp
index 03dc55938..2b92e9663 100644
--- a/src/esengine/renderer/RenderTypePlugin.hpp
+++ b/src/esengine/renderer/RenderTypePlugin.hpp
@@ -50,6 +50,9 @@ enum class RenderPurpose : u8 {
/// post, drawing what belongs to the screen rather than the world. It has no
/// camera — the projection it is given comes from the layout domain.
ScreenOverlay,
+ /// The pass that writes how far each moving surface travelled since last frame, for the temporal
+ /// effects: the scene's meshes again, only those that moved, through a program that writes motion.
+ Motion,
};
struct RenderFrameContext {
@@ -72,6 +75,16 @@ struct RenderFrameContext {
/// The pass these draws execute in has a second colour attachment for the surface's material, so a
/// shader that can write it is asked for the variant that does. False for every other collect.
bool gbuffer = false;
+ /// A temporal effect in the chain reads per-object motion, so the scene collect remembers where each mesh
+ /// was, and the Motion collect has something to compare against. Per view: `motion_key` names which.
+ bool track_motion = false;
+ u64 motion_key = 0;
+ u64 motion_frame = 0;
+ /// Last frame's view-projection (the engine's convention, unjittered) and the shift in NDC that frame was
+ /// drawn with: what the Motion collect writes into each record, so a displacement lands on the raster the
+ /// previous picture was actually drawn on.
+ glm::mat4 motion_prev_view_projection{1.0f};
+ glm::vec2 motion_prev_jitter{0.0f};
};
/** @brief What every plugin needs to know about the camera, derived once per collect
@@ -433,6 +446,9 @@ public:
* depth — so the default is no, and a plugin that casts says so.
*/
virtual bool castsShadows() const { return false; }
+ /// Whether the Motion collect asks this plugin: what it draws can move by itself, and the picture
+ /// after it needs to know by how much.
+ virtual bool drawsMotion() const { return false; }
/**
* @brief Whether this plugin can draw an entity that lives in a UI tree.
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.cpp b/src/esengine/renderer/frame/PostProcessPipeline.cpp
index 62fe451c9..d336eee78 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.cpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.cpp
@@ -316,6 +316,14 @@ bool PostProcessPipeline::temporalActive() {
return false;
}
+bool PostProcessPipeline::velocityWanted() {
+ if (!initialized_ || bypass_ || !device_.supportsFloatTargets()) return false;
+ for (const auto& pass : passes_) {
+ if (pass.enabled && passHasTexture(pass, "u_velocity")) return true;
+ }
+ return false;
+}
+
namespace {
/// Van der Corput in @p base: the low-discrepancy sequence a jitter walks the pixel with.
f32 halton(u32 index, u32 base) {
@@ -350,6 +358,48 @@ glm::vec2 PostProcessPipeline::beginTemporalView(const glm::mat4& viewProjection
return {jx * 2.0f / w, jy * 2.0f / h};
}
+PostProcessPipeline::TemporalView PostProcessPipeline::peekTemporalView(const glm::mat4& viewProjection,
+ u64 key) const {
+ const auto it = temporal_.find(key);
+ const u32 frame = it == temporal_.end() ? 0u : it->second.frame;
+ TemporalView view;
+ view.previous = frame > 0 ? it->second.current : viewProjection;
+ // The step beginTemporalView will take next: one frame on, the same phase of the same sequence.
+ const u32 phase = ((frame + 1) % kJitterPhases) + 1;
+ const f32 w = static_cast<f32>(std::max(width_, 1u));
+ const f32 h = static_cast<f32>(std::max(height_, 1u));
+ view.jitter = {(halton(phase, 2) - 0.5f) * 2.0f / w, (halton(phase, 3) - 0.5f) * 2.0f / h};
+ if (frame > 0) {
+ const u32 before = (frame % kJitterPhases) + 1;
+ view.previousJitter = {(halton(before, 2) - 0.5f) * 2.0f / w, (halton(before, 3) - 0.5f) * 2.0f / h};
+ }
+ return view;
+}
+
+rg::ResourceId PostProcessPipeline::declareVelocity() {
+ velocity_active_ = false;
+ if (!graph_ || !device_.supportsFloatTargets()) return rg::kNoResource;
+ const u32 w = std::max(width_, 1u);
+ const u32 h = std::max(height_, 1u);
+ if (!velocity_fbo_ || velocity_fbo_->getWidth() != w || velocity_fbo_->getHeight() != h) {
+ FramebufferSpec spec;
+ spec.width = w;
+ spec.height = h;
+ spec.colorFormat = GfxPixelFormat::RGBA32F;
+ spec.depthStencil = true;
+ spec.linearFilter = false;
+ velocity_fbo_ = Framebuffer::create(device_, spec);
+ if (!velocity_fbo_) {
+ ES_LOG_ERROR("PostProcessPipeline: Failed to create the motion target");
+ return rg::kNoResource;
+ }
+ }
+ velocity_resource_ = graph_->importTarget(velocity_fbo_->handle(), w, h, /*output=*/false,
+ velocity_fbo_->getColorAttachment());
+ velocity_active_ = true;
+ return velocity_resource_;
+}
+
PostProcessPipeline::TemporalState* PostProcessPipeline::prepareHistory(TemporalState& state,
u32 w, u32 h) {
w = std::max(w, 1u);
@@ -551,6 +601,7 @@ void PostProcessPipeline::declareChain() {
}
void PostProcessPipeline::chainDone() {
+ velocity_active_ = false;
if (!initialized_ || !inFrame_) return;
device_.invalidatePipelineCache();
inFrame_ = false;
@@ -668,6 +719,9 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
if (slot.name == "u_environment" && environment_texture_ != 0) {
gpu = TextureHandle{environment_texture_};
}
+ if (slot.name == "u_velocity" && velocity_active_ && velocity_fbo_) {
+ gpu = velocity_fbo_->getColorAttachment();
+ }
for (const auto& [name, glId] : pass.textureUniforms) {
if (name == slot.name && glId != 0) { gpu = TextureHandle{glId}; break; }
}
@@ -773,6 +827,11 @@ void PostProcessPipeline::runChain(std::vector<PostProcessPass>& passes, rg::Res
// the last pass that reads its colour: returned to the pool any earlier,
// an SSAO behind a colour grade found no depth and occluded nothing.
if (scene_needs_depth_ || scene_gbuffer_) node.dependencies = {scene};
+ // The motion pass wrote a target this one reads by name, not by unit: the graph owes it ordering
+ // (and the pass its place in the culled set), and the pipeline binds it.
+ if (velocity_active_ && velocity_resource_ != rg::kNoResource && passHasTexture(pass, "u_velocity")) {
+ node.dependencies.push_back(velocity_resource_);
+ }
// The last pass feeds the blit, which copies into the output rect at the
// chain's size — so it draws full-size whatever fraction it asked for.
rg::TargetDesc writeDesc = (i == lastEnabled) ? blitDesc : desc;
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.hpp b/src/esengine/renderer/frame/PostProcessPipeline.hpp
index ecd34796c..4f8a56e19 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.hpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.hpp
@@ -220,6 +220,34 @@ public:
*/
bool temporalActive();
+ /**
+ * @brief Whether an enabled pass reads how far each moving surface travelled (`u_velocity`).
+ *
+ * @details The frame then draws its moving meshes a second time into a target of its own, so a
+ * temporal effect can follow an object by its own displacement instead of the camera's.
+ */
+ bool velocityWanted();
+
+ /// What a temporal view would be given by its NEXT begin: the sub-pixel shift and last frame's camera.
+ struct TemporalView {
+ glm::vec2 jitter{0.0f};
+ glm::mat4 previous{1.0f};
+ /// The shift the previous frame was drawn with.
+ glm::vec2 previousJitter{0.0f};
+ };
+ /// Read before the scene is drawn, because the motion pass is collected with the scene and the shift
+ /// is only applied when it is drawn. Changes nothing: beginTemporalView still takes the same step.
+ TemporalView peekTemporalView(const glm::mat4& viewProjection, u64 key) const;
+
+ /**
+ * @brief The motion pass's target for this frame, declared in the graph (kNoResource when it cannot be made).
+ *
+ * @details RGBA32F: xy = how far the surface moved in uv, z = its window depth, w = 1 where something wrote.
+ * Depth in a float because the reader compares it with the scene's, and a half cannot tell two
+ * surfaces apart near the far plane.
+ */
+ rg::ResourceId declareVelocity();
+
/**
* @brief Whether this frame's scene target carries a second attachment for the surface's material.
*
@@ -524,6 +552,10 @@ private:
bool passesWantGBuffer();
bool scene_gbuffer_ = false;
u32 environment_texture_ = 0;
+ /// The motion pass's target, kept across frames for its size, and whether this frame drew into it.
+ Unique<Framebuffer> velocity_fbo_;
+ rg::ResourceId velocity_resource_ = rg::kNoResource;
+ bool velocity_active_ = false;
bool passHasTexture(const PostProcessPass& pass, const char* name);
/// A history made (or kept) at @p w x @p h, or null when it cannot be.
TemporalState* prepareHistory(TemporalState& state, u32 w, u32 h);
diff --git a/src/esengine/renderer/frame/RenderFrame.cpp b/src/esengine/renderer/frame/RenderFrame.cpp
index e64828aba..ded1ecb60 100644
--- a/src/esengine/renderer/frame/RenderFrame.cpp
+++ b/src/esengine/renderer/frame/RenderFrame.cpp
@@ -131,7 +131,8 @@ RenderFrame::RenderFrame(GfxDevice& device, RenderContext& context,
, pool_(device)
, target_pool_(device)
, graph_(device, target_pool_)
- , shadow_pool_(device) {
+ , shadow_pool_(device)
+ , velocity_pool_(device) {
target_manager_.setDevice(device);
}
@@ -163,6 +164,7 @@ void RenderFrame::init(u32 width, u32 height) {
pool_.init();
shadow_pool_.init();
+ velocity_pool_.init();
planar_pool_.init();
batch_shader_id_ = initBatchShader();
@@ -188,6 +190,7 @@ void RenderFrame::shutdown() {
plugins_.clear();
pool_.shutdown();
shadow_pool_.shutdown();
+ velocity_pool_.shutdown();
planar_pool_.shutdown();
releasePlanarCaptures();
releaseShadowFrameUbos();
@@ -628,6 +631,25 @@ void RenderFrame::end() {
graph_.addPass(std::move(scene));
}
+#ifdef ES_ENABLE_POSTPROCESS
+ // What the scene drew, drawn again for how far each moving surface travelled: a temporal pass in the
+ // chain reads it, and nothing else does, so a frame whose meshes all stood still declares nothing.
+ if (usePostProcess && scene_resource_ != rg::kNoResource && velocity_list_.commandCount() > 0) {
+ const rg::ResourceId velocity = post_process_->declareVelocity();
+ if (velocity != rg::kNoResource) {
+ rg::PassDesc motion;
+ motion.name = "motion";
+ motion.write = velocity;
+ motion.clear = true;
+ motion.clearDepth = true;
+ // Nothing written is alpha 0: a reader treats it as "the camera's motion, not an object's".
+ motion.clearColor[0] = motion.clearColor[1] = motion.clearColor[2] = motion.clearColor[3] = 0.0f;
+ motion.execute = [this](const rg::PassContext&) { executeMotionPass(); };
+ graph_.addPass(std::move(motion));
+ }
+ }
+#endif
+
if (usePostProcess) {
#ifdef ES_ENABLE_POSTPROCESS
post_process_->declareChain();
@@ -2162,6 +2184,13 @@ void RenderFrame::collectAll(ecs::Registry& registry, bool sceneGBuffer) {
// Asked at the collect, because it decides which program each material draw is compiled as; the
// scene pass that executes them is the one the answer is about.
ctx.gbuffer = sceneGBuffer && post_process_ && post_process_->sceneHasGBuffer();
+ // A temporal effect that follows moving surfaces: the scene collect remembers where each mesh stands,
+ // and the motion collect below compares.
+ const bool followMotion = sceneGBuffer && post_process_ && post_process_->isEngaged()
+ && post_process_->velocityWanted();
+ ctx.track_motion = followMotion;
+ ctx.motion_key = static_cast<u64>(current_target_);
+ ctx.motion_frame = planar_frame_;
#else
(void)sceneGBuffer;
#endif
@@ -2191,6 +2220,11 @@ void RenderFrame::collectAll(ecs::Registry& registry, bool sceneGBuffer) {
stats_.culled += collectCtx.culled;
stats_.occluded += collectCtx.occluded;
+ velocity_list_.clear();
+#ifdef ES_ENABLE_POSTPROCESS
+ if (followMotion) collectMotion(registry);
+#endif
+
// OR-ed across the frame's cameras, applied at the next beginFrame and never
// mid-frame: the capture is scratch each camera composites out of, and the
// editor's grid draws into it between begin() and this collect.
@@ -2198,6 +2232,51 @@ void RenderFrame::collectAll(ecs::Registry& registry, bool sceneGBuffer) {
frame_depth_seen_ = frame_depth_seen_ || draw_list_.needsDepth();
}
+void RenderFrame::collectMotion(ecs::Registry& registry) {
+ ES_PROFILE_SCOPE("render.collect.motion");
+#ifdef ES_ENABLE_POSTPROCESS
+ const auto view = post_process_->peekTemporalView(view_projection_, static_cast<u64>(current_target_));
+ velocity_jitter_ = view.jitter;
+ velocity_pool_.beginFrame();
+ velocity_list_.clear();
+
+ auto ctx = makeContext();
+ ctx.purpose = RenderPurpose::Motion;
+ ctx.track_motion = true;
+ ctx.motion_key = static_cast<u64>(current_target_);
+ ctx.motion_frame = planar_frame_;
+ ctx.motion_prev_view_projection = view.previous;
+ ctx.motion_prev_jitter = view.previousJitter;
+ RenderCollectContext collectCtx{registry, frustum_, clip_state_, velocity_pool_, velocity_list_, ctx,
+ computeCameraView(ctx.view_projection)};
+ collectCtx.screen_ui = screen_domain_;
+ // The level the scene settled on, not a new choice, and not counted a second time.
+ collectCtx.lod = {view_id_, &lod_view_state_, nullptr};
+ collectCtx.occlusion = &occlusion_;
+ for (auto& plugin : plugins_) {
+ if (plugin->drawsMotion()) plugin->collect(collectCtx);
+ }
+ velocity_list_.finalize(velocity_pool_);
+ velocity_pool_.upload();
+#else
+ (void)registry;
+#endif
+}
+
+void RenderFrame::executeMotionPass() {
+ if (velocity_list_.commandCount() == 0) return;
+ ES_PROFILE_SCOPE("render.motion");
+ // Through the camera the scene was drawn with, jitter included: a surface's depth here has to be the
+ // scene's, since that is what the reader compares it with.
+ context_.updateCameraConstants(view_projection_, velocity_jitter_);
+ // The program is still a Lit shader and still declares the light block (see executeShadowPass).
+ context_.lights().uploadAndBind();
+ velocity_list_.execute(device_, velocity_pool_, context_.materials(), context_.getWhiteTextureId(),
+ nullptr, context_.perDrawBlocks());
+ ES_PROFILE_COUNTER("render.motion.draws", velocity_list_.mergedDrawCallCount());
+ device_.invalidatePipelineCache();
+}
+
glm::vec3 srgbToLinearCpu(const glm::vec3& c) {
auto conv = [](f32 v) {
return v <= 0.04045f ? v / 12.92f : std::pow((v + 0.055f) / 1.055f, 2.4f);
diff --git a/src/esengine/renderer/frame/RenderFrame.hpp b/src/esengine/renderer/frame/RenderFrame.hpp
index 82fb625c5..f8465a106 100644
--- a/src/esengine/renderer/frame/RenderFrame.hpp
+++ b/src/esengine/renderer/frame/RenderFrame.hpp
@@ -954,6 +954,17 @@ private:
* vertices the scene is about to draw from.
*/
TransientBufferPool shadow_pool_;
+ /**
+ * @brief The motion pass's own buffers and draw list, for the same reason as the shadow pass's:
+ * the scene's list is finalized and uploaded before this one is collected.
+ */
+ TransientBufferPool velocity_pool_;
+ DrawList velocity_list_;
+ /// This frame's shift, kept from the collect so the pass draws through the camera the scene did.
+ glm::vec2 velocity_jitter_{0.0f};
+ /// Collects the meshes that moved by themselves since the last frame into velocity_list_.
+ void collectMotion(ecs::Registry& registry);
+ void executeMotionPass();
/// One per view, kept across frames for the capacity. The whole pass is collected
/// before any of it is drawn, so a view's commands have to outlive its collect.
std::vector<DrawList> shadow_view_lists_;
diff --git a/src/esengine/renderer/plugins/MeshPlugin.cpp b/src/esengine/renderer/plugins/MeshPlugin.cpp
index d378027d9..2c2ec54c0 100644
--- a/src/esengine/renderer/plugins/MeshPlugin.cpp
+++ b/src/esengine/renderer/plugins/MeshPlugin.cpp
@@ -247,10 +247,12 @@ void reportLodAuthoring(const LodLevels& levels, bool& warned) {
}
u32 meshVariant(bool normals, bool lit, bool normalMapped, bool skinned, bool depthOnly,
- bool envMapped, bool lightmapped, bool weightHeat = false, bool gbuffer = false) {
+ bool envMapped, bool lightmapped, bool weightHeat = false, bool gbuffer = false,
+ bool motion = false) {
return (normals ? 1u : 0u) | (lit ? 2u : 0u) | (normalMapped ? 4u : 0u)
| (skinned ? 8u : 0u) | (depthOnly ? 16u : 0u) | (envMapped ? 32u : 0u)
- | (lightmapped ? 64u : 0u) | (weightHeat ? 128u : 0u) | (gbuffer ? 256u : 0u);
+ | (lightmapped ? 64u : 0u) | (weightHeat ? 128u : 0u) | (gbuffer ? 256u : 0u)
+ | (motion ? 512u : 0u);
}
/**
@@ -378,7 +380,7 @@ u32 MeshPlugin::meshProgram(RenderFrameContext& ctx, u32 variant) {
const bool normalMapped = (variant & 4u) != 0, skinned = (variant & 8u) != 0;
const bool depthOnly = (variant & 16u) != 0, envMapped = (variant & 32u) != 0;
const bool lightmapped = (variant & 64u) != 0, weightHeat = (variant & 128u) != 0;
- const bool gbuffer = (variant & 256u) != 0;
+ const bool gbuffer = (variant & 256u) != 0, motion = (variant & 512u) != 0;
if (mesh_compiled_[variant]) return mesh_programs_[variant];
mesh_compiled_[variant] = true;
// Counted because the cost of this function is entirely the times it does
@@ -396,7 +398,7 @@ u32 MeshPlugin::meshProgram(RenderFrameContext& ctx, u32 variant) {
if (lightmapped) features.emplace_back("MESH_LIGHTMAP");
if (weightHeat) features.emplace_back("WEIGHT_HEAT");
if (depthOnly) features.emplace_back("SHADOW_DEPTH");
- if (gbuffer) features.emplace_back("ES_GBUFFER");
+ if (motion) features.emplace_back("ES_VELOCITY");
// Resident geometry owns its texture slots, so it is the vertex source that can
// carry a shadow map and a reflection — the batch stream's are a per-vertex
// merge product, and a sampler pinned there would read someone's sprite.
@@ -408,6 +410,9 @@ u32 MeshPlugin::meshProgram(RenderFrameContext& ctx, u32 variant) {
// the plane on a surface that HAS depth is the 2D convention misapplied.
features.emplace_back("ES_SURFACE_3D");
}
+ // After the receiving features, not between them and the `else` they hang from: a lit draw that writes the
+ // surface record still receives shadows and reflections.
+ if (gbuffer) features.emplace_back("ES_GBUFFER");
// Authored as mesh.esshader, WGSL twin included. Its vertex stage is the one
// that reads a model matrix, which is what lets the vertices stay local.
@@ -511,6 +516,16 @@ RenderPrewarmResult MeshPlugin::prewarm(RenderFrameContext& ctx, ecs::Registry&
anyResident);
}
+void MeshPlugin::trackPose(const RenderFrameContext& ctx, Entity entity, const glm::mat4& model) {
+ Pose& pose = poses_[poseKey(ctx, entity)];
+ // Last frame's, when the view drew this mesh last frame; otherwise it has no past worth following, and
+ // standing where it stands says "did not move".
+ if (pose.frame != 0 && pose.frame + 1 == ctx.motion_frame) pose.previous = pose.current;
+ else if (pose.frame != ctx.motion_frame) pose.previous = model;
+ pose.current = model;
+ pose.frame = ctx.motion_frame;
+}
+
void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
auto& registry = collect_ctx.registry;
auto& clips = collect_ctx.clip_state;
@@ -520,10 +535,19 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
auto meshView = registry.activeView<ecs::Transform, ecs::MeshRenderer>();
// Read once: the phase is the same for every mesh this collect walks.
const bool shadowDepth = ctx.purpose == RenderPurpose::ShadowDepth;
+ // The motion pass draws the same meshes a second time, only those that moved, with no material and no light
+ // — what shadows and motion have in common is that neither is the surface's own look.
+ const bool motion = ctx.purpose == RenderPurpose::Motion;
+ const bool depthPass = shadowDepth || motion;
+ if (!motion && ctx.track_motion && ctx.motion_frame % 256 == 0) {
+ for (auto it = poses_.begin(); it != poses_.end();) {
+ it = it->second.frame + 8 < ctx.motion_frame ? poses_.erase(it) : std::next(it);
+ }
+ }
u32 litProgram = 0;
// The mesh as this view's mode describes it; one object reused across the walk.
- const ShadingMode shading = shadowDepth ? ShadingMode::Shaded : collect_ctx.shading;
+ const ShadingMode shading = depthPass ? ShadingMode::Shaded : collect_ctx.shading;
ecs::MeshRenderer shaded;
// Accumulated, not scoped per mesh: a ScopeTimer pair per phase per entity
@@ -544,7 +568,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
++walked;
const auto& authored = meshView.get<ecs::MeshRenderer>(entity);
const ecs::MeshRenderer* described = &authored;
- const bool heated = !shadowDepth && collect_ctx.weights.entity == entity;
+ const bool heated = !depthPass && collect_ctx.weights.entity == entity;
if (heated) {
shaded = authored;
applyShading(shaded, ShadingMode::Albedo);
@@ -581,7 +605,10 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
// Only GPU-resident geometry casts: the inline payload takes the CPU path
// below, which bakes world space into vertices for the BATCH shader — a
// shader that writes colour, not the depth this pass is here to collect.
- if (shadowDepth && !resident) continue;
+ if (depthPass && !resident) continue;
+ // Only a surface that draws opaque and unblended has a motion worth writing: what is blended over
+ // something has two, and the picture can follow one.
+ if (motion && !mesh.opaque) continue;
// A collect with no LOD memory must not select: a shadow map's own screen
// size means nothing to a player, and borrowing a camera's answer would
@@ -657,8 +684,22 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
textureId = tex->getId();
}
}
- const MeshDrawKeys keys =
- meshDrawFor(ctx, registry, entity, mesh, resident, shadowDepth, warned_bones_, heated);
+ MeshDrawKeys keys{};
+ if (motion) {
+ // What was drawn last frame and what is drawn now, for THIS view. Nothing known, or nothing changed,
+ // means the camera's own motion says it all and the temporal pass already has that.
+ const auto pose = poses_.find(poseKey(ctx, entity));
+ if (pose == poses_.end() || pose->second.frame != ctx.motion_frame
+ || pose->second.previous == pose->second.current) {
+ continue;
+ }
+ const MeshProgramFacts facts =
+ meshFactsFromEntity(ctx, registry, entity, mesh, resident, warned_bones_);
+ if (facts.skinned) continue;
+ keys = { meshVariant(facts.hasNormals, false, false, false, false, false, false, false, false, true), 0 };
+ } else {
+ keys = meshDrawFor(ctx, registry, entity, mesh, resident, shadowDepth, warned_bones_, heated);
+ }
const u32 normalTextureId = keys.normalTextureId;
BatchDrawKey key{
@@ -676,7 +717,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
.type = RenderType::Mesh,
};
- if (!shadowDepth && ctx.frame) {
+ if (!depthPass && ctx.frame) {
const u32 reflection = ctx.frame->planarTextureFor(entity);
if (reflection != 0) {
key.envTextureId = reflection;
@@ -716,7 +757,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
// Casting into a shadow map: opaque and depth-tested whatever the camera's
// pass makes it, and none of the three ways to describe a surface — lit, a
// normal map, a material. LAST, so the pass outranks all three.
- if (shadowDepth) {
+ if (depthPass) {
key.stage = RenderStage::Opaque;
key.blend = BlendMode::None;
key.depthTest = true;
@@ -747,7 +788,7 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
// The shape the mesh was REALIZED in, not a second derivation:
// the record is read through that layout, so the fact that built
// it must pack it. (It already excludes a skinned mesh.)
- const bool lightmapped = resident->hasLightmapUV;
+ const bool lightmapped = resident->hasLightmapUV && !motion;
u32 instOffset = buffers.allocVertices(LayoutId::MeshInstance,
MESH_INSTANCE_STRIDE);
auto* dst = buffers.vertexData(LayoutId::MeshInstance) + instOffset;
@@ -799,11 +840,28 @@ void MeshPlugin::collect(RenderCollectContext& collect_ctx) {
std::memcpy(dst + MESH_INSTANCE_TEXEL_LIGHTMAP * 16, &rect, 16);
}
}
+ if (motion) {
+ // This record is the motion pass's own, so the texels a lit draw keeps its tint, normal
+ // matrix and bake in are free: last frame's shift in the tint's, and last frame's
+ // view-projection times last frame's model — as rows, so the shader dots them with the local position.
+ const Pose& pose = poses_.find(poseKey(ctx, entity))->second;
+ const glm::vec4 jitter{ctx.motion_prev_jitter, 0.0f, 0.0f};
+ std::memcpy(dst + MESH_INSTANCE_TEXEL_TINT * 16, &jitter, 16);
+ const glm::mat4 before = ctx.motion_prev_view_projection * pose.previous;
+ for (u32 row = 0; row < 4; ++row) {
+ const glm::vec4 r{before[0][row], before[1][row], before[2][row], before[3][row]};
+ std::memcpy(dst + (MESH_INSTANCE_TEXEL_NORMAL + row) * 16, &r, 16);
+ }
+ } else if (ctx.track_motion && !shadowDepth) {
+ trackPose(ctx, entity, glm::translate(glm::mat4(1.0f), position)
+ * glm::mat4_cast(rotation)
+ * glm::scale(glm::mat4(1.0f), scale));
+ }
// A draw reading its own atlas does not ask for IRRADIANCE, and asks
// for its reflection either way — a lightmapped wall mirrors the room
// like anything else. At the ENTITY's position: bones place a skin.
- if (!shadowDepth) {
+ if (!depthPass) {
ProbeConstants probe;
const bool volume = key.lightmapTextureId == 0
&& ctx.render_context.probes().sample(position, probe);
diff --git a/src/esengine/renderer/plugins/MeshPlugin.hpp b/src/esengine/renderer/plugins/MeshPlugin.hpp
index 9da16db5c..7ce5817cd 100644
--- a/src/esengine/renderer/plugins/MeshPlugin.hpp
+++ b/src/esengine/renderer/plugins/MeshPlugin.hpp
@@ -7,6 +7,7 @@
#include "../draw/BatchVertex.hpp"
#include "../../resource/Handle.hpp"
+#include <unordered_map>
#include <vector>
namespace esengine {
@@ -24,6 +25,7 @@ public:
/// Resident geometry is the only thing the engine draws depth-only. A flat
/// renderable's shadow is ShadowCaster2D's to cast, not the map's.
bool castsShadows() const override { return true; }
+ bool drawsMotion() const override { return true; }
void collect(RenderCollectContext& ctx) override;
RenderPrewarmResult prewarm(RenderFrameContext& ctx, ecs::Registry& registry,
const Entity* entities, u32 count, bool shadowPasses) override;
@@ -53,6 +55,19 @@ private:
/// One entity's bone matrices while they are being built; reused per draw.
std::vector<glm::mat4> pose_scratch_;
u32 meshProgram(RenderFrameContext& ctx, u32 variant);
+
+ /// Where one mesh stood the last two times a view drew it: the motion between them is what a temporal
+ /// effect follows. Per view and entity, kept only while a temporal effect asks for it.
+ struct Pose {
+ glm::mat4 previous{1.0f};
+ glm::mat4 current{1.0f};
+ u64 frame = 0;
+ };
+ std::unordered_map<u64, Pose> poses_;
+ void trackPose(const RenderFrameContext& ctx, Entity entity, const glm::mat4& model);
+ static u64 poseKey(const RenderFrameContext& ctx, Entity entity) {
+ return ctx.motion_key * 0x9E3779B97F4A7C15ull ^ static_cast<u64>(entity.id());
+ }
bool warned_material_ = false; ///< A material on resident geometry is said once.
bool warned_bones_ = false; ///< A skin over the pose budget is said once.
bool warned_lod_ = false; ///< A group that cannot be honoured is said once.
diff --git a/src/esengine/renderer/rhi/GfxEnums.hpp b/src/esengine/renderer/rhi/GfxEnums.hpp
index adc8921c7..8cfe843e9 100644
--- a/src/esengine/renderer/rhi/GfxEnums.hpp
+++ b/src/esengine/renderer/rhi/GfxEnums.hpp
@@ -138,9 +138,9 @@ static constexpr u32 MORPH_TEXTURE_WIDTH = 2048;
static constexpr u32 MORPH_MAX_TEXELS = MORPH_TEXTURE_WIDTH * MORPH_TEXTURE_WIDTH;
/// Programs the resident-mesh shader has: one per combination of {normals, lit,
/// normal-mapped, skinned, depth-only, env-mapped, lightmapped, weight-heat,
-/// writes-the-surface-record}. Nine bits, not one: the geometry, the draw, the frame
+/// writes-the-surface-record, writes-the-motion}. Ten bits, not one: the geometry, the draw, the frame
/// and the pass each answer a different.
-static constexpr u32 MESH_VARIANTS = 512;
+static constexpr u32 MESH_VARIANTS = 1024;
/**
* @brief What a mesh vertex channel MEANS, and the attribute location it binds to.
* @details One vocabulary for the file format, the layout and the shaders: a
diff --git a/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp b/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp
index 68176bdde..3c826901e 100644
--- a/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp
+++ b/src/esengine/renderer/rhi/ShaderEmbeds.generated.hpp
@@ -507,6 +507,14 @@ out highp vec3 v_worldPos;
out highp vec4 v_shadowClip;
#endif
+#ifdef ES_VELOCITY
+
+
+
+
+out highp vec4 v_velCur;
+out highp vec4 v_velPrev;
+#endif
@@ -558,6 +566,18 @@ void main() {
#ifdef SHADOW_DEPTH
v_shadowClip = gl_Position;
#endif
+#ifdef ES_VELOCITY
+ {
+
+
+ highp vec4 L = vec4(local, 1.0);
+ highp vec4 before = esInstanceTint();
+ v_velCur = gl_Position;
+ highp vec4 prev = vec4(dot(esInstanceTexel(4), L), dot(esInstanceTexel(5), L),
+ dot(esInstanceTexel(6), L), dot(esInstanceTexel(7), L));
+ v_velPrev = vec4(prev.xy + before.xy * prev.w, prev.zw);
+ }
+#endif
v_probeSlot = float(ES_INSTANCE_ID);
@@ -587,6 +607,10 @@ in highp vec3 v_worldPos;
#ifdef SHADOW_DEPTH
in highp vec4 v_shadowClip;
#endif
+#ifdef ES_VELOCITY
+in highp vec4 v_velCur;
+in highp vec4 v_velPrev;
+#endif
uniform sampler2D u_texture;
#ifdef NORMAL_MAP
@@ -612,7 +636,14 @@ out vec4 fragColor;
#endif
void main() {
-#ifdef SHADOW_DEPTH
+#ifdef ES_VELOCITY
+
+
+
+ highp vec2 now = v_velCur.xy / v_velCur.w;
+ highp vec2 was = v_velPrev.xy / max(v_velPrev.w, 1e-6);
+ fragColor = vec4((was - now) * 0.5, gl_FragCoord.z, v_velPrev.w > 1e-6 ? 1.0 : 0.0);
+#elif defined(SHADOW_DEPTH)
@@ -701,6 +732,10 @@ struct VSOut {
@location(4) v_shadowClip : vec4f,
#endif
+#ifdef ES_VELOCITY
+ @location(4) v_velCur : vec4f,
+ @location(7) v_velPrev : vec4f,
+#endif
#ifdef MESH_LIGHTMAP
@location(5) v_lightmap : vec3f,
#endif
@@ -770,6 +805,17 @@ struct VSOut {
#ifdef SHADOW_DEPTH
out.v_shadowClip = out.pos;
#endif
+#ifdef ES_VELOCITY
+ {
+
+ let L = vec4f(local, 1.0);
+ let before = esInstanceTint();
+ out.v_velCur = out.pos;
+ let prev = vec4f(dot(esInstanceTexel(4), L), dot(esInstanceTexel(5), L),
+ dot(esInstanceTexel(6), L), dot(esInstanceTexel(7), L));
+ out.v_velPrev = vec4f(prev.xy + before.xy * prev.w, prev.zw);
+ }
+#endif
out.v_probeSlot = f32(inst);
#ifdef LIT
@@ -830,6 +876,10 @@ struct VSOut {
#ifdef SHADOW_DEPTH
@location(4) v_shadowClip : vec4f,
#endif
+#ifdef ES_VELOCITY
+ @location(4) v_velCur : vec4f,
+ @location(7) v_velPrev : vec4f,
+#endif
#ifdef MESH_LIGHTMAP
@location(5) v_lightmap : vec3f,
#endif
@@ -863,7 +913,12 @@ fn packNormal(n : vec3f) -> vec2f {
#ifdef MESH_LIGHTMAP
g_lightmap = v.v_lightmap;
#endif
-#ifdef SHADOW_DEPTH
+#ifdef ES_VELOCITY
+
+ let now = v.v_velCur.xy / v.v_velCur.w;
+ let was = v.v_velPrev.xy / max(v.v_velPrev.w, 1e-6);
+ return vec4f((was - now) * 0.5, v.pos.z, select(0.0, 1.0, v.v_velPrev.w > 1e-6));
+#elif defined(SHADOW_DEPTH)
diff --git a/src/esengine/renderer/webgpu/WebGPUDevice.cpp b/src/esengine/renderer/webgpu/WebGPUDevice.cpp
index 0d4a2af52..1873f34e3 100644
--- a/src/esengine/renderer/webgpu/WebGPUDevice.cpp
+++ b/src/esengine/renderer/webgpu/WebGPUDevice.cpp
@@ -860,6 +860,7 @@ u32 WebGPUDevice::colorVariantOf(WGPUTextureFormat format) {
switch (format) {
case WGPUTextureFormat_RGBA8UnormSrgb: return kColorSrgb8;
case WGPUTextureFormat_RGBA16Float: return kColorRgba16f;
+ case WGPUTextureFormat_RGBA32Float: return kColorRgba32f;
case WGPUTextureFormat_BGRA8Unorm: return kColorBgra8;
default: return kColorRgba8;
}
diff --git a/src/esengine/renderer/webgpu/WebGPUDevice.hpp b/src/esengine/renderer/webgpu/WebGPUDevice.hpp
index 08eff5b4a..e62b25c0b 100644
--- a/src/esengine/renderer/webgpu/WebGPUDevice.hpp
+++ b/src/esengine/renderer/webgpu/WebGPUDevice.hpp
@@ -55,7 +55,7 @@ public:
* are 8-bit unorm: a surface is usually the former, a render target
* the latter, so they cannot share a slot. */
enum ColorVariant : u32 {
- kColorRgba8 = 0, kColorBgra8 = 1, kColorSrgb8 = 2, kColorRgba16f = 3, kColorVariantCount = 4,
+ kColorRgba8 = 0, kColorBgra8 = 1, kColorSrgb8 = 2, kColorRgba16f = 3, kColorRgba32f = 4, kColorVariantCount = 5,
};
/** @brief Sample count of a pass — a third thing WebGPU validates a pipeline
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 67f087765..31c1f4f61 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2487,6 +2487,8 @@ export const NOTED = {
{"note":"Settings → Project → Rendering edits each quality level, including which effects it runs."},
"feat(render): the scene can draw a G-buffer beside its colour, and reflections read each surface's normal, roughness and metalness from it":
{"note":"Reflections now know how rough and how metal each surface is, from a G-buffer the frame draws on request."},
+ "feat(render): Temporal Anti-Aliasing follows meshes that move by themselves, from a motion pass that draws only what moved":
+ {"note":"Temporal Anti-Aliasing now follows meshes that move by themselves, from a motion pass that draws only what moved, so a thin fast object keeps its contrast."},
"feat(render): reflections no longer guess a surface, plain meshes record themselves, and what the ray never finds is the sky":
{"note":"Screen-space reflections leave a surface that recorded nothing alone, plain meshes now record a rough dielectric, and a puddle reflects the sky and what lies off screen."},
};