Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): a Temporal Anti-Aliasing effect accumulates frames, with the camera shifted a sub-pixel amount each one

yhh6c274a810e4b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9e5ba91c1..a08981f2e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- **Temporal Anti-Aliasing.** A Temporal Anti-Aliasing post-process effect smooths edges across frames. Add `taa` to a `PostProcessVolume` and the engine shifts the scene's picture by a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's last output as its history, and blends the two — reaching what multisampling cannot: shader aliasing and thin geometry. Where a pixel was last frame comes from the depth buffer and the previous camera, so a moving camera keeps its history; the history is clipped to the colours around it, so what no longer matches is dropped rather than smeared, and each camera keeps a history of its own. One `blend` setting says how much of each frame is new. Measured in the editor on a lit scene with multisampling off, in-between edge pixels went from 2% of edge pixels to 43% on WebGL2 and 44% on WebGPU, still frames held still, and a picture taken mid-slide of the camera matched the plain one to within a fraction of a level. Objects that move by themselves are not tracked — they are handled by the clip, so fast movers can look slightly soft.
- **Smaller meshes on disk.** Newly imported meshes are about 9% smaller on disk. A mesh whose indices all fit in 16 bits is now written with 16-bit indices (`.esmesh` version 4), half the index bytes, and read back as 32-bit so nothing downstream changes: the renderer, the lightmap and decal bakers and static batching see the same indices as before. A mesh that needs 32 bits is written exactly as before, on the version every earlier reader knows, and files written before this change still load. Measured on the Bistro meshes (1143 files, 116 MB), re-encoding saves 10.4 MB; existing `.esmesh` files are not rewritten until the model is re-imported.
- **Levels of detail generated at import.** Imported models can generate cheaper levels of detail automatically. Set **Generate LODs** to 1–3 levels in a model's import settings (or pass `--lods <n>` to `import-model`) and each mesh is collapsed into copies with about 50%, 20% and 7% of its triangles, written as `<mesh>_lod<n>.esmesh`, with a `LODGroup` on the entity that swaps them in as it shrinks on screen. The collapse bounds how far the surface may move and will not pull apart a UV seam or a hard edge; vertex colours and the lightmap UV come along unchanged. Skinned and morphing meshes, and meshes under a minimum triangle count, keep one level. Measured on the Bistro exterior (1143 meshes, 2.83M triangles), the 621 meshes large enough to get levels hold 1.35M, 0.44M and 0.10M triangles at levels 1, 2 and 3; the levels are extra files, so they make a large scene cheaper to draw rather than smaller to ship.
- **Steadier highlights on curved and bevelled surfaces.** Lit models no longer sparkle on rails, bevels and rounded trim. Where a surface's normal turns faster than a pixel can resolve, the highlight is the sum of many normals the pixel cannot tell apart, and shading it with one made it flicker as the camera moved. The Model shader now widens a highlight by how fast the vertex normal turns across the pixel — a steady, slightly rougher highlight in place of a flickering sharp one — on WebGL2 and WebGPU alike. Measured on a street scene shifted by sub-pixel steps, frame-to-frame shimmer on rails and lamp trim fell by about 5–8%; a surface that is flat or smooth is not touched.
diff --git a/desktop b/desktop
index 5a40236f2..9ec2e6596 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 5a40236f2ccde460164dc23ed94b4d13eddc581f
+Subproject commit 9ec2e659696086757dd704eb9af80681dd32c365
diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdx
index 05a4f9689..1b4c45f7d 100644
--- a/docs/astro/src/content/docs/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx
@@ -22,6 +22,7 @@ several can stack in one frame at no extra target cost beyond the passes.
| `chromaticAberration` | RGB fringing | `intensity` |
| `blur` | Gaussian blur | `intensity` |
| `fxaa` | Fast anti-aliasing | `intensity` |
+| `taa` | Temporal anti-aliasing: edges smoothed over several frames | `blend` |
| `lensDistortion` | Barrel / pincushion warp | `strength`, `zoom` |
| `pixelate` | Block quantization | `pixelSize` |
| `outline` | Ink-edge outlines (Sobel edge darkening) | `intensity`, `threshold`, `thickness` |
@@ -36,6 +37,23 @@ Android, desktop, Steam) runs on. `createWebApp({ msaaSamples })` sizes it, and
`fxaa` is the cheaper, blurrier alternative for devices where even one sample
extra is too much.
+### Temporal anti-aliasing
+
+`taa` smooths what MSAA cannot reach — shader aliasing and thin geometry — by
+accumulating frames. While the effect is in the chain the engine shifts the picture by
+a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's
+last output, and blends the two. Where a pixel was last frame comes from the depth
+buffer: the pixel is un-projected to the world and projected with the previous
+frame's camera, so a moving camera keeps its history. `blend` is the share of each
+frame that is new: lower is smoother and slower to follow a change, 1 is no accumulation.
+
+Put it **first** in the chain, so everything after it sees the settled image, and keep
+`msaaSamples` on — the two add. Each camera keeps its own history. What it does not
+know is how an object moves by itself: an object that moves is a different surface
+from its history, which is clipped to the colours around it, so fast movers can look
+slightly soft. It needs scene depth like SSAO does, so on WebGPU it cannot run with
+multisampling on a scene that has no depth resolve.
+
The uniform names above are the friendly labels; **in code they are `u_`-prefixed**
(`u_intensity`, `u_exposure`, …). Most effects are identity at their defaults
(`colorGrade`, `blur`, `chromaticAberration`, `fxaa`, `lensDistortion`, `lutGrade`),
diff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
index 1b4f1f0da..13142ef93 100644
--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
@@ -21,6 +21,7 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
| `chromaticAberration` | RGB 色散 | `intensity` |
| `blur` | 高斯模糊 | `intensity` |
| `fxaa` | 快速抗锯齿 | `intensity` |
+| `taa` | 时间性抗锯齿:边缘跨多帧平滑 | `blend` |
| `lensDistortion` | 桶形 / 枕形畸变 | `strength`、`zoom` |
| `pixelate` | 块状量化 | `pixelSize` |
| `outline` | 墨线描边(Sobel 边缘压暗) | `intensity`、`threshold`、`thickness` |
@@ -33,6 +34,18 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
`createWebApp({ msaaSamples })` 调它的大小,`msaaSamples: 1` 在填充率比边缘更要紧的设备上关掉它。
`fxaa` 是更便宜也更糊的替代品,留给连多一个采样都吃不下的设备。
+### 时间性抗锯齿
+
+`taa` 抹平 MSAA 够不到的东西——着色锯齿和细几何——办法是跨帧累积。效果在链路里时,引擎每帧把画面
+偏移一个不同的亚像素量(八个位置循环),保留这个效果上一帧的输出,再把两者混合。像素上一帧在哪,
+由深度缓冲得出:把像素反投影到世界,再用上一帧的相机投影回去,所以相机移动时历史还在。`blend`
+是每帧里新画面的占比:越低越平滑、对变化跟得越慢,1 就是不累积。
+
+把它放在链路**最前**,后面的效果看到的就是稳定后的画面,同时保留 `msaaSamples`——两者叠加。
+每个相机有自己的历史。它不知道物体自己怎么动:自己动的物体与它的历史是不同的表面,历史会被
+夹进周围的颜色里,所以快速移动的物体可能略软。它和 SSAO 一样要读场景深度,因此在 WebGPU 上,
+没有深度解析的多重采样场景里无法运行。
+
上面是友好标签;**在代码里它们带 `u_` 前缀**(`u_intensity`、`u_exposure`……)。多数效果
在默认值下是恒等(`colorGrade`、`blur`、`chromaticAberration`、`fxaa`、`lensDistortion`、
`lutGrade`),加进去不调参不改画面。少数一加就可见——`grayscale`(完全去色)、`vignette`、
diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.ts
index 7ea4b5eca..5857c892e 100644
--- a/sdk/src/postprocess/effects.ts
+++ b/sdk/src/postprocess/effects.ts
@@ -250,6 +250,19 @@ registerEffect({
],
});
+registerEffect({
+ type: 'taa',
+ label: 'Temporal Anti-Aliasing',
+ factory: () => postProcessEffects.createTaa(),
+ // One knob: how much of each frame is the new one. The camera shift, the history
+ // and last frame's camera are the engine's, not the effect's — they are not settings.
+ uniforms: [
+ // Lower is smoother and slower to follow a change; 1 is no accumulation at all,
+ // which is what a blend toward "no effect" fades to.
+ { name: 'u_blend', label: 'Blend', min: 0.02, max: 1, step: 0.01, defaultValue: 0.1, neutralValue: 1 },
+ ],
+});
+
registerEffect({
type: 'lutGrade',
label: 'LUT Grade',
diff --git a/sdk/src/postprocess/postProcessEffects.ts b/sdk/src/postprocess/postProcessEffects.ts
index 93b409cab..dfa3c0207 100644
--- a/sdk/src/postprocess/postProcessEffects.ts
+++ b/sdk/src/postprocess/postProcessEffects.ts
@@ -131,6 +131,158 @@ fn sampleLut(c : vec3f) -> vec3f {
return Material.compileShader(source);
},
+ /**
+ * Temporal anti-aliasing. The engine shifts the picture by a sub-pixel amount each
+ * frame and keeps this pass's last output as `u_history`; this blends the two.
+ *
+ * Where last frame's point was comes from the depth buffer: un-project the pixel to
+ * the world, project it with last frame's camera. That is exact for everything the
+ * camera moves past and wrong for an object that moved by itself — there the history
+ * is a different surface, so it is clipped into the colours the neighbourhood holds,
+ * which hides the mismatch instead of smearing it. The blend is done in a space where
+ * a bright pixel weighs less, or one firefly outlives the frame it appeared in.
+ */
+ createTaa(): ShaderHandle {
+ const GLSL = `
+vec3 toYCoCg(vec3 c) {
+ return vec3(0.25 * c.r + 0.5 * c.g + 0.25 * c.b, 0.5 * c.r - 0.5 * c.b, -0.25 * c.r + 0.5 * c.g - 0.25 * c.b);
+}
+vec3 fromYCoCg(vec3 c) {
+ return vec3(c.x + c.y - c.z, c.x + c.z, c.x - c.y - c.z);
+}
+/// Weighs a pixel by 1 / (1 + peak): an HDR highlight cannot dominate the average.
+vec3 compress(vec3 c) { return c / (1.0 + max(max(c.r, c.g), c.b)); }
+vec3 expand(vec3 c) { return c / max(1.0 - max(max(c.r, c.g), c.b), 1e-4); }`;
+ const WGSL = `
+fn toYCoCg(c : vec3f) -> vec3f {
+ return vec3f(0.25 * c.r + 0.5 * c.g + 0.25 * c.b, 0.5 * c.r - 0.5 * c.b, -0.25 * c.r + 0.5 * c.g - 0.25 * c.b);
+}
+fn fromYCoCg(c : vec3f) -> vec3f {
+ return vec3f(c.x + c.y - c.z, c.x + c.z, c.x - c.y - c.z);
+}
+fn compress(c : vec3f) -> vec3f { return c / (1.0 + max(max(c.r, c.g), c.b)); }
+fn expand(c : vec3f) -> vec3f { return c / max(1.0 - max(max(c.r, c.g), c.b), 1e-4); }`;
+ const source = `#pragma shader "PP TAA"
+#pragma version 300 es
+#pragma domain PostProcess
+#pragma param u_blend float default(0.1) range(0.02,1)
+#pragma param u_historyValid float default(0) range(0,1)
+#pragma param u_prevViewProjection0 color default(1,0,0,0)
+#pragma param u_prevViewProjection1 color default(0,1,0,0)
+#pragma param u_prevViewProjection2 color default(0,0,1,0)
+#pragma param u_prevViewProjection3 color default(0,0,0,1)
+#pragma param u_history texture default(black)
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+uniform highp sampler2D u_sceneDepth;
+out vec4 fragColor;
+
+${SSAO_GLSL}
+${GLSL}
+
+void main() {
+ ivec2 size = textureSize(u_texture, 0);
+ vec2 texel = 1.0 / vec2(size);
+ vec4 cur = texture(u_texture, v_texCoord);
+
+ // The neighbourhood the history has to look like: its mean and spread, in a
+ // space where colour and brightness are apart.
+ vec3 m1 = vec3(0.0);
+ vec3 m2 = vec3(0.0);
+ for (int y = -1; y <= 1; ++y) {
+ for (int x = -1; x <= 1; ++x) {
+ vec3 s = toYCoCg(compress(texture(u_texture, v_texCoord + vec2(float(x), float(y)) * texel).rgb));
+ m1 += s;
+ m2 += s * s;
+ }
+ }
+ vec3 mean = m1 / 9.0;
+ vec3 sigma = sqrt(max(m2 / 9.0 - mean * mean, vec3(0.0)));
+ vec3 curY = toYCoCg(compress(cur.rgb));
+
+ ivec2 dsize = textureSize(u_sceneDepth, 0);
+ float d = depthAt(screenTexel(v_texCoord, dsize), dsize);
+ vec3 world = worldFromDepth(v_texCoord, d);
+ vec4 prev = mat4(u_prevViewProjection0, u_prevViewProjection1,
+ u_prevViewProjection2, u_prevViewProjection3) * vec4(world, 1.0);
+ vec2 puv = prev.xy / max(prev.w, 1e-6) * 0.5 + 0.5;
+ bool seen = u_historyValid > 0.5 && prev.w > 1e-6
+ && puv.x >= 0.0 && puv.x <= 1.0 && puv.y >= 0.0 && puv.y <= 1.0;
+ if (!seen) { fragColor = cur; return; }
+
+ vec3 hist = toYCoCg(compress(texture(u_history, puv).rgb));
+ // Pulled toward the neighbourhood's mean until it sits inside the box the
+ // current frame spans: what it clips is what no longer matches.
+ vec3 lo = mean - sigma * 1.25;
+ vec3 hi = mean + sigma * 1.25;
+ vec3 dir = hist - mean;
+ vec3 over = max(abs(dir) / max((hi - lo) * 0.5, vec3(1e-5)), vec3(1.0));
+ hist = mean + dir / max(max(over.x, over.y), over.z);
+
+ // A fast-moving pixel trusts the new frame more: its history is thin.
+ float speed = length((puv - v_texCoord) / texel);
+ float blend = mix(u_blend, min(u_blend * 4.0, 1.0), clamp(speed, 0.0, 1.0));
+ vec3 outY = mix(hist, curY, blend);
+ fragColor = vec4(expand(fromYCoCg(outY)), cur.a);
+}
+#pragma end
+
+#pragma fragment wgsl
+${SSAO_WGSL}
+${WGSL}
+
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let size = vec2i(textureDimensions(t0, 0));
+ let texel = 1.0 / vec2f(size);
+ let cur = textureSampleLevel(t0, s0, v.v_texCoord, 0.0);
+
+ var m1 = vec3f(0.0);
+ var m2 = vec3f(0.0);
+ for (var y : i32 = -1; y <= 1; y = y + 1) {
+ for (var x : i32 = -1; x <= 1; x = x + 1) {
+ let s = toYCoCg(compress(textureSampleLevel(t0, s0, v.v_texCoord + vec2f(f32(x), f32(y)) * texel, 0.0).rgb));
+ m1 = m1 + s;
+ m2 = m2 + s * s;
+ }
+ }
+ let mean = m1 / 9.0;
+ let sigma = sqrt(max(m2 / 9.0 - mean * mean, vec3f(0.0)));
+ let curY = toYCoCg(compress(cur.rgb));
+
+ let dsize = vec2i(textureDimensions(t7, 0));
+ let d = depthAt(screenTexel(v.v_texCoord, dsize), dsize);
+ let world = worldFromDepth(v.v_texCoord, d);
+ let vp = mat4x4f(mc.u_prevViewProjection0, mc.u_prevViewProjection1,
+ mc.u_prevViewProjection2, mc.u_prevViewProjection3);
+ let prev = vp * vec4f(world, 1.0);
+ // This device's uv runs down the rows, so clip y is flipped on the way back.
+ let pw = max(prev.w, 1e-6);
+ let puv = vec2f(prev.x / pw * 0.5 + 0.5, 0.5 - prev.y / pw * 0.5);
+ let seen = mc.u_historyValid > 0.5 && prev.w > 1e-6
+ && puv.x >= 0.0 && puv.x <= 1.0 && puv.y >= 0.0 && puv.y <= 1.0;
+ if (!seen) { return cur; }
+
+ var hist = toYCoCg(compress(textureSampleLevel(u_history, u_history_s, puv, 0.0).rgb));
+ let lo = mean - sigma * 1.25;
+ let hi = mean + sigma * 1.25;
+ let dir = hist - mean;
+ let over = max(abs(dir) / max((hi - lo) * 0.5, vec3f(1e-5)), vec3f(1.0));
+ hist = mean + dir / max(max(over.x, over.y), over.z);
+
+ let speed = length((puv - v.v_texCoord) / texel);
+ let blend = mix(mc.u_blend, min(mc.u_blend * 4.0, 1.0), clamp(speed, 0.0, 1.0));
+ let outY = mix(hist, curY, blend);
+ return vec4f(expand(fromYCoCg(outY)), cur.a);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
createBlur(): ShaderHandle {
const source = `#pragma shader "PP Blur"
#pragma version 300 es
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.cpp b/src/esengine/renderer/frame/PostProcessPipeline.cpp
index 183a05936..05089a7ac 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.cpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.cpp
@@ -179,6 +179,7 @@ void PostProcessPipeline::shutdown() {
for (auto& pass : screenPasses_) releasePassResources(pass);
passes_.clear();
screenPasses_.clear();
+ temporal_.clear();
auto* device = &device_;
if (screen_quad_vbo_ != BufferHandle::Invalid && device) {
@@ -225,6 +226,8 @@ void PostProcessPipeline::resize(u32 width, u32 height) {
// handed out again — they would just sit there holding memory.
if (graph_) graph_->releasePool();
sceneResource_ = rg::kNoResource;
+ // History at the old size names texels this picture no longer has.
+ temporal_.clear();
ensureGraph();
@@ -275,6 +278,89 @@ void PostProcessPipeline::setPassContinuesEffect(const std::string& passName) {
if (auto* pass = findPass(passName)) pass->continuesEffect = true;
}
+bool PostProcessPipeline::passIsTemporal(const PostProcessPass& pass) {
+ const MaterialUniformLayout* layout = context_.materials().layoutFor(pass.shader);
+ if (!layout) return false;
+ for (const auto& slot : layout->textures) {
+ if (slot.name == "u_history") return true;
+ }
+ return false;
+}
+
+bool PostProcessPipeline::temporalActive() {
+ if (!initialized_ || bypass_) return false;
+ for (const auto& pass : passes_) {
+ if (pass.enabled && passIsTemporal(pass)) return true;
+ }
+ return false;
+}
+
+namespace {
+/// Van der Corput in @p base: the low-discrepancy sequence a jitter walks the pixel with.
+f32 halton(u32 index, u32 base) {
+ f32 f = 1.0f, r = 0.0f;
+ while (index > 0) {
+ f /= static_cast<f32>(base);
+ r += f * static_cast<f32>(index % base);
+ index /= base;
+ }
+ return r;
+}
+/// Samples before the pattern repeats. Eight covers a pixel evenly enough that the
+/// accumulated image is stable, and repeats fast enough to settle in a few frames.
+constexpr u32 kJitterPhases = 8;
+} // namespace
+
+glm::vec2 PostProcessPipeline::beginTemporalView(const glm::mat4& viewProjection, u64 key) {
+ TemporalState& state = temporal_[key];
+ temporalKey_ = key;
+ if (state.frame > 0) state.previous = state.current;
+ else state.previous = viewProjection;
+ state.current = viewProjection;
+ ++state.frame;
+ // A chain that did not write last frame leaves history of a picture this one no
+ // longer follows (a camera that was off, a frame the host skipped).
+ if (state.writtenFrame + 1 != state.frame) state.valid = false;
+ const u32 phase = (state.frame % kJitterPhases) + 1;
+ const f32 jx = halton(phase, 2) - 0.5f;
+ const f32 jy = halton(phase, 3) - 0.5f;
+ const f32 w = static_cast<f32>(std::max(width_, 1u));
+ const f32 h = static_cast<f32>(std::max(height_, 1u));
+ return {jx * 2.0f / w, jy * 2.0f / h};
+}
+
+PostProcessPipeline::TemporalState* PostProcessPipeline::prepareTemporal() {
+ auto it = temporal_.find(temporalKey_);
+ // Never begun: nothing shifted this frame's picture, and nothing is remembered.
+ if (it == temporal_.end() || it->second.frame == 0) return nullptr;
+ TemporalState& state = it->second;
+ if (state.history[0] && state.width == width_ && state.height == height_
+ && state.format == interFormat()) {
+ return &state;
+ }
+ for (auto& fbo : state.history) {
+ FramebufferSpec spec;
+ spec.colorFormat = interFormat();
+ spec.width = width_;
+ spec.height = height_;
+ spec.depthStencil = false;
+ // The reprojected read lands between texels, so it wants the filter.
+ spec.linearFilter = true;
+ fbo = Framebuffer::create(device_, spec);
+ if (!fbo) {
+ ES_LOG_ERROR("PostProcessPipeline: Failed to create a temporal history target");
+ state.history[0].reset();
+ state.history[1].reset();
+ return nullptr;
+ }
+ }
+ state.width = width_;
+ state.height = height_;
+ state.format = interFormat();
+ state.valid = false;
+ return &state;
+}
+
void PostProcessPipeline::releasePassResources(PostProcessPass& pass) {
if (pass.paramUbo != BufferHandle::Invalid) {
device_.deleteBuffer(pass.paramUbo);
@@ -472,6 +558,20 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
// layout when the effect compiled. Resolution-style inputs come from the
// injected u_viewport, so nothing per-pass remains loose.
const MaterialUniformLayout* layout = context_.materials().layoutFor(pass.shader);
+ TemporalState* temporal = nullptr;
+ if (pass.temporal) {
+ if (auto it = temporal_.find(temporalKey_); it != temporal_.end()) temporal = &it->second;
+ }
+ if (temporal) {
+ // The previous picture's camera, as columns: a world point times it is where
+ // that point was one frame ago. Re-sent each frame because it is each frame's.
+ for (int c = 0; c < 4; ++c) {
+ pass.vec4Uniforms["u_prevViewProjection" + std::to_string(c)] =
+ glm::vec4(temporal->previous[c]);
+ }
+ pass.floatUniforms["u_historyValid"] = temporal->valid ? 1.0f : 0.0f;
+ pass.paramDirty = true;
+ }
if (layout) {
if (layout->blockSize > 0) {
if (pass.paramDirty || pass.paramBytes.size() < layout->blockSize) {
@@ -517,6 +617,9 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
// an unset param gets its declared default (white/black/flatnormal).
for (const auto& slot : layout->textures) {
TextureHandle gpu = context_.materials().builtinDefault(slot.defaultTexture);
+ if (temporal && slot.name == "u_history" && temporal->history[temporal->readIndex]) {
+ gpu = temporal->history[temporal->readIndex]->getColorAttachment();
+ }
for (const auto& [name, glId] : pass.textureUniforms) {
if (name == slot.name && glId != 0) { gpu = TextureHandle{glId}; break; }
}
@@ -626,8 +729,25 @@ void PostProcessPipeline::runChain(std::vector<PostProcessPass>& passes, rg::Res
// chain's size — so it draws full-size whatever fraction it asked for.
rg::TargetDesc writeDesc = (i == lastEnabled) ? blitDesc : desc;
if (i != lastEnabled) writeDesc.scale = pass.scale;
- node.write = graph_->createTarget(writeDesc);
- node.execute = [this, &pass](const rg::PassContext& ctx) { renderPass(pass, ctx); };
+ // A temporal pass writes the history target itself, so that what it wrote is
+ // still there next frame, and the passes after it read it from there.
+ TemporalState* temporal = (&passes == &passes_ && passIsTemporal(pass)) ? prepareTemporal() : nullptr;
+ pass.temporal = temporal != nullptr;
+ if (temporal) {
+ const u32 write = temporal->readIndex ^ 1u;
+ Framebuffer& target = *temporal->history[write];
+ node.write = graph_->importTarget(target.handle(), width_, height_, /*output=*/false,
+ target.getColorAttachment());
+ node.execute = [this, &pass, temporal, write](const rg::PassContext& ctx) {
+ renderPass(pass, ctx);
+ temporal->readIndex = write;
+ temporal->valid = true;
+ temporal->writtenFrame = temporal->frame;
+ };
+ } else {
+ node.write = graph_->createTarget(writeDesc);
+ node.execute = [this, &pass](const rg::PassContext& ctx) { renderPass(pass, ctx); };
+ }
input = node.write;
graph_->addPass(std::move(node));
}
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.hpp b/src/esengine/renderer/frame/PostProcessPipeline.hpp
index 5513c1e4f..490e9dfee 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.hpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.hpp
@@ -63,6 +63,10 @@ struct PostProcessPass {
/// (0 = input, 1 = scene).
std::vector<std::pair<std::string, u32>> textureUniforms;
+ /// Set by the chain when the pass keeps its output for the next frame (its
+ /// shader declares `u_history`); the pass then reads the previous frame there.
+ bool temporal = false;
+
/// Fraction of the chain's size this pass draws at; the pass consuming it
/// upsamples. The graph resolves the fraction, and u_viewport still reports
/// the CANVAS, so a fractional pass must read its extent off its texture.
@@ -200,6 +204,25 @@ public:
/** Closes the capture once the graph has run it. */
void chainDone();
+ /**
+ * @brief Whether this frame's chain holds a temporal effect.
+ *
+ * @details An effect is temporal when its shader declares a `u_history` texture:
+ * the engine then keeps the pass's last output for it and shifts the
+ * picture by a sub-pixel amount each frame. The pass list is the
+ * answer, so there is no flag to disagree with it.
+ */
+ bool temporalActive();
+
+ /**
+ * @brief Starts a frame of the temporal view @p key, and answers how far to shift it.
+ *
+ * @param key Names the view whose history this is — two cameras must not
+ * blend into one history. 0 for the screen.
+ * @return The shift in clip-square units, to hand to the camera constants.
+ */
+ glm::vec2 beginTemporalView(const glm::mat4& viewProjection, u64 key);
+
/**
* @brief Checks if pipeline is initialized
*/
@@ -454,6 +477,26 @@ private:
u32 output_vp_w_ = 0;
u32 output_vp_h_ = 0;
+ /// What a temporal pass remembers of one view between frames.
+ struct TemporalState {
+ glm::mat4 current{1.0f};
+ glm::mat4 previous{1.0f};
+ u32 frame = 0;
+ /// The frame whose chain last wrote the history; a gap means the history
+ /// is of a picture this one no longer follows.
+ u32 writtenFrame = 0;
+ Unique<Framebuffer> history[2];
+ u32 readIndex = 0;
+ bool valid = false;
+ u32 width = 0;
+ u32 height = 0;
+ GfxPixelFormat format = GfxPixelFormat::RGBA8;
+ };
+ std::unordered_map<u64, TemporalState> temporal_;
+ u64 temporalKey_ = 0;
+ bool passIsTemporal(const PostProcessPass& pass);
+ TemporalState* prepareTemporal();
+
Unique<Framebuffer> screenFBO_;
bool screenFBOCreated_ = false;
bool screenCaptureActive_ = false;
diff --git a/src/esengine/renderer/frame/RenderContext.cpp b/src/esengine/renderer/frame/RenderContext.cpp
index 0e3c86a13..3ba058c1d 100644
--- a/src/esengine/renderer/frame/RenderContext.cpp
+++ b/src/esengine/renderer/frame/RenderContext.cpp
@@ -166,6 +166,17 @@ void RenderContext::updateCameraConstants(const glm::mat4& viewProjection) {
viewProjection);
}
+void RenderContext::updateCameraConstants(const glm::mat4& viewProjection, const glm::vec2& jitterNdc) {
+ // Added to clip xy in proportion to w, so it is the same shift in NDC under a
+ // perspective divide and under none: row 0 and 1 each gain a multiple of row 3.
+ glm::mat4 jittered = viewProjection;
+ for (int col = 0; col < 4; ++col) {
+ jittered[col][0] += jitterNdc.x * viewProjection[col][3];
+ jittered[col][1] += jitterNdc.y * viewProjection[col][3];
+ }
+ uploadFrameConstants(toClipDepthRange(jittered, device_.clipDepthRange()), viewProjection);
+}
+
void RenderContext::updateFrameConstants(const glm::mat4& viewProjection) {
uploadFrameConstants(viewProjection, viewProjection);
}
diff --git a/src/esengine/renderer/frame/RenderContext.hpp b/src/esengine/renderer/frame/RenderContext.hpp
index d17c9ab03..21fcc163b 100644
--- a/src/esengine/renderer/frame/RenderContext.hpp
+++ b/src/esengine/renderer/frame/RenderContext.hpp
@@ -149,6 +149,14 @@ public:
* the device's. This is the one place that conversion happens.
*/
void updateCameraConstants(const glm::mat4& viewProjection);
+ /**
+ * @brief The same, with the picture shifted by @p jitterNdc (a fraction of the
+ * clip square) — what temporal anti-aliasing samples a different
+ * sub-pixel position with each frame.
+ * @details Only the UPLOADED matrix moves. The engine's own copy, which culling
+ * and the camera position are read from, is the unshifted one.
+ */
+ void updateCameraConstants(const glm::mat4& viewProjection, const glm::vec2& jitterNdc);
/**
* @brief Point the frame at a pass whose projection already suits the device.
diff --git a/src/esengine/renderer/frame/RenderFrame.cpp b/src/esengine/renderer/frame/RenderFrame.cpp
index dedbc911d..a2855aaac 100644
--- a/src/esengine/renderer/frame/RenderFrame.cpp
+++ b/src/esengine/renderer/frame/RenderFrame.cpp
@@ -220,7 +220,16 @@ void RenderFrame::resize(u32 width, u32 height) {
// The scene pass's body. The graph has bound the target and set the camera's
// viewport; this is the draw and the constants it needs current.
void RenderFrame::drawScene() {
- context_.updateCameraConstants(view_projection_);
+ // A temporal effect in the chain asks for the picture to be shifted by a different
+ // sub-pixel amount each frame, so its history samples what a single frame cannot.
+ // The scene only: the overlay and the shadow maps are drawn with constants of their own.
+ glm::vec2 jitter{0.0f};
+#ifdef ES_ENABLE_POSTPROCESS
+ if (post_process_ && post_process_->isEngaged() && post_process_->temporalActive()) {
+ jitter = post_process_->beginTemporalView(view_projection_, current_target_);
+ }
+#endif
+ context_.updateCameraConstants(view_projection_, jitter);
// Where THIS camera's pixels sit inside the 2D shadow mask. Per camera because the
// mask is screen space: two cameras drew into two parts of one target, and a
// fragment has to read the part its own camera filled.
diff --git a/src/esengine/renderer/graph/RenderGraph.cpp b/src/esengine/renderer/graph/RenderGraph.cpp
index 84f280cc4..edb8a5ac0 100644
--- a/src/esengine/renderer/graph/RenderGraph.cpp
+++ b/src/esengine/renderer/graph/RenderGraph.cpp
@@ -53,10 +53,12 @@ ResourceId RenderGraph::importTexture(TextureHandle texture, u32 width, u32 heig
return static_cast<ResourceId>(resources_.size() - 1);
}
-ResourceId RenderGraph::importTarget(FramebufferHandle target, u32 width, u32 height, bool output) {
+ResourceId RenderGraph::importTarget(FramebufferHandle target, u32 width, u32 height, bool output,
+ TextureHandle readable) {
Resource res;
res.kind = Kind::ImportedTarget;
res.target = target;
+ res.texture = readable;
res.width = width;
res.height = height;
resources_.push_back(res);
diff --git a/src/esengine/renderer/graph/RenderGraph.hpp b/src/esengine/renderer/graph/RenderGraph.hpp
index efdc6e4fa..8b5f45729 100644
--- a/src/esengine/renderer/graph/RenderGraph.hpp
+++ b/src/esengine/renderer/graph/RenderGraph.hpp
@@ -155,8 +155,12 @@ public:
* here did not contribute to the image and is dropped. Set output
* false for a persistent intermediate capture: importing it must
* not move the final output away from the scene or its post stack.
+ * @p readable is the colour texture behind @p target, so a later pass
+ * can read what this one wrote — a history buffer is imported to be
+ * written and then sampled, by the next pass and by the next frame.
*/
- ResourceId importTarget(FramebufferHandle target, u32 width, u32 height, bool output = true);
+ ResourceId importTarget(FramebufferHandle target, u32 width, u32 height, bool output = true,
+ TextureHandle readable = TextureHandle::Invalid);
/** A target the graph allocates from the pool and recycles after its last read. */
ResourceId createTarget(const TargetDesc& desc);
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 1acae2f01..34b33707b 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2471,4 +2471,6 @@ export const NOTED = {
{"note":"Imported models can generate cheaper levels of detail automatically."},
"feat(assets): meshes with fewer than 65536 vertices are written with 16-bit indices":
{"note":"Newly imported meshes are about 9% smaller on disk."},
+ "feat(render): a Temporal Anti-Aliasing effect accumulates frames, with the camera shifted a sub-pixel amount each one":
+ {"note":"A Temporal Anti-Aliasing post-process effect smooths edges across frames."},
};