Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): Auto Exposure and Depth of Field effects, and a pass that remembers its last output without shifting the picture

yhhedd814a63bd8
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 539e67ec1..57672089e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- **Auto Exposure and Depth of Field.** Auto Exposure and Depth of Field post-process effects are available. `autoExposure` measures the average log luminance of the picture (leaving out pixels darker than 2^-8, so a black backdrop does not open the exposure to its limit), turns it into the exposure that puts it at middle grey, and moves toward that over time from where the last frame left it — brighter faster than darker, per second, within `minEV` and `maxEV`, with a `bias` in stops. To keep the settled value between frames a post-process shader can now declare a `u_feedback` texture: the engine keeps that pass's last output, at the pass's own size, and binds it there, without shifting the picture the way `u_history` does. `depthOfField` blurs by distance from `focusDistance`, with a sharp slab of `focusRange`, a blur radius that grows with how far a surface is from focus up to `maxBlur`, and a gather that counts a neighbour only where its own blur reaches back, so a sharp object does not bleed into a blurred background. Verified in the editor on both backends: a dim scene brightens (mean 6.9 to 16.5) and a bright one darkens (30.7 to 20.4), a drop in light is first dark and then recovered over real time (5.2 then 15.8), and focusing close takes the squared edge energy down to 21% of the sharp frame.
- **Height Fog and Light Shafts.** Height Fog and Light Shafts post-process effects add atmosphere. `heightFog` is exponential height fog with a closed-form integral — no marching — so a valley fills and a hilltop clears, the horizon turns hazy while the zenith stays clear, and looking toward the sun the haze takes the sun's colour; sky pixels run through the same formula with a very long ray, and `skyFog` sets how much haze the sky keeps. `lightShafts` marches each pixel toward the sun's place on screen and counts how much of the way is open sky, leaving rays through whatever stands in front of the sun. The sun's direction is set on the effects (azimuth and elevation), not read from a light. Neither is a shadowed volume: fog is lit evenly. Both run on WebGL2 and WebGPU; with an output transform of None, a floor fully taken by the fog reads exactly the authored colour on both.
- **Screen Space Reflections.** A Screen Space Reflections post-process effect mirrors props in floors. Add `ssr` to a `PostProcessVolume`: with no G-buffer, the surface normal is rebuilt from the depth buffer, the reflected ray is marched through the world and projected back to the screen at every step, and a step landing just behind the visible surface is a hit, refined by bisection. The reflection is blended over the pixel by a Schlick Fresnel term — faint face-on, strong at a grazing angle — and blurred by how far the ray travelled, which stands in for roughness. `intensity`, `maxDistance`, `thickness` and `floorBias` are the settings; `floorBias` at 1 lets only up-facing surfaces reflect, since the chain does not know a material's roughness. What is off screen or in the sky is not reflected; the pixel is left as it was. It reads the same depth buffer as ambient occlusion, runs on WebGL2 and WebGPU alike, and its per-pixel start offset is meant to be averaged away by `taa`. Verified in the editor on a lit scene with both backends: the props appear in the floor they stand on and nothing else on screen changes.
- **Temporal Anti-Aliasing.** A Temporal Anti-Aliasing post-process effect smooths edges across frames. Add `taa` to a `PostProcessVolume` and the engine shifts the scene's picture by a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's last output as its history, and blends the two — reaching what multisampling cannot: shader aliasing and thin geometry. Where a pixel was last frame comes from the depth buffer and the previous camera, so a moving camera keeps its history; the history is clipped to the colours around it, so what no longer matches is dropped rather than smeared, and each camera keeps a history of its own. One `blend` setting says how much of each frame is new. Measured in the editor on a lit scene with multisampling off, in-between edge pixels went from 2% of edge pixels to 43% on WebGL2 and 44% on WebGPU, still frames held still, and a picture taken mid-slide of the camera matched the plain one to within a fraction of a level. Objects that move by themselves are not tracked — they are handled by the clip, so fast movers can look slightly soft.
diff --git a/desktop b/desktop
index 01f419305..d273eb368 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 01f41930550a72eea767fd7ba2a9b5ced69b88b5
+Subproject commit d273eb36826f137461d297fa00d6aad0b9c8c55b
diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdx
index 6374aba11..1434cb253 100644
--- a/docs/astro/src/content/docs/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx
@@ -25,6 +25,8 @@ several can stack in one frame at no extra target cost beyond the passes.
| `distanceFog` | Linear fog by distance from the camera | `near`, `far`, `intensity`, colour |
| `heightFog` | Exponential height fog with the sun's glow and a hazy horizon | `density`, `heightFalloff`, `baseHeight`, `sunAzimuth`, `sunElevation`, `skyFog`, colours |
| `lightShafts` | Sun rays through whatever stands in front of it | `intensity`, `sunAzimuth`, `sunElevation`, `decay`, `reach` |
+| `autoExposure` | Eye adaptation: exposure follows how bright the picture is | `bias`, `speedUp`, `speedDown`, `minEV`, `maxEV` |
+| `depthOfField` | Blur by distance from a focus | `focusDistance`, `focusRange`, `aperture`, `maxBlur` |
| `ssr` | Screen-space reflections: props mirrored in the floor they stand on | `intensity`, `maxDistance`, `thickness`, `floorBias` |
| `taa` | Temporal anti-aliasing: edges smoothed over several frames | `blend` |
| `lensDistortion` | Barrel / pincushion warp | `strength`, `zoom` |
@@ -41,6 +43,27 @@ Android, desktop, Steam) runs on. `createWebApp({ msaaSamples })` sizes it, and
`fxaa` is the cheaper, blurrier alternative for devices where even one sample
extra is too much.
+### Auto exposure
+
+`autoExposure` is the eye: it measures how bright the picture is and settles the exposure that puts
+it at middle grey. The measurement is the average of the **log** of the luminance, since brightness
+is judged by ratio and a raw average is decided by whatever lamp is in view; pixels darker than 2^-8
+are left out, because a black backdrop is not a dark scene. The result moves toward its target over time
+from where the last frame left it — brighter faster (`speedUp`) than darker (`speedDown`), per second, so it does not
+depend on the frame rate — and `minEV` / `maxEV` bound how far it may go. `bias` adds stops to the result for a
+scene that should sit darker or brighter than grey. It works in the HDR chain; in an 8-bit chain the
+measurement is coarse and the adaptation steps are visible. Put it before the output transform and
+before `bloom`, so the glow sees the exposure the eye settled on.
+
+### Depth of field
+
+`depthOfField` blurs what is not at `focusDistance` (a world length from the camera), leaving a slab `focusRange` deep
+sharp. The blur radius, in pixels, grows with how far a surface is from the focus as a share of its distance, up to `maxBlur`;
+`aperture` is the radius of something infinitely far, and the sky counts as that. Each pixel gathers its neighbours over its own
+circle, counting a neighbour only to the extent that neighbour's own blur reaches back: a sharp object in front of a
+blurred background does not bleed into it, while a blurred foreground spreads over what is behind it. The gather is
+rotated per pixel, so it leaves a fine grain that `taa` averages away.
+
### Fog and atmosphere
`heightFog` is fog that thins with height: a valley fills, a hilltop clears. Its density falls
diff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
index 5a6cecd97..832c96515 100644
--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
@@ -24,6 +24,8 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
| `distanceFog` | 按到相机的距离做线性雾 | `near`、`far`、`intensity`、颜色 |
| `heightFog` | 指数高度雾,带太阳辉光和朦胧的地平线 | `density`、`heightFalloff`、`baseHeight`、`sunAzimuth`、`sunElevation`、`skyFog`、颜色 |
| `lightShafts` | 太阳光穿过前方遮挡物形成的光束 | `intensity`、`sunAzimuth`、`sunElevation`、`decay`、`reach` |
+| `autoExposure` | 眼睛适应:曝光跟随画面亮度 | `bias`、`speedUp`、`speedDown`、`minEV`、`maxEV` |
+| `depthOfField` | 按到焦点的距离模糊 | `focusDistance`、`focusRange`、`aperture`、`maxBlur` |
| `ssr` | 屏幕空间反射:道具映在它们所站的地面上 | `intensity`、`maxDistance`、`thickness`、`floorBias` |
| `taa` | 时间性抗锯齿:边缘跨多帧平滑 | `blend` |
| `lensDistortion` | 桶形 / 枕形畸变 | `strength`、`zoom` |
@@ -38,6 +40,21 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
`createWebApp({ msaaSamples })` 调它的大小,`msaaSamples: 1` 在填充率比边缘更要紧的设备上关掉它。
`fxaa` 是更便宜也更糊的替代品,留给连多一个采样都吃不下的设备。
+### 自动曝光
+
+`autoExposure` 就是眼睛:它测量画面有多亮,并稳定到把画面放在中灰的曝光上。测量的是亮度**对数**的平均,
+因为亮度是按比例判断的,而原始平均会被画面里的某盏灯决定;暗于 2^-8 的像素不计入,因为黑色背景不等于暗场景。
+结果从上一帧的位置随时间趋近目标——变亮比变暗快(`speedUp` / `speedDown`,按秒计,与帧率无关)——
+`minEV` / `maxEV` 限定范围,`bias` 给结果加几档,用于想比中灰更暗或更亮的场景。它在 HDR 链路里工作;
+8 位链路里测量粗糙、适应的台阶可见。放在输出变换之前、`bloom` 之前,让辉光看到眼睛稳定后的曝光。
+
+### 景深
+
+`depthOfField` 模糊不在 `focusDistance`(到相机的世界长度)的东西,`focusRange` 深的一片保持清晰。模糊半径(像素)随表面
+离焦点的距离占其自身距离的比例增长,封顶 `maxBlur`;`aperture` 是无限远处东西的半径,天空算作这样。每个像素在自己的圆里
+收集邻居,只在邻居自己的模糊够得着这里时才计入:清晰的前景物体不会渗进模糊的背景,而模糊的前景会铺到它后面的东西上。
+收集按像素旋转,留下一层细颗粒,由 `taa` 平均掉。
+
### 雾与大气
`heightFog` 是随高度变薄的雾:山谷被填满,山顶是清朗的。它的密度按指数下降,所以射线穿过多少雾有
diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.ts
index 11e351e23..2d8ef9187 100644
--- a/sdk/src/postprocess/effects.ts
+++ b/sdk/src/postprocess/effects.ts
@@ -264,6 +264,44 @@ registerEffect({
],
});
+registerEffect({
+ type: 'autoExposure',
+ label: 'Auto Exposure',
+ factory: () => postProcessEffects.createAutoExposureApply(),
+ uniforms: [
+ // Stops added to the exposure the measurement chose: the look of a scene that should be
+ // darker or brighter than middle grey.
+ { name: 'u_bias', label: 'Compensation (EV)', min: -4, max: 4, step: 0.1, defaultValue: 0, neutralValue: 0 },
+ // Stops per second's worth of rate, in each direction: an eye adapts to light faster than to dark.
+ { name: 'u_speedUp', label: 'Brighten Speed', min: 0.1, max: 20, step: 0.1, defaultValue: 3 },
+ { name: 'u_speedDown', label: 'Darken Speed', min: 0.1, max: 20, step: 0.1, defaultValue: 1 },
+ { name: 'u_minEV', label: 'Min EV', min: -10, max: 0, step: 0.5, defaultValue: -4 },
+ { name: 'u_maxEV', label: 'Max EV', min: 0, max: 10, step: 0.5, defaultValue: 4 },
+ ],
+ // A few texels measure the picture, one pass keeps the settled exposure from frame to frame,
+ // and the last applies it. The first two are small; the engine keeps the second's output.
+ multiPass: [
+ { name: 'ae_measure', factory: () => postProcessEffects.createAutoExposureMeasure(), scale: 0.0625 },
+ { name: 'ae_adapt', factory: () => postProcessEffects.createAutoExposureAdapt(), scale: 0.01 },
+ { name: 'ae_apply', factory: () => postProcessEffects.createAutoExposureApply() },
+ ],
+});
+
+registerEffect({
+ type: 'depthOfField',
+ label: 'Depth of Field',
+ factory: () => postProcessEffects.createDepthOfField(),
+ uniforms: [
+ // A WORLD length from the camera to what is in focus.
+ { name: 'u_focusDistance', label: 'Focus Distance', min: 1, max: 5000, step: 1, defaultValue: 600 },
+ // How deep a slab around it stays sharp, before the blur starts.
+ { name: 'u_focusRange', label: 'Focus Range', min: 0, max: 2000, step: 1, defaultValue: 100 },
+ // The blur, in pixels, of something infinitely far: a bigger lens blurs sooner and harder.
+ { name: 'u_aperture', label: 'Aperture (px)', min: 0, max: 64, step: 0.5, defaultValue: 24, neutralValue: 0 },
+ { name: 'u_maxBlur', label: 'Max Blur (px)', min: 1, max: 32, step: 0.5, defaultValue: 14 },
+ ],
+});
+
registerEffect({
type: 'ssao',
label: 'Ambient Occlusion',
diff --git a/sdk/src/postprocess/postProcessEffects.ts b/sdk/src/postprocess/postProcessEffects.ts
index cf28308fc..413466be2 100644
--- a/sdk/src/postprocess/postProcessEffects.ts
+++ b/sdk/src/postprocess/postProcessEffects.ts
@@ -1442,6 +1442,286 @@ fn ign(p : vec2f) -> f32 {
return Material.compileShader(source);
},
+ /**
+ * Auto exposure, pass 1 of 3: how bright is the picture, in a few texels. Each texel of this
+ * small target averages the LOG of the luminance over a block of the scene — the log, because
+ * the eye judges brightness by ratio, and an average of raw values is decided by whatever
+ * lamp or sun is in view. Pixels darker than 2^-8 are left out of the average — a black
+ * backdrop is not a dark scene, and counting it would open the exposure to the limit — and
+ * what share of the block counted travels beside the average. Both are stored as 0..1 codes
+ * (the average over -12..8 stops), because the target may be 8-bit when the project is not HDR.
+ */
+ createAutoExposureMeasure(): ShaderHandle {
+ const source = `#pragma shader "PP Auto Exposure Measure"
+#pragma version 300 es
+#pragma domain PostProcess
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+out vec4 fragColor;
+
+void main() {
+ vec2 texel = 1.0 / vec2(textureSize(u_texture, 0));
+ float sum = 0.0;
+ float counted = 0.0;
+ // Sixteen bilinear taps, four pixels apart: each reads 2x2 pixels and together they
+ // cover the 16x16 block this texel stands for.
+ for (int j = 0; j < 4; ++j) {
+ for (int i = 0; i < 4; ++i) {
+ vec3 c = texture(u_texture, v_texCoord + (vec2(float(i), float(j)) - 1.5) * 4.0 * texel).rgb;
+ float lum = dot(c, vec3(0.2126, 0.7152, 0.0722));
+ if (lum > 0.0039) { sum += log2(lum); counted += 1.0; }
+ }
+ }
+ float code = counted > 0.0 ? clamp((sum / counted + 12.0) / 20.0, 0.0, 1.0) : 0.0;
+ fragColor = vec4(code, counted / 16.0, 0.0, 1.0);
+}
+#pragma end
+
+#pragma fragment wgsl
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let texel = 1.0 / vec2f(vec2i(textureDimensions(t0, 0)));
+ var sum = 0.0;
+ var counted = 0.0;
+ for (var j : i32 = 0; j < 4; j = j + 1) {
+ for (var i : i32 = 0; i < 4; i = i + 1) {
+ let c = textureSampleLevel(t0, s0, v.v_texCoord + (vec2f(f32(i), f32(j)) - 1.5) * 4.0 * texel, 0.0).rgb;
+ let lum = dot(c, vec3f(0.2126, 0.7152, 0.0722));
+ if (lum > 0.0039) { sum = sum + log2(lum); counted = counted + 1.0; }
+ }
+ }
+ var code = 0.0;
+ if (counted > 0.0) { code = clamp((sum / counted + 12.0) / 20.0, 0.0, 1.0); }
+ return vec4f(code, counted / 16.0, 0.0, 1.0);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
+ /**
+ * Auto exposure, pass 2 of 3: the exposure the eye has settled on. The measurement is averaged
+ * to one number, turned into the exposure that would put that brightness at middle grey, and
+ * moved toward over time from where the last frame left it — that last value is this pass's own
+ * previous output, which the engine keeps for it (`u_feedback`). Brightening is allowed to
+ * be faster than darkening, as an eye adapts, and the rate is per second so it does not depend on
+ * the frame rate.
+ */
+ createAutoExposureAdapt(): ShaderHandle {
+ const source = `#pragma shader "PP Auto Exposure Adapt"
+#pragma version 300 es
+#pragma domain PostProcess
+#pragma param u_bias float default(0) range(-4,4)
+#pragma param u_speedUp float default(3) range(0.1,20)
+#pragma param u_speedDown float default(1) range(0.1,20)
+#pragma param u_minEV float default(-4) range(-10,0)
+#pragma param u_maxEV float default(4) range(0,10)
+#pragma param u_historyValid float default(0) range(0,1)
+#pragma param u_feedback texture default(black)
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+out vec4 fragColor;
+
+void main() {
+ float sum = 0.0;
+ float weight = 0.0;
+ for (int j = 0; j < 8; ++j) {
+ for (int i = 0; i < 8; ++i) {
+ vec2 m = texture(u_texture, (vec2(float(i), float(j)) + 0.5) / 8.0).rg;
+ sum += m.r * m.g;
+ weight += m.g;
+ }
+ }
+ // Weighted by how much of each block counted; a picture with nothing to measure
+ // is left at the exposure the author's compensation alone asks for.
+ float meanLog = weight > 1e-3 ? (sum / weight) * 20.0 - 12.0 : log2(0.18);
+ // The exposure that takes the scene's mean to 18% grey, plus what the author asked for.
+ float target = clamp(log2(0.18) - meanLog + u_bias, u_minEV, u_maxEV);
+ float prev = texture(u_feedback, vec2(0.5)).r * 20.0 - 10.0;
+ float ev = target;
+ if (u_historyValid > 0.5) {
+ float rate = target > prev ? u_speedUp : u_speedDown;
+ ev = prev + (target - prev) * (1.0 - exp(-clamp(u_time.y, 0.0, 0.25) * rate));
+ }
+ float code = clamp((ev + 10.0) / 20.0, 0.0, 1.0);
+ fragColor = vec4(code, code, code, 1.0);
+}
+#pragma end
+
+#pragma fragment wgsl
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ var sum = 0.0;
+ var weight = 0.0;
+ for (var j : i32 = 0; j < 8; j = j + 1) {
+ for (var i : i32 = 0; i < 8; i = i + 1) {
+ let m = textureSampleLevel(t0, s0, (vec2f(f32(i), f32(j)) + 0.5) / 8.0, 0.0).rg;
+ sum = sum + m.r * m.g;
+ weight = weight + m.g;
+ }
+ }
+ var meanLog = log2(0.18);
+ if (weight > 1e-3) { meanLog = (sum / weight) * 20.0 - 12.0; }
+ let target_ev = clamp(log2(0.18) - meanLog + mc.u_bias, mc.u_minEV, mc.u_maxEV);
+ let prev = textureSampleLevel(u_feedback, u_feedback_s, vec2f(0.5), 0.0).r * 20.0 - 10.0;
+ var ev = target_ev;
+ if (mc.u_historyValid > 0.5) {
+ var rate = mc.u_speedDown;
+ if (target_ev > prev) { rate = mc.u_speedUp; }
+ ev = prev + (target_ev - prev) * (1.0 - exp(-clamp(tc.u_time.y, 0.0, 0.25) * rate));
+ }
+ let code = clamp((ev + 10.0) / 20.0, 0.0, 1.0);
+ return vec4f(code, code, code, 1.0);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
+ /** Auto exposure, pass 3 of 3: the scene, at unit 1, scaled by the exposure the last pass settled on. */
+ createAutoExposureApply(): ShaderHandle {
+ const source = `#pragma shader "PP Auto Exposure Apply"
+#pragma version 300 es
+#pragma domain PostProcess
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+uniform sampler2D u_sceneTexture;
+out vec4 fragColor;
+
+void main() {
+ float ev = texture(u_texture, vec2(0.5)).r * 20.0 - 10.0;
+ vec4 scene = texture(u_sceneTexture, v_texCoord);
+ fragColor = vec4(scene.rgb * exp2(ev), scene.a);
+}
+#pragma end
+
+#pragma fragment wgsl
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let ev = textureSampleLevel(t0, s0, vec2f(0.5), 0.0).r * 20.0 - 10.0;
+ let scene = textureSampleLevel(t1, s1, v.v_texCoord, 0.0);
+ return vec4f(scene.rgb * exp2(ev), scene.a);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
+ /**
+ * Depth of field. Each pixel's circle of confusion comes from how far its surface is from the
+ * focus distance, and a pixel outside focus is blurred by gathering its neighbours over that
+ * circle. A neighbour counts only to the extent its OWN circle reaches this pixel: a sharp
+ * object in front of a blurred background does not bleed into it, and a blurred foreground
+ * does spread over what is behind. The sky counts as infinitely far.
+ */
+ createDepthOfField(): ShaderHandle {
+ const GLSL = `
+const int TAPS = 32;
+float ign(vec2 p) {
+ return fract(52.9829189 * fract(dot(p, vec2(0.06711056, 0.00583715))));
+}
+/// The blur radius in pixels at a uv: how far the surface is from the focus, past the slab
+/// that stays sharp, as a share of its distance — the thin-lens ratio, capped.
+float cocAt(vec2 uv, float depth, vec3 eye) {
+ if (depth >= 1.0) return min(u_aperture, u_maxBlur);
+ float dist = length(worldFromDepth(uv, depth) - eye);
+ return min(u_aperture * max(abs(dist - u_focusDistance) - u_focusRange, 0.0) / max(dist, 1e-3), u_maxBlur);
+}`;
+ const WGSL = `
+const TAPS : i32 = 32;
+fn ign(p : vec2f) -> f32 {
+ return fract(52.9829189 * fract(dot(p, vec2f(0.06711056, 0.00583715))));
+}
+fn cocAt(uv : vec2f, depth : f32, eye : vec3f) -> f32 {
+ if (depth >= 1.0) { return min(mc.u_aperture, mc.u_maxBlur); }
+ let dist = length(worldFromDepth(uv, depth) - eye);
+ return min(mc.u_aperture * max(abs(dist - mc.u_focusDistance) - mc.u_focusRange, 0.0) / max(dist, 1e-3), mc.u_maxBlur);
+}`;
+ const source = `#pragma shader "PP Depth of Field"
+#pragma version 300 es
+#pragma domain PostProcess
+#pragma param u_focusDistance float default(600) range(1,5000)
+#pragma param u_focusRange float default(100) range(0,2000)
+#pragma param u_aperture float default(24) range(0,64)
+#pragma param u_maxBlur float default(14) range(1,32)
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+uniform highp sampler2D u_sceneDepth;
+out vec4 fragColor;
+
+${SSAO_GLSL}
+${GLSL}
+
+void main() {
+ vec4 src = texture(u_texture, v_texCoord);
+ ivec2 size = textureSize(u_sceneDepth, 0);
+ vec3 eye = u_camera.w > 0.5 ? u_camera.xyz : worldFromDepth(v_texCoord, 0.0);
+ float coc = cocAt(v_texCoord, depthAt(screenTexel(v_texCoord, size), size), eye);
+ if (coc < 0.5) { fragColor = src; return; }
+
+ vec2 texel = 1.0 / vec2(textureSize(u_texture, 0));
+ float rot = ign(gl_FragCoord.xy) * 6.2831853;
+ vec3 sum = src.rgb;
+ float wsum = 1.0;
+ for (int i = 0; i < TAPS; ++i) {
+ float r = coc * sqrt((float(i) + 0.5) / float(TAPS));
+ float a = float(i) * 2.39996323 + rot;
+ vec2 uv = v_texCoord + vec2(cos(a), sin(a)) * r * texel;
+ float tapCoc = cocAt(uv, depthAt(screenTexel(uv, size), size), eye);
+ // Gathered only to the extent that tap's own blur reaches back here.
+ float w = clamp(tapCoc / max(r, 0.5), 0.0, 1.0);
+ sum += texture(u_texture, uv).rgb * w;
+ wsum += w;
+ }
+ fragColor = vec4(sum / wsum, src.a);
+}
+#pragma end
+
+#pragma fragment wgsl
+${SSAO_WGSL}
+${WGSL}
+
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let src = textureSampleLevel(t0, s0, v.v_texCoord, 0.0);
+ let size = vec2i(textureDimensions(t7, 0));
+ var eye = worldFromDepth(v.v_texCoord, 0.0);
+ if (frame.camera.w > 0.5) { eye = frame.camera.xyz; }
+ let coc = cocAt(v.v_texCoord, depthAt(screenTexel(v.v_texCoord, size), size), eye);
+ if (coc < 0.5) { return src; }
+
+ let texel = 1.0 / vec2f(vec2i(textureDimensions(t0, 0)));
+ let rot = ign(v.pos.xy) * 6.2831853;
+ var sum = src.rgb;
+ var wsum = 1.0;
+ for (var i : i32 = 0; i < TAPS; i = i + 1) {
+ let r = coc * sqrt((f32(i) + 0.5) / f32(TAPS));
+ let a = f32(i) * 2.39996323 + rot;
+ let uv = v.v_texCoord + vec2f(cos(a), sin(a)) * r * texel;
+ let tapCoc = cocAt(uv, depthAt(screenTexel(uv, size), size), eye);
+ let w = clamp(tapCoc / max(r, 0.5), 0.0, 1.0);
+ sum = sum + textureSampleLevel(t0, s0, uv, 0.0).rgb * w;
+ wsum = wsum + w;
+ }
+ return vec4f(sum / wsum, src.a);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
createDistanceFog(): ShaderHandle {
// Fog is a DISTANCE and a depth sample is not one: the same 0.9 is a
// metre away up close and a hundred out. `worldFromDepth` (injected with
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.cpp b/src/esengine/renderer/frame/PostProcessPipeline.cpp
index 05089a7ac..d45e13ef4 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.cpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.cpp
@@ -180,6 +180,7 @@ void PostProcessPipeline::shutdown() {
passes_.clear();
screenPasses_.clear();
temporal_.clear();
+ feedback_.clear();
auto* device = &device_;
if (screen_quad_vbo_ != BufferHandle::Invalid && device) {
@@ -228,6 +229,7 @@ void PostProcessPipeline::resize(u32 width, u32 height) {
sceneResource_ = rg::kNoResource;
// History at the old size names texels this picture no longer has.
temporal_.clear();
+ feedback_.clear();
ensureGraph();
@@ -278,15 +280,23 @@ void PostProcessPipeline::setPassContinuesEffect(const std::string& passName) {
if (auto* pass = findPass(passName)) pass->continuesEffect = true;
}
-bool PostProcessPipeline::passIsTemporal(const PostProcessPass& pass) {
+bool PostProcessPipeline::passHasTexture(const PostProcessPass& pass, const char* name) {
const MaterialUniformLayout* layout = context_.materials().layoutFor(pass.shader);
if (!layout) return false;
for (const auto& slot : layout->textures) {
- if (slot.name == "u_history") return true;
+ if (slot.name == name) return true;
}
return false;
}
+bool PostProcessPipeline::passIsTemporal(const PostProcessPass& pass) {
+ return passHasTexture(pass, "u_history");
+}
+
+u64 PostProcessPipeline::feedbackKey(const std::string& passName) const {
+ return temporalKey_ * 1000003ull + std::hash<std::string>{}(passName);
+}
+
bool PostProcessPipeline::temporalActive() {
if (!initialized_ || bypass_) return false;
for (const auto& pass : passes_) {
@@ -329,20 +339,19 @@ glm::vec2 PostProcessPipeline::beginTemporalView(const glm::mat4& viewProjection
return {jx * 2.0f / w, jy * 2.0f / h};
}
-PostProcessPipeline::TemporalState* PostProcessPipeline::prepareTemporal() {
- auto it = temporal_.find(temporalKey_);
- // Never begun: nothing shifted this frame's picture, and nothing is remembered.
- if (it == temporal_.end() || it->second.frame == 0) return nullptr;
- TemporalState& state = it->second;
- if (state.history[0] && state.width == width_ && state.height == height_
+PostProcessPipeline::TemporalState* PostProcessPipeline::prepareHistory(TemporalState& state,
+ u32 w, u32 h) {
+ w = std::max(w, 1u);
+ h = std::max(h, 1u);
+ if (state.history[0] && state.width == w && state.height == h
&& state.format == interFormat()) {
return &state;
}
for (auto& fbo : state.history) {
FramebufferSpec spec;
spec.colorFormat = interFormat();
- spec.width = width_;
- spec.height = height_;
+ spec.width = w;
+ spec.height = h;
spec.depthStencil = false;
// The reprojected read lands between texels, so it wants the filter.
spec.linearFilter = true;
@@ -354,8 +363,8 @@ PostProcessPipeline::TemporalState* PostProcessPipeline::prepareTemporal() {
return nullptr;
}
}
- state.width = width_;
- state.height = height_;
+ state.width = w;
+ state.height = h;
state.format = interFormat();
state.valid = false;
return &state;
@@ -561,8 +570,10 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
TemporalState* temporal = nullptr;
if (pass.temporal) {
if (auto it = temporal_.find(temporalKey_); it != temporal_.end()) temporal = &it->second;
+ } else if (pass.feedback) {
+ if (auto it = feedback_.find(pass.historyKey); it != feedback_.end()) temporal = &it->second;
}
- if (temporal) {
+ if (temporal && pass.temporal) {
// The previous picture's camera, as columns: a world point times it is where
// that point was one frame ago. Re-sent each frame because it is each frame's.
for (int c = 0; c < 4; ++c) {
@@ -571,6 +582,9 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
}
pass.floatUniforms["u_historyValid"] = temporal->valid ? 1.0f : 0.0f;
pass.paramDirty = true;
+ } else if (temporal) {
+ pass.floatUniforms["u_historyValid"] = temporal->valid ? 1.0f : 0.0f;
+ pass.paramDirty = true;
}
if (layout) {
if (layout->blockSize > 0) {
@@ -617,7 +631,8 @@ void PostProcessPipeline::renderPass(PostProcessPass& pass, const rg::PassContex
// an unset param gets its declared default (white/black/flatnormal).
for (const auto& slot : layout->textures) {
TextureHandle gpu = context_.materials().builtinDefault(slot.defaultTexture);
- if (temporal && slot.name == "u_history" && temporal->history[temporal->readIndex]) {
+ if (temporal && (slot.name == "u_history" || slot.name == "u_feedback")
+ && temporal->history[temporal->readIndex]) {
gpu = temporal->history[temporal->readIndex]->getColorAttachment();
}
for (const auto& [name, glId] : pass.textureUniforms) {
@@ -731,13 +746,30 @@ void PostProcessPipeline::runChain(std::vector<PostProcessPass>& passes, rg::Res
if (i != lastEnabled) writeDesc.scale = pass.scale;
// A temporal pass writes the history target itself, so that what it wrote is
// still there next frame, and the passes after it read it from there.
- TemporalState* temporal = (&passes == &passes_ && passIsTemporal(pass)) ? prepareTemporal() : nullptr;
- pass.temporal = temporal != nullptr;
+ TemporalState* temporal = nullptr;
+ pass.temporal = false;
+ pass.feedback = false;
+ if (&passes == &passes_) {
+ if (passIsTemporal(pass)) {
+ // Never begun: nothing shifted this frame's picture, and nothing is remembered.
+ if (auto it = temporal_.find(temporalKey_); it != temporal_.end() && it->second.frame > 0) {
+ temporal = prepareHistory(it->second, width_, height_);
+ pass.temporal = temporal != nullptr;
+ }
+ } else if (passHasTexture(pass, "u_feedback")) {
+ // At the pass's own size: a measurement is a few texels, not a picture.
+ pass.historyKey = feedbackKey(pass.name);
+ const u32 fw = static_cast<u32>(static_cast<f32>(width_) * pass.scale);
+ const u32 fh = static_cast<u32>(static_cast<f32>(height_) * pass.scale);
+ temporal = prepareHistory(feedback_[pass.historyKey], fw, fh);
+ pass.feedback = temporal != nullptr;
+ }
+ }
if (temporal) {
const u32 write = temporal->readIndex ^ 1u;
Framebuffer& target = *temporal->history[write];
- node.write = graph_->importTarget(target.handle(), width_, height_, /*output=*/false,
- target.getColorAttachment());
+ node.write = graph_->importTarget(target.handle(), temporal->width, temporal->height,
+ /*output=*/false, target.getColorAttachment());
node.execute = [this, &pass, temporal, write](const rg::PassContext& ctx) {
renderPass(pass, ctx);
temporal->readIndex = write;
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.hpp b/src/esengine/renderer/frame/PostProcessPipeline.hpp
index 490e9dfee..6f1f7d141 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.hpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.hpp
@@ -66,6 +66,12 @@ struct PostProcessPass {
/// Set by the chain when the pass keeps its output for the next frame (its
/// shader declares `u_history`); the pass then reads the previous frame there.
bool temporal = false;
+ /// Set instead of `temporal` when the shader declares `u_feedback`: the pass keeps
+ /// its own last output without shifting the picture, and reads it there. An
+ /// auto-exposure's adapted value is remembered this way.
+ bool feedback = false;
+ /// Which remembered state the pass uses, resolved when the chain is declared.
+ u64 historyKey = 0;
/// Fraction of the chain's size this pass draws at; the pass consuming it
/// upsamples. The graph resolves the fraction, and u_viewport still reports
@@ -495,7 +501,12 @@ private:
std::unordered_map<u64, TemporalState> temporal_;
u64 temporalKey_ = 0;
bool passIsTemporal(const PostProcessPass& pass);
- TemporalState* prepareTemporal();
+ bool passHasTexture(const PostProcessPass& pass, const char* name);
+ /// A history made (or kept) at @p w x @p h, or null when it cannot be.
+ TemporalState* prepareHistory(TemporalState& state, u32 w, u32 h);
+ /// Per pass, per view: what a `u_feedback` pass remembers.
+ std::unordered_map<u64, TemporalState> feedback_;
+ u64 feedbackKey(const std::string& passName) const;
Unique<Framebuffer> screenFBO_;
bool screenFBOCreated_ = false;
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index dff1c818c..6ef2d73e0 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2477,4 +2477,6 @@ export const NOTED = {
{"note":"A Screen Space Reflections post-process effect mirrors props in floors."},
"feat(render): Height Fog and Light Shafts effects give a hazy horizon, the sun's glow and rays through what stands in front of it":
{"note":"Height Fog and Light Shafts post-process effects add atmosphere."},
+ "feat(render): Auto Exposure and Depth of Field effects, and a pass that remembers its last output without shifting the picture":
+ {"note":"Auto Exposure and Depth of Field post-process effects are available."},
};