Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): ground-truth ambient occlusion joins the post-process effects

yhh7e176fb7d6fc
diff --git a/desktop b/desktop
index 6babe436b..37a13e7c5 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 6babe436b43e78517bfd15ac92f00ee706de67c9
+Subproject commit 37a13e7c526b2651fd59a28960ccc3a7f3085e98
diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdx
index 87817d0bd..3ddddcc74 100644
--- a/docs/astro/src/content/docs/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx
@@ -28,6 +28,8 @@ several can stack in one frame at no extra target cost beyond the passes.
| `voxelGI` | Diffuse global illumination from a world-space voxel cache the camera remembers (WebGPU only) | `intensity`, `rays`, `voxelSize`, `range`, `maxBounce`, `decay`, `blend`, `divisor` |
| `sharpen` | Sharpening that cannot make a halo (a compute effect: WebGPU only) | `intensity`, `limit` |
| `lightShafts` | Sun rays through whatever stands in front of it | `intensity`, `sunAzimuth`, `sunElevation`, `decay`, `reach` |
+| `ssao` | Ambient occlusion from a ring of depth samples around each point | `radius`, `intensity`, `bias`, `power` |
+| `gtao` | Ground-truth ambient occlusion: the horizon each point can see, integrated against its own normal | `radius`, `intensity`, `power`, `thickness` |
| `contactShadows` | The small sharp shadows at a contact a shadow map is too coarse to hold | `intensity`, `length`, `thickness` |
| `ssgi` | One bounce of diffuse light between the surfaces the camera sees | `intensity`, `radius`, `thickness`, `maxBounce` |
| `autoExposure` | Eye adaptation: exposure follows how bright the picture is | `bias`, `speedUp`, `speedDown`, `minEV`, `maxEV` |
@@ -96,6 +98,16 @@ The clock is the game's. A capture that has to be the same on every run sets `ti
wants. What this is not: the ground's shape is read from what the camera sees (a hollow behind a wall is not known),
and the rain is a screen-space layer, not particles that collide with the world.
+### Ambient occlusion
+
+Two estimators share one blur and one composite. `ssao` counts how many of a ring of nearby points stand above the
+surface; it is the cheaper of the two and needs a `bias` so that a flat floor does not shade itself. `gtao` marches
+the depth buffer along a few slice planes through each pixel, finds how high the horizon climbs on both sides, and
+integrates the arc that is left open, weighted by the surface's own normal — so there is no bias to set, a flat floor
+stays white, and a crease darkens by how closed it really is. `radius` is a world length in both. `thickness` (GTAO) lets
+a thin occluder such as a railing be seen past. Both read scene depth, run at half resolution and blur without crossing
+a depth edge. Quality levels govern both through the one ambient-occlusion switch, and the low level leaves GTAO out.
+
### Contact shadows and bounce light
`contactShadows` marches a short ray from each pixel toward the sun (the scene's own directional light, like the fog) in the depth
diff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
index d4eec3b4c..f4bcc4654 100644
--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
@@ -30,6 +30,8 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
| `voxelGI` | 基于相机记住的世界空间体素缓存的漫反射全局光照(仅 WebGPU) | `intensity`、`rays`、`voxelSize`、`range`、`maxBounce`、`decay`、`blend`、`divisor` |
| `sharpen` | 不会出光晕的锐化(计算特效:仅 WebGPU) | `intensity`、`limit` |
| `lightShafts` | 太阳光穿过前方遮挡物形成的光束 | `intensity`、`sunAzimuth`、`sunElevation`、`decay`、`reach` |
+| `ssao` | 由每个点周围一圈深度采样得出的环境光遮蔽 | `radius`、`intensity`、`bias`、`power` |
+| `gtao` | 真值环境光遮蔽:每个点能看见的地平线,按它自身法线积分 | `radius`、`intensity`、`power`、`thickness` |
| `contactShadows` | 阴影贴图太粗放不下的接触处的小而锐利的阴影 | `intensity`、`length`、`thickness` |
| `ssgi` | 相机看得见的表面之间的一次漫反射光弹射 | `intensity`、`radius`、`thickness`、`maxBounce` |
| `autoExposure` | 眼睛适应:曝光跟随画面亮度 | `bias`、`speedUp`、`speedDown`、`minEV`、`maxEV` |
@@ -75,6 +77,10 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
时钟是游戏的。要在每次运行都相同的截图,把 `timeScale` 设为 0,`phase` 设为想要的时间。它不是什么:地面的形状是从相机看到的东西里读出来的(墙后面的凹处它不知道),
雨是屏幕空间的一层,不是会与世界碰撞的粒子。
+### 环境光遮蔽
+
+两种估计器共用一套模糊和合成。`ssao` 数的是周围一圈点里有多少高于表面;它更便宜,需要 `bias` 来避免平坦的地面给自己上阴影。`gtao` 沿每个像素的几个切片平面在深度缓冲里步进,找出两侧地平线升到多高,再对剩下的开阔弧段按表面自身法线积分——所以没有要调的 bias,平坦的地面保持白色,夹角处的暗度就是它实际有多封闭。两者的 `radius` 都是世界长度。`thickness`(GTAO)让栏杆这类很薄的遮挡物可以被看穿。两者都读场景深度、以半分辨率运行,并且模糊时不会跨过深度边缘。质量档位用同一个环境光遮蔽开关管理两者,低档位不包含 GTAO。
+
### 接触阴影与弹射光
`contactShadows` 从每个像素朝太阳(场景自己的方向光,和雾一样)在深度缓冲里步进一条长 `length` 世界单位的短射线,射线刚好经过深度缓冲所示表面
diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.ts
index b156f3df8..efc966fdc 100644
--- a/sdk/src/postprocess/effects.ts
+++ b/sdk/src/postprocess/effects.ts
@@ -3,6 +3,7 @@
import type { ShaderHandle } from '../render/material';
import { postProcessEffects } from './postProcessEffects';
import { lensEffects } from './lensEffects';
+import { gtaoPasses } from './gtao';
import { SHARPEN_WGSL } from './computeEffects';
export interface EffectUniformDef {
@@ -545,6 +546,27 @@ registerEffect({
],
});
+registerEffect({
+ type: 'gtao',
+ label: 'Ground-Truth Ambient Occlusion',
+ factory: () => gtaoPasses.createGtaoAo(),
+ // The horizon-based estimator in place of the ring of samples: no bias to tune against a surface occluding itself,
+ // so a flat floor stays white and a crease darkens by how closed it really is. The blur and the composite are SSAO's.
+ uniforms: [
+ { name: 'u_radius', label: 'Radius', min: 1, max: 500, step: 1, defaultValue: 80 },
+ { name: 'u_intensity', label: 'Intensity', min: 0, max: 2, step: 0.01, defaultValue: 1 },
+ { name: 'u_power', label: 'Power', min: 0.5, max: 4, step: 0.05, defaultValue: 1.4, neutralValue: 1 },
+ // 0 trusts every occluder to be as tall as it looks; 1 lets a thin one (a railing, a leaf) be seen past.
+ { name: 'u_thickness', label: 'Thickness', min: 0, max: 1, step: 0.01, defaultValue: 0.5 },
+ ],
+ multiPass: [
+ { name: 'gtao_ao', factory: () => gtaoPasses.createGtaoAo(), scale: 0.5 },
+ { name: 'gtao_blur_x', factory: () => postProcessEffects.createSsaoBlur(0), scale: 0.5 },
+ { name: 'gtao_blur_y', factory: () => postProcessEffects.createSsaoBlur(1), scale: 0.5 },
+ { name: 'gtao_composite', factory: () => postProcessEffects.createSsaoComposite() },
+ ],
+});
+
registerEffect({
type: 'ssr',
label: 'Screen Space Reflections',
diff --git a/sdk/src/postprocess/gtao.ts b/sdk/src/postprocess/gtao.ts
new file mode 100644
index 000000000..14442850a
--- /dev/null
+++ b/sdk/src/postprocess/gtao.ts
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file gtao.ts
+ * @brief Ground-truth ambient occlusion: the occlusion pass of the `gtao` effect.
+ * @details The `ssao` effect asks, of a ring of nearby points, how many stand above the surface; this asks what a person
+ * standing on the surface would see. For a few slice planes through the pixel it finds the highest the horizon
+ * climbs on each side (a march over the depth buffer) and integrates the visible arc under it, cosine-weighted by
+ * the surface's own normal (Jimenez et al., "Practical Realtime Strategies for Accurate Indirect Occlusion"). The
+ * result has no tuning constant standing in for a bias, which is the visible difference: a flat floor stays white
+ * and a crease darkens by how closed it actually is. The blur and the composite are the `ssao` effect's own.
+ */
+import type { ShaderHandle } from '../render/material';
+import { Material } from '../render/material';
+import { SSAO_GLSL, SSAO_WGSL } from './postProcessEffects';
+
+export const gtaoPasses = {
+ createGtaoAo(): ShaderHandle {
+ const source = `#pragma shader "PP GTAO"
+#pragma version 300 es
+#pragma domain PostProcess
+#pragma param u_radius float default(80) range(1,500)
+#pragma param u_intensity float default(1) range(0,2)
+#pragma param u_power float default(1.4) range(0.5,4)
+#pragma param u_thickness float default(0.5) range(0,1)
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform highp sampler2D u_sceneDepth;
+out vec4 fragColor;
+
+${SSAO_GLSL}
+
+const int SLICES = 3;
+const int STEPS = 6;
+const float PI = 3.14159265359;
+const float HALF_PI = 1.57079632679;
+
+float fastAcos(float x) { return acos(clamp(x, -1.0, 1.0)); }
+
+void main() {
+ ivec2 size = textureSize(u_sceneDepth, 0);
+ ivec2 c = screenTexel(v_texCoord, size);
+ float d = depthAt(c, size);
+ if (d >= 1.0) { fragColor = vec4(1.0); return; }
+ vec2 cuv = uvOfTexel(c, size);
+ vec3 P = worldFromDepth(cuv, d);
+
+ // The nearer neighbour on each axis, as the SSAO normal is built: across a silhouette the far side is another surface.
+ float dl = depthAt(c + ivec2(-1, 0), size);
+ float dr = depthAt(c + ivec2( 1, 0), size);
+ float dd = depthAt(c + ivec2(0, -1), size);
+ float du = depthAt(c + ivec2(0, 1), size);
+ vec3 ax = abs(dr - d) < abs(dl - d) ? worldAt(c + ivec2(1, 0), size) - P : P - worldAt(c + ivec2(-1, 0), size);
+ vec3 ay = abs(du - d) < abs(dd - d) ? worldAt(c + ivec2(0, 1), size) - P : P - worldAt(c + ivec2(0, -1), size);
+ vec3 N = normalize(cross(ax, ay));
+ vec3 V = viewDirection(P);
+ if (dot(N, V) < 0.0) N = -N;
+
+ // What a texel step is in the world along each screen axis: the frame the slices are laid in.
+ vec3 right = normalize(worldFromDepth(cuv + vec2(1.0 / float(size.x), 0.0), d) - P);
+ vec3 up = normalize(worldFromDepth(cuv + vec2(0.0, 1.0 / float(size.y)), d) - P);
+ float worldPerTexel = max(length(worldFromDepth(cuv + vec2(1.0 / float(size.x), 0.0), d) - P), 1e-6);
+ float radiusTexels = clamp(u_radius / worldPerTexel, 3.0, 128.0);
+
+ // A 4x4 pattern of slice angles and march offsets, keyed to this pass's own pixel (c counts full-resolution texels).
+ ivec2 rp = c / 2;
+ int idx = ((rp.x & 3) * 4 + (rp.y & 3)) * 7;
+ float turn = float(idx - (idx / 16) * 16) / 16.0;
+ float jitter = fract(float(((rp.x * 5 + rp.y * 3) & 15)) * 0.0625 + 0.03125);
+
+ float visibility = 0.0;
+ for (int slice = 0; slice < SLICES; ++slice) {
+ float phi = (float(slice) + turn) * (PI / float(SLICES));
+ vec2 dir = vec2(cos(phi), sin(phi));
+ vec3 dirW = right * dir.x + up * dir.y;
+ // The slice plane holds the view vector and this direction; its normal is the axis the normal is projected off.
+ vec3 ortho = dirW - V * dot(dirW, V);
+ vec3 axis = normalize(cross(ortho, V));
+ vec3 projN = N - axis * dot(N, axis);
+ float projLen = length(projN);
+ float signN = sign(dot(ortho, projN));
+ float cosN = clamp(dot(projN, V) / max(projLen, 1e-5), 0.0, 1.0);
+ float n = signN * fastAcos(cosN);
+
+ float lowCos0 = cos(n + HALF_PI);
+ float lowCos1 = cos(n - HALF_PI);
+ float hCos0 = lowCos0;
+ float hCos1 = lowCos1;
+ for (int i = 0; i < STEPS; ++i) {
+ float f = (float(i) + jitter) / float(STEPS);
+ f = f * f + 1.0 / radiusTexels;
+ vec2 off = dir * (f * radiusTexels);
+ ivec2 s0 = c + ivec2(floor(off + 0.5));
+ ivec2 s1 = c - ivec2(floor(off + 0.5));
+ float z0 = depthAt(s0, size);
+ float z1 = depthAt(s1, size);
+ if (z0 < 1.0) {
+ vec3 delta = worldAt(s0, size) - P;
+ float len = length(delta);
+ // Past the radius a sample is another surface: faded, not cut, or the term pops as geometry crosses it.
+ float weight = clamp(2.0 * (1.0 - len / u_radius), 0.0, 1.0);
+ float shc = dot(delta, V) / max(len, 1e-5);
+ hCos0 = max(hCos0, mix(lowCos0, shc, weight));
+ }
+ if (z1 < 1.0) {
+ vec3 delta = worldAt(s1, size) - P;
+ float len = length(delta);
+ float weight = clamp(2.0 * (1.0 - len / u_radius), 0.0, 1.0);
+ float shc = dot(delta, V) / max(len, 1e-5);
+ hCos1 = max(hCos1, mix(lowCos1, shc, weight));
+ }
+ }
+ // Thin occluders: a horizon that has just climbed a wall does not climb forever, so it falls back a little.
+ hCos0 = mix(hCos0, lowCos0, u_thickness * 0.35);
+ hCos1 = mix(hCos1, lowCos1, u_thickness * 0.35);
+
+ float h0 = -fastAcos(hCos1);
+ float h1 = fastAcos(hCos0);
+ h0 = n + clamp(h0 - n, -HALF_PI, HALF_PI);
+ h1 = n + clamp(h1 - n, -HALF_PI, HALF_PI);
+ float arc0 = (cosN + 2.0 * h0 * sin(n) - cos(2.0 * h0 - n)) * 0.25;
+ float arc1 = (cosN + 2.0 * h1 * sin(n) - cos(2.0 * h1 - n)) * 0.25;
+ visibility += projLen * (arc0 + arc1);
+ }
+ visibility /= float(SLICES);
+ float ao = 1.0 - clamp((1.0 - clamp(visibility, 0.0, 1.0)) * u_intensity, 0.0, 1.0);
+ fragColor = vec4(vec3(pow(ao, u_power)), 1.0);
+}
+#pragma end
+
+#pragma fragment wgsl
+${SSAO_WGSL}
+
+const SLICES : i32 = 3;
+const STEPS : i32 = 6;
+const PI : f32 = 3.14159265359;
+const HALF_PI : f32 = 1.57079632679;
+
+fn fastAcos(x : f32) -> f32 { return acos(clamp(x, -1.0, 1.0)); }
+
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let size = vec2i(textureDimensions(t7, 0));
+ let c = screenTexel(v.v_texCoord, size);
+ let d = depthAt(c, size);
+ if (d >= 1.0) { return vec4f(1.0); }
+ let cuv = uvOfTexel(c, size);
+ let P = worldFromDepth(cuv, d);
+
+ let dl = depthAt(c + vec2i(-1, 0), size);
+ let dr = depthAt(c + vec2i( 1, 0), size);
+ let dd = depthAt(c + vec2i(0, -1), size);
+ let du = depthAt(c + vec2i(0, 1), size);
+ var ax = P - worldAt(c + vec2i(-1, 0), size);
+ if (abs(dr - d) < abs(dl - d)) { ax = worldAt(c + vec2i(1, 0), size) - P; }
+ var ay = P - worldAt(c + vec2i(0, -1), size);
+ if (abs(du - d) < abs(dd - d)) { ay = worldAt(c + vec2i(0, 1), size) - P; }
+ var N = normalize(cross(ax, ay));
+ let V = viewDirection(P);
+ if (dot(N, V) < 0.0) { N = -N; }
+
+ let right = normalize(worldFromDepth(cuv + vec2f(1.0 / f32(size.x), 0.0), d) - P);
+ let up = normalize(worldFromDepth(cuv + vec2f(0.0, 1.0 / f32(size.y)), d) - P);
+ let worldPerTexel = max(length(worldFromDepth(cuv + vec2f(1.0 / f32(size.x), 0.0), d) - P), 1e-6);
+ let radiusTexels = clamp(mc.u_radius / worldPerTexel, 3.0, 128.0);
+
+ let rp = c / 2;
+ let idx = ((rp.x & 3) * 4 + (rp.y & 3)) * 7;
+ let turn = f32(idx - (idx / 16) * 16) / 16.0;
+ let jitter = fract(f32((rp.x * 5 + rp.y * 3) & 15) * 0.0625 + 0.03125);
+
+ var visibility = 0.0;
+ for (var slice : i32 = 0; slice < SLICES; slice = slice + 1) {
+ let phi = (f32(slice) + turn) * (PI / f32(SLICES));
+ let dir = vec2f(cos(phi), sin(phi));
+ let dirW = right * dir.x + up * dir.y;
+ let ortho = dirW - V * dot(dirW, V);
+ let axis = normalize(cross(ortho, V));
+ let projN = N - axis * dot(N, axis);
+ let projLen = length(projN);
+ let signN = sign(dot(ortho, projN));
+ let cosN = clamp(dot(projN, V) / max(projLen, 1e-5), 0.0, 1.0);
+ let n = signN * fastAcos(cosN);
+
+ let lowCos0 = cos(n + HALF_PI);
+ let lowCos1 = cos(n - HALF_PI);
+ var hCos0 = lowCos0;
+ var hCos1 = lowCos1;
+ for (var i : i32 = 0; i < STEPS; i = i + 1) {
+ var f = (f32(i) + jitter) / f32(STEPS);
+ f = f * f + 1.0 / radiusTexels;
+ let off = dir * (f * radiusTexels);
+ let p0 = c + vec2i(floor(off + vec2f(0.5)));
+ let p1 = c - vec2i(floor(off + vec2f(0.5)));
+ let z0 = depthAt(p0, size);
+ let z1 = depthAt(p1, size);
+ if (z0 < 1.0) {
+ let delta = worldAt(p0, size) - P;
+ let len = length(delta);
+ let weight = clamp(2.0 * (1.0 - len / mc.u_radius), 0.0, 1.0);
+ let shc = dot(delta, V) / max(len, 1e-5);
+ hCos0 = max(hCos0, mix(lowCos0, shc, weight));
+ }
+ if (z1 < 1.0) {
+ let delta = worldAt(p1, size) - P;
+ let len = length(delta);
+ let weight = clamp(2.0 * (1.0 - len / mc.u_radius), 0.0, 1.0);
+ let shc = dot(delta, V) / max(len, 1e-5);
+ hCos1 = max(hCos1, mix(lowCos1, shc, weight));
+ }
+ }
+ hCos0 = mix(hCos0, lowCos0, mc.u_thickness * 0.35);
+ hCos1 = mix(hCos1, lowCos1, mc.u_thickness * 0.35);
+
+ var h0 = -fastAcos(hCos1);
+ var h1 = fastAcos(hCos0);
+ h0 = n + clamp(h0 - n, -HALF_PI, HALF_PI);
+ h1 = n + clamp(h1 - n, -HALF_PI, HALF_PI);
+ let arc0 = (cosN + 2.0 * h0 * sin(n) - cos(2.0 * h0 - n)) * 0.25;
+ let arc1 = (cosN + 2.0 * h1 * sin(n) - cos(2.0 * h1 - n)) * 0.25;
+ visibility = visibility + projLen * (arc0 + arc1);
+ }
+ visibility = visibility / f32(SLICES);
+ let ao = 1.0 - clamp((1.0 - clamp(visibility, 0.0, 1.0)) * mc.u_intensity, 0.0, 1.0);
+ return vec4f(vec3f(pow(ao, mc.u_power)), 1.0);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+};
diff --git a/sdk/src/postprocess/qualityFilter.ts b/sdk/src/postprocess/qualityFilter.ts
index ab729cf06..819ed7825 100644
--- a/sdk/src/postprocess/qualityFilter.ts
+++ b/sdk/src/postprocess/qualityFilter.ts
@@ -16,7 +16,8 @@ export function qualityPassAllowed(profile: Readonly<QualityProfile> | null, pas
if (profile === null) return true;
if (!profile.postProcess) return false;
const name = passName.toLowerCase();
- if (!profile.ssao && effectPassNames('ssao').some((pass) => pass.toLowerCase() === name)) return false;
+ // The profile's own switch for ambient occlusion governs both estimators: it was written for the one and means the other.
+ if (!profile.ssao && ['ssao', 'gtao'].some((type) => effectPassNames(type).some((pass) => pass.toLowerCase() === name))) return false;
for (const skipped of profile.skipEffects) {
const passes = effectPassNames(skipped);
if (passes.length > 0) {
diff --git a/sdk/src/render/quality.ts b/sdk/src/render/quality.ts
index 45bdfddab..52d8ecf9e 100644
--- a/sdk/src/render/quality.ts
+++ b/sdk/src/render/quality.ts
@@ -60,7 +60,7 @@ const PROFILES: Record<QualityLevel, QualityProfile> = {
shadowAtlasSize: 1024, shadowCellSize: 256, shadowCascades: 2,
planarReflectionScale: 0.25, planarReflectionMaxSize: 512, planarReflectionUpdatesPerSecond: 0,
shadowDistance: 0, particleLimit: 1000, postProcess: true, ssao: false,
- skipEffects: ['ssr', 'taa', 'depthOfField', 'lightShafts', 'motionBlur', 'lensFlare'],
+ skipEffects: ['ssr', 'taa', 'depthOfField', 'lightShafts', 'motionBlur', 'lensFlare', 'gtao'],
},
medium: {
renderScale: 1, minRenderScale: 0.65, msaaSamples: 2,