A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(render): ground-truth ambient occlusion joins the post-process effects
yhh7e176fb7d6fc
diff --git a/desktop b/desktopindex 6babe436b..37a13e7c5 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit 6babe436b43e78517bfd15ac92f00ee706de67c9+Subproject commit 37a13e7c526b2651fd59a28960ccc3a7f3085e98diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdxindex 87817d0bd..3ddddcc74 100644--- a/docs/astro/src/content/docs/graphics/post-processing.mdx+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx@@ -28,6 +28,8 @@ several can stack in one frame at no extra target cost beyond the passes.| `voxelGI` | Diffuse global illumination from a world-space voxel cache the camera remembers (WebGPU only) | `intensity`, `rays`, `voxelSize`, `range`, `maxBounce`, `decay`, `blend`, `divisor` || `sharpen` | Sharpening that cannot make a halo (a compute effect: WebGPU only) | `intensity`, `limit` || `lightShafts` | Sun rays through whatever stands in front of it | `intensity`, `sunAzimuth`, `sunElevation`, `decay`, `reach` |+| `ssao` | Ambient occlusion from a ring of depth samples around each point | `radius`, `intensity`, `bias`, `power` |+| `gtao` | Ground-truth ambient occlusion: the horizon each point can see, integrated against its own normal | `radius`, `intensity`, `power`, `thickness` || `contactShadows` | The small sharp shadows at a contact a shadow map is too coarse to hold | `intensity`, `length`, `thickness` || `ssgi` | One bounce of diffuse light between the surfaces the camera sees | `intensity`, `radius`, `thickness`, `maxBounce` || `autoExposure` | Eye adaptation: exposure follows how bright the picture is | `bias`, `speedUp`, `speedDown`, `minEV`, `maxEV` |@@ -96,6 +98,16 @@ The clock is the game's. A capture that has to be the same on every run sets `tiwants. What this is not: the ground's shape is read from what the camera sees (a hollow behind a wall is not known),and the rain is a screen-space layer, not particles that collide with the world.+### Ambient occlusion++Two estimators share one blur and one composite. `ssao` counts how many of a ring of nearby points stand above the+surface; it is the cheaper of the two and needs a `bias` so that a flat floor does not shade itself. `gtao` marches+the depth buffer along a few slice planes through each pixel, finds how high the horizon climbs on both sides, and+integrates the arc that is left open, weighted by the surface's own normal — so there is no bias to set, a flat floor+stays white, and a crease darkens by how closed it really is. `radius` is a world length in both. `thickness` (GTAO) lets+a thin occluder such as a railing be seen past. Both read scene depth, run at half resolution and blur without crossing+a depth edge. Quality levels govern both through the one ambient-occlusion switch, and the low level leaves GTAO out.+### Contact shadows and bounce light`contactShadows` marches a short ray from each pixel toward the sun (the scene's own directional light, like the fog) in the depthdiff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdxindex d4eec3b4c..f4bcc4654 100644--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx@@ -30,6 +30,8 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess| `voxelGI` | 基于相机记住的世界空间体素缓存的漫反射全局光照(仅 WebGPU) | `intensity`、`rays`、`voxelSize`、`range`、`maxBounce`、`decay`、`blend`、`divisor` || `sharpen` | 不会出光晕的锐化(计算特效:仅 WebGPU) | `intensity`、`limit` || `lightShafts` | 太阳光穿过前方遮挡物形成的光束 | `intensity`、`sunAzimuth`、`sunElevation`、`decay`、`reach` |+| `ssao` | 由每个点周围一圈深度采样得出的环境光遮蔽 | `radius`、`intensity`、`bias`、`power` |+| `gtao` | 真值环境光遮蔽:每个点能看见的地平线,按它自身法线积分 | `radius`、`intensity`、`power`、`thickness` || `contactShadows` | 阴影贴图太粗放不下的接触处的小而锐利的阴影 | `intensity`、`length`、`thickness` || `ssgi` | 相机看得见的表面之间的一次漫反射光弹射 | `intensity`、`radius`、`thickness`、`maxBounce` || `autoExposure` | 眼睛适应:曝光跟随画面亮度 | `bias`、`speedUp`、`speedDown`、`minEV`、`maxEV` |@@ -75,6 +77,10 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess时钟是游戏的。要在每次运行都相同的截图,把 `timeScale` 设为 0,`phase` 设为想要的时间。它不是什么:地面的形状是从相机看到的东西里读出来的(墙后面的凹处它不知道),雨是屏幕空间的一层,不是会与世界碰撞的粒子。+### 环境光遮蔽++两种估计器共用一套模糊和合成。`ssao` 数的是周围一圈点里有多少高于表面;它更便宜,需要 `bias` 来避免平坦的地面给自己上阴影。`gtao` 沿每个像素的几个切片平面在深度缓冲里步进,找出两侧地平线升到多高,再对剩下的开阔弧段按表面自身法线积分——所以没有要调的 bias,平坦的地面保持白色,夹角处的暗度就是它实际有多封闭。两者的 `radius` 都是世界长度。`thickness`(GTAO)让栏杆这类很薄的遮挡物可以被看穿。两者都读场景深度、以半分辨率运行,并且模糊时不会跨过深度边缘。质量档位用同一个环境光遮蔽开关管理两者,低档位不包含 GTAO。+### 接触阴影与弹射光`contactShadows` 从每个像素朝太阳(场景自己的方向光,和雾一样)在深度缓冲里步进一条长 `length` 世界单位的短射线,射线刚好经过深度缓冲所示表面diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.tsindex b156f3df8..efc966fdc 100644--- a/sdk/src/postprocess/effects.ts+++ b/sdk/src/postprocess/effects.ts@@ -3,6 +3,7 @@import type { ShaderHandle } from '../render/material';import { postProcessEffects } from './postProcessEffects';import { lensEffects } from './lensEffects';+import { gtaoPasses } from './gtao';import { SHARPEN_WGSL } from './computeEffects';export interface EffectUniformDef {@@ -545,6 +546,27 @@ registerEffect({],});+registerEffect({+ type: 'gtao',+ label: 'Ground-Truth Ambient Occlusion',+ factory: () => gtaoPasses.createGtaoAo(),+ // The horizon-based estimator in place of the ring of samples: no bias to tune against a surface occluding itself,+ // so a flat floor stays white and a crease darkens by how closed it really is. The blur and the composite are SSAO's.+ uniforms: [+ { name: 'u_radius', label: 'Radius', min: 1, max: 500, step: 1, defaultValue: 80 },+ { name: 'u_intensity', label: 'Intensity', min: 0, max: 2, step: 0.01, defaultValue: 1 },+ { name: 'u_power', label: 'Power', min: 0.5, max: 4, step: 0.05, defaultValue: 1.4, neutralValue: 1 },+ // 0 trusts every occluder to be as tall as it looks; 1 lets a thin one (a railing, a leaf) be seen past.+ { name: 'u_thickness', label: 'Thickness', min: 0, max: 1, step: 0.01, defaultValue: 0.5 },+ ],+ multiPass: [+ { name: 'gtao_ao', factory: () => gtaoPasses.createGtaoAo(), scale: 0.5 },+ { name: 'gtao_blur_x', factory: () => postProcessEffects.createSsaoBlur(0), scale: 0.5 },+ { name: 'gtao_blur_y', factory: () => postProcessEffects.createSsaoBlur(1), scale: 0.5 },+ { name: 'gtao_composite', factory: () => postProcessEffects.createSsaoComposite() },+ ],+});+registerEffect({type: 'ssr',label: 'Screen Space Reflections',diff --git a/sdk/src/postprocess/gtao.ts b/sdk/src/postprocess/gtao.tsnew file mode 100644index 000000000..14442850a--- /dev/null+++ b/sdk/src/postprocess/gtao.ts@@ -0,0 +1,233 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file gtao.ts+ * @brief Ground-truth ambient occlusion: the occlusion pass of the `gtao` effect.+ * @details The `ssao` effect asks, of a ring of nearby points, how many stand above the surface; this asks what a person+ * standing on the surface would see. For a few slice planes through the pixel it finds the highest the horizon+ * climbs on each side (a march over the depth buffer) and integrates the visible arc under it, cosine-weighted by+ * the surface's own normal (Jimenez et al., "Practical Realtime Strategies for Accurate Indirect Occlusion"). The+ * result has no tuning constant standing in for a bias, which is the visible difference: a flat floor stays white+ * and a crease darkens by how closed it actually is. The blur and the composite are the `ssao` effect's own.+ */+import type { ShaderHandle } from '../render/material';+import { Material } from '../render/material';+import { SSAO_GLSL, SSAO_WGSL } from './postProcessEffects';++export const gtaoPasses = {+ createGtaoAo(): ShaderHandle {+ const source = `#pragma shader "PP GTAO"+#pragma version 300 es+#pragma domain PostProcess+#pragma param u_radius float default(80) range(1,500)+#pragma param u_intensity float default(1) range(0,2)+#pragma param u_power float default(1.4) range(0.5,4)+#pragma param u_thickness float default(0.5) range(0,1)++#pragma fragment+precision highp float;++in vec2 v_texCoord;+uniform highp sampler2D u_sceneDepth;+out vec4 fragColor;++${SSAO_GLSL}++const int SLICES = 3;+const int STEPS = 6;+const float PI = 3.14159265359;+const float HALF_PI = 1.57079632679;++float fastAcos(float x) { return acos(clamp(x, -1.0, 1.0)); }++void main() {+ ivec2 size = textureSize(u_sceneDepth, 0);+ ivec2 c = screenTexel(v_texCoord, size);+ float d = depthAt(c, size);+ if (d >= 1.0) { fragColor = vec4(1.0); return; }+ vec2 cuv = uvOfTexel(c, size);+ vec3 P = worldFromDepth(cuv, d);++ // The nearer neighbour on each axis, as the SSAO normal is built: across a silhouette the far side is another surface.+ float dl = depthAt(c + ivec2(-1, 0), size);+ float dr = depthAt(c + ivec2( 1, 0), size);+ float dd = depthAt(c + ivec2(0, -1), size);+ float du = depthAt(c + ivec2(0, 1), size);+ vec3 ax = abs(dr - d) < abs(dl - d) ? worldAt(c + ivec2(1, 0), size) - P : P - worldAt(c + ivec2(-1, 0), size);+ vec3 ay = abs(du - d) < abs(dd - d) ? worldAt(c + ivec2(0, 1), size) - P : P - worldAt(c + ivec2(0, -1), size);+ vec3 N = normalize(cross(ax, ay));+ vec3 V = viewDirection(P);+ if (dot(N, V) < 0.0) N = -N;++ // What a texel step is in the world along each screen axis: the frame the slices are laid in.+ vec3 right = normalize(worldFromDepth(cuv + vec2(1.0 / float(size.x), 0.0), d) - P);+ vec3 up = normalize(worldFromDepth(cuv + vec2(0.0, 1.0 / float(size.y)), d) - P);+ float worldPerTexel = max(length(worldFromDepth(cuv + vec2(1.0 / float(size.x), 0.0), d) - P), 1e-6);+ float radiusTexels = clamp(u_radius / worldPerTexel, 3.0, 128.0);++ // A 4x4 pattern of slice angles and march offsets, keyed to this pass's own pixel (c counts full-resolution texels).+ ivec2 rp = c / 2;+ int idx = ((rp.x & 3) * 4 + (rp.y & 3)) * 7;+ float turn = float(idx - (idx / 16) * 16) / 16.0;+ float jitter = fract(float(((rp.x * 5 + rp.y * 3) & 15)) * 0.0625 + 0.03125);++ float visibility = 0.0;+ for (int slice = 0; slice < SLICES; ++slice) {+ float phi = (float(slice) + turn) * (PI / float(SLICES));+ vec2 dir = vec2(cos(phi), sin(phi));+ vec3 dirW = right * dir.x + up * dir.y;+ // The slice plane holds the view vector and this direction; its normal is the axis the normal is projected off.+ vec3 ortho = dirW - V * dot(dirW, V);+ vec3 axis = normalize(cross(ortho, V));+ vec3 projN = N - axis * dot(N, axis);+ float projLen = length(projN);+ float signN = sign(dot(ortho, projN));+ float cosN = clamp(dot(projN, V) / max(projLen, 1e-5), 0.0, 1.0);+ float n = signN * fastAcos(cosN);++ float lowCos0 = cos(n + HALF_PI);+ float lowCos1 = cos(n - HALF_PI);+ float hCos0 = lowCos0;+ float hCos1 = lowCos1;+ for (int i = 0; i < STEPS; ++i) {+ float f = (float(i) + jitter) / float(STEPS);+ f = f * f + 1.0 / radiusTexels;+ vec2 off = dir * (f * radiusTexels);+ ivec2 s0 = c + ivec2(floor(off + 0.5));+ ivec2 s1 = c - ivec2(floor(off + 0.5));+ float z0 = depthAt(s0, size);+ float z1 = depthAt(s1, size);+ if (z0 < 1.0) {+ vec3 delta = worldAt(s0, size) - P;+ float len = length(delta);+ // Past the radius a sample is another surface: faded, not cut, or the term pops as geometry crosses it.+ float weight = clamp(2.0 * (1.0 - len / u_radius), 0.0, 1.0);+ float shc = dot(delta, V) / max(len, 1e-5);+ hCos0 = max(hCos0, mix(lowCos0, shc, weight));+ }+ if (z1 < 1.0) {+ vec3 delta = worldAt(s1, size) - P;+ float len = length(delta);+ float weight = clamp(2.0 * (1.0 - len / u_radius), 0.0, 1.0);+ float shc = dot(delta, V) / max(len, 1e-5);+ hCos1 = max(hCos1, mix(lowCos1, shc, weight));+ }+ }+ // Thin occluders: a horizon that has just climbed a wall does not climb forever, so it falls back a little.+ hCos0 = mix(hCos0, lowCos0, u_thickness * 0.35);+ hCos1 = mix(hCos1, lowCos1, u_thickness * 0.35);++ float h0 = -fastAcos(hCos1);+ float h1 = fastAcos(hCos0);+ h0 = n + clamp(h0 - n, -HALF_PI, HALF_PI);+ h1 = n + clamp(h1 - n, -HALF_PI, HALF_PI);+ float arc0 = (cosN + 2.0 * h0 * sin(n) - cos(2.0 * h0 - n)) * 0.25;+ float arc1 = (cosN + 2.0 * h1 * sin(n) - cos(2.0 * h1 - n)) * 0.25;+ visibility += projLen * (arc0 + arc1);+ }+ visibility /= float(SLICES);+ float ao = 1.0 - clamp((1.0 - clamp(visibility, 0.0, 1.0)) * u_intensity, 0.0, 1.0);+ fragColor = vec4(vec3(pow(ao, u_power)), 1.0);+}+#pragma end++#pragma fragment wgsl+${SSAO_WGSL}++const SLICES : i32 = 3;+const STEPS : i32 = 6;+const PI : f32 = 3.14159265359;+const HALF_PI : f32 = 1.57079632679;++fn fastAcos(x : f32) -> f32 { return acos(clamp(x, -1.0, 1.0)); }++@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {+ let size = vec2i(textureDimensions(t7, 0));+ let c = screenTexel(v.v_texCoord, size);+ let d = depthAt(c, size);+ if (d >= 1.0) { return vec4f(1.0); }+ let cuv = uvOfTexel(c, size);+ let P = worldFromDepth(cuv, d);++ let dl = depthAt(c + vec2i(-1, 0), size);+ let dr = depthAt(c + vec2i( 1, 0), size);+ let dd = depthAt(c + vec2i(0, -1), size);+ let du = depthAt(c + vec2i(0, 1), size);+ var ax = P - worldAt(c + vec2i(-1, 0), size);+ if (abs(dr - d) < abs(dl - d)) { ax = worldAt(c + vec2i(1, 0), size) - P; }+ var ay = P - worldAt(c + vec2i(0, -1), size);+ if (abs(du - d) < abs(dd - d)) { ay = worldAt(c + vec2i(0, 1), size) - P; }+ var N = normalize(cross(ax, ay));+ let V = viewDirection(P);+ if (dot(N, V) < 0.0) { N = -N; }++ let right = normalize(worldFromDepth(cuv + vec2f(1.0 / f32(size.x), 0.0), d) - P);+ let up = normalize(worldFromDepth(cuv + vec2f(0.0, 1.0 / f32(size.y)), d) - P);+ let worldPerTexel = max(length(worldFromDepth(cuv + vec2f(1.0 / f32(size.x), 0.0), d) - P), 1e-6);+ let radiusTexels = clamp(mc.u_radius / worldPerTexel, 3.0, 128.0);++ let rp = c / 2;+ let idx = ((rp.x & 3) * 4 + (rp.y & 3)) * 7;+ let turn = f32(idx - (idx / 16) * 16) / 16.0;+ let jitter = fract(f32((rp.x * 5 + rp.y * 3) & 15) * 0.0625 + 0.03125);++ var visibility = 0.0;+ for (var slice : i32 = 0; slice < SLICES; slice = slice + 1) {+ let phi = (f32(slice) + turn) * (PI / f32(SLICES));+ let dir = vec2f(cos(phi), sin(phi));+ let dirW = right * dir.x + up * dir.y;+ let ortho = dirW - V * dot(dirW, V);+ let axis = normalize(cross(ortho, V));+ let projN = N - axis * dot(N, axis);+ let projLen = length(projN);+ let signN = sign(dot(ortho, projN));+ let cosN = clamp(dot(projN, V) / max(projLen, 1e-5), 0.0, 1.0);+ let n = signN * fastAcos(cosN);++ let lowCos0 = cos(n + HALF_PI);+ let lowCos1 = cos(n - HALF_PI);+ var hCos0 = lowCos0;+ var hCos1 = lowCos1;+ for (var i : i32 = 0; i < STEPS; i = i + 1) {+ var f = (f32(i) + jitter) / f32(STEPS);+ f = f * f + 1.0 / radiusTexels;+ let off = dir * (f * radiusTexels);+ let p0 = c + vec2i(floor(off + vec2f(0.5)));+ let p1 = c - vec2i(floor(off + vec2f(0.5)));+ let z0 = depthAt(p0, size);+ let z1 = depthAt(p1, size);+ if (z0 < 1.0) {+ let delta = worldAt(p0, size) - P;+ let len = length(delta);+ let weight = clamp(2.0 * (1.0 - len / mc.u_radius), 0.0, 1.0);+ let shc = dot(delta, V) / max(len, 1e-5);+ hCos0 = max(hCos0, mix(lowCos0, shc, weight));+ }+ if (z1 < 1.0) {+ let delta = worldAt(p1, size) - P;+ let len = length(delta);+ let weight = clamp(2.0 * (1.0 - len / mc.u_radius), 0.0, 1.0);+ let shc = dot(delta, V) / max(len, 1e-5);+ hCos1 = max(hCos1, mix(lowCos1, shc, weight));+ }+ }+ hCos0 = mix(hCos0, lowCos0, mc.u_thickness * 0.35);+ hCos1 = mix(hCos1, lowCos1, mc.u_thickness * 0.35);++ var h0 = -fastAcos(hCos1);+ var h1 = fastAcos(hCos0);+ h0 = n + clamp(h0 - n, -HALF_PI, HALF_PI);+ h1 = n + clamp(h1 - n, -HALF_PI, HALF_PI);+ let arc0 = (cosN + 2.0 * h0 * sin(n) - cos(2.0 * h0 - n)) * 0.25;+ let arc1 = (cosN + 2.0 * h1 * sin(n) - cos(2.0 * h1 - n)) * 0.25;+ visibility = visibility + projLen * (arc0 + arc1);+ }+ visibility = visibility / f32(SLICES);+ let ao = 1.0 - clamp((1.0 - clamp(visibility, 0.0, 1.0)) * mc.u_intensity, 0.0, 1.0);+ return vec4f(vec3f(pow(ao, mc.u_power)), 1.0);+}+#pragma end+`;+ return Material.compileShader(source);+ },+};diff --git a/sdk/src/postprocess/qualityFilter.ts b/sdk/src/postprocess/qualityFilter.tsindex ab729cf06..819ed7825 100644--- a/sdk/src/postprocess/qualityFilter.ts+++ b/sdk/src/postprocess/qualityFilter.ts@@ -16,7 +16,8 @@ export function qualityPassAllowed(profile: Readonly<QualityProfile> | null, pasif (profile === null) return true;if (!profile.postProcess) return false;const name = passName.toLowerCase();- if (!profile.ssao && effectPassNames('ssao').some((pass) => pass.toLowerCase() === name)) return false;+ // The profile's own switch for ambient occlusion governs both estimators: it was written for the one and means the other.+ if (!profile.ssao && ['ssao', 'gtao'].some((type) => effectPassNames(type).some((pass) => pass.toLowerCase() === name))) return false;for (const skipped of profile.skipEffects) {const passes = effectPassNames(skipped);if (passes.length > 0) {diff --git a/sdk/src/render/quality.ts b/sdk/src/render/quality.tsindex 45bdfddab..52d8ecf9e 100644--- a/sdk/src/render/quality.ts+++ b/sdk/src/render/quality.ts@@ -60,7 +60,7 @@ const PROFILES: Record<QualityLevel, QualityProfile> = {shadowAtlasSize: 1024, shadowCellSize: 256, shadowCascades: 2,planarReflectionScale: 0.25, planarReflectionMaxSize: 512, planarReflectionUpdatesPerSecond: 0,shadowDistance: 0, particleLimit: 1000, postProcess: true, ssao: false,- skipEffects: ['ssr', 'taa', 'depthOfField', 'lightShafts', 'motionBlur', 'lensFlare'],+ skipEffects: ['ssr', 'taa', 'depthOfField', 'lightShafts', 'motionBlur', 'lensFlare', 'gtao'],},medium: {renderScale: 1, minRenderScale: 0.65, msaaSamples: 2,