Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): a Screen Space Reflections effect mirrors the scene in the surfaces that face up

yhhea26bb1707c1
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a08981f2e..f1de2bfa9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -16,6 +16,7 @@ published separately; it ships inside the editor.
### Added
+- **Screen Space Reflections.** A Screen Space Reflections post-process effect mirrors props in floors. Add `ssr` to a `PostProcessVolume`: with no G-buffer, the surface normal is rebuilt from the depth buffer, the reflected ray is marched through the world and projected back to the screen at every step, and a step landing just behind the visible surface is a hit, refined by bisection. The reflection is blended over the pixel by a Schlick Fresnel term — faint face-on, strong at a grazing angle — and blurred by how far the ray travelled, which stands in for roughness. `intensity`, `maxDistance`, `thickness` and `floorBias` are the settings; `floorBias` at 1 lets only up-facing surfaces reflect, since the chain does not know a material's roughness. What is off screen or in the sky is not reflected; the pixel is left as it was. It reads the same depth buffer as ambient occlusion, runs on WebGL2 and WebGPU alike, and its per-pixel start offset is meant to be averaged away by `taa`. Verified in the editor on a lit scene with both backends: the props appear in the floor they stand on and nothing else on screen changes.
- **Temporal Anti-Aliasing.** A Temporal Anti-Aliasing post-process effect smooths edges across frames. Add `taa` to a `PostProcessVolume` and the engine shifts the scene's picture by a different sub-pixel amount each frame (eight positions, repeating), keeps the effect's last output as its history, and blends the two — reaching what multisampling cannot: shader aliasing and thin geometry. Where a pixel was last frame comes from the depth buffer and the previous camera, so a moving camera keeps its history; the history is clipped to the colours around it, so what no longer matches is dropped rather than smeared, and each camera keeps a history of its own. One `blend` setting says how much of each frame is new. Measured in the editor on a lit scene with multisampling off, in-between edge pixels went from 2% of edge pixels to 43% on WebGL2 and 44% on WebGPU, still frames held still, and a picture taken mid-slide of the camera matched the plain one to within a fraction of a level. Objects that move by themselves are not tracked — they are handled by the clip, so fast movers can look slightly soft.
- **Smaller meshes on disk.** Newly imported meshes are about 9% smaller on disk. A mesh whose indices all fit in 16 bits is now written with 16-bit indices (`.esmesh` version 4), half the index bytes, and read back as 32-bit so nothing downstream changes: the renderer, the lightmap and decal bakers and static batching see the same indices as before. A mesh that needs 32 bits is written exactly as before, on the version every earlier reader knows, and files written before this change still load. Measured on the Bistro meshes (1143 files, 116 MB), re-encoding saves 10.4 MB; existing `.esmesh` files are not rewritten until the model is re-imported.
- **Levels of detail generated at import.** Imported models can generate cheaper levels of detail automatically. Set **Generate LODs** to 1–3 levels in a model's import settings (or pass `--lods <n>` to `import-model`) and each mesh is collapsed into copies with about 50%, 20% and 7% of its triangles, written as `<mesh>_lod<n>.esmesh`, with a `LODGroup` on the entity that swaps them in as it shrinks on screen. The collapse bounds how far the surface may move and will not pull apart a UV seam or a hard edge; vertex colours and the lightmap UV come along unchanged. Skinned and morphing meshes, and meshes under a minimum triangle count, keep one level. Measured on the Bistro exterior (1143 meshes, 2.83M triangles), the 621 meshes large enough to get levels hold 1.35M, 0.44M and 0.10M triangles at levels 1, 2 and 3; the levels are extra files, so they make a large scene cheaper to draw rather than smaller to ship.
diff --git a/desktop b/desktop
index 9ec2e6596..f2d16e7cf 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 9ec2e659696086757dd704eb9af80681dd32c365
+Subproject commit f2d16e7cf27d79bc97020fd7dbc006efe41a733a
diff --git a/docs/astro/src/content/docs/graphics/post-processing.mdx b/docs/astro/src/content/docs/graphics/post-processing.mdx
index 1b4c45f7d..6ca62cfac 100644
--- a/docs/astro/src/content/docs/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/graphics/post-processing.mdx
@@ -22,6 +22,7 @@ several can stack in one frame at no extra target cost beyond the passes.
| `chromaticAberration` | RGB fringing | `intensity` |
| `blur` | Gaussian blur | `intensity` |
| `fxaa` | Fast anti-aliasing | `intensity` |
+| `ssr` | Screen-space reflections: props mirrored in the floor they stand on | `intensity`, `maxDistance`, `thickness`, `floorBias` |
| `taa` | Temporal anti-aliasing: edges smoothed over several frames | `blend` |
| `lensDistortion` | Barrel / pincushion warp | `strength`, `zoom` |
| `pixelate` | Block quantization | `pixelSize` |
@@ -37,6 +38,23 @@ Android, desktop, Steam) runs on. `createWebApp({ msaaSamples })` sizes it, and
`fxaa` is the cheaper, blurrier alternative for devices where even one sample
extra is too much.
+### Screen-space reflections
+
+`ssr` reflects what is on screen in the surfaces that face up: a wet street, a polished
+floor, a lake's edge. There is no G-buffer, so it works from what the chain already has — the
+surface normal is rebuilt from the depth buffer, the reflected ray is marched through the world
+and projected back at every step, and a step that lands just behind what the depth buffer shows
+is a hit. What it finds is blended over the pixel by a Schlick Fresnel term: faint face-on,
+strong at a grazing angle, so the floor shines toward the horizon. A farther hit is blurred more,
+which stands in for roughness.
+
+Nothing in the chain knows a material's roughness, so `floorBias` decides what may reflect: at 1
+only up-facing surfaces do, at 0 every surface reflects by its Fresnel term alone. It cannot show
+what is off screen, behind another object or in the sky — there it leaves the pixel as it was, so
+it sits on top of the environment's own specular rather than replacing it. `maxDistance` is a
+world length; the step count is fixed, so a longer reach is a coarser march, and the per-pixel
+start offset that hides the banding is what `taa` averages away — use them together.
+
### Temporal anti-aliasing
`taa` smooths what MSAA cannot reach — shader aliasing and thin geometry — by
diff --git a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
index 13142ef93..738b65b5b 100644
--- a/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
+++ b/docs/astro/src/content/docs/zh-cn/graphics/post-processing.mdx
@@ -21,6 +21,7 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
| `chromaticAberration` | RGB 色散 | `intensity` |
| `blur` | 高斯模糊 | `intensity` |
| `fxaa` | 快速抗锯齿 | `intensity` |
+| `ssr` | 屏幕空间反射:道具映在它们所站的地面上 | `intensity`、`maxDistance`、`thickness`、`floorBias` |
| `taa` | 时间性抗锯齿:边缘跨多帧平滑 | `blend` |
| `lensDistortion` | 桶形 / 枕形畸变 | `strength`、`zoom` |
| `pixelate` | 块状量化 | `pixelSize` |
@@ -34,6 +35,18 @@ Estella 通过 ping-pong 帧缓冲运行全屏后处理。加一个 `PostProcess
`createWebApp({ msaaSamples })` 调它的大小,`msaaSamples: 1` 在填充率比边缘更要紧的设备上关掉它。
`fxaa` 是更便宜也更糊的替代品,留给连多一个采样都吃不下的设备。
+### 屏幕空间反射
+
+`ssr` 把屏幕上已有的东西反射到朝上的表面里:湿街道、抛光地板、湖岸。没有 G-buffer,所以只用链路
+里已有的东西——法线由深度缓冲重建,反射射线在世界里步进、每一步投影回屏幕,落在深度缓冲所示表面
+刚好后面的一步算命中。找到的颜色按 Schlick 菲涅尔项混到像素上:正对时很弱,掠射时很强,所以地面
+越靠近地平线越亮。命中越远越模糊,这代替了粗糙度。
+
+链路里没有任何东西知道材质的粗糙度,所以由 `floorBias` 决定谁可以反射:为 1 时只有朝上的表面反射,
+为 0 时每个表面仅按菲涅尔项反射。它看不到屏幕外、被别的物体挡住的或天空里的东西——那里像素保持
+原样,所以它叠在环境自身的高光之上而不是取代它。`maxDistance` 是世界长度;步数固定,所以距离越长
+步进越粗,而用来藏起条带的逐像素起始偏移要靠 `taa` 平均掉——两者配合使用。
+
### 时间性抗锯齿
`taa` 抹平 MSAA 够不到的东西——着色锯齿和细几何——办法是跨帧累积。效果在链路里时,引擎每帧把画面
diff --git a/sdk/src/postprocess/effects.ts b/sdk/src/postprocess/effects.ts
index 5857c892e..a9f81922a 100644
--- a/sdk/src/postprocess/effects.ts
+++ b/sdk/src/postprocess/effects.ts
@@ -250,6 +250,23 @@ registerEffect({
],
});
+registerEffect({
+ type: 'ssr',
+ label: 'Screen Space Reflections',
+ factory: () => postProcessEffects.createSsr(),
+ uniforms: [
+ { name: 'u_intensity', label: 'Intensity', min: 0, max: 2, step: 0.01, defaultValue: 1, neutralValue: 0 },
+ // A WORLD length, like SSAO's radius: how far a reflected ray may travel before the
+ // search gives up. Step count is a constant, so a longer reach is a coarser step.
+ { name: 'u_maxDistance', label: 'Max Distance', min: 10, max: 3000, step: 10, defaultValue: 400 },
+ // How far behind a visible surface a ray may pass and still count as having hit it.
+ { name: 'u_thickness', label: 'Thickness', min: 1, max: 200, step: 1, defaultValue: 12 },
+ // 1 = only surfaces that face up reflect (wet ground, polished floors); 0 = every surface
+ // reflects by its Fresnel term alone.
+ { name: 'u_floorBias', label: 'Floor Bias', min: 0, max: 1, step: 0.01, defaultValue: 1 },
+ ],
+});
+
registerEffect({
type: 'taa',
label: 'Temporal Anti-Aliasing',
diff --git a/sdk/src/postprocess/postProcessEffects.ts b/sdk/src/postprocess/postProcessEffects.ts
index dfa3c0207..39a3d5dc9 100644
--- a/sdk/src/postprocess/postProcessEffects.ts
+++ b/sdk/src/postprocess/postProcessEffects.ts
@@ -283,6 +283,239 @@ ${WGSL}
return Material.compileShader(source);
},
+ /**
+ * Screen-space reflections. No G-buffer: the surface normal is rebuilt from depth the way
+ * SSAO does, the reflected ray is marched in the world and projected back to the screen at
+ * every step, and a step that lands behind what the depth buffer shows by less than
+ * `u_thickness` is a hit. What the ray found is blended over the pixel by a Schlick
+ * Fresnel term — weak face-on, strong at a grazing angle — so a wet floor shines toward the
+ * horizon and a wall facing the camera does not.
+ *
+ * Nothing here knows a surface's roughness, which is why `u_floorBias` exists: with it up,
+ * only surfaces that face up reflect, which is what a wet street or a polished floor wants
+ * and what a brick wall does not.
+ */
+ createSsr(): ShaderHandle {
+ const GLSL = `
+const int STEPS = 48;
+const int REFINE = 6;
+
+/// Interleaved gradient noise: a different start offset per pixel, so the stepping
+/// bands dissolve into noise that a temporal pass then averages away.
+float ign(vec2 p) {
+ return fract(52.9829189 * fract(dot(p, vec2(0.06711056, 0.00583715))));
+}
+/// The screen uv a world point lands on, and whether it is in front of the camera.
+vec3 project(vec3 P) {
+ vec4 c = u_projection * vec4(P, 1.0);
+ return vec3(c.xy / max(c.w, 1e-6) * 0.5 + 0.5, c.w);
+}`;
+ const WGSL = `
+const STEPS : i32 = 48;
+const REFINE : i32 = 6;
+fn ign(p : vec2f) -> f32 {
+ return fract(52.9829189 * fract(dot(p, vec2f(0.06711056, 0.00583715))));
+}
+fn project(P : vec3f) -> vec3f {
+ let c = frame.projection * vec4f(P, 1.0);
+ let w = max(c.w, 1e-6);
+ return vec3f(c.x / w * 0.5 + 0.5, 0.5 - c.y / w * 0.5, c.w);
+}`;
+ const source = `#pragma shader "PP SSR"
+#pragma version 300 es
+#pragma domain PostProcess
+#pragma param u_intensity float default(1) range(0,2)
+#pragma param u_maxDistance float default(400) range(10,3000)
+#pragma param u_thickness float default(12) range(1,200)
+#pragma param u_floorBias float default(1) range(0,1)
+
+#pragma fragment
+precision highp float;
+
+in vec2 v_texCoord;
+uniform sampler2D u_texture;
+uniform highp sampler2D u_sceneDepth;
+out vec4 fragColor;
+
+${SSAO_GLSL}
+${GLSL}
+
+void main() {
+ vec4 scene = texture(u_texture, v_texCoord);
+ ivec2 size = textureSize(u_sceneDepth, 0);
+ ivec2 c = screenTexel(v_texCoord, size);
+ float d = depthAt(c, size);
+ if (d >= 1.0 || u_intensity <= 0.0) { fragColor = scene; return; }
+
+ vec2 cuv = uvOfTexel(c, size);
+ vec3 P = worldFromDepth(cuv, d);
+
+ // The nearer neighbour on each axis, as SSAO takes it: across a silhouette the far
+ // side is another surface and a centred derivative would average the two.
+ float dl = depthAt(c + ivec2(-1, 0), size);
+ float dr = depthAt(c + ivec2( 1, 0), size);
+ float dd = depthAt(c + ivec2(0, -1), size);
+ float du = depthAt(c + ivec2(0, 1), size);
+ vec3 ax = abs(dr - d) < abs(dl - d) ? worldAt(c + ivec2(1, 0), size) - P
+ : P - worldAt(c + ivec2(-1, 0), size);
+ vec3 ay = abs(du - d) < abs(dd - d) ? worldAt(c + ivec2(0, 1), size) - P
+ : P - worldAt(c + ivec2(0, -1), size);
+ vec3 N = normalize(cross(ax, ay));
+ vec3 V = viewDirection(P);
+ if (dot(N, V) < 0.0) N = -N;
+
+ // Fresnel at this angle, Schlick with the dielectric 4% a surface without a metal in
+ // it reflects face-on; and the floor mask that keeps walls from shining.
+ float cosT = clamp(dot(N, V), 0.0, 1.0);
+ float fresnel = 0.04 + 0.96 * pow(1.0 - cosT, 5.0);
+ float up = mix(1.0, smoothstep(0.55, 0.9, N.y), u_floorBias);
+ float weight = fresnel * up * u_intensity;
+ if (weight < 0.002) { fragColor = scene; return; }
+
+ vec3 R = reflect(-V, N);
+ // A ray heading back at the camera has nothing on screen to find.
+ if (dot(R, V) > 0.97) { fragColor = scene; return; }
+
+ float stepLen = u_maxDistance / float(STEPS);
+ float jitter = ign(gl_FragCoord.xy);
+ vec3 eye = u_camera.w > 0.5 ? u_camera.xyz : P + u_camera.xyz * 1e4;
+ vec3 hitColor = vec3(0.0);
+ float hit = 0.0;
+ float travelled = 0.0;
+ vec3 prevP = P;
+ for (int i = 0; i < STEPS; ++i) {
+ float t = (float(i) + jitter) * stepLen + stepLen * 0.5;
+ vec3 Q = P + R * t;
+ vec3 s = project(Q);
+ if (s.z <= 0.0 || s.x < 0.0 || s.x > 1.0 || s.y < 0.0 || s.y > 1.0) break;
+ float ds = depthAt(screenTexel(s.xy, size), size);
+ if (ds >= 1.0) { prevP = Q; continue; }
+ vec3 S = worldFromDepth(uvOfTexel(screenTexel(s.xy, size), size), ds);
+ // How far the ray has gone past the surface the depth buffer shows there,
+ // measured along the view: positive is behind it.
+ float behind = length(Q - eye) - length(S - eye);
+ if (behind > 0.0 && behind < u_thickness + stepLen) {
+ // Bisect between the last step in front and this one behind.
+ vec3 a = prevP;
+ vec3 b = Q;
+ for (int k = 0; k < REFINE; ++k) {
+ vec3 m = 0.5 * (a + b);
+ vec3 sm = project(m);
+ float dm = depthAt(screenTexel(sm.xy, size), size);
+ vec3 Sm = worldFromDepth(uvOfTexel(screenTexel(sm.xy, size), size), dm);
+ if (length(m - eye) - length(Sm - eye) > 0.0) b = m; else a = m;
+ }
+ vec3 sh = project(b);
+ // The reflected surface must face the ray; one seen edge-on is an
+ // artefact of the depth buffer's stair-stepping, not a surface.
+ vec2 e = min(sh.xy, 1.0 - sh.xy);
+ float edge = smoothstep(0.0, 0.1, min(e.x, e.y));
+ // A farther hit is a rougher reflection: blur it by how far it is.
+ float blur = (length(b - P) / max(u_maxDistance, 1.0)) * 6.0;
+ vec2 texel = 1.0 / vec2(textureSize(u_texture, 0));
+ vec3 col = texture(u_texture, sh.xy).rgb * 0.4;
+ col += texture(u_texture, sh.xy + vec2( blur, 0.0) * texel).rgb * 0.15;
+ col += texture(u_texture, sh.xy + vec2(-blur, 0.0) * texel).rgb * 0.15;
+ col += texture(u_texture, sh.xy + vec2(0.0, blur) * texel).rgb * 0.15;
+ col += texture(u_texture, sh.xy + vec2(0.0, -blur) * texel).rgb * 0.15;
+ travelled = length(b - P);
+ hit = edge * (1.0 - smoothstep(0.6, 1.0, travelled / u_maxDistance));
+ hitColor = col;
+ break;
+ }
+ prevP = Q;
+ }
+ fragColor = vec4(mix(scene.rgb, hitColor, clamp(weight * hit, 0.0, 1.0)), scene.a);
+}
+#pragma end
+
+#pragma fragment wgsl
+${SSAO_WGSL}
+${WGSL}
+
+@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+ let scene = textureSampleLevel(t0, s0, v.v_texCoord, 0.0);
+ let size = vec2i(textureDimensions(t7, 0));
+ let c = screenTexel(v.v_texCoord, size);
+ let d = depthAt(c, size);
+ if (d >= 1.0 || mc.u_intensity <= 0.0) { return scene; }
+
+ let cuv = uvOfTexel(c, size);
+ let P = worldFromDepth(cuv, d);
+
+ let dl = depthAt(c + vec2i(-1, 0), size);
+ let dr = depthAt(c + vec2i( 1, 0), size);
+ let dd = depthAt(c + vec2i(0, -1), size);
+ let du = depthAt(c + vec2i(0, 1), size);
+ var ax = P - worldAt(c + vec2i(-1, 0), size);
+ if (abs(dr - d) < abs(dl - d)) { ax = worldAt(c + vec2i(1, 0), size) - P; }
+ var ay = P - worldAt(c + vec2i(0, -1), size);
+ if (abs(du - d) < abs(dd - d)) { ay = worldAt(c + vec2i(0, 1), size) - P; }
+ var N = normalize(cross(ax, ay));
+ let V = viewDirection(P);
+ if (dot(N, V) < 0.0) { N = -N; }
+
+ let cosT = clamp(dot(N, V), 0.0, 1.0);
+ let fresnel = 0.04 + 0.96 * pow(1.0 - cosT, 5.0);
+ let up = mix(1.0, smoothstep(0.55, 0.9, N.y), mc.u_floorBias);
+ let weight = fresnel * up * mc.u_intensity;
+ if (weight < 0.002) { return scene; }
+
+ let R = reflect(-V, N);
+ if (dot(R, V) > 0.97) { return scene; }
+
+ let stepLen = mc.u_maxDistance / f32(STEPS);
+ let jitter = ign(v.pos.xy);
+ var eye = P + frame.camera.xyz * 1e4;
+ if (frame.camera.w > 0.5) { eye = frame.camera.xyz; }
+ var hitColor = vec3f(0.0);
+ var hit = 0.0;
+ var prevP = P;
+ for (var i : i32 = 0; i < STEPS; i = i + 1) {
+ let t = (f32(i) + jitter) * stepLen + stepLen * 0.5;
+ let Q = P + R * t;
+ let s = project(Q);
+ if (s.z <= 0.0 || s.x < 0.0 || s.x > 1.0 || s.y < 0.0 || s.y > 1.0) { break; }
+ let st = screenTexel(s.xy, size);
+ let ds = depthAt(st, size);
+ if (ds >= 1.0) { prevP = Q; continue; }
+ let S = worldFromDepth(uvOfTexel(st, size), ds);
+ let behind = length(Q - eye) - length(S - eye);
+ if (behind > 0.0 && behind < mc.u_thickness + stepLen) {
+ var a = prevP;
+ var b = Q;
+ for (var k : i32 = 0; k < REFINE; k = k + 1) {
+ let m = 0.5 * (a + b);
+ let sm = project(m);
+ let mt = screenTexel(sm.xy, size);
+ let dm = depthAt(mt, size);
+ let Sm = worldFromDepth(uvOfTexel(mt, size), dm);
+ if (length(m - eye) - length(Sm - eye) > 0.0) { b = m; } else { a = m; }
+ }
+ let sh = project(b);
+ let e = min(sh.xy, vec2f(1.0) - sh.xy);
+ let edge = smoothstep(0.0, 0.1, min(e.x, e.y));
+ let travelled = length(b - P);
+ let blur = (travelled / max(mc.u_maxDistance, 1.0)) * 6.0;
+ let texel = 1.0 / vec2f(vec2i(textureDimensions(t0, 0)));
+ var col = textureSampleLevel(t0, s0, sh.xy, 0.0).rgb * 0.4;
+ col = col + textureSampleLevel(t0, s0, sh.xy + vec2f( blur, 0.0) * texel, 0.0).rgb * 0.15;
+ col = col + textureSampleLevel(t0, s0, sh.xy + vec2f(-blur, 0.0) * texel, 0.0).rgb * 0.15;
+ col = col + textureSampleLevel(t0, s0, sh.xy + vec2f(0.0, blur) * texel, 0.0).rgb * 0.15;
+ col = col + textureSampleLevel(t0, s0, sh.xy + vec2f(0.0, -blur) * texel, 0.0).rgb * 0.15;
+ hit = edge * (1.0 - smoothstep(0.6, 1.0, travelled / mc.u_maxDistance));
+ hitColor = col;
+ break;
+ }
+ prevP = Q;
+ }
+ return vec4f(mix(scene.rgb, hitColor, clamp(weight * hit, 0.0, 1.0)), scene.a);
+}
+#pragma end
+`;
+ return Material.compileShader(source);
+ },
+
createBlur(): ShaderHandle {
const source = `#pragma shader "PP Blur"
#pragma version 300 es
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 34b33707b..b768fcf65 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2473,4 +2473,6 @@ export const NOTED = {
{"note":"Newly imported meshes are about 9% smaller on disk."},
"feat(render): a Temporal Anti-Aliasing effect accumulates frames, with the camera shifted a sub-pixel amount each one":
{"note":"A Temporal Anti-Aliasing post-process effect smooths edges across frames."},
+ "feat(render): a Screen Space Reflections effect mirrors the scene in the surfaces that face up":
+ {"note":"A Screen Space Reflections post-process effect mirrors props in floors."},
};