Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(render): a scene drawn below the screen's size is stretched back with a sharper resampler

yhhae6d85395dc0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index be99d4c55..56cee4c80 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,8 @@ published separately; it ships inside the editor.
### Added
- **Contact shadows and screen-space bounce light.** Contact Shadows and Screen Space GI effects add small sharp shadows at contacts and one bounce of colour between surfaces, and the weather softens the sun's shadows. `contactShadows` marches a short ray toward the scene's sun in the depth buffer; `ssgi` gathers eight rays a pixel over the G-buffer's normals at half size into a float target and averages the neighbours on the same surface before laying the light, times the receiver's colour, over the picture. The Weather component raises the sun's `sourceAngle` and lowers its `shadowStrength` as the sky clouds over.
+- **Ground-truth ambient occlusion.** Ground-truth ambient occlusion (GTAO) is a post-process effect beside SSAO: a crease darkens by how closed it really is and a flat floor stays white, with no bias to tune. It marches the depth buffer along a few slice planes through each pixel, finds the horizon on both sides and integrates the open arc against the surface's own normal; the blur and composite are SSAO's, the low quality level leaves it out, and the ambient-occlusion switch governs both.
+- **A sharper upscale.** A scene drawn at a render scale below 1 is stretched back to the screen with a resampler that keeps edges steeper than a bilinear copy. The resampler is Catmull-Rom clamped to the texels around each point, so it draws no halo; a whole-number stretch is still an exact copy. Measured on a sprite edge at the low level's 0.75 scale: the edge's 10-90% width is 1.04 px against 1.15 px bilinear (0.69 px at full size).
- **Motion blur, film grain and lens flare.** Motion blur, film grain and lens flare are post-process effects, and a moving mesh's motion now reaches effects on WebGPU too. Motion blur follows the camera by the depth buffer and a moving mesh by its own motion, with the shutter and a cap on the streak as its settings; it needs no temporal anti-aliasing and does not shift the picture. Film grain is mid-tone noise that crawls with the clock; lens flare throws ghosts and a halo from what is brighter than a threshold. The low quality level skips motion blur and lens flare. Also fixed: a 32-bit float texture an effect only reads texel by texel (the motion pass's) is no longer paired with a filtering sampler, which WebGPU refused — temporal anti-aliasing's object motion logged an error every frame there.
- **Shading layers.** The Model material has shading layers — clear coat, sheen, anisotropy, translucency, thin film, index of refraction and transmission — and a glTF with the matching extensions arrives using them. Clear coat puts a second, sharper highlight over the base and darkens what is under it; sheen is the soft grazing-angle light of cloth; anisotropy stretches a highlight across brushed grooves; translucency lets light wrap past the terminator and show through from behind; thin film shifts the reflection's hue with the angle; the index of refraction sets how strongly a non-metal reflects; transmission lets a surface show what is behind it, drawn blended with no refraction. Custom Lit shaders reach the same lighting through `applyLightingSurface` and a `SurfaceExt`. A layer's textures are not read — its factors are — and the import says so; a material that sets none of them draws as before.
- **Many lights.** A frame now carries up to 112 lights: sixteen are read for every pixel and the rest are tiled, so a street of lamps lights what each one reaches. The sun, the lights that cast a shadow and then the brightest go in the lighting loop as before; every other point and spot light without a shadow map is recorded per screen tile (16 x 12) and shaded only by the pixels it reaches, so the cost follows the lights near a pixel and not the lights in the scene. Tiled lights do not cast shadows, and the whole lighting block still fits the 16 KB a uniform block is guaranteed, so it runs on WebGL2 and WebGPU alike.
diff --git a/docs/astro/src/content/docs/performance/quality-levels.mdx b/docs/astro/src/content/docs/performance/quality-levels.mdx
index 24f76f2c3..16b0739c1 100644
--- a/docs/astro/src/content/docs/performance/quality-levels.mdx
+++ b/docs/astro/src/content/docs/performance/quality-levels.mdx
@@ -20,7 +20,7 @@ three profiles — **Low, Medium, High** — each saying what the renderer spend
| Auto | The level comes from the device: the first [device rule](#device-rules) that matches, otherwise memory and core count. |
**Dynamic resolution** lowers the scene's resolution under GPU pressure, down to the level's *Min render scale*, and
-brings it back when there is room; the UI stays at display resolution. **Target frame rate** is the frame time it works
+brings it back when there is room; the UI stays at display resolution. A scene drawn below the screen's size is stretched back with a Catmull-Rom resampler clamped to the texels around it, which keeps an edge steeper than a bilinear copy does without drawing a halo; a whole-number stretch is an exact copy instead. **Target frame rate** is the frame time it works
to.
## What a level holds
diff --git a/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx b/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
index 7a228003c..b6067ee4f 100644
--- a/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
+++ b/docs/astro/src/content/docs/zh-cn/performance/quality-levels.mdx
@@ -18,7 +18,7 @@ import { Aside } from '@astrojs/starlight/components';
| 低 / 中 / 高 | 所有设备都用该档位。 |
| 自动 | 档位由设备决定:第一条匹配的[设备规则](#设备规则),否则看内存和核心数。 |
-**动态分辨率**在 GPU 吃紧时降低场景分辨率,最低降到该档位的*最小渲染比例*,有余量时再升回来;UI 保持显示分辨率。
+**动态分辨率**在 GPU 吃紧时降低场景分辨率,最低降到该档位的*最小渲染比例*,有余量时再升回来;UI 保持显示分辨率。低于屏幕大小绘制的场景会用一个被周围纹素夹取的 Catmull-Rom 重采样器拉伸回来,边缘比双线性拷贝更陡,又不会画出光晕;整数倍的拉伸则是精确拷贝。
**目标帧率**是它要维持的帧时间。
## 一个档位里有什么
diff --git a/src/esengine/data/shaders/blit.esshader b/src/esengine/data/shaders/blit.esshader
index 328589ef4..18e3ee45e 100644
--- a/src/esengine/data/shaders/blit.esshader
+++ b/src/esengine/data/shaders/blit.esshader
@@ -9,8 +9,47 @@ in vec2 v_texCoord;
uniform sampler2D u_texture;
out vec4 fragColor;
+#ifdef ES_UPSCALE
+// The picture is being stretched over more pixels than it has. A bilinear copy softens every edge by the stretch; this is
+// Catmull-Rom (Jimenez's five taps of the hardware filter) clamped to what the four nearest texels span, so it sharpens an
+// edge without drawing the halo a plain cubic does.
+vec4 upscale(vec2 uv) {
+ vec2 size = vec2(textureSize(u_texture, 0));
+ vec2 pos = uv * size;
+ vec2 centre = floor(pos - 0.5) + 0.5;
+ vec2 f = pos - centre;
+ vec2 f2 = f * f;
+ vec2 f3 = f2 * f;
+ vec2 w0 = f2 - 0.5 * (f3 + f);
+ vec2 w1 = 1.5 * f3 - 2.5 * f2 + 1.0;
+ vec2 w3 = 0.5 * (f3 - f2);
+ vec2 w2 = 1.0 - w0 - w1 - w3;
+ vec2 w12 = w1 + w2;
+ vec2 t0 = (centre - 1.0) / size;
+ vec2 t3 = (centre + 2.0) / size;
+ vec2 t12 = (centre + w2 / w12) / size;
+ vec4 sum = texture(u_texture, vec2(t12.x, t0.y)) * (w12.x * w0.y)
+ + texture(u_texture, vec2(t0.x, t12.y)) * (w0.x * w12.y)
+ + texture(u_texture, t12) * (w12.x * w12.y)
+ + texture(u_texture, vec2(t3.x, t12.y)) * (w3.x * w12.y)
+ + texture(u_texture, vec2(t12.x, t3.y)) * (w12.x * w3.y);
+ sum /= (w12.x * w0.y + w0.x * w12.y + w12.x * w12.y + w3.x * w12.y + w12.x * w3.y);
+ ivec2 base = ivec2(centre - 0.5);
+ ivec2 hi = ivec2(size) - 1;
+ vec4 a = texelFetch(u_texture, clamp(base, ivec2(0), hi), 0);
+ vec4 b = texelFetch(u_texture, clamp(base + ivec2(1, 0), ivec2(0), hi), 0);
+ vec4 d = texelFetch(u_texture, clamp(base + ivec2(0, 1), ivec2(0), hi), 0);
+ vec4 e = texelFetch(u_texture, clamp(base + ivec2(1, 1), ivec2(0), hi), 0);
+ return clamp(sum, min(min(a, b), min(d, e)), max(max(a, b), max(d, e)));
+}
+#endif
+
void main() {
+#ifdef ES_UPSCALE
+ vec4 c = upscale(v_texCoord);
+#else
vec4 c = texture(u_texture, v_texCoord);
+#endif
// The editor view's exposure (u_time.z, stops): zero outside the editor, where
// exp2(0) = 1 leaves every pixel as it was. Before the curve, like any exposure.
c = vec4(c.rgb * exp2(u_time.z), c.a);
@@ -34,8 +73,44 @@ void main() {
#pragma end
#pragma fragment wgsl
+#ifdef ES_UPSCALE
+fn upscale(uv : vec2f) -> vec4f {
+ let size = vec2f(textureDimensions(t0, 0));
+ let pos = uv * size;
+ let centre = floor(pos - vec2f(0.5)) + vec2f(0.5);
+ let f = pos - centre;
+ let f2 = f * f;
+ let f3 = f2 * f;
+ let w0 = f2 - 0.5 * (f3 + f);
+ let w1 = 1.5 * f3 - 2.5 * f2 + vec2f(1.0);
+ let w3 = 0.5 * (f3 - f2);
+ let w2 = vec2f(1.0) - w0 - w1 - w3;
+ let w12 = w1 + w2;
+ let tapLo = (centre - vec2f(1.0)) / size;
+ let tapHi = (centre + vec2f(2.0)) / size;
+ let tapMid = (centre + w2 / w12) / size;
+ var sum = textureSampleLevel(t0, s0, vec2f(tapMid.x, tapLo.y), 0.0) * (w12.x * w0.y)
+ + textureSampleLevel(t0, s0, vec2f(tapLo.x, tapMid.y), 0.0) * (w0.x * w12.y)
+ + textureSampleLevel(t0, s0, tapMid, 0.0) * (w12.x * w12.y)
+ + textureSampleLevel(t0, s0, vec2f(tapHi.x, tapMid.y), 0.0) * (w3.x * w12.y)
+ + textureSampleLevel(t0, s0, vec2f(tapMid.x, tapHi.y), 0.0) * (w12.x * w3.y);
+ sum = sum / (w12.x * w0.y + w0.x * w12.y + w12.x * w12.y + w3.x * w12.y + w12.x * w3.y);
+ let base = vec2i(centre - vec2f(0.5));
+ let hi = vec2i(size) - vec2i(1);
+ let a = textureLoad(t0, clamp(base, vec2i(0), hi), 0);
+ let b = textureLoad(t0, clamp(base + vec2i(1, 0), vec2i(0), hi), 0);
+ let d = textureLoad(t0, clamp(base + vec2i(0, 1), vec2i(0), hi), 0);
+ let e = textureLoad(t0, clamp(base + vec2i(1, 1), vec2i(0), hi), 0);
+ return clamp(sum, min(min(a, b), min(d, e)), max(max(a, b), max(d, e)));
+}
+#endif
+
@fragment fn fs_main(v : VSOut) -> @location(0) vec4f {
+#ifdef ES_UPSCALE
+ var c = upscale(v.v_texCoord);
+#else
var c = textureSampleLevel(t0, s0, v.v_texCoord, 0.0);
+#endif
c = vec4f(c.rgb * exp2(tc.u_time.z), c.a);
#ifdef ES_TONEMAP
c = vec4f(acesFilmic(c.rgb), c.a);
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.cpp b/src/esengine/renderer/frame/PostProcessPipeline.cpp
index fd9e2d8aa..aeb08e1ea 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.cpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.cpp
@@ -98,6 +98,17 @@ void PostProcessPipeline::setViewExposure(f32 ev) {
* than at init: a project that never asks for a curve never pays for one.
*/
resource::ShaderHandle PostProcessPipeline::outputShader() {
+ if (presentUpscales()) {
+ const usize which = output_transform_ == OutputTransform::Neutral ? 1 : output_transform_ == OutputTransform::ACES ? 2 : 0;
+ if (!upscaleShaders_[which].isValid()) {
+ std::vector<std::string> features{"ES_UPSCALE"};
+ if (which == 1) features.push_back("ES_TONEMAP_NEUTRAL");
+ if (which == 2) features.push_back("ES_TONEMAP");
+ upscaleShaders_[which] = compileBlit(features);
+ if (!upscaleShaders_[which].isValid()) ES_LOG_ERROR("PostProcessPipeline: no upscaling blit; the present stays bilinear");
+ }
+ if (upscaleShaders_[which].isValid()) return upscaleShaders_[which];
+ }
if (output_transform_ == OutputTransform::Neutral) {
if (!neutralShader_.isValid()) {
neutralShader_ = compileBlit({"ES_TONEMAP_NEUTRAL"});
@@ -216,6 +227,11 @@ void PostProcessPipeline::shutdown() {
resourceManager_.releaseShader(tonemapShader_);
tonemapShader_ = resource::ShaderHandle{};
}
+ for (auto& shader : upscaleShaders_) {
+ if (!shader.isValid()) continue;
+ resourceManager_.releaseShader(shader);
+ shader = resource::ShaderHandle{};
+ }
if (neutralShader_.isValid()) {
resourceManager_.releaseShader(neutralShader_);
neutralShader_ = resource::ShaderHandle{};
diff --git a/src/esengine/renderer/frame/PostProcessPipeline.hpp b/src/esengine/renderer/frame/PostProcessPipeline.hpp
index 46849f574..b3f3d8483 100644
--- a/src/esengine/renderer/frame/PostProcessPipeline.hpp
+++ b/src/esengine/renderer/frame/PostProcessPipeline.hpp
@@ -401,6 +401,13 @@ public:
* one to disagree with. A whole multiple copies exactly, and
* interpolating it blurs a picture that needed no resampling.
*/
+ /// A present that stretches the chain's picture over a bigger rect by a factor that is not a whole number: where a
+ /// bilinear copy would blur it, and the blit resamples with a sharper filter instead.
+ bool presentUpscales() const {
+ return presentScales() && width_ > 0 && height_ > 0 && !presentIsWholeMultiple()
+ && (output_vp_w_ > width_ || output_vp_h_ > height_);
+ }
+
bool presentIsWholeMultiple() const {
if (!presentScales() || width_ == 0 || height_ == 0) return false;
if (output_vp_w_ % width_ || output_vp_h_ % height_) return false;
@@ -578,6 +585,8 @@ private:
/// The ES_TONEMAP variant of the same file. Invalid until a chain asks for it.
resource::ShaderHandle tonemapShader_;
resource::ShaderHandle neutralShader_;
+ /// The same three blits (plain, neutral, ACES) with the resampling upscaler, compiled when a present first needs one.
+ resource::ShaderHandle upscaleShaders_[3];
std::vector<PostProcessPass> passes_;
u32 width_ = 0;
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 6feabb869..9c60a7ac7 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -2493,6 +2493,10 @@ export const NOTED = {
{"note":"Skinned meshes are followed by their bones, from the palette they were last drawn with."},
"feat(render): fog follows the scene's sun and can walk the air against its shadow map":
{"note":"Height Fog and Light Shafts follow the scene's own sun, and Height Fog can carry the sun's shadows as volumetric light."},
+ "feat(render): ground-truth ambient occlusion joins the post-process effects":
+ {"note":"Ground-truth ambient occlusion (GTAO) is a post-process effect beside SSAO: a crease darkens by how closed it really is and a flat floor stays white, with no bias to tune."},
+ "feat(render): a scene drawn below the screen's size is stretched back with a sharper resampler":
+ {"note":"A scene drawn at a render scale below 1 is stretched back to the screen with a resampler that keeps edges steeper than a bilinear copy."},
"feat(render): motion blur, film grain and lens flare join the post-process effects":
{"note":"Motion blur, film grain and lens flare are post-process effects, and a moving mesh's motion now reaches effects on WebGPU too."},
"feat(render): materials carry a clear coat, sheen, anisotropy, translucency, thin film, index of refraction and transmission":